Often, after implementing a new system or application on BizTalk, we need to provide a report to management of how many files were processed.

If you keep an archive of all files in or out, you can use that with a simple PowerShell to accomplish the report. Here’s an example I found:

PowerShell Example 1 – Files by Date

$count = @{}
$size  = @{}

get-childitem d:\BizTalk\MyApp\Archive\EDI850Order\*.txt |
 foreach {
           $date = $_.lastwritetime.tostring('dd-MMM')
           $count[$date]++
           $size[$date] += $_.length
         }
 $count.keys |
  sort|
  foreach {
           [PSCustomObject]@{
            Date = $_
            'Number of files' = $Count[$_]
            Size = $Size[$_]
           } 
         } | format-table -AutoSize

Sample Results of Above Powershell

Date   Number of files   Size
----   ---------------   ----
08-Sep              14  37263
11-Sep              19  53761
12-Sep               7  26984
13-Sep              45 147575
14-Sep              44 154050

Example 2 – Multiple Folders – Files by App and Date

#################################################################
#
# 
# Neal Walters - 09/25/2017
# Script: FileCountByDateAllDirs.ps1 
# Purpose: Get a summary report of files processed 
#  by each app on each date 
#  based on archive files 
#
#################################################################
cls 
$count = @{}
$size  = @{}

$items  = @(Get-ChildItem 'd:\BizTalk\EDIHorizon\Archive\EDI850Order' -r)
$items += @(Get-ChildItem 'd:\BizTalk\ZLien\ArchiveOutbound' -r)

$items |
 foreach {
           $posFirstSlash  = $_.fullname.indexOf("\") 
           $posSecondSlash = $_.fullname.indexOf("\", $posFirstSlash+1) 
           $posThirdSlash  = $_.fullname.indexOf("\", $posSecondSlash+1) 
           $lenApp = $posThirdSlash - $posSecondSlash - 1 
           # Write-Host $app $posSecondSlash $posThirdSlash $_.fullname 
           $date = $_.lastwritetime.tostring('yyyy-MMM-dd')
           $app = $_.fullname.substring($posSecondSlash+1,$lenApp)
           $key = $app+" "+$date
           $count[$key]++
           $size[$key] += $_.length
         }

 $count.keys |
  sort|
  foreach {
           [PSCustomObject]@{
            App_Date = $_
            'Number of files' = $Count[$_]
            Size = $Size[$_]
           } 
         } | format-table -AutoSize

Sample Results of Above Powershell

App_Date         Number of files     Size
--------         ---------------     ----
App1 2017-Sep-18              35   119159
App1 2017-Sep-19              32   105811
App1 2017-Sep-20              35   116315
App1 2017-Sep-21              12    34450
App1 2017-Sep-22              24    85952
App2 2017-Sep-20               2 15460798
App2 2017-Sep-21               2 15457187
App2 2017-Sep-22               2 15472012

cls
$filename = "c:\Users\Neal\OneDrive\Documents\myFile.html"

#example of what I'm trying to pick out
<strong>mydomainname.com</strong>
$regexPattern = "<strong>(.*?)</strong></a>"

gc $filename | Select-String -Pattern $regexPattern -AllMatches | ForEach-Object {$_.matches.groups[1].value}

Note that the Matches returns two groups with subscripts 0 and 1. The subscript 0 contains the tags “strong” around the match. the subscript 1 contains just the captured text. Thus I put groups[1].value in the logic above. Groups is an object that has several variables; “Value” is the one we need here (see related blogs below).

When can take it to the next level and generate SQL statements to insert those domains into a SQL table.
This is done with one long line of code and using the pipeline (piping).

gc $filename | Select-String -Pattern $regexPattern -AllMatches  |  ForEach-Object {Write-Host "insert into domains values ('$($_.matches.groups[1].value)')"} 

Output is a list or the matching domain names to the console.

References that helped me get this:

https://powershell.org/forums/topic/how-to-get-a-regex-group-from-a-select-string-cmdlet/

https://stackoverflow.com/questions/25064249/command-line-to-extract-all-domain-names-referenced-in-a-file

See also my related blog on Powershell Regex and the objects that it returns (below).

 

 

If you are interesting in using C# to Monitor Biztalk, instead of Powershell to Monitor BizTalk, or Powershell to Monitor Diskspace, or Powershell to Monitor EventLogs (and send out emails) then check out this blog and sample code:
https://prashantbiztalkblogs.wordpress.com/2017/08/30/monitoring-biztalk-resources-programmatically-using-c/

I didn’t run it, but it looks like a good sample to get started with.

 

With the SFTP ports in BizTalk 2013, we are using the log function. BizTalk appends each SFTP log to the end of the existing log file.
So I created a Powershell Script to take an array of directory names, and run the same re-run and delete logic for each.
I also include the logic to delete a file over a certain retention period (see also shorter code for that in prior post: “Powershell to Delete Old Files“).


#################################################################
#
#
# Neal Walters - 09/05/2017
# Script: RenameSFTPLogsToAddDate.ps1
# Purpose: Rename a file such as "receive_internal.txt" to
# "receive_internal_2017_09_01.txt"
# so that each day has it's own separate file of SFTP messages.
# Also purge files over $retentionDays
# Use Task Scheduler to schedlue this script
# to run once late a night or early morning.
#
#################################################################

cls
$retentionDays = 30
$formatDateTime = get-date -f _yyyy_MM_dd__HH_mm_ss
$formatDate = get-date -f _yyyy_MM_dd
Write-Host “formatDateTime= $formatDateTime”
Write-Host “——————————————————-”
$renameCount = 0
$skipCount = 0
$deleteCount = 0

# Array of Directory Names (logic below will process each directory you specify here)
$DirNames = “d:\BizTalk\App1\SFTPLogs\”,
“d:\BizTalk\App2\SFTPLogs\”

Foreach ($DirName in $DirNames)
{
Write-Host “DirName=$DirName”
Get-ChildItem $DirName -Filter *.txt |
Foreach-Object
{

$fullname = $_.FullName.ToString();
$dirname = $_.Directory.ToString();
$filename = $_.Name.ToString();
$filenameOnly = [io.path]::GetFileNameWithoutExtension($filename)
$ext = [io.path]::GetExtension($filename) # this includes the ., for example .txt

#If filename contains an underscore, then we think it has a date in it.
#If no date found, we want to rename the file, otherwise leave it as it was.

# looking for 20 as the century… part of date 2017, 2018, 2019 etc…
# if (-Not ($filenameOnly.toString().Contains(“_20”) ) )
if (-Not ($filenameOnly -Match “_20”) )
{
Write-Host “OldName $fullname”
$content = Get-Content $_.FullName
$formatDate = get-date -f _yyyy_MM_dd

Write-Host “Filename=$filename”
$newFileName = $dirname + “\” + $filenameOnly + $formatDate + $ext
Write-Host “NewName $newFileName`n”
Rename-Item $fullname $newFileName
$renameCount++
}
else
{
#Write-Host “Skipping file=$filenameOnly because no match”
$skipCount++
}

#purge files over retentionDays days old
if ($_.LastWriteTime.AddDays($retentionDays) -lt (GET-DATE) -and -not $_.psiscontainer)
{
REMOVE-ITEM $_.fullname -force
$deleteCount++
}
} #end of Foreach-Object
Write-Host “Number of Files Renamed = $renameCount”
Write-Host “Number of Files Skipped = $skipCount (for rename)”
Write-Host “Number of Files Deleted = $deleteCount”
Write-Host “——————————————————-”
}