$filename = "c:\Users\Neal\OneDrive\Documents\myFile.html"

#example of what I'm trying to pick out
$regexPattern = "<strong>(.*?)</strong></a>"

gc $filename | Select-String -Pattern $regexPattern -AllMatches | ForEach-Object {$_.matches.groups[1].value}

Note that the Matches returns two groups with subscripts 0 and 1. The subscript 0 contains the tags “strong” around the match. the subscript 1 contains just the captured text. Thus I put groups[1].value in the logic above. Groups is an object that has several variables; “Value” is the one we need here (see related blogs below).

When can take it to the next level and generate SQL statements to insert those domains into a SQL table.
This is done with one long line of code and using the pipeline (piping).

gc $filename | Select-String -Pattern $regexPattern -AllMatches  |  ForEach-Object {Write-Host "insert into domains values ('$($_.matches.groups[1].value)')"} 

Output is a list or the matching domain names to the console.

References that helped me get this:

See also my related blog on Powershell Regex and the objects that it returns (below).



If you are interesting in using C# to Monitor Biztalk, instead of Powershell to Monitor BizTalk, or Powershell to Monitor Diskspace, or Powershell to Monitor EventLogs (and send out emails) then check out this blog and sample code:

I didn’t run it, but it looks like a good sample to get started with.


With the SFTP ports in BizTalk 2013, we are using the log function. BizTalk appends each SFTP log to the end of the existing log file.
So I created a Powershell Script to take an array of directory names, and run the same re-run and delete logic for each.
I also include the logic to delete a file over a certain retention period (see also shorter code for that in prior post: “Powershell to Delete Old Files“).

# Neal Walters - 09/05/2017
# Script: RenameSFTPLogsToAddDate.ps1
# Purpose: Rename a file such as "receive_internal.txt" to
# "receive_internal_2017_09_01.txt"
# so that each day has it's own separate file of SFTP messages.
# Also purge files over $retentionDays
# Use Task Scheduler to schedlue this script
# to run once late a night or early morning.

$retentionDays = 30
$formatDateTime = get-date -f _yyyy_MM_dd__HH_mm_ss
$formatDate = get-date -f _yyyy_MM_dd
Write-Host “formatDateTime= $formatDateTime”
Write-Host “——————————————————-”
$renameCount = 0
$skipCount = 0
$deleteCount = 0

# Array of Directory Names (logic below will process each directory you specify here)
$DirNames = “d:\BizTalk\App1\SFTPLogs\”,

Foreach ($DirName in $DirNames)
Write-Host “DirName=$DirName”
Get-ChildItem $DirName -Filter *.txt |

$fullname = $_.FullName.ToString();
$dirname = $_.Directory.ToString();
$filename = $_.Name.ToString();
$filenameOnly = [io.path]::GetFileNameWithoutExtension($filename)
$ext = [io.path]::GetExtension($filename) # this includes the ., for example .txt

#If filename contains an underscore, then we think it has a date in it.
#If no date found, we want to rename the file, otherwise leave it as it was.

# looking for 20 as the century… part of date 2017, 2018, 2019 etc…
# if (-Not ($filenameOnly.toString().Contains(“_20”) ) )
if (-Not ($filenameOnly -Match “_20”) )
Write-Host “OldName $fullname”
$content = Get-Content $_.FullName
$formatDate = get-date -f _yyyy_MM_dd

Write-Host “Filename=$filename”
$newFileName = $dirname + “\” + $filenameOnly + $formatDate + $ext
Write-Host “NewName $newFileName`n”
Rename-Item $fullname $newFileName
#Write-Host “Skipping file=$filenameOnly because no match”

#purge files over retentionDays days old
if ($_.LastWriteTime.AddDays($retentionDays) -lt (GET-DATE) -and -not $_.psiscontainer)
REMOVE-ITEM $_.fullname -force
} #end of Foreach-Object
Write-Host “Number of Files Renamed = $renameCount”
Write-Host “Number of Files Skipped = $skipCount (for rename)”
Write-Host “Number of Files Deleted = $deleteCount”
Write-Host “——————————————————-”

You get the following error when you do a BTDF deploy.


x does not belong to the same application as “Y” or its references.

Full example of error

Information: Importing bindings "C:\Users\...\Projects\MyEDIProject\MyEDIProject.Deployment\PortBindings.xml" into application "MyEDIProject" in BizTalk configuration database (server="MyServer1", database="BizTalkMgmtDb")...
Error: Failed to update binding information.
"Microsoft.BizTalk.Edi.DefaultPipelines.EdiSend" could not be bound to "sp_MyProj_Process_EDI_File". The artifact "Microsoft.BizTalk.Edi.DefaultPipelines.EdiSend" does not belong to the same application as "sp_MyProj_Process_EDI_File" or its references.


The pipeline is in a different project. When BTDF deletes your project and redploys it, it loses the project references. You must add the project cross references in the BTDFProj file, as shown below.

<AppsToReference Include=?MyProj2? />

The above shows you how to add a “Project Reference” to your BTDF configuration file.

The one thing I didn’t account for (yet), is the case where different files can have different EDI delimiters. Technically, you should look for the end of the ISA segment to find the delimiters, and use those in the RegEx match. For now, I’m assuming the field delimiter is * and the segment delimiter is the tilda (~).


I was archiving the EDI files in BizTalk with the filename set to “%datetime%_%MessageID%_EDI.txt”. I decided it would be better to name the files COMPANYNAME_DOCTYPE_ORDERNO_ORDERDATE_%datetime%_%MessageID%_EDI.txt.
NOTE: I could have done this logic in a custom C# BizTalk Pipeline, but decided to do it after the fact with a more simple Powershell than would be easier for administrative staff to maintain and update.

Sample 1 – Just test the parsing

With this sample, you can copy the contents of a file into the $ediText string, and test.

#Note subsituted " with `" in the string to escape the quotes within quotes issue 
$ediText = "ISA*00*          *00*          *ZZ*MYCUSTOMER*ZZ*MYCOUNTRY*170823*1610*U*00401*000000117*0*T*:~GS*PO*BTS-SENDER*RECEIVE-APP*170823*1610*117*T*00401~ST*850*0117~BEG*00*NE*391949**20170828~N1*BY*DELIVERY-ADDRESS~N1*ST*DELIVERY-ADDRESS~N3*1420 MAINSTREET DR~N4*DALLAS*TX*12345~PO1*1*5.00*EA*4.350**IN*106889~PID*F****SAND MIX ( SSM80 )~PO1*2*1.00*etc...~"; 

$CompanyID  = [regex]::match($ediText,'.*ISA\*.*?\*.*?\*.*?\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
$OrderNum   = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
$OrderDate  = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*.*?\*.*?\*(.*?)[~\*].*').Groups[1].Value
$EdiDocType = [regex]::match($content,'.~ST\*(.*?)[~\*].*').Groups[1].Value

Write-Host "CompanyID = $CompanyID"; 
Write-Host "OrderNum = $OrderNum"; 
Write-Host "OrderDate= $OrderDate"; 
Write-Host "EdiDocType= $EdiDocType"; 

Sample 2 – Renaming Files Based on EDI Key Fields


$DirName = "d:\BizTalk\EDIHorizon\Archive\EDI850Order\"

#only rename files that start with the year, 2017, 2018, etc...  thus 20*.txt 
Get-ChildItem $Dirname -Filter 20*.txt | 
Foreach-Object {

    $fullname = $_.FullName.ToString();  
    $dirname = $_.Directory.ToString(); 
    $filename = $_.Name.ToString(); 

    Write-Host "OldName $fullname"
    $content = Get-Content $_.FullName

    $CompanyID  = [regex]::match($content,'.*ISA\*.*?\*.*?\*.*?\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
    $OrderNum   = [regex]::match($content,'.*BEG\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
    $OrderDate  = [regex]::match($content,'.*BEG\*.*?\*.*?\*.*?\*.*?\*(.*?)[~\*].*').Groups[1].Value
    $EdiDocType = [regex]::match($content,'.~ST\*(.*?)[~\*].*').Groups[1].Value
    Write-Host "$OrderNum $OrderDate"

    Write-Host "Filename=$filename"
    $newFileName = $dirname + "\" + $CompanyID + "_" + $EdiDocType + "_" + $OrderNum + "_" + $OrderDate  + "_" + $filename
    Write-Host "NewName $newFileName`n" 
    Rename-Item $fullname $newFileName 

Having a filename like this will make it faster to search the archives for certain types of orders or files from a certain partner, or do do quick counts, based on the filename alone. For example, how many files did we get from XYZ company yesterday and today? This could be done in BizTalk with BAM as well, but my current client opted out of the overhead and complexity of BAM, especially since BizTalk was (for the most part), just passing the files around, not creating them.

The variable $EdiDocType above represents something like and 850, 855, 856, 810, 997, etc…

I might add one more feature. Many of the trading partner don’t use name, but some Dun number, phone number, or other ID number. I might have a lookup table to translate the code to a shortname that represents that trading partner.

Business Requirement:

Only send an email to the credit department if the filename contains the word “EXPORT” and and ends with .txt.

We cannot put *EXPORT*.txt in the Receive Location, because there are other files coming in with different patterns.

Question: How would you do that in BizTalk?

BizTalk has filters on the send ports. So in theory, you could filter on the filename. But guess what?
There is no operator for “contains”, or “matches”, or any RegEx (Regular Expressions).

The image below shows the field we would use: FILE.ReceivedFileName, but there is no operator that helps solve this problem.

My Solution:

On the ReceivePort, I created my own custom pipeline component, did the matching myself, and promoted fields if the results were true. The admin/developer can identify the name of the promoted field as one of the values in my Pipeline Component. He also provides the filemask he wants to match.

The receive port pipeline configuration is shown below.

I created a Property Schema that corresponds:

Then in the SendPort, I can test the promoted fields that were promoted by that Receive Pipeline component.

The pipeline component is too big to include here, but here are some key fragments.

First, I found the code below on StackOverflow to do file matching like DOS does:

        private static bool FitsMask(string fileName, string fileMask)
            string pattern =
                 '^' +
                 Regex.Escape(fileMask.Replace(".", "__DOT__")
                                 .Replace("*", "__STAR__")
                                 .Replace("?", "__QM__"))
                     .Replace("__DOT__", "[.]")
                     .Replace("__STAR__", ".*")
                     .Replace("__QM__", ".")
                 + '$';
            bool result = new Regex(pattern, RegexOptions.IgnoreCase).IsMatch(fileName);
            return result; 

setPromotedField is a common function that I could easily call over and over, as shown in the code bock following this one.

 * Neal Walters 
 * 05/19/2017 
 * The purpose of this is to allow for more complex file matching, 
 * and allow the SendPort to do a subscription when the file does NOT contain a given mask. 
 * For example, in BillTrust, there was need to write file to disk if filename 
 * contains "Export" but not "*.txt".
 * So this component is put in a pipeline that runs on the Receive, and sets 
 * a series of promoted fields FileMatchGroup## (01-07) 
 * that can be used as Filters in the SendPort. 
 * Each of the 7 fields come in a pair, e.g. 
 * FileMask01Matches and FileMask01Group 
 * The matches field will be something like *.txt, and when true, the value of 
 * FileMask01Group will be set in the promoted fields, otherwise it is created with a value of blank. 
 * For further flexibility, The FileMask01Matches can contain an array of Matches, such as: 
 * *.txt,*.csv,*.xls 
 * The program will split on the comma, and if any one of the masks is true, 
 * then the FileMask01Group will be set. 

        private static void setPromotedField(string seqNum,
                                                string strMatches,
                                                string strMaskGroup,
                                                string argReceivedFileName,
                                                IBaseMessage pInMsg)
            string promotedFieldNamespace = "https://MyCompany.Common.PropertySchemas.CommonPropertySchema";

            // See the comments at the top of this pipeline component to explain this logic. 

            if (strMatches != null && strMatches.Length > 0)
                //Before doing the split/array, this was the logic: 
                //bool fitsMaskResult = FitsMask(argReceivedFileName, strMatches);

                // allow multiple patterns, comma separated 
                // Pipe symbol was used, because Windows files cannot have pipe symbol, 
                // and because comma looks a lot like a comma in the 
                // BizTalk Admin Console configuration screens 

                string[] matchPatterns = strMatches.Split('|');
                bool fitsMaskResult = false;
                bool foundAtLeastOneMatch = false; 
                string debugSummary = "";

                // Loop through each of the split items (each mask/pattern) 
                // If any one of them is true, we count it as a match. 
                foreach (string matchPattern in matchPatterns)
                    fitsMaskResult = FitsMask(argReceivedFileName, matchPattern);
                    debugSummary = debugSummary + matchPattern + "=" + fitsMaskResult + ",";
                    if (fitsMaskResult)
                        foundAtLeastOneMatch = true; 
                string strSetValue = "";
                if (foundAtLeastOneMatch)
                    strSetValue = strMaskGroup;

                // Example: FileMask##Group - we substituted the two char seqNum to get the field name 
                // Note: We create a promoted field even if the match is false; 
                // this is to ensure that we can test the field in subequent SendPort Filter statements. 
                string varName = "FileMask" + seqNum + "Group";
                pInMsg.Context.Promote(varName, promotedFieldNamespace, strSetValue);


The above “setPromotedField” function is called like this:

	string receivedFileName = pInMsg.Context.Read(

	// Repeat exact same process for 7 propertyBag pairs of fields 

	setPromotedField("01", this.FileMask01Matches, this.FileMask01Group, receivedFileName, pInMsg);
	setPromotedField("02", this.FileMask02Matches, this.FileMask02Group, receivedFileName, pInMsg);
	setPromotedField("03", this.FileMask03Matches, this.FileMask03Group, receivedFileName, pInMsg);
	setPromotedField("04", this.FileMask04Matches, this.FileMask04Group, receivedFileName, pInMsg);
	setPromotedField("05", this.FileMask05Matches, this.FileMask05Group, receivedFileName, pInMsg);
	setPromotedField("06", this.FileMask06Matches, this.FileMask06Group, receivedFileName, pInMsg);
	setPromotedField("07", this.FileMask07Matches, this.FileMask07Group, receivedFileName, pInMsg);

The underlying connection was closed: The connection was closed unexpectedly.

I’m sure there are numerous reasons for this error, but here’s one I encountered today.

First the background. We have BizTalk server connecting to an internal Open/AS2 server running on Linux. That Linux server was re-installed on a Virtual Machine; and I had agreed with the Linux guy to change the AS2-ID (defined in the Trading Parties) from BiztalkTest to BizTalkProd. However, he didn’t make the change on his side.

So when he looked at the logs, he could see that it was receiving data, but as he described it “the server didn’t know what to do with it.” That means, it didn’t generate the MDN and send it back to BizTalk.

I had defined my SendPort was a two-way (Static Solicit-Reponse) and was using content-based routing. So we sent the message to the AS2 server, and didn’t get a response back, giving the “underlying connection closed unexpectedly” error.

Here are some samples you can use to list, start/stop, enable/disable, enlist/unenlist and so on your Send Ports, Receive Ports, and Recieve Locations.

Note the path hierarchy. You must specify you application name. I’m using “BizTalk EDI Application” since anyone with EDI installed will have the same results.

Sample Script

Add-PSSnapIn -Name BiztalkFactory.PowerShell.Extensions  #NOTE: Must be in 32-bit version of Powershellto use this SnapIn

cd "Biztalk:\Applications\BizTalk EDI Application\Send Ports"
Get-ChildItem | ft -auto      # or just type in "DIR", the "| ft -auto" avoids the Name being cut off with the elipse ... 
Write-Host "`nSend Ports - List Completed"
Get-ChildItem |Where-Object { $_.Name -match 'ResendPort' } | Unenlist-SendPort
Get-ChildItem |Where-Object { $_.Name -match 'ResendPort' } | Start-SendPort

cd "Biztalk:\Applications\BizTalk EDI Application\Receive Ports"
Get-ChildItem | ft -auto      # or just type in "DIR", the "| ft -auto" avoids the Name being cut off with the elipse ... 
Write-Host "`nReceive Ports - List Completed"

cd "Biztalk:\Applications\BizTalk EDI Application\Receive Locations"
Get-ChildItem | ft -auto      # or just type in "DIR", the "| ft -auto" avoids the Name being cut off with the elipse ... 
Write-Host "`nReceive Ports - List Completed"

cd "Biztalk:\Applications\BizTalk EDI Application\Receive Locations"
Get-ChildItem |Where-Object { $_.Name -match 'ResendReceiveLocation' } | Disable-ReceiveLocation
Get-ChildItem |Where-Object { $_.Name -match 'ResendReceiveLocation' } | Enable-ReceiveLocation

Note: I didn't put any Write-Host to show when the ports were succesfully stopped or started, you could add that i you want it.

Output of running the above script

    Path: BiztalkFactory.PowerShell.Extensions\BizTalk::Biztalk:\Applications\BizTalk EDI Application\Send Ports

Name       Status  Application            
----       ------  -----------            
ResendPort Started BizTalk EDI Application

Send Ports - List Completed

    Path: BiztalkFactory.PowerShell.Extensions\BizTalk::Biztalk:\Applications\BizTalk EDI Application\Receive Ports

Name                        Application            
----                        -----------            
BatchControlMessageRecvPort BizTalk EDI Application
ResendReceivePort           BizTalk EDI Application

Receive Ports - List Completed

    Path: BiztalkFactory.PowerShell.Extensions\BizTalk::Biztalk:\Applications\BizTalk EDI Application\Receive Locations

Name                       Enabled Application            
----                       ------- -----------            
BatchControlMessageRecvLoc True    BizTalk EDI Application
ResendReceiveLocation      True    BizTalk EDI Application

Receive Ports - List Completed

PS BizTalk:\Applications\BizTalk EDI Application\Receive Locations> 

When you install the Powershell Extensions for BizTalk, there is a readme.txt file created ("d:\Program Files (x86)\Microsoft BizTalk Server 2016\SDK\Utilities\PowerShell\readme.txt") that will show you all the cmdlets that are available (such as Start-SendPort, Unenlist-SendPort, etc...)

When dealing with Host Instances, you must specify the fully-qualified name, or use a couple of tricks I show in this blog.

I’ve been setting up some pre-canned scripts that I can just open when needed, and that have already been tested. Today I was looking at listing, starting, and stopping BizTalk Host Instances.

Powershell Script

Add-PSSnapIn -Name BiztalkFactory.PowerShell.Extensions  #NOTE: Must be in 32-bit version of Powershellto use this SnapIn

cd "Biztalk:\Platform Settings\Host Instances"
Get-ChildItem | ft -auto      # or just type in "DIR", the "| ft -auto" avoids the Name being cut off with the elipse ... 
Write-Host "`nList Completed"

#Demonstrating 3 ways: 
#  1) Use fully qualified host instance name (a big pain to type it correctly) 
#  2) Use the -match operator to match on a string 
#  3) Use the Host name instead
# With last two, it will start/stop all host-instances on all machines

Stop-HostInstance "Microsoft BizTalk Server OrchHost SERVER01"
Write-Host "Stopped"

#Get-ChildItem |Where-Object { $_.Name -match 'OrchHost' } |Start-HostInstance
Get-ChildItem |Where-Object { $_.Host -eq 'OrchHost' } |Start-HostInstance
Write-Host "Started"

Results of running the above Powershell Script

    Path: BiztalkFactory.PowerShell.Extensions\BizTalk::Biztalk:\Platform Settings\Host Instances

Name                                                           Host Name                 Windows Group                         Running Server
----                                                           ---------                 -------------                         --------------
Microsoft BizTalk Server BizTalkServerApplication SERVER01  BizTalkServerApplication  BizTalk Application Users             SERVER01   
Microsoft BizTalk Server BizTalkServerIsolatedHost SERVER01 BizTalkServerIsolatedHost BizTalk Isolated Host Users           SERVER01   
Microsoft BizTalk Server TrackingHost SERVER01              TrackingHost              SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server AS2HostReceive SERVER01            AS2HostReceive            SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server AS2HostSend SERVER01               AS2HostSend               SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server SFTPHostReceive SERVER01           SFTPHostReceive           SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server SFTPHostSend SERVER01              SFTPHostSend              SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server FileHostReceive SERVER01           FileHostReceive           SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server FileHostSend SERVER01              FileHostSend              SERVER01\BizTalk Application Users SERVER01   
Microsoft BizTalk Server OrchHost SERVER01                  OrchHost                  SERVER01\BizTalk Application Users SERVER01   

List Completed

PS BizTalk:\Platform Settings\Host Instances&gt; 

The use of “-ft auto” is used to avoid the ellipsis and show the full name (rather than truncating it).

After you add Host and Host-Instances to BizTalk, they are not useful and cannot be associated with Send/Receive ports until you relate the Host to the Adapter. This can be done in BizTalk Admin Console, but below is a program that allows you to do it with the BizTalk Powershell Extensions.

See related script to use Powershell to add new BizTalk Hosts and Host Instances.

Add-PSSnapIn -Name BiztalkFactory.PowerShell.Extensions  #NOTE: Must be in 32-bit version of Powershellto use this SnapIn
#get-PsSnapIn -registered   ### list registered Snap-In's

function UpdateAdapter($AdapterName, $HostName, $Direction)
        # modeled after code from: 

        Write-Host "Starting of `$Adapter=$AdapterName and `$Host=$HostName `$Direction=$Direction"
        #Example: Set-Location –Path '..\SFTP'
        $Pathname = "BizTalk:\Platform Settings\Adapters\$AdapterName"
        Write-Host "`$PathName=$PathName"
        Set-Location –Path $PathName 

        if ($Direction -eq "Send" -or $Direction -eq "Both") 
            Write-Host "Setup Sending" 
            ### Sending 
                                                           #Example:  -eq "SFTP Send Handler (Sending_64)"
            $VarAdapterHandler = Get-ChildItem | Where-Object{$_.Name -eq "$AdapterName Send Handler ($HostName)"}
            if($VarAdapterHandler.Name -eq $null)
                   Write-Host "Adding $AdapterName handler for Host=$HostName"
                   #New-Item  -Path .\Sending_64 -HostName Sending_64 -Direction Send 
                   New-Item  -Path .\$HostName -HostName $HostName -Direction Send

        if ($Direction -eq "Receive" -or $Direction -eq "Both") 
            Write-Host "Setup Receiving" 
            ### Receiving
            $VarAdapterHandler = Get-ChildItem | Where-Object{$_.Name -eq "$AdapterName Receive Handler ($HostName)"}
            if($VarAdapterHandler.Name -eq $null)
                   Write-Host "Adding SFTP Receive handler for Receiving_64 Host"
                   New-Item  -Path .\$Hostname -HostName $HostName -Direction Receive
        Write-Host "End of $AdapterName and $HostName"
        Write-Host "----"


### MAIN CODE HERE - Calls Function Above once per Host/Host-Instance ### 

#Pass Three Parms: 1) AdapterName, 2) HostName, 3) Direction [Send/Recive/Both]
#UpdateAdapter  "TrackingHost"   "Both" 

UpdateAdapter "HTTP" "AS2HostReceive" "Receive"
UpdateAdapter "HTTP" "AS2HostSend"    "Send"

UpdateAdapter "SFTP" "SFTPHostReceive" "Receive"
UpdateAdapter "SFTP" "SFTPHostSend"    "Send"

UpdateAdapter "FILE" "FileHostReceive" "Receive"
UpdateAdapter "FILE" "FileHostSend"    "Send"

Write-Host "Script Completed"