You get the following error when you do a BTDF deploy.

Error:

x does not belong to the same application as “Y” or its references.

Full example of error


Information: Importing bindings "C:\Users\...\Projects\MyEDIProject\MyEDIProject.Deployment\PortBindings.xml" into application "MyEDIProject" in BizTalk configuration database (server="MyServer1", database="BizTalkMgmtDb")...
Error: Failed to update binding information.
"Microsoft.BizTalk.Edi.DefaultPipelines.EdiSend" could not be bound to "sp_MyProj_Process_EDI_File". The artifact "Microsoft.BizTalk.Edi.DefaultPipelines.EdiSend" does not belong to the same application as "sp_MyProj_Process_EDI_File" or its references.

Solution:

The pipeline is in a different project. When BTDF deletes your project and redploys it, it loses the project references. You must add the project cross references in the BTDFProj file, as shown below.

<code>
<ItemGroup>
<AppsToReference Include=?MyProj2? />
</ItemGroup>
</code>

The above shows you how to add a “Project Reference” to your BTDF configuration file.

The one thing I didn’t account for (yet), is the case where different files can have different EDI delimiters. Technically, you should look for the end of the ISA segment to find the delimiters, and use those in the RegEx match. For now, I’m assuming the field delimiter is * and the segment delimiter is the tilda (~).

Requirement

I was archiving the EDI files in BizTalk with the filename set to “%datetime%_%MessageID%_EDI.txt”. I decided it would be better to name the files COMPANYNAME_DOCTYPE_ORDERNO_ORDERDATE_%datetime%_%MessageID%_EDI.txt.
NOTE: I could have done this logic in a custom C# BizTalk Pipeline, but decided to do it after the fact with a more simple Powershell than would be easier for administrative staff to maintain and update.

Sample 1 – Just test the parsing

With this sample, you can copy the contents of a file into the $ediText string, and test.

cls
#Note subsituted " with `" in the string to escape the quotes within quotes issue 
$ediText = "ISA*00*          *00*          *ZZ*MYCUSTOMER*ZZ*MYCOUNTRY*170823*1610*U*00401*000000117*0*T*:~GS*PO*BTS-SENDER*RECEIVE-APP*170823*1610*117*T*00401~ST*850*0117~BEG*00*NE*391949**20170828~N1*BY*DELIVERY-ADDRESS~N1*ST*DELIVERY-ADDRESS~N3*1420 MAINSTREET DR~N4*DALLAS*TX*12345~PO1*1*5.00*EA*4.350**IN*106889~PID*F****SAND MIX ( SSM80 )~PO1*2*1.00*etc...~"; 

$CompanyID  = [regex]::match($ediText,'.*ISA\*.*?\*.*?\*.*?\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
$OrderNum   = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
$OrderDate  = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*.*?\*.*?\*(.*?)[~\*].*').Groups[1].Value
$EdiDocType = [regex]::match($content,'.~ST\*(.*?)[~\*].*').Groups[1].Value

Write-Host "CompanyID = $CompanyID"; 
Write-Host "OrderNum = $OrderNum"; 
Write-Host "OrderDate= $OrderDate"; 
Write-Host "EdiDocType= $EdiDocType"; 

Sample 2 – Renaming Files Based on EDI Key Fields

cls

$DirName = "d:\BizTalk\EDIHorizon\Archive\EDI850Order\"

#only rename files that start with the year, 2017, 2018, etc...  thus 20*.txt 
Get-ChildItem $Dirname -Filter 20*.txt | 
Foreach-Object {

    $fullname = $_.FullName.ToString();  
    $dirname = $_.Directory.ToString(); 
    $filename = $_.Name.ToString(); 

    Write-Host "OldName $fullname"
    $content = Get-Content $_.FullName

    $CompanyID  = [regex]::match($content,'.*ISA\*.*?\*.*?\*.*?\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
    $OrderNum   = [regex]::match($content,'.*BEG\*.*?\*.*?\*(.*?)\*.*').Groups[1].Value
    $OrderDate  = [regex]::match($content,'.*BEG\*.*?\*.*?\*.*?\*.*?\*(.*?)[~\*].*').Groups[1].Value
    $EdiDocType = [regex]::match($content,'.~ST\*(.*?)[~\*].*').Groups[1].Value
    Write-Host "$OrderNum $OrderDate"

    Write-Host "Filename=$filename"
    $newFileName = $dirname + "\" + $CompanyID + "_" + $EdiDocType + "_" + $OrderNum + "_" + $OrderDate  + "_" + $filename
    Write-Host "NewName $newFileName`n" 
    Rename-Item $fullname $newFileName 
}

Having a filename like this will make it faster to search the archives for certain types of orders or files from a certain partner, or do do quick counts, based on the filename alone. For example, how many files did we get from XYZ company yesterday and today? This could be done in BizTalk with BAM as well, but my current client opted out of the overhead and complexity of BAM, especially since BizTalk was (for the most part), just passing the files around, not creating them.

The variable $EdiDocType above represents something like and 850, 855, 856, 810, 997, etc…

I might add one more feature. Many of the trading partner don’t use name, but some Dun number, phone number, or other ID number. I might have a lookup table to translate the code to a shortname that represents that trading partner.

Business Requirement:

Only send an email to the credit department if the filename contains the word “EXPORT” and and ends with .txt.

We cannot put *EXPORT*.txt in the Receive Location, because there are other files coming in with different patterns.

Question: How would you do that in BizTalk?

BizTalk has filters on the send ports. So in theory, you could filter on the filename. But guess what?
There is no operator for “contains”, or “matches”, or any RegEx (Regular Expressions).

The image below shows the field we would use: FILE.ReceivedFileName, but there is no operator that helps solve this problem.

My Solution:

On the ReceivePort, I created my own custom pipeline component, did the matching myself, and promoted fields if the results were true. The admin/developer can identify the name of the promoted field as one of the values in my Pipeline Component. He also provides the filemask he wants to match.

The receive port pipeline configuration is shown below.

I created a Property Schema that corresponds:

Then in the SendPort, I can test the promoted fields that were promoted by that Receive Pipeline component.

The pipeline component is too big to include here, but here are some key fragments.

First, I found the code below on StackOverflow to do file matching like DOS does:

        private static bool FitsMask(string fileName, string fileMask)
        {
            string pattern =
                 '^' +
                 Regex.Escape(fileMask.Replace(".", "__DOT__")
                                 .Replace("*", "__STAR__")
                                 .Replace("?", "__QM__"))
                     .Replace("__DOT__", "[.]")
                     .Replace("__STAR__", ".*")
                     .Replace("__QM__", ".")
                 + '$';
            bool result = new Regex(pattern, RegexOptions.IgnoreCase).IsMatch(fileName);
            return result; 
        }

setPromotedField is a common function that I could easily call over and over, as shown in the code bock following this one.

/*
 * Neal Walters 
 * 05/19/2017 
 * 
 * The purpose of this is to allow for more complex file matching, 
 * and allow the SendPort to do a subscription when the file does NOT contain a given mask. 
 * For example, in BillTrust, there was need to write file to disk if filename 
 * contains "Export" but not "*.txt".
 * So this component is put in a pipeline that runs on the Receive, and sets 
 * a series of promoted fields FileMatchGroup## (01-07) 
 * that can be used as Filters in the SendPort. 
 * 
 * Each of the 7 fields come in a pair, e.g. 
 * FileMask01Matches and FileMask01Group 
 * The matches field will be something like *.txt, and when true, the value of 
 * FileMask01Group will be set in the promoted fields, otherwise it is created with a value of blank. 
 * 
 * For further flexibility, The FileMask01Matches can contain an array of Matches, such as: 
 * *.txt,*.csv,*.xls 
 * The program will split on the comma, and if any one of the masks is true, 
 * then the FileMask01Group will be set. 
 * 
 * 
 */

        private static void setPromotedField(string seqNum,
                                                string strMatches,
                                                string strMaskGroup,
                                                string argReceivedFileName,
                                                IBaseMessage pInMsg)
        {
            string promotedFieldNamespace = "https://MyCompany.Common.PropertySchemas.CommonPropertySchema";

            // See the comments at the top of this pipeline component to explain this logic. 

            if (strMatches != null && strMatches.Length > 0)
            {
                //Before doing the split/array, this was the logic: 
                //bool fitsMaskResult = FitsMask(argReceivedFileName, strMatches);

                // allow multiple patterns, comma separated 
                // Pipe symbol was used, because Windows files cannot have pipe symbol, 
                // and because comma looks a lot like a comma in the 
                // BizTalk Admin Console configuration screens 

                string[] matchPatterns = strMatches.Split('|');
                bool fitsMaskResult = false;
                bool foundAtLeastOneMatch = false; 
                string debugSummary = "";

                // Loop through each of the split items (each mask/pattern) 
                // If any one of them is true, we count it as a match. 
                foreach (string matchPattern in matchPatterns)
                {
                    fitsMaskResult = FitsMask(argReceivedFileName, matchPattern);
                    debugSummary = debugSummary + matchPattern + "=" + fitsMaskResult + ",";
                    if (fitsMaskResult)
                    {
                        foundAtLeastOneMatch = true; 
                    }
                }
                string strSetValue = "";
                if (foundAtLeastOneMatch)
                {
                    strSetValue = strMaskGroup;
                }

                // Example: FileMask##Group - we substituted the two char seqNum to get the field name 
                // Note: We create a promoted field even if the match is false; 
                // this is to ensure that we can test the field in subequent SendPort Filter statements. 
                string varName = "FileMask" + seqNum + "Group";
                pInMsg.Context.Promote(varName, promotedFieldNamespace, strSetValue);
            }

        }

The above “setPromotedField” function is called like this:

	string receivedFileName = pInMsg.Context.Read(
		   "ReceivedFileName",
		   "http://schemas.microsoft.com/BizTalk/2003/file-properties")
		   .ToString();


	// Repeat exact same process for 7 propertyBag pairs of fields 

	setPromotedField("01", this.FileMask01Matches, this.FileMask01Group, receivedFileName, pInMsg);
	setPromotedField("02", this.FileMask02Matches, this.FileMask02Group, receivedFileName, pInMsg);
	setPromotedField("03", this.FileMask03Matches, this.FileMask03Group, receivedFileName, pInMsg);
	setPromotedField("04", this.FileMask04Matches, this.FileMask04Group, receivedFileName, pInMsg);
	setPromotedField("05", this.FileMask05Matches, this.FileMask05Group, receivedFileName, pInMsg);
	setPromotedField("06", this.FileMask06Matches, this.FileMask06Group, receivedFileName, pInMsg);
	setPromotedField("07", this.FileMask07Matches, this.FileMask07Group, receivedFileName, pInMsg);


The underlying connection was closed: The connection was closed unexpectedly.

I’m sure there are numerous reasons for this error, but here’s one I encountered today.

First the background. We have BizTalk server connecting to an internal Open/AS2 server running on Linux. That Linux server was re-installed on a Virtual Machine; and I had agreed with the Linux guy to change the AS2-ID (defined in the Trading Parties) from BiztalkTest to BizTalkProd. However, he didn’t make the change on his side.

So when he looked at the logs, he could see that it was receiving data, but as he described it “the server didn’t know what to do with it.” That means, it didn’t generate the MDN and send it back to BizTalk.

I had defined my SendPort was a two-way (Static Solicit-Reponse) and was using content-based routing. So we sent the message to the AS2 server, and didn’t get a response back, giving the “underlying connection closed unexpectedly” error.