The following may help with a variety of custom pipeline component issues, including why your “Debug” is not catching when you expect it to. There were times today, where I would do “Debug Attach” to the BizTalk Host Instance, and the component would never go into debug mode.

1. Check that date/time of the .DLL in the GAC. There’s a chance you compiled it and though you GAC’ed it but it didn’t. Here’s the folder name to check:

c:\Windows\Microsoft.NET\assembly\GAC_MSIL

You might be using a Post Build Event like this to GAC the .DLL:


which is placed in the project properties dialog as shown below:

Check your View/Output in Visual Studio to see if you have message that it did or didn’t GAC successfully:

Example Successful GAC
1>  Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.0
1>  Copyright (c) Microsoft Corporation.  All rights reserved.
1>  
1>  Assembly successfully added to the cache
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
Example Failed GAC
1>  Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.0
1>  Copyright (c) Microsoft Corporation.  All rights reserved.
1>  
1>  Failure adding assembly to the cache:   The process cannot access the file because it is being used by another process.
1>  
1>C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets(4714,5): error MSB3073: The command "CALL "%VS140COMNTOOLS%vsvars32.bat"
1>C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets(4714,5): error MSB3073: gacutil.exe /i "C:\Users\nrwalters\Documents\Visual Studio 2015\Projects\ABC.Common\ABC.Common.BizTalkPipelineComponentEmail\bin\Debug\ABC.Common.BizTalkPipelineComponentEmail.dll"" exited with code 1.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped =========

Sometimes you have to close Visual Studio, then re-open it and do the ReBuild.
Once I was just doing “Build” and since the code didn’t change, it wasn’t rebuilding, and thus it wasn’t running the Post-Build Event, and thus the GAC didn’t get changed!

Other Steps

2. Always restart your host instance. If you have more than one, make sure you picked the right one.

3. When you do Debug Attach – you also have to make sure you select the proper BTNTSvc.exe. Use Task Manager to get the process ID based on the name, and then in Visual Studio, select based on the “ID” column.

Pipeline Components Directory?

Some blogs say that since BT 2006 this isn’t really needed; that you should just GAC your pipeline components.
Other people argue that it is still necessary. I seem to have success with the first otpion.

I started with this excellent working unzip pipeline component here (by Rui Machado).

I made the following changes to that code:
1) Put it in MyCompany.Common.etc… namespace and project (not shown here)
2) Added a Property Bag Item called FileMask (and a one for future use called “ParmReserved1” (this is standard coding, so I didn’t include here).
3) Added the FitsMask function from StackOverflow
4) Added the functionality to set the %SourceFileName% (aka ReceivedFileName, based on the files inside the zip instead of the zip file dropped).

public void Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
  IBaseMessage Temp = inmsg;
  // uses Ionic.Zip 
  using (ZipFile zip = ZipFile.Read(inmsg.BodyPart.GetOriginalDataStream()))
  {
    foreach (ZipEntry e in zip)
    {
      string strCheckFileName = System.IO.Path.GetFileName(e.FileName);

      if (FitsMask(strCheckFileName, FileMask))
      {
         var ms = new MemoryStream();
         IBaseMessage outMsg;
         outMsg = pc.GetMessageFactory().CreateMessage();
         outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);
         outMsg.Context = inmsg.Context;

        // Neal Walters - 05/08/2017 - added promoted field with the filename - 
        // This allows us to use the %sourcefilenam% macro in the SendPort to name the file. 
        outMsg.Context.Write("ReceivedFileName",
             "http://schemas.microsoft.com/BizTalk/2003/file-properties", 
             strCheckFileName);

        e.Extract(ms);
        string XMLMessage = Encoding.UTF8.GetString(ms.ToArray());

        MemoryStream mstemp = new System.IO.MemoryStream(
        System.Text.Encoding.UTF8.GetBytes(XMLMessage));

        outMsg.GetPart("Body").Data = mstemp;

       _msgs.Enqueue(outMsg);
      }
   }
 }
}

        private static bool FitsMask(string fileName, string fileMask)
        {
            string pattern =
                 '^' +
                 Regex.Escape(fileMask.Replace(".", "__DOT__")
                                 .Replace("*", "__STAR__")
                                 .Replace("?", "__QM__"))
                     .Replace("__DOT__", "[.]")
                     .Replace("__STAR__", ".*")
                     .Replace("__QM__", ".")
                 + '$';
            return new Regex(pattern, RegexOptions.IgnoreCase).IsMatch(fileName);
        }

The screen shot below show how the FileMask can be specified in the RecieveLocation (via use of the standard Property Bag interface):

(Sorry, don't have SnagIt yet at this client, so have to use the poor man's Snipping Tool).

I recently wrote a pipeline that needed to access the data, in order to use that data for setting one of the context properties (I can explain why in a future blog).  In the past, I might have used XmlDoc.Load and used XPath to process the document, but these days, I’m more savvy on how to use XmlTextReader.

I want to summarize and add some comments about an issue I recently had (reported on MSDN forum, but solved myself):

https://social.msdn.microsoft.com/Forums/en-US/8043cdef-de37-48f2-bbdc-a792d8dc286d/constructing-xmltextreader-causes-root-element-is-missing-in-biztalk-wcf-send-pipeline?forum=biztalkr2adapters

Why should we be using XmlTextReader in a pipeline instead of a System.Xml.XmlDocument?  As this StackOverflow article explains, it’s about memory usage.  If you know your BizTalk messages are small (and to me, that probably means under a megabyte), you could probably use XmlDocument and not see much of an issue (unless you were doing extremely high volume of messages).  XmlDocument will load the entire message in memory at one time.  So if your message was huge (like 1 Gig for example), each running instance of your pipeline would need 1 Gig of memory, and that could result in performance issues.  What if your trading partners sent you 30 1 Gig messages all within the same few minutes?

By using a “Stream,” that data can be processed in small chunks. With the BizTalk “VirtualStream” class/object, you can even set the buffer size. For more info, check out this blog for great background info on .NET Streams.

So here’s my tale of woe.  I created a pipeline, processed the stream using XmlTestReader, found the desired data and used it to set the OutboundTransportLocation (the URL of a WCF service, that I was trying to make dynamic, while yet using a static port).  My code worked fine when I hardcoded a value, but once I put in the XmlTextReader, it started blowing up with this error:

System.Xml.XmlException: Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.XmlReader.MoveToContent()
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkBodyWriter.ValidateStreamContainsXml(Stream stream)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfMarshaller.CreateWcfMessage(CreateWcfMessageSettings settings)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendRequestMessage(IBaseMessage bizTalkMessage, IRequestChannel channel)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendMessage(IBaseMessage bizTalkMessage)

I searched the web for solutions, and didn’t find anything obvious, which is why I’m creating this blog. Somewhere from back in the deep dark memories of my brain, I recalled something about doing a Seek in pipelines, so I “googled” and found this article titled: Handling Incoming Data Streams in Pipeline Components.

That article made perfect sense. It says:

If you do not do this and the stream is read to the end in the current component, the next component receives what appears to be an empty stream because the data stream pointer was not set to the start of the stream. This can cause unexpected parsing and validation errors in subsequent pipeline components.

So if I’m at the end of the stream, and the stream gets passed on to the next step, and that step tried to read the stream, it would start at the current position of the stream, which could be the middle or end of the stream (depending on how I coded my XmlReader loop). Actually, I stopped as soon as I found the value I needed, so most likely, I left the seek position in the middle of the stream. So the next XmlTextReader starts from there, and it finds no root element, or basically invalid XML. If you take the second half of an XML file, it would not be well-formed; it would be missing it’s root element, and the final element wouldn’t match back to the root.

I cannot include my entire pipeline here, but here is short sample code using Virtual Stream in a BizTalk Pipeline. Instead of using the “Seek” method, that codes does the following to set the “Position” property, which accomplishes the same:

vStream.Position = 0;

For further info: MSDN Article: Optimizing Pipeline Performance

Pipeline components are an advanced aspect of BizTalk often overlooked or often implemented in a manner that breaks the careful streaming model of BizTalk Server.  Implementing pipeline components in a truly stream based way allows BizTalk to process messages of arbitrary and effectively unlimited size.  This example shows how to use a stream to archive received messages in a way that avoids reading the entire message into memory. 

https://code.msdn.microsoft.com/BizTalk-stream-based-d6ef0f1f

You can an “Invalid Pipeline Component” error when you browse to a .DLL in the “Choose Toolbox Items” dialog box.

Error

“You have selected an invalid pipeline component assembly. Please check security settings for the assembly if you are loading it from an UNC path.”

Pipeline Component Error

Pipeline Component Error

Solution

This Patric Wellink blog put on the right path to solving, but it was not very clear exactly what to do.

The following steps solved for me:
1. Find your namespace in your code, and copy it:

PipelineComponent_Namespace

2. Right click on the Pipeline Components Project in Visual Studio Solution Explorer.
Paste the namespace above into the “Default namespace” below.

See below, how the value in the red box is different from the namespace in the red box above.

Change the Default Namespace to Match

Change the Default Namespace to Match

I used the BizTalk Pipeline Component Wizard available for free download here: https://btsplcw.codeplex.com.  It automatically creates the .resx (resource) file. But then I changed the namespace in the code, but not in the project properties.  I believe the resource file inherits the namespace from the project properties.  And if your code has a different namespace, that is the cause of the problem.

You can also work around by not using the resource manager.  You can hard-code some of the names that are in the resource, such as: ComponentDescription, ComponentName, ComponentVersion.

ResourceFile_Sample

Example Resource File for BizTalk Pipeline Component

At first I started in that direction, but hit a road block on how to load an icon properly.

You can also get the error if you don’t have the appropriate attributes “mark up”, such as:
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]  (see http://bencops.blogspot.com/2006/02/you-have-selected-invalid-pipeline.html).

 

I got this error today, and it was very difficult to figure out. I wasted about two hours on it.

There was a failure executing the receive pipeline: “pipeline name” Source: “Pipeline ” Receive Port: “SQLPollingWire” URI: “SQL://Server/ECData_SharedDev/4” Reason: Input string was not in a correct format.

I was using the polling of the built-in SQL adapter, and add a new element to my SQL query, and thus manually added the same element to the schema.

In the schem a, the new field w as defined as a string, not a number, so why would this blow-up in the pipeline?

The error is the same error that is thrown when you would get if you tried this: System.Convert.ToInt32(“test”).

The answer is that I did a quick-promote on it, and accidentally associated it with field in the property schema that was an “int”. Thus, when the pipeline was doing the promotion of the field, that’s when it died.

Another error solved, and more time wasted. Hope this helps if you get this error.

If more than one borrower occurs under “Borrowers”, we get the following error:

A message sent to adapter “FILE” on send port “YourSendPortName” with URI “c:BiztalkDemosYourSubFolder%SourceFileName%” is suspended.
Error details: Unable to read the stream produced by the pipeline.
Details: Cannot find definition for the input: BORROWER

The m ap w as using a Logical-If and a Value-Flattening Functoid.
The incoming test data was auto-generated by doing a right-click generate on a schema,
thus we had duplicate children (Borrowers) in the incoming data file.

By changing the test data, and setting various flags so that the Logical-If functoid only selected one child amongst the children, the problem went away.

This error was only detected when running via the pipeline. The error did not occur when doing a “Test Map” inside Visual Studio.