Sometimes, you want to create a new SendPort that is an exact clone or duplicate of another one in BizTalk Admin Console.

In July, 2014, Richard Seroter provided a BizTalk SendPort Duplicator tool to do this for you. So I recommend you check it out, but in this blog, I want to go through the basics, in a teach mode, of how to do it the “old fashioned way”, and at the same time, familiarize you with editing the dreaded BizTalk Binding files.

Hopefully you already know how to create a binding file, but just in case you don’t, open the BizTalk Admin Console, select the desired application (the binding file is unique to each application).

Export Bindings

By default it will go to this directory: C:\Users\YourUserName\Local\BizTalk\Binding Files\YourAppName.BindingInfo.xml, but you can override on the following screen:

Export Binding Parms

I had an app with about 6 orchestrations, 10 receive ports, and 16 send ports, and this spewed out a file that was 8413 lines long.  Binding files can be scary and intimidating the first time you open one with an editor.

So to duplicate my SendPort, I opened the above file with NotePad++. Obviously, I’m not going to show that whole file here.  What you need to know are the major structural elements of the file.
BizTalk Binding File Structure of XML

So based on the above, if I’m looking for a SendPort, you can probably guess I would find it in the SendPortCollection node.

So basically, my goal is to create a binding file with just one SendPort in it.  It still needs to be inside the SendPortCollection tag.  But all the other tags (elements) can be deleted, except the root element (with the name BindingInfo).  So I delete from the Timestamp down to the closing tag of the ModuleRefCollection.

Then I find the SendPort I want, and carefully delete the others. Here’s a brief example of an SMTP (email) SendPort.

Sample SendPort in BizTalk XML Binding File

So if this is the port you want to duplicate, you would delete all the other SendPorts and keep the close tag SendPortCollection.  You could then delete everything below that down to close root element tag named “BindingInfo”.

Make some change to the SendPort name, so that when  you load it back in, you will have a duplicate, but under a new name. (You can’t have two with the exact same name.)

Save the file, either in the same name or a new file name.

Then you are ready to import it, which is basically the opposite of the earlier export.

BizTalk Admin Console - Import Bindings

A file picker dialog box will appear, and you can pick the file or past the name, then click “Open” and that will start the upload.

If any error occurs, the entire import will fail, a dialog box will pop-up with a description of the error (hopefully not to cryptic), then you have to edit the file and try again.  You might want to make sure the file is valid XML before you import it.

After that, go back and refresh your application, and you should see your duplicated SendPort.

Why duplicate a SendPort? You might have a WCF SendPort, which has a rather complex set of properties.  Maybe you just need to have a duplicate of it, to change the URL to point to a different environment, for example TEST vs your QA (Quality Assurance) system.

Had a case today where I opened BizTalk Map, and the three fields “Script Assembly”, “Script Class”, and “Script Method” for a “Scripting Functoid” were all blank.  The “Select script type” was set to “External Assembly” (meaning a C# .DLL that you usually write yourself).  Yet I could compile and deploy the orchestration  Below is a sample screen shot:

Scripting_Functoid_Missing_External_Assembly

This apparently happened because the map was copied from one BizTalk Project to another.  The new project didn’t have the references set up.

So I copied the .DLL to a Libraries subfolder, then in Visual Studio Solution Explorer, I clicked on “References” under my project, and added the reference to the .DLL.  Then next time I opened that scripting functoid, the fields were all filled in properly.

Apparently that .DLL was in the GAC, which is why the compile/deploy worked.

Have you ever been confused by the long XPaths found in BizTalk? XPath is a great syntax used to select or query XML and return XML nodes or specific element/attribute values. XPath is good to know for BizTalk, .NET programming, and XSLT.

Example 1

For example, look at the schema below.

Demo BizTalk Schema

Demo BizTalk Schema

If you click on the “Name” element, then go to the properties window, you will see the following Xpath in the value of the field called “Property XPath”.

/*[local-name()='SchemaDemo2' and namespace-uri()='http://BizTalkSampleProject.SchemaDemo2']/*[local-name()='ShippingAddress' and namespace-uri()='']/*[local-name()='Name' and namespace-uri()='']

What does the above actually mean? W3Schools has a good tutorial on XPath, but it doesn’t cover dealing with namespaces. On this xpath syntax page, it does however define a predicate which is the XPath code in between the square brackes [].

Predicates are used to find a specific node or a node that contains a specific value.

Predicates are always embedded in square brackets.

In the table below we have listed some path expressions with predicates and the result of the expressions:

 

And here are the example they give, but no examples with namespaces!

Xpath_Predicate_Examples

So think of a predicate as a “Where Clause”. Even in SQL terminology, a predicate is something that results in a TRUE or FALSE.

So when you see the typical Microsoft generated XPath, it can be explained as follows. The / is the normal separator, and starting point from the current node. The “*” is the abbreviation for the “child” axis (explained more here: https://www.stylusstudio.com/docs/v2007/d_xpath95.html, http://www.informit.com/articles/article.aspx?p=102644, http://www.w3schools.com/xsl/xpath_axes.asp). It selects all element children of the context node.

But then we don’t really want all the nodes, so we apply the predicate or “where clause” – only give me the nodes where the local-name()=’SchemaDemo2′ and namespace-uri()=’http://BizTalkSampleProject.SchemaDemo2′ (and that must be enclosed in brackets). local-name() and namespace-uri() are functions (or internal variables?) that return the current element name and the current namespace). If you have the same namespace all the way through your schema, then you can leave off the namespace-uri() predicate.

In other words, the longer full syntax is often shortened to this:

/*[local-name()='SchemaDemo2']/*[local-name()='ShippingAddress']/*[local-name()='Name']

The only time this shortening would get you in trouble is if you had the same element under more than one namespace.

Example 2

Consider this example, where I created an Address schema, then I imported it into the Labels Schema, then made one Billing Address and one Shipping Address.

Nested_Schema

/*[local-name()='Labels' and namespace-uri()='http://BizTalkSampleProject.Labels']/*[local-name()='BillingAddress' and namespace-uri()='']/*[local-name()='Address' and namespace-uri()='http://BizTalkSampleProject.AddressSchema']/*[local-name()='Name' and namespace-uri()='']

It turns out that even in this case, you could shorten the XPath to the following:

/*[local-name()='Labels']/*[local-name()='BillingAddress']/*[local-name()='Address']/*[local-name()='Name']

So one case where you couldn’t shorten it would be when you had two Address nodes with different namespaces.

Testing XPath Online

Now, I want to go into some reasons why I wrote this blog. I was using C# to loop through XML of a bigger document.
Inside the loop, I have this chunk of XML in an XmlNode.

I’ve been testing XPath Online with http://www.xpathtester.com/xpath.
If I start my XPath with two slashes, it seems to be going back to the top of the entire XML document, rather than just searching the current node.

I originally had this xpath:

//*[local-name()='Property'][@Name='Name']

So basically, I my aim is to pick out the orchestration variable name, which is the example below is “strYNTraceOnFromSSO”;
It works fine in Xpath Tester because I have just the above text, but in C# it is returning some node outside of that text. (I guess I could reload the XML in a new XML document, but using this as an opportunity to get a better graso my XPath).

This was my XML, which actually came out of an .odx file.


Here is what the query looks like in XPath Tester: Xpath Tester So if I start my Xpath with /*, I would need to specify the parent node, right? This example returns the following error:

    /*[local-name()='Element']/[local-name()='Property'][@Name='Name']

returns error: ERROR – Failed to evaluate XPath expression: javax.xml.transform.TransformerException: A location step was expected following the ‘/’ or ‘//’ token.

Looking back, it’s an obvious error, but it wasn’t at the time. In easy XPath, you can just put a slash between the elements. But when using the predicates, you have to put /* to indicate “all elements where the name=’Property'”.

Why is there no slash or /* before the attribute? Because it’s a predicate on the predicate. Only return for me Property elements that have an attribute called “Name” that also has a value of “Name”.

Here is the correct result in XPath Tester, returning the selected Property element:
XPathTester_Example2

C# Code

Above example found the desired Property element only. In the sample code below I go down and pick off the value of the attribute.  This code basically loops through all the variables in an orchestration.

            string xpathTopLevelVariables = @"//*[local-name()='Element'][@Type='VariableDeclaration']";
            XmlNodeList xmlNodeList = xmlNodeServiceDeclaration.SelectNodes(xpathTopLevelVariables);

            foreach (XmlNode xmlNode in xmlNodeList)
            {
                XmlDocument xmlDoc2 = new XmlDocument();
                xmlDoc2.LoadXml(xmlNode.OuterXml);   // this seems unnecessary to me, but it works 
                string xpathProperty = "/*[local-name()='Element']/*[local-name()='Property'][@Name='Name']/@Value";
                string variableName = xmlDoc2.SelectSingleNode(xpathProperty).Value;
                // other unrelated code omitted 
           }

 


Note: For some reason, I had to reload the chunk of XML into a new xml document, otherwise I was getting all data from the top rather from the current node downward. I wanted to put the SelectSingleNode method directly on the xmlNode variable, but I didn’t solve that challenge.

I recently wrote a pipeline that needed to access the data, in order to use that data for setting one of the context properties (I can explain why in a future blog).  In the past, I might have used XmlDoc.Load and used XPath to process the document, but these days, I’m more savvy on how to use XmlTextReader.

I want to summarize and add some comments about an issue I recently had (reported on MSDN forum, but solved myself):

https://social.msdn.microsoft.com/Forums/en-US/8043cdef-de37-48f2-bbdc-a792d8dc286d/constructing-xmltextreader-causes-root-element-is-missing-in-biztalk-wcf-send-pipeline?forum=biztalkr2adapters

Why should we be using XmlTextReader in a pipeline instead of a System.Xml.XmlDocument?  As this StackOverflow article explains, it’s about memory usage.  If you know your BizTalk messages are small (and to me, that probably means under a megabyte), you could probably use XmlDocument and not see much of an issue (unless you were doing extremely high volume of messages).  XmlDocument will load the entire message in memory at one time.  So if your message was huge (like 1 Gig for example), each running instance of your pipeline would need 1 Gig of memory, and that could result in performance issues.  What if your trading partners sent you 30 1 Gig messages all within the same few minutes?

By using a “Stream,” that data can be processed in small chunks. With the BizTalk “VirtualStream” class/object, you can even set the buffer size. For more info, check out this blog for great background info on .NET Streams.

So here’s my tale of woe.  I created a pipeline, processed the stream using XmlTestReader, found the desired data and used it to set the OutboundTransportLocation (the URL of a WCF service, that I was trying to make dynamic, while yet using a static port).  My code worked fine when I hardcoded a value, but once I put in the XmlTextReader, it started blowing up with this error:

System.Xml.XmlException: Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.XmlReader.MoveToContent()
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.BizTalkBodyWriter.ValidateStreamContainsXml(Stream stream)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfMarshaller.CreateWcfMessage(CreateWcfMessageSettings settings)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendRequestMessage(IBaseMessage bizTalkMessage, IRequestChannel channel)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendMessage(IBaseMessage bizTalkMessage)

I searched the web for solutions, and didn’t find anything obvious, which is why I’m creating this blog. Somewhere from back in the deep dark memories of my brain, I recalled something about doing a Seek in pipelines, so I “googled” and found this article titled: Handling Incoming Data Streams in Pipeline Components.

That article made perfect sense. It says:

If you do not do this and the stream is read to the end in the current component, the next component receives what appears to be an empty stream because the data stream pointer was not set to the start of the stream. This can cause unexpected parsing and validation errors in subsequent pipeline components.

So if I’m at the end of the stream, and the stream gets passed on to the next step, and that step tried to read the stream, it would start at the current position of the stream, which could be the middle or end of the stream (depending on how I coded my XmlReader loop). Actually, I stopped as soon as I found the value I needed, so most likely, I left the seek position in the middle of the stream. So the next XmlTextReader starts from there, and it finds no root element, or basically invalid XML. If you take the second half of an XML file, it would not be well-formed; it would be missing it’s root element, and the final element wouldn’t match back to the root.

I cannot include my entire pipeline here, but here is short sample code using Virtual Stream in a BizTalk Pipeline. Instead of using the “Seek” method, that codes does the following to set the “Position” property, which accomplishes the same:

vStream.Position = 0;

For further info: MSDN Article: Optimizing Pipeline Performance

While there can probably be many reasons for this error, here’s one that I had:

msgNew() = “http://MySite.com/MyApp/MyService.svc”;

There was not value in the parentheses. For some reason, this does NOT put the red exclamation point on the Message Assignment shape.

 

I had been testing the following, then changed it, and forgot and left it blank

msgNew(BTS.OutboundTransportLocation) = “http://MySite.com/MyApp/MyService.svc”;

 

 

 

 

Pipeline components are an advanced aspect of BizTalk often overlooked or often implemented in a manner that breaks the careful streaming model of BizTalk Server.  Implementing pipeline components in a truly stream based way allows BizTalk to process messages of arbitrary and effectively unlimited size.  This example shows how to use a stream to archive received messages in a way that avoids reading the entire message into memory. 

https://code.msdn.microsoft.com/BizTalk-stream-based-d6ef0f1f

XpatherTester.com is the current “go to” tool I use when I want to test XPath (online and for free). I prefer the one built-in to Stylus Studio, but that’s a paid for product that I cannot install on each client’s computer.

To actually learn XPath, I suggest the W3Schools site: http://www.w3schools.com/xsl/xpath_intro.asp

To test the tool, you can copy/paste a sample XML from here: https://msdn.microsoft.com/en-us/library/ms762271%28v=vs.85%29.aspx

Step 1 – Paste in your XML to the tool

XPathTester_Step1_Paste_Your_XML

Step 2- Type in your XPath

For example, I typed in //book/title below. I have highlighted in yellow, because at first, it’s not intuitively obvious where to actually type in the XPath. Note: Do not type in the XPath in the Namespaces: box!

XPathTester_Step2_Type_Your_XPath

Step 3- View your results

The tool will leave your original XML on one tab of your browser, then the results will be opened in a new tab on your browser. This picture shows the original tab on the left, and the new tab on the right.

XPathTester_broiwser_tabs
Note that the “” xml wrapper is added to keep the results as well-formed XML. When you run your XPath in C# or other tools, that will not happen.

XPathTester_Step3_Results

Step 4- Retry as needed

When doing a complicated XPath, you might have to try several times. Close the results tab in your browser, and go back to your original tab to try again.

If you want to specify something like //ns1:books you can define namespace ns1 in the namespaces textarea at the top of the screen.  I haven’t placed with that much.  I tend to create XPath that uses the “localname” syntax, which is a little more difficult.  Use of this syntax is explained well on this StackOverflow page: http://stackoverflow.com/questions/10813653/xpath-select-node-based-in-a-condition-with-local-name.

For example:

//*[local-name()='a'][*[local-name()='aCode']='aaa']

Additional XPATH resources/samples for dealing with local-name and namespaces

Here are some old posts of mine when learning to do XPath back in 2004, it hasn’t really changed since then:

  1. http://www.stylusstudio.com/SSDN/default.asp?action=9&fid=23&read=3074
  2. http://www.stylusstudio.com/SSDN/default.asp?action=9&read=5018&fid=48&FirstTopic=1420&LastTopic=1439&style=WebBoard

One of the characteristics that separates the good from the great Biztalk developers and architects, is handling of errors, particularly SOAP errors.  This is often not tested sufficiently.

This blog by Shashidharan Krishnan does a good job of explaining how to catch a SOAP error in a BizTalk Orchestration:
http://blogs.msdn.com/b/biztalknotes/archive/2013/02/12/how-to-handle-fault-message-in-biztalk-server.aspx

I’d like to throw in some ideas that I implemented today.
Based on the blog above, I created the xpath below, a slightly simpler form to grab two of the main SOAP Fields. I only wanted the two most critical, in order to save space (I’ll discuss more about my column size limitations below). I didn’t want all the namespaces and SOAP wrappers, that just cluttered up any error that the user might see.

//SOAPEXECPTION_11 is my 'Exception Object Name' specified in the Catch (of the Scope) Properties 
strSoapFaultcode   = xpath(SOAPEXCEPTION_11, "string(//*[local-name()='faultcode'])");
strSoapFaultstring = xpath(SOAPEXCEPTION_11, "string(//*[local-name()='faultstring'])");

// format my own condensed error consisting of the above two parts 
strValue = "SoapFaultCode=" + strSoapFaultcode + " SoapFaultString=" + strSoapFaultstring; 

// Single quotes cause SQL to blow up, 
// truncate to 1000 (so we can put two fault codes in one DB field 
strValue = strValue.Replace("'","''");
vWebServiceAResponseMessage = strValue.Substring
   (0,System.Math.Min(strValue.Length,vMaxSizeEachSoapError));

The site where I’m working wanted to store the error back in a database table via a web service. Basically, the is a table that represents the canonical schema. A “poller” job extracts data from that table, writing a canonical XML to disk. An orchestration picks it up, and processes it. That same orchestration then calls a web services that was provided to me to update the same row of that table with the success or failure codes. If any webservice or SOAP errors are encountered, an flag is set, and that row can be re-queued to be be polled and processed again later after those errors have been fixed.

The database column for the error was a max of 2000 characters, and was not an XML column, just a varchar2(2000) column in an Oracle database. So if I passed to big of a field, the web service would blow up with a size issue on the SQL column. So it became my job to make sure my errors fit.

Now, to further complicate matters, imagine this scenario. I called WebService-A, then WebService-B. Each could have a SOAP error. Then I call WebService-C to actually update the table. So basically, I had to get a max of 1000 characters of error for each of the WebServices A and B, for a max of 2000. In some cases, you would not even call WebService-B if it were dependent on results from A; but in my case, they wanted to store the error from A into the backend ERP system (which is what WebService-B was updating).

So I was also dealing with a scenario where A could succeed and B could fail, A could fail and B could succeed, both could succeed or both could fail. I did this with just a normal decide shape with 4 or 5 branches. I set string variables there, then had one construct/message assignment afterward, using those string variables to construct the message request for WebService-C.

Further, if the SOAP message had single quotes in it, that was messing up SQL, so I had turn each single quote into two single quotes. (If you are not familiar with this, the classic case is LastName = O’Brian needs to be treated as O”Brian. Typcially, the database server is set up to handle to consecutive single quotes as one single quote, and helps avoid confusion with the single quotes used in the SQL syntax.

Here are two other thoughts to consider. Should your orchestration be flexible enough to handle SOAP 1.1 and SOAP 1.2 errors? I usually don’t go that far, but the guy I worked with did. It’s nice and flexible, but causes a little extra bloat in the orchestration.

For the final thought, there can be “hard” SOAP errors, or soft business/errors. At this client, the web service typically return a Result that was either set to “Ok” or some error. So in addition to checking the SOAP errors, I had to check that error as well. This is fairly typical.

In other orchestrations that I’ve recently written, I created a common error handler, then threw an internal exception or set an “IsError” variable/flag, so that the common error handler could run once as needed. That way, I could treat SOAP 1.1, SOAP 1.2, or soft/business errors using the same logic. Orchestrations don’t allow easily for re-useable code. You either have to call a sub-orchestration, use direct-binding to call another orchestration, call C# code, or try to create a code block that runs based on flags. Having separate orchestrations can make restart a big pain.

And what do you do when your exception handling webservice itself has an error?  I’ll leave that for you and your architect.  Here, we “pretty print” all XML request/responses that have errors, and email them to a distribution group.

How to cause a SOAP error to Test a BizTalk Orchestration

Changing the URL didn’t do it, that caused an XLANG/s exception, with a service not found exception, as shown below:

Error Description: System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://mysite.com/MyService/MyServiceName2.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more 

 


In the real world, an error like this would be fixed by changing the URL and resuming the orchestration.

One thing I do to force a SOAP exception is to edit the WCF SendPort, and modify the “Action” block on the General tab. I save the original values to NotePad++, then remove or change the values of the: <Operation Name=”value”…
You may have to restart the host instance, then run the test.

This results in the following error:

  a:ActionNotSupported
  The message with Action '&lt;BtsActionMapping xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"&gt;
  &lt;Operation Name="MyMethod" Action="http://tempuri.org/IMyService/MyMethod" /&gt;
&lt;/BtsActionMapping&gt;' cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and receiver) or a binding/security mismatch between the sender and the receiver.  Check that sender and receiver have the same contract and the same binding (including security requirements, e.g. Message, Transport, None).

When you are done testing your exceptions, then copy back what you saved in NotePad++.

If you happen to be the author of the webservice (or the author is inclined to work closely with you), you might want to code in a web service exception that will be sent back as SOAP exception. Use some flag or freaky/sneaky set of values to trigger the exception (make sure those values would never occur in the real production environment).