Issue/Scenario:

I have a new application that I’m building. I have for example three projects, App.Common, App.In, and App.Out.

We have a somewhat debatable policy of keeping one app separate from the others and almost no application cross-references (still on BizTalk 2010 – but in 2013 this is supposed alleviated, partially or in whole).

I had a web service schema copied in to App.Out. And I developed the proof of concept in App.Out.

Now, it turns out we have about 6 output messages.  There was a also some debate as to whether they should go in the single AppOut project, or in six different projects (App.Out1, App.Out2, etc…).  The architect wanted the second approach, so different developers could be assigned to each one in order to possible meet a tight development deadline.

Now, I don’t want that same web service schema to be deploy 6 times in the same application. So I moved the schemas to the App.Common.  And of course the map doesn’t compile because I change the location of the schemas.

Solution

I was afraid I was going to have to do map surgery, but it turns out I could just re-reference the schema in the map.  Since the schema and nodes are the same, none of the mapping was lost.

Internals of the Map

I was afraid I was going to have to do map surgery, but it turns out I could just re-reference the schema in the map.  Since the schema and nodes are the same, none of the mapping was lost.

But for fun, if I had to do map surgery, this is chat change. The Original “Before” is on top, and the “After” is on the bottom”.

NOTE: When you open a map there are no line breaks.  I use the XML tools in NotePad++ to format the XML (but don’t save it after formatting, always do the formatting on a copy).

You can see below how the “Reference Location” attribute changed.
I’m mapping from our Canonical schema to an internal web service schema.

 

<SrcTree RootNode_Name="Canonical_DATA">
<Reference Location="CanonicalArtifacts.Schemas.MY_CANONICAL_SCHEMA_V4" />
</SrcTree>
<TrgTree RootNode_Name="OrderUpdateHeaderByMessageID_New">
<Reference Location="SchemasWebSvc\B2BData_tempuri_org.xsd" />
</TrgTree>

<SrcTree RootNode_Name="Canonical_DATA">
<Reference Location="CanonicalArtifacts.Schemas.MY_CANONICAL_SCHEMA_V4" />
</SrcTree>
<TrgTree RootNode_Name="OrderUpdateHeaderByMessageID_New">
<Reference Location="AppOut.SchemasWebSvc.B2BData_tempuri_org" />
</TrgTree>

 

I was getting this strange error yesterday:

Error

map_X_to_Y.btm: error btm1023: Exception Caught: Object reference not set to an instance of an object.

Solution

What I had done was copied a map from one project to another.  I had either not copied the schema, or I had copied the map to a different folder where it couldn’t find one of the referenced schemas.   So either I moved the map or added the missing schema, and all was well.

At first I was focusing on Functoids and possible logic errors, but then I expanded the schema(s) on the left, and noticed the following:

Map_Missing_Schema

You can see in the picture above that InputMessagePart_0 was missing the schema.

 

 

Sample Code

XSLT_Bad_Value-Of_element

Error

MyMap.btm: error btm1023: Exception Caught: The ‘=’ character, hexadecimal value 0x3D, cannot be included in a name. Line 4, position 23.

Solution

This was a simple typo. I typed in “value-of-select=” instead of “value-of select=” (I erroneously included an extra dash between the “of” and the “select”). The parser saw the entire element name as “value-of-select=”, so of course, you cannot have an equal sign in an element name.

Context

I was trying to merge data from two incoming messages in a BizTalk Map, using the XSLT technique described here:
https://blogs.msdn.microsoft.com/biztalkcpr/2009/06/01/how-to-join-two-schemas-in-a-map-when-they-contain-namespaces/

Two things that I would like to add to that blog:

  1. You need to use an XSLT Template instead of just inline XSLT because you need to pass a parameter.
  2. When using the XSLT Template, you need to connect the Scripting Functoid to a field on the right side of the map; otherwise the Template will never get called.

 

How do you test a multi-part map, i.e. a map with multiple input messages?

To test a normal map, you just take an sample XML file, click the map (in Visual Studio Solution Explorer), set the property of the filename into the value of the property “TestMap Input”, then right-click on the map and select “Test Map”. The output is shown in the Output window, and you can do a CNTL-Click on the output file to view the results.

Maps created in the normal mapper usually just have one input message; but maps created in an orchestration can easily have multiple input messages (just by selecting more than one incoming message in Transform shape). This creates a special structure, or wrapper around the messages. You can still do a test-map, but the file you input to the test map must be in the special format as follows:

<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
	<InputMessagePart_0>
		<ns1:Schema1RootElement xmlns:ns1="http://MyNamespace.Schema1">
		</ns1:Schema1RootElement>
	</InputMessagePart_0>
	<InputMessagePart_1>
		<ns2:Schema2RootElement xmlns:ns2="http://MyNamespace.Schema2">
		</ns2:Schema2RootElement>
	</InputMessagePart_1>
</ns0:Root>

You would put your own elements and attribute between Schema1RootElement and it’s closing element, as well as between Schema1RootElement2 and it’s closing element.

So normally, you use some text or XML editor (like NotePad++) and paste data samples, then wrap them with the XML element wrappers shown above.

If your schemas don’t have target namespaces, you can simplify as follows:

<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
	<InputMessagePart_0>
		<Schema1RootElement>
		</Schema1RootElement>
	</InputMessagePart_0>
	<InputMessagePart_1>
		<Schema2RootElement>
		</Schema2RootElement>
	</InputMessagePart_1>
</ns0:Root>

So basically the root element is part of a special namespace called the “aggschema” (aggregate schema). Then, as many message as you added each are found, but each is wrapped with N) where N start with 0, and increments once per each message.

I had a case today where I was mapping two fields N01 and N02 to two side by side fields in the output schema.

N01 was working, and N02 wasn’t.  Both were under an condition that a certain variable MESSAGE_TYPE was equal to a certain value.  Yet one worked, the other didn’t?   How did I go about debugging this?

I did a right click “Validate Map”, and opened the XSLT.  I searched for N01, and found the following.

XSLT_Debug_Sample_1

Can you spot any clues to the issue?  If not scroll on down…

 

 

 

 

 

 

 

 

 

 

 

 

 

In the image below, I have highlighted some of the code. The N01 is wrapped by one IF statement, so I would expect the same IF statement around N02.

But when I look at N02, I see to nested IF statements.

XSLT_Debug_Sample_2

You can find the variables to understand what the tests are.  So I searched for the extra test around N02, i.e. the variable v16, and found this.

XSLT_Debug_Sample_3

That jogged my memory.  I had another page on the map, where we were using a “trick” to not map certain fields.
This was a RosettaNet schema, and it contained many nodes with default values that were filling up the output document with unnecessary junk. The “trick” is to map “false” to the fields you don’t want to map.  So I had essentially told the map that I did not want to output this element (even though I had mapped something to it).

Of course, this is only one small case, but I hope you like the methodology, and can use it to apply to your own debugging.  Learning XSLT of course helps.  The XSLT generated by BizTalk can be a little confusing because of all the “userCSharp” routines, and all the variables, but it can help you find issues in your map while testing.

Also note, I would not have found this issue unless I was carefully unit-testing.  I hand-crafted 5 different input test files, and was running each one of them through the “Test Map” feature, and manually scanning the output.

 

Had a case today where I opened BizTalk Map, and the three fields “Script Assembly”, “Script Class”, and “Script Method” for a “Scripting Functoid” were all blank.  The “Select script type” was set to “External Assembly” (meaning a C# .DLL that you usually write yourself).  Yet I could compile and deploy the orchestration  Below is a sample screen shot:

Scripting_Functoid_Missing_External_Assembly

This apparently happened because the map was copied from one BizTalk Project to another.  The new project didn’t have the references set up.

So I copied the .DLL to a Libraries subfolder, then in Visual Studio Solution Explorer, I clicked on “References” under my project, and added the reference to the .DLL.  Then next time I opened that scripting functoid, the fields were all filled in properly.

Apparently that .DLL was in the GAC, which is why the compile/deploy worked.

Sometimes, you might need to get the SQL Server Name of the SQL Server where the BizTalkMgmtDB

The C# code below, uses the registry to get that server name, and also builds a connection screen to some database on that server (where you pass the Database Name as a parameter). MgmtDBServer (in the rk.GetValue below), referes to the BizTalkMgtmDB.

        public static string DynamicSQLConnectionStringOLEDB(string DBName)
        {
            try
            {
                Microsoft.Win32.RegistryKey rk = Microsoft.Win32.Registry.LocalMachine.OpenSubKey("SOFTWARE\\Microsoft\\BizTalk Server\\3.0\\Administration", false);
                string ConnString = 
                  "Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=" + 
                  DBName + ";Data Source=" + rk.GetValue("MgmtDBServer").ToString();
                return ConnString;
            }
            catch
            {
                return "";
            }
        }

Here’s one example how this could be used. Suppose you have a database that contains lookup tables, and that database is on the same SQL server as the BizTalkMgmtDB database. The Scripting Functoid below calls the C# to get the connection string, and becomes the second parameter to the Database Lookup Functoid.

GetSQLServerForBizTalkFunction

The scripting functoid is setup as shown in the picture below. First, make sure you reference the .net assembly created from the C# code above. Then select that in “Script Assembly”, and pick the class in “Script Class”.

ScriptingFunctoidSQLServer

Related blogs:

Keith Lim

MSDN Restoring the BizTalk Configuration Database

Xref Database Routines

 

What do you when you cannot connect an element on the left to one on the right in a Biztalk map?  Today, I had a case where I was trying to update a map created by another developer.  I thought the values on the right side were missing, but later I noticed they had a value in the property.  At first, I thought these values were defaulted in the schema, but a closer look at the schema showed no default values.

So this comes to show that you can set the value of an element by just typing in the value in the “Value” property.  Once you have set that property, you can no longer drag and drop any value into that element on the GUI part of the map.

Here’s an example with a small schema with just two elements:

BizTalk Map - Setting Value Property

BizTalk Map – Setting Value Property

Below is an example what happens when you try to connect an element (after the value is set).  The icon will change to the “no drop” (circle with a slash through it) symbol.

Cannot Map to Target Element

Cannot Map to Target Element

So this goes to show that after 15 years or more of BizTalk, you can still learn new tricks and find new issues.

 

BizTalk Mapper and Altova MapForce are both tools used to data from one schema to another. A schema internally is treated as XML, but may represent a flat file, CSV, a SQL table to stored proc, or a web service request/response message. Typically the job of a B2B developer is to map data from one format to the other. In this blog, I compare mapping from schema “Flight” to schema “FlightNew” using both tools.

BizTalk

Below is a sample BizTalk Map.

BizTalk Example Map

BizTalk Example Map

BizTalk has various functions called Functoids. Internally, these are implemented in C# embedded in the XSLT code. There is a red concatenate Functoid in the sample above, and two Scripting Component Functoids. A Scripting Component allows you to type or paste-in your own custom C# code. So this is how I implemented the difference in the two dates.

BizTalk_Script_Functoid_CSharp

You can right-click on the map properties in the Solution Explorer, and set the desired XML input file. Then right-click the map name in Solution Explorer, and click “Test Map”. After the map runs, you will see some text in the Output window. You can CNTL-CLICK on the output file, then you will then see the results below.

BizTalk - Results of "Test Map"

BizTalk – Results of “Test Map”

Altova MapForce

Altova MapForce - Full Screen

Altova MapForce – Full Screen

MapForce has a “datetime-diff” function, but the output has to go to a field defined with a type of xs:duration. I had actually never heard of this data type before. You can see the “concat” function in XSLT is essentially identical to the one in BizTalk.

Altova MapForce - Zoom In on Mapping

Altova MapForce – Zoom In on Mapping

Click the “Output” tab under the map to view the conversion of the data to the new schema format:

Altova MapForce - Sample Output

Altova MapForce – Sample Output

 

However, some functions are only supported in C# and Java. When switching to XSLT, I found the function below to be unavailable. You can see the striped lines appear in the function below.

Altova_MapForce_NonXSLT_Function

I didn’t have time to try to reimplement that function in XSLT.  So below is the same map without being able to compute the difference between ScheduledDateTIme and ActualDateTime.

When in XSLT mode, and XSLT tab appears below the mapper. Below is the view of the XSLT code generated from the visual tool.

One of the advantages of MapForce is that it can build XSLT, and has a two-way editor. You can edit the visual mapper, or edit the XSLT code directly, and the other automatically syncs.

Altova MapForce XSLT Tab

Altova MapForce XSLT Tab

Altova can also generate Java, C# or C++ code to perform the transformation.  You can take that code, and call it from your application.  MapForce has EDI functionality built in, so you can map for example 850 Purchase Orders into XML data.

One of the advantages of MapForce is that it can build XSLT, and has a two-way editor. You can edit the visual mapper, or edit the XSLT code directly, and the other automatically syncs.

By the way, BizTalk does support XSLT, but once you switch to what they call “Custom XSLT”, you can no longer use the visual “GUI” editor.

So in this overview, we see two ways to map data using two different but similar tools.  We saw that the scripting functions can be quite different in the two products.

Yes, today we are stepping into our magic time machine, and going back to BizTalk 2004. Why? Because that’s what I had to do at work today. And I forgot a few basics from that long ago time. I needed to test a map in Visual Studio – and I was totally lost. I googled, couldn’t find anything easily. Had to ask a co-worker, even though I used BizTalk 2004 and even used to teach it. Thus the reason for this blog.

The trick is that you have to do right-click (on the map) then click “Properties”. A separate “Properties Window” pops-up, different than the Visual Studio component properties (that we are used to seeing by default in the lower right corner of the screen).

Here’s what it looks like:

BT2004 Map Properties

BT2004 Map Properties

The properties screen looks like this:

BT2004 Map Property Pages

BT2004 Map Property Pages

From there just type or paste your filename in the “TestMap Input Instance”. Then later, when you do a right-click “Test Map” on your map, that’s the file it will use for input.

Just to compare, in BT2010 with VS2010, it looks like this:

BT2010 - Test Map - Map Properties

BT2010 – Test Map – Map Properties

You still use the Output window to see the results of the test-map, CNTL-CLICK on the output file name to open it and see the results. If you get a lot of errors or warnings, you could try setting “Validate TestMap Input” and “Validate TestMap Output” to False.