If you get the error “An xpath expression must be of the form …” (in an expression shape of a BizTalk orchestration), here’s one possible solution.

I was using a multi-part message, so instead of msgCreateSPServiceOrderAckResp, I had had to specify msgCreateSPServiceOrderAckResp.parameters.

The compiler just shows the error “The expression that you have entered is not valid”. But when you find the small red mark, and mouse over it, you see the following:

Here is the corrected version:

 

 

 

Issue/Scenario:

I have a new application that I’m building. I have for example three projects, App.Common, App.In, and App.Out.

We have a somewhat debatable policy of keeping one app separate from the others and almost no application cross-references (still on BizTalk 2010 – but in 2013 this is supposed alleviated, partially or in whole).

I had a web service schema copied in to App.Out. And I developed the proof of concept in App.Out.

Now, it turns out we have about 6 output messages.  There was a also some debate as to whether they should go in the single AppOut project, or in six different projects (App.Out1, App.Out2, etc…).  The architect wanted the second approach, so different developers could be assigned to each one in order to possible meet a tight development deadline.

Now, I don’t want that same web service schema to be deploy 6 times in the same application. So I moved the schemas to the App.Common.  And of course the map doesn’t compile because I change the location of the schemas.

Solution

I was afraid I was going to have to do map surgery, but it turns out I could just re-reference the schema in the map.  Since the schema and nodes are the same, none of the mapping was lost.

Internals of the Map

I was afraid I was going to have to do map surgery, but it turns out I could just re-reference the schema in the map.  Since the schema and nodes are the same, none of the mapping was lost.

But for fun, if I had to do map surgery, this is chat change. The Original “Before” is on top, and the “After” is on the bottom”.

NOTE: When you open a map there are no line breaks.  I use the XML tools in NotePad++ to format the XML (but don’t save it after formatting, always do the formatting on a copy).

You can see below how the “Reference Location” attribute changed.
I’m mapping from our Canonical schema to an internal web service schema.

 

<SrcTree RootNode_Name="Canonical_DATA">
<Reference Location="CanonicalArtifacts.Schemas.MY_CANONICAL_SCHEMA_V4" />
</SrcTree>
<TrgTree RootNode_Name="OrderUpdateHeaderByMessageID_New">
<Reference Location="SchemasWebSvc\B2BData_tempuri_org.xsd" />
</TrgTree>

<SrcTree RootNode_Name="Canonical_DATA">
<Reference Location="CanonicalArtifacts.Schemas.MY_CANONICAL_SCHEMA_V4" />
</SrcTree>
<TrgTree RootNode_Name="OrderUpdateHeaderByMessageID_New">
<Reference Location="AppOut.SchemasWebSvc.B2BData_tempuri_org" />
</TrgTree>

 

I was getting this strange error yesterday:

Error

map_X_to_Y.btm: error btm1023: Exception Caught: Object reference not set to an instance of an object.

Solution

What I had done was copied a map from one project to another.  I had either not copied the schema, or I had copied the map to a different folder where it couldn’t find one of the referenced schemas.   So either I moved the map or added the missing schema, and all was well.

At first I was focusing on Functoids and possible logic errors, but then I expanded the schema(s) on the left, and noticed the following:

Map_Missing_Schema

You can see in the picture above that InputMessagePart_0 was missing the schema.

 

 

Sample Code

XSLT_Bad_Value-Of_element

Error

MyMap.btm: error btm1023: Exception Caught: The ‘=’ character, hexadecimal value 0x3D, cannot be included in a name. Line 4, position 23.

Solution

This was a simple typo. I typed in “value-of-select=” instead of “value-of select=” (I erroneously included an extra dash between the “of” and the “select”). The parser saw the entire element name as “value-of-select=”, so of course, you cannot have an equal sign in an element name.

Context

I was trying to merge data from two incoming messages in a BizTalk Map, using the XSLT technique described here:
https://blogs.msdn.microsoft.com/biztalkcpr/2009/06/01/how-to-join-two-schemas-in-a-map-when-they-contain-namespaces/

Two things that I would like to add to that blog:

  1. You need to use an XSLT Template instead of just inline XSLT because you need to pass a parameter.
  2. When using the XSLT Template, you need to connect the Scripting Functoid to a field on the right side of the map; otherwise the Template will never get called.

 

How do you test a multi-part map, i.e. a map with multiple input messages?

To test a normal map, you just take an sample XML file, click the map (in Visual Studio Solution Explorer), set the property of the filename into the value of the property “TestMap Input”, then right-click on the map and select “Test Map”. The output is shown in the Output window, and you can do a CNTL-Click on the output file to view the results.

Maps created in the normal mapper usually just have one input message; but maps created in an orchestration can easily have multiple input messages (just by selecting more than one incoming message in Transform shape). This creates a special structure, or wrapper around the messages. You can still do a test-map, but the file you input to the test map must be in the special format as follows:

<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
	<InputMessagePart_0>
		<ns1:Schema1RootElement xmlns:ns1="http://MyNamespace.Schema1">
		</ns1:Schema1RootElement>
	</InputMessagePart_0>
	<InputMessagePart_1>
		<ns2:Schema2RootElement xmlns:ns2="http://MyNamespace.Schema2">
		</ns2:Schema2RootElement>
	</InputMessagePart_1>
</ns0:Root>

You would put your own elements and attribute between Schema1RootElement and it’s closing element, as well as between Schema1RootElement2 and it’s closing element.

So normally, you use some text or XML editor (like NotePad++) and paste data samples, then wrap them with the XML element wrappers shown above.

If your schemas don’t have target namespaces, you can simplify as follows:

<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
	<InputMessagePart_0>
		<Schema1RootElement>
		</Schema1RootElement>
	</InputMessagePart_0>
	<InputMessagePart_1>
		<Schema2RootElement>
		</Schema2RootElement>
	</InputMessagePart_1>
</ns0:Root>

So basically the root element is part of a special namespace called the “aggschema” (aggregate schema). Then, as many message as you added each are found, but each is wrapped with N) where N start with 0, and increments once per each message.


USE BizTalkDTADb
--select top 10 * from dbo.dta_DebugTrace
select COUNT(*) from dbo.dta_DebugTrace as TotalRowCount

SELECT YEAR(dtBeginTimeStamp) AS Yr,
Month(dtBeginTimeStamp) AS Mo,
COUNT(*) AS [RowCount]
FROM dbo.dta_DebugTrace nolock
GROUP BY
YEAR(dtBeginTimeStamp),
DatePart(mm,dtBeginTimeStamp)
Order BY
YEAR(dtBeginTimeStamp),
Month(dtBeginTimeStamp)

The results show you by year/month how many rows. In most cases, you will find you need to purge data prior the current month. At least on a test system, I can think of any possible use of month old orchestration trace data (or even on Production for that matter).  But if you have long-running dehydrated orhestrations you might need it).

dta_DebugTrace_Counts

Orchestrations write data to these tables each time they run (when Trace is enabled).

Go to SQL Agent on the SQL Server that supports your BizTalk server, check the job entitled: DTA Purge and Archive (BizTalkDTADb)
SQLAgent_Jobs_BizTalk_2010
I take the original code that is there, and copy it to the line below, then comment out the code that is there with the T-SQL comment (two dashes).
Then change the second line to the parameters you want. Follow-that by running the job manually, or set it up to run on a scheduled basis.


--exec dtasp_BackupAndPurgeTrackingDatabase
0, --@nLiveHours tinyint, --Any completed instance older than the live hours +live days
1, --@nLiveDays tinyint = 0, --will be deleted along with all associated data
30, --@nHardDeleteDays tinyint = 0, --all data older than this will be deleted.
null, --@nvcFolder nvarchar(1024) = null, --folder for backup files
null, --@nvcValidatingServer sysname = null,
0 --@fForceBackup int = 0 --


exec dtasp_BackupAndPurgeTrackingDatabase 24, 30, 32, 'e:\Backup\BizTalkDatabases\', null, 0

MSDN Reference: https://msdn.microsoft.com/library/aa558715.aspx

NOTE: Running the job creates a backup, so if your DTA database is large, and you are short on disk space, you will have to make sure you find disk space for the backup to be written.

If you get odd errors, try running the command directly in a SQL query window, then the a more useful error may be displayed than what you would see in the “View History” of the SQL Agent Job.

dtasp_BackupAndPurgeTrackingDatabase_Errors


The hard purge window cannot be less than the live data window [SQLSTATE 42000] (Error 50000).
The step failed.

After changing the second 30 to 32, I re-ran it, and saw this result:

Successful_Run_of_dtasp_BackupAndPurgeTrackingDatabase

 

The Parms are as follows:
dtasp_BackupAndPurgeTrackingDatabase_storedProc_Parms

 

However, after running it, I still saw many of the same rows there (this is on BT2010 by the way). I did some more research, found Sandro Pereira’s blog and ran the following:


declare @dtLastBackup datetime
set @dtLastBackup = GetUTCDate() exec dtasp_PurgeTrackingDatabase 1, 0, 7, @dtLastBackup

I brought a Rosetta .xsd schema into a new BizTalk Project, gave it a compile and got 1524 occurences of the following error:

The Build Errors


------ Rebuild All started: Project: BTS2C7Outbound, Configuration: Debug Any CPU ------
e:\MyDir\BTS2C7Outbound\Schemas\2C7\Domain\Logistics\CodeList\RN_ResponseCode_01_00.xsd.cs(13,17): error CS0234: The type or namespace name 'NonSerializedAttribute' does not exist in the namespace 'BTS2C7Outbound.Schemas._2C7.System' (are you missing an assembly reference?)
e:\MyDir\BTS2C7Outbound\Schemas\2C7\Domain\Logistics\CodeList\RN_ResponseCode_01_00.xsd.cs(13,17): error CS0234: The type or namespace name 'NonSerializedAttributeAttribute' does not exist in the namespace 'BTS2C7Outbound.Schemas._2C7.System' (are you missing an assembly reference?)
...
e:\MyDir\BTS2C7Outbound\Schemas\2C7\Domain\Logistics\CodeList\RN_ContainerType_01_02.xsd.cs(201,17): error CS0234: The type or namespace name 'SerializableAttribute' does not exist in the namespace 'BTS2C7Outbound.Schemas._2C7.System' (are you missing an assembly reference?)
e:\MyDir\BTS2C7Outbound\Schemas\2C7\Domain\Logistics\CodeList\RN_ContainerType_01_02.xsd.cs(201,17): error CS0234: The type or namespace name 'SerializableAttributeAttribute' does not exist in the namespace 'BTS2C7Outbound.Schemas._2C7.System' (are you missing an assembly reference?)

Compile complete — 1524 errors, 0 warnings
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

 

RosettaNet_type_Namespace_Serializable.AttributeAttribute_does_not_exit

Solution

You must change the “Namespace” property on the following two schemas in the “System” folder; one directly in that folder, and one the “CodeList” subfolder.
Two_Rosetta_Schemas

Here is an example of the original values:
Rosetta_Schema_Namespace_Fix

Change the .System in yellow above to ._System. or some other value.

Rebuild, and your error should go away.

For additional information about bringing RosettaNet PIPs into BizTalk projects, see this link:

https://msdn.microsoft.com/en-us/library/ff720376.aspx

I had a case today where I was mapping two fields N01 and N02 to two side by side fields in the output schema.

N01 was working, and N02 wasn’t.  Both were under an condition that a certain variable MESSAGE_TYPE was equal to a certain value.  Yet one worked, the other didn’t?   How did I go about debugging this?

I did a right click “Validate Map”, and opened the XSLT.  I searched for N01, and found the following.

XSLT_Debug_Sample_1

Can you spot any clues to the issue?  If not scroll on down…

 

 

 

 

 

 

 

 

 

 

 

 

 

In the image below, I have highlighted some of the code. The N01 is wrapped by one IF statement, so I would expect the same IF statement around N02.

But when I look at N02, I see to nested IF statements.

XSLT_Debug_Sample_2

You can find the variables to understand what the tests are.  So I searched for the extra test around N02, i.e. the variable v16, and found this.

XSLT_Debug_Sample_3

That jogged my memory.  I had another page on the map, where we were using a “trick” to not map certain fields.
This was a RosettaNet schema, and it contained many nodes with default values that were filling up the output document with unnecessary junk. The “trick” is to map “false” to the fields you don’t want to map.  So I had essentially told the map that I did not want to output this element (even though I had mapped something to it).

Of course, this is only one small case, but I hope you like the methodology, and can use it to apply to your own debugging.  Learning XSLT of course helps.  The XSLT generated by BizTalk can be a little confusing because of all the “userCSharp” routines, and all the variables, but it can help you find issues in your map while testing.

Also note, I would not have found this issue unless I was carefully unit-testing.  I hand-crafted 5 different input test files, and was running each one of them through the “Test Map” feature, and manually scanning the output.

 

In the past, I admit, I was a GAC’cer.  Yes, that means that I would run GACUTIL to deploy minor code changes to BizTalk Ochestrations.

Where I currently work, that is not the best practice for several reasons.  One reason is that if you  create an MSI after doing the GAC, the MSI will not contain the newest DLL.  At other places I worked, this was not an issue, because we used the BizTalk Deployment Framework (BTDF).

Last week, I had an orchestration that was rather difficult to test in my own environment.  So I wanted to compile it, deploy the .DLL to our QA environment, and test it there.  But I had to do this over-and-over, making minor little changes.

The manual process was:

  1. Stop the orchestration (and make sure to terminate any suspended instances).
  2. Go to the “Add Resource”, click a button to find the .DLL, wait a few seconds, then click a few check boxes, and click the “OK” button, and wait a few more seconds.
  3. Since we had two machines in our QA environment, I had to repeat Step 2 on the second machine.
  4. Then start the orchestration
  5. Then restart the host instances (so the new code would be picked up)

I vaguely remember the BTSTASK command, but haven’t used it for years outside of BTDF, so I looked it up, and created the following .cmd file:

cscript "c:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Samples\Admin\WMI\Stop Orchestration\VBScript\StopOrch.vbs" "MyProjNamespace.MyOrchName" "MyAppName" Unenlist
"c:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSTask.exe" AddResource /ApplicationName:MyAppName /Type:System.BizTalk:BizTalkAssembly /Overwrite /Source:MyProjNamespace.dll /Options:GacOnAdd,GacOnInstall,GacOnImport &gt;ReplaceResources_Log.txt
cscript "VBScript\StartOrch.vbs" "MyProjNamespace.MyOrchName" "MyAppName"

The program called StopOrch.vbs existed in the BizTalk provided SDK/examples. However, there was no StartOrch.vbs, so I had to create the VBScript to start an orchestration (see link to separate blog on that).  Heres the MSDN Doc for BTSTask with the AddResource parameter.

I’m still doing Step 5 manually.  I copy the .DLL to server1 and server 2 in the BizTalk group.  I then run the above script on each server.  Then I restart the host instances manually.

Today, I was tasked with upgrading an orchestration from calling one version of a web service (version 3), to a newer version 4.

Fortunately, it was backward compatible, i.e. I didn’t have to remap any fields. However, the schema was renamed, as it actually had the version number in the web service name. So how does one do this in BizTalk?

In this case, I’m sure there are many paths to the same destination, but here are the steps I followed.

1. I opened a new temporary Visual Studio BizTalk project and consumed the new web service (using the “Add Generated Items” / “Consume WCF Service”). The reason I did it in a separate new project is that it creates the dummy orchestration, and the binding files.

2. I then copied the 2 or 3 schema files to my actual project.

3. I checked my map out (from source control), and opened it in NotePad++ and had to do one small surgery. I had to find Reference Location=”..\WebServices\PlaceOrder_V3.xsd” and replace it with “..\WebServices\PlaceOrder_V4.xsd”. I could have tried it in map, but I’ve seen too many cases where you lose all your mappings, so I did it this way. I then opened the map to verify, and all looked good there.

I then removed the old web service schema, and did a Rebuild, just to see what would happen, which lead to the next two steps.

4. I had to reassociate my Multi-part Message types. Since the messages were using multi-part message types, they were okay as is. But I had two messages types (one for the web service request, and one for the response) that were “red” and had to change. So I just had to carefully pick the new .xsd name for each. (Could have done this also by editing the .odx file, if done carefully.)

5. If you had promoted (or distinguished fields) on the web service schema, you would have to redo those, and that was the case in this orchestration. Then, a really weird scenario happened. Every expression shape that was using the distinguished fields was getting an error. There was not red squiggly line under the items, just a compile error. Even closing and reopening Visual Studio did not solve it.

In one case, I re-typed the msg.property_name using Intellisense, and that solved the issue. For another shape, I did the same thing and it wouldn’t go away. I took a wild stab at it, and created a new expression shape, cut and pasted the statements from the old shape to the new shape, then it compiled clean. I cannot explain that part at all!

6. I’m about to change the URL in the SendPort now, and start testing. I’ll update if anything else unusual happens. I did not use the binding files created by the “Add Generated Items” in the first step above.

6a. Things went bad during testing, took me way too long, like 2 hours to get it working. I had to deploy to QA environment, because it was easier to test.
6b. One issue I had forgot was that the SOAP Action Headers needed to change (in the General configuration tab of the SendPort), for example:
<Operation Name=”PlaceOrder” Action=”http://mysite.com/webservices/neworder_v3/2013/01/PlaceOrder” />
<Operation Name=”PlaceOrder” Action=”http://mysite.com/webservices/neworder_v4/2015/11/INewOrder/PlaceOrder” />

6c. For some reason, it still didn’t work. I’m checking with the authors of the webservice if they change anything else. To finally get it to work, I had to take the CustomBindings.xml file, and run it on the QA system, then change my orchestration bindings to use the new SendPort.