Using the following Reg-Ex (Regular Expression) to find UTF-8 or UTF-16 characters in a text file;
this should work with any editor that supports Reg-Ex, such as NotePad++ or Visual Studio.

Be sure and check the “Regular Expression” option on the search, then type in the find string: [x80-xFF]

FULL ERROR

Exception: System.ServiceModel.CommunicationException: There was an error while trying to serialize parameter http://tempuri.org/:ExportValuationPolicyNumberResult. The InnerException message was ‘Type ‘TFBIC.RCT.WCFWebServices.ExpressLync.MainStreetValuation’ with data contract name ‘MainStreetValuation:http://schemas.datacontract.org/2004/07/TFBIC.RCT.WCFWebServices.ExpressLync’ is not expected. Add any types not known statically to the list of known types – for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.’. Please see InnerException for more details. —> System.Runtime.Serialization.SerializationException: Type ‘TFBIC.RCT.WCFWebServices.ExpressLync.MainStreetValuation’ with data contract name ‘MainStreetValuation:http://schemas.datacontract.org/2004/07/TFBIC.RCT.WCFWebServices.ExpressLync’ is not expected. Add any types not known statically to the list of known types – for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.

SOLUTION

1. Manually create a proxy to the WCF Service from the command line like so:
svcutil /serializer:XmlSerializer http://localhost:46122/ValuationService.svc

2. Add the [XmlSerializerFormat] to your ServiceContract on the WCF Service

By doing this, we force our WCF Service to use the XmlSerializer rather than the default DataContractSerializer. Note: you?ll need to call the WCF service from you new proxy in your sample code.

Converts several date/time formats to standard XML Date/ Time


        /// <summary>
        /// This takes a vendor's non-standard dates, such as 5/1/2008 and formats as XML Date.
        /// (i.e. the month may one or two digits, the day may be one or two digits).
        /// Neal Walters - changed on 09/03/2008 to  throw exception. 
        /// also tested to make sure it handles all the following dates (and it does)
        ///    string testDate1 = "03/09/2008";
        ///    string testDate2 = "1999-05-31";
        ///    string testDate3 = "03/09/2008 10:50:16.157";
        ///    string testDate4 = "1999-05-31T13:20:00.000-05:00";
        ///    string testDate5 = "2008-09-03 10:50:16.157";
        ///    string testDate6 = "2008-09-32";
        /// </summary>
        /// <param name="inDate"></param>
        /// <returns></returns>
        public static string ConvertDateTimetoXmlDateTime(string inDate)
        {
            try
            {
                DateTime dtIn = DateTime.Parse(inDate);
                return System.Xml.XmlConvert.ToString(dtIn, System.Xml.XmlDateTimeSerializationMode.Local);

            }
            catch
            {
                string errorMessage = "invalid-Date/Time-conversion from value='" + inDate + "'";
                throw new Exception(errorMessage);
                //return errorMessage;
            }
        }

This code builds an xml document in memory, then returns it as an System.Xml.XmlDocument. It also show the proper way to build an XML date.

        /// <summary>
        /// This routine is needed when an in-bound endpoint has a map failure when translating
        /// inbound schema to  TMXML. 
        /// </summary>
        /// <returns></returns>
        public static XmlDocument ReturnNewTMXML()
        {
            DateTime currentDateTime = DateTime.Now;
            string strDateTime = currentDateTime.ToLongDateString();

            string strMyDoc =
            "<n2:MyRootEl xsi:schemaLocation='http://MyNamespace.MySchema.xsd' +
'xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'>" +
                "<source>" +
                  "<objectId>NULL</objectId>" +
                  "<objectType>NULL</objectType>" +
                  "<requestor>NULL</requestor>" +
                  "<transactionGuid>NULL</transactionGuid>" +
                  "<requestType>NULL</requestType>" +
                  "<dateTimeReceived>" + TFBIC.Common.BizTalk.Components.MapHelper.ConvertDateTimetoXmlDateTime(strDateTime) + "</dateTimeReceived>" +
                  "<isResponseExpected>false</isResponseExpected>" +
                "</source>" +
            "</n2:MyRootEl>";

            XmlDocument xmlDoc = new XmlDocument();
            xmlDoc.PreserveWhitespace = true; //making sure no whitespaces get added
            xmlDoc.LoadXml(strTMXML);
            return xmlDoc;

        }

This sample shows how to build an XML node and add it as a new child in an existing

        /// <summary>
        /// This routine is needed when an in-bound endpoint Sends TMXML with no Itinerary Node in the Header
        /// The Receiver must make sure that the Itinerary node is present in the Header.
        /// </summary>
        /// <returns></returns>
        public static XmlDocument AddItineraryNode(XmlDocument xmlDoc)
        {
            xmlDoc.PreserveWhitespace = true; //making sure no whitespaces get added
            string strXPath = "//*[local-name()='ItineraryStep']";
            string strItinerary =
                    "<policyName>NULL</policyName>" +
                     "<status>NULL</status>" +
                     "<currentStepName>NULL</currentStepName>" +
                     "<currentStepStatus>NULL</currentStepStatus>" +
                     "<currentStepIndex>0</currentStepIndex>" +
                     "<currentStepErrorType>NULL</currentStepErrorType>" +
                     "<currentProcessCode>NULL</currentProcessCode>" +
                     "<currentProcessMethod>NULL</currentProcessMethod>" +
                     "<itinerarySteps></itinerarySteps>";
            XmlNamespaceManager iMgr = new XmlNamespaceManager(xmlDoc.NameTable);
            XmlNode nodeItin = xmlDoc.SelectSingleNode(strXPath, iMgr);

            if (nodeItin == null)
            {
                XmlElement newItinerary = xmlDoc.CreateElement("Itinerary");
                newItinerary.InnerXml = strItinerary;
                xmlDoc.DocumentElement.FirstChild.AppendChild(newItinerary);
            }

            return xmlDoc;

        }


In the following example, ExceptionLog table has an xml column called TMXML. This contains a canonical xml message. The SQL can “peer” into the xml column, and pull out selected fields.

<code>
SELECT
  eventDateTime,
  TMXML.value('/*[local-name()="TMXML"][1]/*[local-name()="Header"][1]/*[local-name()="Source"][1]/requestType[1]',
  'varchar(24)') AS RequestType,
  TMXML.value('/*[local-name()="TMXML"][1]/*[local-name()="Header"][1]/*[local-name()="Itinerary"][1]/currentStepName[1]',
  'varchar(24)') AS CurrentStepName,
  TMXML.value('/*[local-name()="TMXML"][1]/*[local-name()="Header"][1]/*[local-name()="Itinerary"][1]/currentProcessCode[1]',
  'varchar(24)') AS CurrentProcessCode
FROM ExceptionLog

</code>

A nifty idea is to put the above SQL in a view, and hide the xquery statements from the common user. Then the user can simply do

<code>
Select * from ViewExceptionLog
</code>

For further info: http://msdn.microsoft.com/en-us/library/ms178030.aspx

What if you need to remove all the empty nodes in an XML document? It’ easy with this small subroutine. This might be useful to set fields to NULL when they have no value, especially when communicating between BizTalk and a WCF web service. You might often get serialization errors when, for example, a date/time field has an empty-string value. Better to pass no element to the web service (i.e. NULL) than a field with a bad value.


        public static XmlDocument RemoveEmptyNodes(XmlDocument txd)
        {

           // make a clean copy so we don't accidentally destroy the byRef txd variable passed to us.
            XmlDocument xmldoc = new XmlDocument();
            xmldoc.LoadXml(txd.OuterXml);
            XmlNodeList emptyElements = xmldoc.SelectNodes(@"//*[not(node())]");
            for (int i = emptyElements.Count - 1; i >= 0; i--)
            {
               emptyElements[i].ParentNode.RemoveChild(emptyElements[i]);
            }
            xmldoc.InnerXml = xmldoc.InnerXml.Replace("", "");
            return xmldoc;
        }

You can call the above code from a message assignment shape in an orchestration.
Just put the above code in a C# helper .DLL, make a reference to it from your orchestration, then you should be able to call it statically.

XML Serializer – Sample Code – How to take a C# object and write it to a file in XML “serialized” format.

NOTE: Serialize means to take an object (in memory) and turn it to a format that can be preserved to disk (or sent over a wire). Deserialize means to take the serialized-format, and turn it back into an object (in memory).

<code>
using System.Xml.Serialization;
using System.IO;
...

//  Serialize an object in memory to disk. 

            XmlSerializer xs1 = new XmlSerializer(typeof(YourClassName));
            StreamWriter sw1 = new StreamWriter(@"c:DeserializeYourObject.xml");
            xs1.Serialize(sw1, objYourObjectFromYourClassName);
            sw1.Close();

// Now to deserialize it - assuming this might be a different program, run later.

            XmlSerializer xs2 = new XmlSerializer(typeof(YourClassName));
            StreamReader sr2 = new StreamReader(@"c:DeserializeYourObject.xml");
            // deserialize and cast back to proper class
            objYourObjectFromYourClassName = (YourClassName) xs2.Deserialize(sr2);
            sr2.Close();
            Console.WriteLine ("Deserialize2 completed");


</code>

NOTE: Serializing to a string in memory is a lot trickier than you might think, because you have to deal with byteArrays, and encoding. Here’s a link to a page that explains how to do it:


http://www.dotnetjohn.com/articles.aspx?articleid=173

Based on the above article, I put the following code in my “Common” C# Component library that is deployed on all my systems:

<code>
        /// <summary>
        /// Method to convert a custom Object to XML string
        /// </summary>
        /// Object that is to be serialized to XML
        /// XML string
        public static String SerializeObject(Object pObject)
        {

            try
            {

                String XmlizedString = null;
                MemoryStream memoryStream = new MemoryStream();
                XmlSerializer xs = new XmlSerializer(pObject.GetType());
                XmlTextWriter xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8);

                xs.Serialize(xmlTextWriter, pObject);
                memoryStream = (MemoryStream)xmlTextWriter.BaseStream;
                XmlizedString = UTF8ByteArrayToString(memoryStream.ToArray());
                return XmlizedString;

            }

            catch (Exception e)
            {
                System.Console.WriteLine(e);
                return null;
            }

        }
        /// <summary>
        /// Method to reconstruct turn XMLString back into Object
        /// </summary>
        /// 
        /// 

        public static Object DeserializeObject(String pXmlizedString, Type classType)
        {

            XmlSerializer xs = new XmlSerializer(classType);
            MemoryStream memoryStream = new MemoryStream(StringToUTF8ByteArray(pXmlizedString));
            XmlTextWriter xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8);
            return xs.Deserialize(memoryStream);

        }

        private static String UTF8ByteArrayToString(Byte[] characters)
        {
            UTF8Encoding encoding = new UTF8Encoding();
            String constructedString = encoding.GetString(characters);
            return (constructedString);
        }

        private static Byte[] StringToUTF8ByteArray(String pXmlString)
        {
            UTF8Encoding encoding = new UTF8Encoding();
            Byte[] byteArray = encoding.GetBytes(pXmlString);
            return byteArray;
        }


</code>

Just wrap the above with your class name, and add these using statements:

<code>
using System;
using System.Collections.Generic;
using System.Collections;
using System.Text;
using System.Xml;
using System.Reflection;
using System.Xml.Schema;
using System.IO;
using System.Xml.Serialization;
using System.Text;
</code>

scheduleREc is a LINQ object, and the field SchedXML is of type System.Xml.Linq.XElement.
strXml is simple a C# string that contains my xml.

<code>
            // These two lines of code took a few hours to figure  out! 
            // How to convert string to XElement
            StringReader stringRdr = new StringReader(strXml);
            scheduleRec.SchedXML = XElement.Load(stringRdr);
</code>

It is possible to build and test an XSLT stylesheet in a non-Biztalk product such as Stylus Studio (or XML SPY). The advantages are the XSLT wonderful debuggers that these products have. You can set breakpoints, walk through the XSLT, and do things not limited by Biztalk’s Functoids. The only downside is you do need to know XSLT.

To include the external stylesheet into a map:

1) You should include the XSLT source file into the project first (especially if you are using Visual SourceSafe)

2) click on the map grid, and set the property ? Custom XSLT Path ? to filename containing the XSLT.

In it’s new release this week, Stylus Studio just added EDI and HL7 mapping. I’m not a big EDI/HL7 user, but this might be an nice alternative or cost savings to what Biztalk currently offers (i.e. Covast and the HL7 Accereraltor).

Here’s the quote from the Stylus Studio website (http://StylusStudio.com).

EDI to XML Mapping

A new EDI-to-XML mapping tool, featuring full support for HL7, UN/EDIFACT and ANSI/X12 specs and more! Build XML enabled systems that leverage existing investments in EDI technologies.