Recently we had to restore an environment for a demo. This environment consists of a couple of servers, the main one (obviously) being a BizTalk 2009 server. Here we ran into this error in the SSO console application:
ERROR: Cannot perform encryption or decryption because the secret is not available from the master secret server.
My first attempt was to restore the secret from an old back-up file as described on MSDN: How to Restore the Master Secret. There you have to enter the password you used when configuring SSO and alas we couldn't retrieve that password.
Basically this means you're up the creek. I could find only one solution to this:
1. Export your bindings, and make notes of the host instances you use;
2. In BizTalk Server Configuration unconfigure Enterprise SSO (which will automatically unconfigure most of the other features);
3. In SQL Server Management Studio delete SSODB and the BizTalk databases (deleting just the SSODB and re-using the BizTalk databases won't work);
4. In BizTalk Server Configuration reconfigure all features from scratch (and this time make sure you have a way to retrieve the SSO Secret backup password!);
5. Recreate the host instances using the exact same names as before (they are hardcoded in the bindings file);
6. Redeploy your application(s);
7. Import your bindings.
If done correctly, you'll have this done in an hour or less. Which sure beats the time I had to spend on this...
Sunday, May 29, 2011
Sunday, March 6, 2011
Receive pipeline component for Excel files
When you want to import an Excel file into BizTalk, you used to have a problem. The old way Excel files used to be 'composed' was almost impossible to translate to an XML structure.
But the times they are a-changin' and nowadays Excel files are stored in the OpenXml format. Download the OpenXML SDK at http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c6e744e5-36e9-45f5-8d8c-331df206e0d0&displaylang=en and you can go ahead building your own pipeline components for importing of .xlsx and .xlsm files.
Here is an example of how you might do it. I highlighted the parts you should take special notice to.
A short explanation of the properties I'm using here:
UseColumnNamesFromFirstRow
If you set this property to 'true', the first row of your sheet is used to define the tag-names of your XML output. Say your spreadsheet looks like this:
With this property set to 'false', the resulting XML looks like this:
When you set it to 'true', the XML will look like this:
Note that I'm adding the column as a prefix to make sure that the tags are different, even if there are 2 columns with the same header.
RootElementName
The name of the root-element of the resulting XML. It will default to 'root'.
RowElementName
The name of the row-elements of the resulting XML. It will default to 'row'.
NamespaceName
The namespace that will be used in the resulting XML. It defaults to 'nonamespace'.
But the times they are a-changin' and nowadays Excel files are stored in the OpenXml format. Download the OpenXML SDK at http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c6e744e5-36e9-45f5-8d8c-331df206e0d0&displaylang=en and you can go ahead building your own pipeline components for importing of .xlsx and .xlsm files.
Here is an example of how you might do it. I highlighted the parts you should take special notice to.
A short explanation of the properties I'm using here:
UseColumnNamesFromFirstRow
| A | B | C | |
|---|---|---|---|
| 1 | Name | Department | Salary |
| 2 | John | Sales | 50000 |
| 3 | Bill | Development | 30000 |
With this property set to 'false', the resulting XML looks like this:
<ns0:root xmlns:ns0='nonamespace'>
<ns0:row>
<ns0:ColA>Name</ns0:ColA>
<ns0:ColB>Department</ns0:ColB>
<ns0:ColC>Salary</ns0:ColC>
</ns0:row>
<ns0:row>
<ns0:ColA>John</ns0:ColA>
<ns0:ColB>Sales</ns0:ColB>
<ns0:ColC>50000</ns0:ColC>
</ns0:row>
<ns0:row>
<ns0:ColA>Bill</ns0:ColA>
<ns0:ColB>Development</ns0:ColB>
<ns0:ColC>30000</ns0:ColC>
</ns0:row>
</ns0:root>
<ns0:row>
<ns0:ColA>Name</ns0:ColA>
<ns0:ColB>Department</ns0:ColB>
<ns0:ColC>Salary</ns0:ColC>
</ns0:row>
<ns0:row>
<ns0:ColA>John</ns0:ColA>
<ns0:ColB>Sales</ns0:ColB>
<ns0:ColC>50000</ns0:ColC>
</ns0:row>
<ns0:row>
<ns0:ColA>Bill</ns0:ColA>
<ns0:ColB>Development</ns0:ColB>
<ns0:ColC>30000</ns0:ColC>
</ns0:row>
</ns0:root>
When you set it to 'true', the XML will look like this:
<ns0:root xmlns:ns0='nonamespace'>
<ns0:row>
<ns0:A_Name>John</ns0:A_Name>
<ns0:B_Department>Sales</ns0:B_Department>
<ns0:C_Salary>50000</ns0:C_Salary>
</ns0:row>
<ns0:row>
<ns0:A_Name>Bill</ns0:A_Name>
<ns0:B_Department>Development</ns0:B_Department>
<ns0:C_Salary>30000</ns0:C_Salary>
</ns0:row>
</ns0:root>
<ns0:row>
<ns0:A_Name>John</ns0:A_Name>
<ns0:B_Department>Sales</ns0:B_Department>
<ns0:C_Salary>50000</ns0:C_Salary>
</ns0:row>
<ns0:row>
<ns0:A_Name>Bill</ns0:A_Name>
<ns0:B_Department>Development</ns0:B_Department>
<ns0:C_Salary>30000</ns0:C_Salary>
</ns0:row>
</ns0:root>
Note that I'm adding the column as a prefix to make sure that the tags are different, even if there are 2 columns with the same header.
RootElementName
The name of the root-element of the resulting XML. It will default to 'root'.
RowElementName
The name of the row-elements of the resulting XML. It will default to 'row'.
NamespaceName
The namespace that will be used in the resulting XML. It defaults to 'nonamespace'.
using System;
using System.IO;
using System.Text;
using System.Drawing;
using System.Resources;
using System.Reflection;
using System.Diagnostics;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Xml;
using System.Xml.Schema;
using System.Data;
using System.Linq;
using Microsoft.BizTalk.Message.Interop;
using Microsoft.BizTalk.Component.Interop;
using Microsoft.BizTalk.Component;
using Microsoft.BizTalk.Messaging;
using Microsoft.BizTalk.Component.Utilities;
namespace RonaldLokers.Blogspot.Com.PipelineComponents
{
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[System.Runtime.InteropServices.Guid("0022db2c-1599-4008-9312-ec3f9b66c356")]
[ComponentCategory(CategoryTypes.CATID_Decoder)]
public class Excel2007XmlExtractor: Microsoft.BizTalk.Component.Interop.IComponent, IBaseComponent, IPersistPropertyBag, IComponentUI
{
#region private variables
private System.Resources.ResourceManagerresourceManager = new System.Resources.ResourceManager("RonaldLokers.Blogspot.Com.PipelineComponents.Excel2007XmlExtractor", Assembly.GetExecutingAssembly());
public bool UseColumnNamesFromFirstRow
{
get { return _UseColumnNamesFromFirstRow; }
set { _UseColumnNamesFromFirstRow = value; }
}
private string _RootElementName;
public string RootElementName
{
get { return string.IsNullOrEmpty(_RootElementName)?"root":_RootElementName; }
set { _RootElementName = value; }
}
private string _RowElementName;
public string RowElementName
{
get { return string.IsNullOrEmpty(_RowElementName)?"row":_RowElementName; }
set { _RowElementName = value; }
}
private string _NamespaceName;
public string NamespaceName
{
get { return string.IsNullOrEmpty(_NamespaceName)?"nonamespace":_NamespaceName; }
set { _NamespaceName = value; }
}
#endregion
#region IBaseComponent members
/// <summary>
/// Name of the component
/// </summary>
[Browsable(false)]
public string Name
{
get
{
returnresourceManager.GetString("COMPONENTNAME", System.Globalization.CultureInfo.InvariantCulture);
}
}
/// <summary>
/// Version of the component
/// </summary>
[Browsable(false)]
public string Version
{
get
{
returnresourceManager.GetString("COMPONENTVERSION", System.Globalization.CultureInfo.InvariantCulture);
}
}
/// <summary>
/// Description of the component
/// </summary>
[Browsable(false)]
public string Description
{
get
{
return resourceManager.GetString("COMPONENTDESCRIPTION", System.Globalization.CultureInfo.InvariantCulture);
}
}
#endregion
#region IPersistPropertyBag members
/// <summary>
/// Gets class ID of component for usage from unmanaged code.
/// </summary>
/// <param name="classid">
/// Class ID of the component
/// </param>
public void GetClassID(outSystem.Guid classid)
{
classid = newSystem.Guid("0022db2c-1599-4008-9312-ec3f9b66c356");
}
/// <summary>
/// not implemented
/// </summary>
public void InitNew()
{
}
/// <summary>
/// Loads configuration properties for the component
/// </summary>
/// <param name="pb">Configuration property bag</param>
/// <param name="errlog">Error status</param>
public virtual void Load(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, int errlog)
{
object val;
val = this.ReadPropertyBag(pb, "UseColumnNamesFromFirstRow");
if (val != null)
{
this.UseColumnNamesFromFirstRow = ((bool)(val));
}
val = this.ReadPropertyBag(pb, "RootElementName");
if (val != null)
{
this.RootElementName = ((string)(val));
}
val = this.ReadPropertyBag(pb, "RowElementName");
if (val != null)
{
this.RowElementName = ((string)(val));
}
val = this.ReadPropertyBag(pb, "NamespaceName");
if (val != null)
{
this.NamespaceName = ((string)(val));
}
}
/// <summary>
/// Saves the current component configuration into the property bag
/// </summary>
/// <param name="pb">Configuration property bag</param>
/// <param name="fClearDirty">not used</param>
/// <param name="fSaveAllProperties">not used</param>
public virtual void Save(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, bool fClearDirty, boolfSaveAllProperties)
{
this.WritePropertyBag(pb, "UseColumnNamesFromFirstRow", this.UseColumnNamesFromFirstRow);
this.WritePropertyBag(pb, "RootElementName", this.RootElementName);
this.WritePropertyBag(pb, "RowElementName", this.RowElementName);
this.WritePropertyBag(pb, "NamespaceName", this.NamespaceName);
}
#region utility functionality
/// <summary>
/// Reads property value from property bag
/// </summary>
/// <param name="pb">Property bag</param>
/// <param name="propName">Name of property</param>
/// <returns>Value of the property</returns>
private object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, string propName)
{
object val = null;
try
{
pb.Read(propName, outval, 0);
}
catch (System.ArgumentException)
{
return val;
}
catch (System.Exception e)
{
throw new System.ApplicationException(e.Message);
}
return val;
}
/// <summary>
/// Writes property values into a property bag.
/// </summary>
/// <param name="pb">Property bag.</param>
/// <param name="propName">Name of property.</param>
/// <param name="val">Value of property.</param>
private void WritePropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, string propName, object val)
{
try
{
pb.Write(propName, refval);
}
catch (System.Exception e)
{
throw new System.ApplicationException(e.Message);
}
}
#endregion
#endregion
#region IComponentUI members
/// <summary>
/// Component icon to use in BizTalk Editor
/// </summary>
[Browsable(false)]
public IntPtr Icon
{
get
{
return((System.Drawing.Bitmap)(this.resourceManager.GetObject("COMPONENTICON", System.Globalization.CultureInfo.InvariantCulture))).GetHicon();
}
}
/// <summary>
/// The Validate method is called by the BizTalk Editor during the build
/// of a BizTalk project.
/// </summary>
/// <param name="obj">An Object containing the configuration properties.</param>
/// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>
public System.Collections.IEnumerator Validate(object obj)
{
// example implementation:
// ArrayList errorList = new ArrayList();
// errorList.Add("This is a compiler error");
// return errorList.GetEnumerator();
return null;
}
#endregion
#region IComponent members
/// <summary>
/// Implements IComponent.Execute method.
/// </summary>
/// <param name="pc">Pipeline context</param>
/// <param name="inmsg">Input message</param>
/// <returns>Original input message</returns>
/// <remarks>
/// IComponent.Execute method is used to initiate
/// the processing of the message in this pipeline component.
/// </remarks>
{
Stream instream = CopyStream(inmsg.BodyPart.GetOriginalDataStream());
//try
//{
string xml = string.Empty;
XmlDocument xmldoc = new XmlDocument();
XmlElement root = xmldoc.CreateElement(RootElementName, NamespaceName);
xmldoc.AppendChild(root);
using (SpreadsheetDocument doc = SpreadsheetDocument.Open(instream, false))
{
WorkbookPart wbp = doc.WorkbookPart;
SharedStringTablePart shareStringPart = doc.WorkbookPart.GetPartsOfType<SharedStringTablePart>().First();
SharedStringItem[] items = shareStringPart.SharedStringTable.Elements<SharedStringItem>().ToArray();
Sheet sheet = wbp.Workbook.Descendants<Sheet>().First();
Worksheet worksheet = ((WorksheetPart)doc.WorkbookPart.GetPartById(sheet.Id)).Worksheet;
Hashtable columnHeaders = new Hashtable();
if(_UseColumnNamesFromFirstRow)
{
foreach(Row row in worksheet.Descendants<Row>())
{
if(row.RowIndex.Value == 1)
{
foreach(Cell cell in row)
{
string columnheader = GetCellValue(cell, items).Replace(' ', '_');
if(!string.IsNullOrEmpty(columnheader))
{
string col = GetCellColumn(cell);
columnHeaders.Add(col, col + "_" + columnheader);
}
}
break;
}
}
}
foreach (Row row in worksheet.Descendants<Row>())
{
if(!(_UseColumnNamesFromFirstRow && row.RowIndex.Value == 1))
{
XmlElement rownode = xmldoc.CreateElement(RowElementName, NamespaceName);
root.AppendChild(rownode);
foreach(Cell cell in row)
{
string col = GetCellColumn(cell);
string nodename;
if(columnHeaders.ContainsKey(col))
nodename = (string)columnHeaders[col];
else
nodename = "Col" + col;
string cellvalue = GetCellValue(cell, items).Trim();
if(!string.IsNullOrEmpty(cellvalue))
{
XmlElement node = xmldoc.CreateElement(nodename, NamespaceName);
node.InnerText = cellvalue;
rownode.AppendChild(node);
}
}
}
}
}
MemoryStream memStr = new MemoryStream();
StreamWriter strOut = new StreamWriter(memStr);
strOut.Write(xmldoc.OuterXml);
strOut.Flush();
memStr.Position = 0;
pc.ResourceTracker.AddResource(memStr);
inmsg.BodyPart.Data = memStr;
//}
//catch (Exception exc)
//{
// You can do your own error handling here...
//}
return inmsg;
}
private Stream CopyStream(Streaminput)
{
Stream output = new MemoryStream();
byte[] buffer = new byte[32768];
int read = 1;
while (read > 0) {
read = input.Read(buffer, 0, buffer.Length);
if (read > 0)
output.Write(buffer, 0, read);
}
return output;
}
private string GetCellColumn(Cellcell)
{
string cellref = cell.CellReference.Value;
int pos = 0;
int column;
while (!int.TryParse(cellref.Substring(pos), out column))
pos++;
returncellref.Substring(0, pos);
}
private string GetCellValue(Cellcell, SharedStringItem[] items)
{
string txt = string.Empty;
if (cell.CellValue != null)
txt = cell.CellValue.Text;
if (cell.DataType != null && cell.DataType.Value == CellValues.SharedString)
txt = items[int.Parse(txt)].InnerText;
return txt;
}
#endregion
}
}
using System.IO;
using System.Text;
using System.Drawing;
using System.Resources;
using System.Reflection;
using System.Diagnostics;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Xml;
using System.Xml.Schema;
using System.Data;
using System.Linq;
using Microsoft.BizTalk.Message.Interop;
using Microsoft.BizTalk.Component.Interop;
using Microsoft.BizTalk.Component;
using Microsoft.BizTalk.Messaging;
using Microsoft.BizTalk.Component.Utilities;
using DocumentFormat.OpenXml.Packaging;
using DocumentFormat.OpenXml.Spreadsheet;
using DocumentFormat.OpenXml.Spreadsheet;
namespace RonaldLokers.Blogspot.Com.PipelineComponents
{
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[System.Runtime.InteropServices.Guid("0022db2c-1599-4008-9312-ec3f9b66c356")]
[ComponentCategory(CategoryTypes.CATID_Decoder)]
public class Excel2007XmlExtractor: Microsoft.BizTalk.Component.Interop.IComponent, IBaseComponent, IPersistPropertyBag, IComponentUI
{
#region private variables
private System.Resources.ResourceManagerresourceManager = new System.Resources.ResourceManager("RonaldLokers.Blogspot.Com.PipelineComponents.Excel2007XmlExtractor", Assembly.GetExecutingAssembly());
// Propertie declarations.
private bool _UseColumnNamesFromFirstRow;public bool UseColumnNamesFromFirstRow
{
get { return _UseColumnNamesFromFirstRow; }
set { _UseColumnNamesFromFirstRow = value; }
}
private string _RootElementName;
public string RootElementName
{
get { return string.IsNullOrEmpty(_RootElementName)?"root":_RootElementName; }
set { _RootElementName = value; }
}
private string _RowElementName;
public string RowElementName
{
get { return string.IsNullOrEmpty(_RowElementName)?"row":_RowElementName; }
set { _RowElementName = value; }
}
private string _NamespaceName;
public string NamespaceName
{
get { return string.IsNullOrEmpty(_NamespaceName)?"nonamespace":_NamespaceName; }
set { _NamespaceName = value; }
}
#endregion
#region IBaseComponent members
/// <summary>
/// Name of the component
/// </summary>
[Browsable(false)]
public string Name
{
get
{
returnresourceManager.GetString("COMPONENTNAME", System.Globalization.CultureInfo.InvariantCulture);
}
}
/// <summary>
/// Version of the component
/// </summary>
[Browsable(false)]
public string Version
{
get
{
returnresourceManager.GetString("COMPONENTVERSION", System.Globalization.CultureInfo.InvariantCulture);
}
}
/// <summary>
/// Description of the component
/// </summary>
[Browsable(false)]
public string Description
{
get
{
return resourceManager.GetString("COMPONENTDESCRIPTION", System.Globalization.CultureInfo.InvariantCulture);
}
}
#endregion
#region IPersistPropertyBag members
/// <summary>
/// Gets class ID of component for usage from unmanaged code.
/// </summary>
/// <param name="classid">
/// Class ID of the component
/// </param>
public void GetClassID(outSystem.Guid classid)
{
classid = newSystem.Guid("0022db2c-1599-4008-9312-ec3f9b66c356");
}
/// <summary>
/// not implemented
/// </summary>
public void InitNew()
{
}
/// <summary>
/// Loads configuration properties for the component
/// </summary>
/// <param name="pb">Configuration property bag</param>
/// <param name="errlog">Error status</param>
public virtual void Load(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, int errlog)
{
object val;
val = this.ReadPropertyBag(pb, "UseColumnNamesFromFirstRow");
if (val != null)
{
this.UseColumnNamesFromFirstRow = ((bool)(val));
}
val = this.ReadPropertyBag(pb, "RootElementName");
if (val != null)
{
this.RootElementName = ((string)(val));
}
val = this.ReadPropertyBag(pb, "RowElementName");
if (val != null)
{
this.RowElementName = ((string)(val));
}
val = this.ReadPropertyBag(pb, "NamespaceName");
if (val != null)
{
this.NamespaceName = ((string)(val));
}
}
/// <summary>
/// Saves the current component configuration into the property bag
/// </summary>
/// <param name="pb">Configuration property bag</param>
/// <param name="fClearDirty">not used</param>
/// <param name="fSaveAllProperties">not used</param>
public virtual void Save(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, bool fClearDirty, boolfSaveAllProperties)
{
this.WritePropertyBag(pb, "UseColumnNamesFromFirstRow", this.UseColumnNamesFromFirstRow);
this.WritePropertyBag(pb, "RootElementName", this.RootElementName);
this.WritePropertyBag(pb, "RowElementName", this.RowElementName);
this.WritePropertyBag(pb, "NamespaceName", this.NamespaceName);
}
#region utility functionality
/// <summary>
/// Reads property value from property bag
/// </summary>
/// <param name="pb">Property bag</param>
/// <param name="propName">Name of property</param>
/// <returns>Value of the property</returns>
private object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, string propName)
{
object val = null;
try
{
pb.Read(propName, outval, 0);
}
catch (System.ArgumentException)
{
return val;
}
catch (System.Exception e)
{
throw new System.ApplicationException(e.Message);
}
return val;
}
/// <summary>
/// Writes property values into a property bag.
/// </summary>
/// <param name="pb">Property bag.</param>
/// <param name="propName">Name of property.</param>
/// <param name="val">Value of property.</param>
private void WritePropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBagpb, string propName, object val)
{
try
{
pb.Write(propName, refval);
}
catch (System.Exception e)
{
throw new System.ApplicationException(e.Message);
}
}
#endregion
#endregion
#region IComponentUI members
/// <summary>
/// Component icon to use in BizTalk Editor
/// </summary>
[Browsable(false)]
public IntPtr Icon
{
get
{
return((System.Drawing.Bitmap)(this.resourceManager.GetObject("COMPONENTICON", System.Globalization.CultureInfo.InvariantCulture))).GetHicon();
}
}
/// <summary>
/// The Validate method is called by the BizTalk Editor during the build
/// of a BizTalk project.
/// </summary>
/// <param name="obj">An Object containing the configuration properties.</param>
/// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>
public System.Collections.IEnumerator Validate(object obj)
{
// example implementation:
// ArrayList errorList = new ArrayList();
// errorList.Add("This is a compiler error");
// return errorList.GetEnumerator();
return null;
}
#endregion
#region IComponent members
/// <summary>
/// Implements IComponent.Execute method.
/// </summary>
/// <param name="pc">Pipeline context</param>
/// <param name="inmsg">Input message</param>
/// <returns>Original input message</returns>
/// <remarks>
/// IComponent.Execute method is used to initiate
/// the processing of the message in this pipeline component.
/// </remarks>
// Here we go... this is where the real work is done!
public Microsoft.BizTalk.Message.Interop.IBaseMessageExecute(Microsoft.BizTalk.Component.Interop.IPipelineContextpc, Microsoft.BizTalk.Message.Interop.IBaseMessageinmsg){
// The 'SpreadsheetDocument.Open' method throws a System.IndexOutOfRange exception when you use the stream from inmsg directly. This is because this is a Microsoft.BizTalk.Message.Interop.StreamViewOfIStream occurence. So first I copy it to a regular MemoryStream.
Stream instream = CopyStream(inmsg.BodyPart.GetOriginalDataStream());
//try
//{
string xml = string.Empty;
XmlDocument xmldoc = new XmlDocument();
XmlElement root = xmldoc.CreateElement(RootElementName, NamespaceName);
xmldoc.AppendChild(root);
using (SpreadsheetDocument doc = SpreadsheetDocument.Open(instream, false))
{
// I'm not going into all the details of the OpenXml format. You can find it all in the SDK.
WorkbookPart wbp = doc.WorkbookPart;
SharedStringTablePart shareStringPart = doc.WorkbookPart.GetPartsOfType<SharedStringTablePart>().First();
SharedStringItem[] items = shareStringPart.SharedStringTable.Elements<SharedStringItem>().ToArray();
Sheet sheet = wbp.Workbook.Descendants<Sheet>().First();
Worksheet worksheet = ((WorksheetPart)doc.WorkbookPart.GetPartById(sheet.Id)).Worksheet;
Hashtable columnHeaders = new Hashtable();
if(_UseColumnNamesFromFirstRow)
{
// Here is the part where I look for the column headers. I loop through the rows of the sheet to find the top row (RowIndex.Value == 1) and then put all cell values into a Hashtable. (There's probably an easier way to get the first row, but I couldn't figure it out, and this works too ;-)
foreach(Row row in worksheet.Descendants<Row>())
{
if(row.RowIndex.Value == 1)
{
foreach(Cell cell in row)
{
string columnheader = GetCellValue(cell, items).Replace(' ', '_');
if(!string.IsNullOrEmpty(columnheader))
{
string col = GetCellColumn(cell);
// As explained above, I'm adding the column as a prefix.
columnHeaders.Add(col, col + "_" + columnheader);
}
}
break;
}
}
}
foreach (Row row in worksheet.Descendants<Row>())
{
if(!(_UseColumnNamesFromFirstRow && row.RowIndex.Value == 1))
{
// And here the XmlDocument is being build.
XmlElement rownode = xmldoc.CreateElement(RowElementName, NamespaceName);
root.AppendChild(rownode);
foreach(Cell cell in row)
{
string col = GetCellColumn(cell);
string nodename;
if(columnHeaders.ContainsKey(col))
nodename = (string)columnHeaders[col];
else
nodename = "Col" + col;
string cellvalue = GetCellValue(cell, items).Trim();
if(!string.IsNullOrEmpty(cellvalue))
{
XmlElement node = xmldoc.CreateElement(nodename, NamespaceName);
node.InnerText = cellvalue;
rownode.AppendChild(node);
}
}
}
}
}
// Finally write the XmlDocument contents to a stream and replace the pipeline stream with this new stream.
MemoryStream memStr = new MemoryStream();
StreamWriter strOut = new StreamWriter(memStr);
strOut.Write(xmldoc.OuterXml);
strOut.Flush();
memStr.Position = 0;
pc.ResourceTracker.AddResource(memStr);
inmsg.BodyPart.Data = memStr;
//}
//catch (Exception exc)
//{
// You can do your own error handling here...
//}
return inmsg;
}
private Stream CopyStream(Streaminput)
{
Stream output = new MemoryStream();
byte[] buffer = new byte[32768];
int read = 1;
while (read > 0) {
read = input.Read(buffer, 0, buffer.Length);
if (read > 0)
output.Write(buffer, 0, read);
}
return output;
}
private string GetCellColumn(Cellcell)
{
string cellref = cell.CellReference.Value;
int pos = 0;
int column;
while (!int.TryParse(cellref.Substring(pos), out column))
pos++;
returncellref.Substring(0, pos);
}
private string GetCellValue(Cellcell, SharedStringItem[] items)
{
string txt = string.Empty;
if (cell.CellValue != null)
txt = cell.CellValue.Text;
if (cell.DataType != null && cell.DataType.Value == CellValues.SharedString)
txt = items[int.Parse(txt)].InnerText;
return txt;
}
#endregion
}
}
Thursday, December 30, 2010
How to ‘loosely couple’ an orchestration to a WCF webservice
Consider the following scenario:
I have an orchestration, which is activated by a Direct receive port, i.e. the orchestration is activated when a certain message (of the type the receive shape filters on) is written to the messagebox. At the end of the orchestration, the output message is sent to a Direct send port, i.e. the message is written to the messagebox.
Now I define a request-response WCF webservice, and what I want is that a message sent to this webservice is picked up by the orchestration and obviously the result message from the orchestration should be sent through the webservice again.
The first steps in this plan are easy: create the orchestration, create a webservice using the BizTalk WCF Service Publishing Wizard, deploy it all and go! But alas, there will be a routing failure… BizTalk doesn’t know where to send the result message from the orchestration.
So how do we tell BizTalk the result message should be send back to the webservice? To cut a long story short, we need a Message Assignment on the result message with this code:
and on the send shape we need a Correlation Set with these properties:
BTS.CorrelationToken,BTS.EpmRRCorrelationToken,BTS.IsRequestResponse,BTS.ReqRespTransmitPipelineID,BTS.RouteDirectToTP
That’s all there’s to it!
If you want the long explanation and the ‘why’ of it all, check out Bram Veldhoen’s blog, which perfectly describes it in full detail. So no need for me to go through all that effort…
I have an orchestration, which is activated by a Direct receive port, i.e. the orchestration is activated when a certain message (of the type the receive shape filters on) is written to the messagebox. At the end of the orchestration, the output message is sent to a Direct send port, i.e. the message is written to the messagebox.
Now I define a request-response WCF webservice, and what I want is that a message sent to this webservice is picked up by the orchestration and obviously the result message from the orchestration should be sent through the webservice again.
The first steps in this plan are easy: create the orchestration, create a webservice using the BizTalk WCF Service Publishing Wizard, deploy it all and go! But alas, there will be a routing failure… BizTalk doesn’t know where to send the result message from the orchestration.
So how do we tell BizTalk the result message should be send back to the webservice? To cut a long story short, we need a Message Assignment on the result message with this code:
| Message_Out(*) = Message_In(*); Message_Out(BTS.RouteDirectToTP) = true; Message_Out(BTS.IsRequestResponse) = true; |
and on the send shape we need a Correlation Set with these properties:
BTS.CorrelationToken,BTS.EpmRRCorrelationToken,BTS.IsRequestResponse,BTS.ReqRespTransmitPipelineID,BTS.RouteDirectToTP
That’s all there’s to it!
If you want the long explanation and the ‘why’ of it all, check out Bram Veldhoen’s blog, which perfectly describes it in full detail. So no need for me to go through all that effort…
Configuring IIS to work with WCF services
The first time you’ll try to access a WCF service, you may get a ‘HTTP Error 404.3 - Not Found’ error. This can be caused by WCF not being installed properly yet. To install WCF, do the following:
At a command prompt, type the following commands:
For more background, check out this blog: http://iweb.adefwebserver.com/Default.aspx?tabid=57&EntryID=34
At a command prompt, type the following commands:
| cd c:\windows\Microsoft.Net\Framework\v3.0\Windows Communication Foundation\ ServiceModelReg –i |
For more background, check out this blog: http://iweb.adefwebserver.com/Default.aspx?tabid=57&EntryID=34
Visual Studio 2008 doesn’t know a BizTalk project
If you have a new Visual Studio 2008 installation, or updated it in any way, you may find that adding a new BizTalk project will fail. The solution is simple: in the registry, change key [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\9.0\Projects\{FAE04EC0-301F-11d3-BF4B-00C04F79EFBC}] from ‘csproj’ to ‘csproj;btproj’. Somehow this key gets reset every time the Visual Studio installation is changed...
Friday, November 12, 2010
Debatching a flat file with multiple lines per identifier
If you're new to debatching a flat file (i.e. you receive one flat file containing multiple messages, which you all have to process separately in BizTalk), there are plenty of good blogs around to get you started. One good example is this one: http://geekswithblogs.net/benny/archive/2006/02/05/68249.aspx. This blog describes how you can debatch a flat file where every line represents one message. Now let's take it a step further...
First let me describe the data for you: let's say it's someone's birthday and me and my friends want to give her presents. And obviously, being a good nerd, I have written an orchestration that processes the data (For what purpose? Use your imagination! I'm already happy I came up with this example!). This orchestration takes as input an XML message like this:
The input for my program
Now a friend of mine heard about my plans and to be helpful he started to collect the data in a flat file that looks like this:
Flat file format 1
There are multiple lines per friend! So I can't use the basic flat file disassembler to convert this flat file into the XML I need!
Of course I could write a custom pipeline component that replaces the standard flat file disassembler component but another thought kept on spinning in my mind.. I could easily use the standard flat file disassembler component if my input file would look like this:
Flat file format 2
So every present is separated by a '|', while the data of each present is separated by a semi-colon.
With this in mind, it was quite easy to write a custom pipeline component for the Decode stage of a pipeline. This component should read the flat file format 1, join the lines, and produce a flat file format 2 as I described above. Here is the code of the component as I wrote it. Just disregard all the 'component' stuff and scroll right down to the Execute method, where all the hard work is done.
As you can see, there is one property called 'NumberOfKeyFields'. In my example, there is only 1 key field (the friend's name) but in case you would have a 'firstname;lastname;' key then by setting this property to 2, the MultiLineJoiner still does the work.
First let me describe the data for you: let's say it's someone's birthday and me and my friends want to give her presents. And obviously, being a good nerd, I have written an orchestration that processes the data (For what purpose? Use your imagination! I'm already happy I came up with this example!). This orchestration takes as input an XML message like this:
| <Friend xmlns="http://ronaldlokers.blogspot.com/birthdaypresents"> <Name>John</Name> <Presents> <Present> <Name>Flowers</Name> <Price>5.99</Price> </Present> <Present> <Name>Teapot</Name> <Price>11.99</Price> </Present> </Presents> </Friend> |
Now a friend of mine heard about my plans and to be helpful he started to collect the data in a flat file that looks like this:
| Mary;Shawl;14.50 Mary;Headphones;12.99 Christian;Ring;99.50 John;Flowers;5.99 John;Teapot;11.99 Marcy;Red lipstick;4.99 Marcy;Purple lipstick;4.99 Marcy;Black lipstick;4.99 |
There are multiple lines per friend! So I can't use the basic flat file disassembler to convert this flat file into the XML I need!
Of course I could write a custom pipeline component that replaces the standard flat file disassembler component but another thought kept on spinning in my mind.. I could easily use the standard flat file disassembler component if my input file would look like this:
| Mary|Shawl;14.50|Headphones;12.99 Christian|Ring;99.50 John|Flowers;5.99|Teapot;11.99 Marcy|Red lipstick;4.99|Purple lipstick;4.99|Black lipstick;4.99 |
So every present is separated by a '|', while the data of each present is separated by a semi-colon.
With this in mind, it was quite easy to write a custom pipeline component for the Decode stage of a pipeline. This component should read the flat file format 1, join the lines, and produce a flat file format 2 as I described above. Here is the code of the component as I wrote it. Just disregard all the 'component' stuff and scroll right down to the Execute method, where all the hard work is done.
using System; using System.IO; using System.Text; using System.Drawing; using System.Resources; using System.Reflection; using System.Diagnostics; using System.Collections; using System.ComponentModel; using Microsoft.BizTalk.Message.Interop; using Microsoft.BizTalk.Component.Interop; using Microsoft.BizTalk.Component; using Microsoft.BizTalk.Messaging; using Microsoft.BizTalk.Component.Utilities; namespace RonaldLokers.Blogspot.Com { [ComponentCategory(CategoryTypes.CATID_PipelineComponent)] [System.Runtime.InteropServices.Guid("79e5a9b2-9abf-40d8-aa02-d4c15b218f88")] [ComponentCategory(CategoryTypes.CATID_Decoder)] public class MultiLineJoiner : Microsoft.BizTalk.Component.Interop.IComponent, IBaseComponent, IPersistPropertyBag, IComponentUI { private System.Resources.ResourceManager resourceManager = new System.Resources.ResourceManager("RonaldLokers.Blogspot.Com.MultiLineJoiner", Assembly.GetExecutingAssembly()); private int _NumberOfKeyFields; public int NumberOfKeyFields { get { return _NumberOfKeyFields; } set { _NumberOfKeyFields = value; } } #region IBaseComponent members /// <summary> /// Name of the component /// </summary> [Browsable(false)] public string Name { get { return resourceManager.GetString("COMPONENTNAME", System.Globalization.CultureInfo.InvariantCulture); } } /// <summary> /// Version of the component /// </summary> [Browsable(false)] public string Version { get { return resourceManager.GetString("COMPONENTVERSION", System.Globalization.CultureInfo.InvariantCulture); } } /// <summary> /// Description of the component /// </summary> [Browsable(false)] public string Description { get { return resourceManager.GetString("COMPONENTDESCRIPTION", System.Globalization.CultureInfo.InvariantCulture); } } #endregion #region IPersistPropertyBag members /// <summary> /// Gets class ID of component for usage from unmanaged code. /// </summary> /// <param name="classid"> /// Class ID of the component /// </param> public void GetClassID(out System.Guid classid) { classid = new System.Guid("79e5a9b2-9abf-40d8-aa02-d4c15b218f88"); } /// <summary> /// not implemented /// </summary> public void InitNew() { } /// <summary> /// Loads configuration properties for the component /// </summary> /// <param name="pb">Configuration property bag</param> /// <param name="errlog">Error status</param> public virtual void Load(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, int errlog) { object val = null; val = this.ReadPropertyBag(pb, "NumberOfKeyFields"); if (val != null) { this._NumberOfKeyFields = ((int)(val)); } } /// <summary> /// Saves the current component configuration into the property bag /// </summary> /// <param name="pb">Configuration property bag</param> /// <param name="fClearDirty">not used</param> /// <param name="fSaveAllProperties">not used</param> public virtual void Save(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, bool fClearDirty, bool fSaveAllProperties) { this.WritePropertyBag(pb, "NumberOfKeyFields", this.NumberOfKeyFields); } #region utility functionality /// <summary> /// Reads property value from property bag /// </summary> /// <param name="pb">Property bag</param> /// <param name="propName">Name of property</param> /// <returns>Value of the property</returns> private object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName) { object val = null; try { pb.Read(propName, out val, 0); } catch (System.ArgumentException) { return val; } catch (System.Exception e) { throw new System.ApplicationException(e.Message); } return val; } /// <summary> /// Writes property values into a property bag. /// </summary> /// <param name="pb">Property bag.</param> /// <param name="propName">Name of property.</param> /// <param name="val">Value of property.</param> private void WritePropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName, object val) { try { pb.Write(propName, ref val); } catch (System.Exception e) { throw new System.ApplicationException(e.Message); } } #endregion #endregion #region IComponentUI members /// <summary> /// Component icon to use in BizTalk Editor /// </summary> [Browsable(false)] public IntPtr Icon { get { return ((System.Drawing.Bitmap)(this.resourceManager.GetObject("COMPONENTICON", System.Globalization.CultureInfo.InvariantCulture))).GetHicon(); } } /// <summary> /// The Validate method is called by the BizTalk Editor during the build /// of a BizTalk project. /// </summary> /// <param name="obj">An Object containing the configuration properties.</param> /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns> public System.Collections.IEnumerator Validate(object obj) { // example implementation: // ArrayList errorList = new ArrayList(); // errorList.Add("This is a compiler error"); // return errorList.GetEnumerator(); return null; } #endregion #region IComponent members /// <summary> /// Implements IComponent.Execute method. /// </summary> /// <param name="pc">Pipeline context</param> /// <param name="inmsg">Input message</param> /// <returns>Original input message</returns> /// <remarks> /// IComponent.Execute method is used to initiate /// the processing of the message in this pipeline component. /// </remarks> public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg) { StreamReader strIn = new StreamReader(inmsg.BodyPart.GetOriginalDataStream()); MemoryStream memStr = new MemoryStream(); StreamWriter strOut = new StreamWriter(memStr); // Read the first line of the input string line = strIn.ReadLine(); // Set an arbitrary key that won't match the first key field string key = "#"; while (line != null) { if (!line.StartsWith(key)) { // If this line doesn't start with the current key, start out by writing the key to the destination, and store the key in the 'key' variable string[] parts = line.Split(new char[] { ';' }); key = parts[0]; for (int i = 1; i < NumberOfKeyFields; i++) key += ";" + parts[i]; strOut.Write(key.Replace(";", "|")); key += ";"; } // Write the data of the non-key part of the line strOut.Write("|" + line.Substring(key.Length)); // Read the next line from our source line = strIn.ReadLine(); if (line == null || !line.StartsWith(key)) { // If we're at the end of the file, or the next line has a different key, go to the next line in our destination strOut.WriteLine(); } } strOut.Flush(); memStr.Position = 0; pc.ResourceTracker.AddResource(memStr); inmsg.BodyPart.Data = memStr; return inmsg; } #endregion } } |
As you can see, there is one property called 'NumberOfKeyFields'. In my example, there is only 1 key field (the friend's name) but in case you would have a 'firstname;lastname;' key then by setting this property to 2, the MultiLineJoiner still does the work.
Introduction
As you will understand from the name of this blog, my name is Ronald Lokers. Let me introduce myself!
I'm a BizTalk developer (amongst a whole lot of other things), but I'm also keeping an eye out for all things related to integration. I live in Montfoort, the Netherlands and I work at a large IT company.
In this blog I will post my 'Adventures in Integration Land'... don't expect any ground-breaking new insights though (or not right away anyway), I intend to use this blog to post things that took me a while to figure out, or to find on the Internet, so I can easily find them again when I need them. And perhaps some of these posts might help YOU out in your search for Integration Enlightenment!
Well, so much for the introduction... if you'd like to know more about me, just ask!
I'm a BizTalk developer (amongst a whole lot of other things), but I'm also keeping an eye out for all things related to integration. I live in Montfoort, the Netherlands and I work at a large IT company.
In this blog I will post my 'Adventures in Integration Land'... don't expect any ground-breaking new insights though (or not right away anyway), I intend to use this blog to post things that took me a while to figure out, or to find on the Internet, so I can easily find them again when I need them. And perhaps some of these posts might help YOU out in your search for Integration Enlightenment!
Well, so much for the introduction... if you'd like to know more about me, just ask!
Subscribe to:
Posts (Atom)