Ben Runchey’s Integration Dev Blog

a.k.a Ben’s spot to put stuff he does not want to forget

What I Learned Yesterday 12

  • Orchestration Throttling within BizTalk – It appears that Scott Colestock has a nice approach to throttling.  Richard Seroter has it outlined on his blog here.  While I like this solution, I still wish that the good guys on the BizTalk Server dev team would provide some sort of functionality for us in this realm.  I realize it is maybe not the most important functionality, but it is probably the only feature from BizTalk 2000/2002 that I miss.
  • Archiving Pipeline Component – Ran into a situation where I wanted to archive outbound flat-files in a send pipeline.  So I looked into using the Archiving component written by Gilles Zunino a few years ago since it has worked so well for me on the inbound files.  Well after some testing it appears that there are some issues using this with the FF Assembler in the same Send Pipeline.  I tried implementing some of the solutions others have posted on their blogs, but none worked for me.  Not exactly sure what causes this, but to handle archiving my file, I used the CForwardOnlyEventingReadStream Class which is found in the Microsoft.BizTalk.Streaming namespace (and assembly of the same name).   I found a great post about using it at Mikael Hakanssons blog here about how he was using it in a component to archive messages to a TEXT field.  Basically what this class provides is the ability to have a callback for when BizTalk reads the stream to give you the opportunity at the same time.
  • Xobni Add in for Outlook – If you are using outlook you might consider trying the addin from Xobni.  It really makes “seeing across” your email intuitive.  It tracks all sorts of “social” information about your inbox. 
  • BizTalk Best Practices Analyzer – While this is not a new application, I have been trying to use it more religiously when setting up a configuration on one my client’s biztalk server group.  It just makes sure to catch things that I may miss that do not adhere to what the BizTalk team has designated as best practices.  Kind of nice to schedule running this on a regular basis (like nightly maybe) to catch when something might have gotten out of wack on your server.  It would be nice to be able to have the report be loaded into the MMC console.  Kind of a one stop shop to manage your BizTalk server. 

May 27, 2008 Posted by | BizTalk, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 11

  • Google and their “exodus” – Found an article on earlier today about “What’s next for Google?”.  Much of the article focused on why many of the most talented people are moving on from Google.  I don’t think that those writing these articles are always in touch with what makes many technical people tick.  Working for Google was probably something that allowed many of these engineers to work on advanced projects, learning new things from people they may not otherwise have run into.  The one down side to this for these engineers is that they are working FOR other people who are making the decisions.  Many of the best engineers would ultimately like to have some sucess on their OWN terms.  This in my opinion is why many of the best people are most likely moving on to start their own thing, or have more say in what goes on.
  • Blogical sFTP adapter for BizTalk – Found a really great sFTP adapter on Codeplex.  While I still love the adapters from /n Software, if you are only going to be using the sFTP adapter, it is kind of expensive to pay the 2500/proc to buy it.  Not to mention the guys that wrote it are very quick to respond to emails (thanks Mikael!).
  • Sql Server Integration Services – I am having to work with this quite a bit these days.  I like it but there are a few things I wish it had
    • The support for Checkpoints is nice, but I wish it were improved to support multiple checkpoints within one package.  I have to basically use a custom built state machine to make this happen for myself right now.
    • Ability to have some common scripts.  I would like to have to only write common code once and share it between all of my script tasks.  I realize that I can use a .NET component to do this, but for small blocks of code I would like to be able to use a module of some sort.

May 14, 2008 Posted by | BizTalk, SSIS, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 10

  • Found a great tool for exporting data from existing SQL Server Tables to a script – The SQL Scripter!  What a clever name.  You can find it at
  • Looking into Ruby on Rails these days for a potential project and I have to say WOW!  this environment is really productive.  Just in the short time that I have spent with it I can tell it was written with how a developer needs to work as it’s main goal.  The strict MVC support, ActiveRecord for the ORM layer, and the built in support for unit testing.  While I really like it, I still really thing .NET and the CLR/DLR is the best of both worlds.  I have not had a chance to see the MVC support in the most recent version of ASP.NET (3 or 3.5, can’t remember), but my advice to Microsoft is to start building your tools to utilize your frameworks with the developer in mind.  I think that is reason number 1 why people are raving about Ruby.  The productivity in this environment is sweet!
  • What is with the script task in SQL Server Integration Services?  I would like to find out why it is not possible to just access any global variable inside of my script task!  For some reason, very smart people like to make me declare what global variables I am going to need access to before I can use them in the code.  I hope they change this in the next version!
  • I have pretty much decided to go with XSL Transformations instead of the biztalk mapper for any maps which start to get complicated for two reasons.
    • First reason is development time.  I can code XSL much faster than I can trial and error my way through using the functoids available.  Also the lack of a simple if then else type functoid can sometimes frustrate me to no end.
    • Being able to debug an XSL is easier when it is written using variable names that have context. 

April 19, 2008 Posted by | BizTalk, Ruby on Rails, T-SQL, What I Learned Yesterday | 1 Comment

What I Learned Yesterday 7

  • It has been a while since I have had to really go at things using T-SQL.  Lots of my recent experiences have been around CRUD type operations where we would read the data into an object model work on it and then save it back to the database.  But I am working on a project where we are loading a temp table and then inserting/updating records into another table.  To get the identity columns of the newly inserted or newly updated records the OUTPUT keyword in T-SQL is a life saver.   This is only available in SQL Server 2005/2008 though.  This link here is a good overview of what it can do.
  • I spoke of how cool the use of Over is with rownumber, but here is an article where the guy is improving on top.  I have to admit the things that are in SQL 2005 and newer from just a T-SQL standpoint are pretty sweet.  They make writing some of the useless looping logic that permiated all aspects of my “business logic” a much easier task. Shows how long I have been doing XML and Integration that I have not spent any large amounts of time learning these new features (which are not so new at the time I am writing this)
  • Hiding the Results Pane (or should I say pain?) in the SQL 2005 Management Studio is something I have wanted for quite some time.  I have always liked being able to have a full screen for editing my T-SQL script.  I have finally figured out how to make this happen thanks to this guy here.
  • To output a null date to a FlatFile Schema in Biztalk set the format of the date field to yyyyMMdd.  Then use the pad character to zero fill if you need. Beautiful

March 7, 2008 Posted by | BizTalk, T-SQL, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 6

  • To enable edit and contiue I had to uninstall the Windows SDK debugging tools.  Apparently this had a profiler running which did not allow for me to do the edit and continue in vs 2005.   Life is much better now.  I had actually never used this in 2005 previously.
  • BizTalk CrossReferencing API – looks like this is the way to solve those pesky file counters that some people like to use to avoid duplicate files or as a reference point. Again since I feel like all I do is regurgitate things that other people have already writted good blog posts on I will just do some simple link forwarding on this one.  The first one is by a guy named Eric (not sure about his last name) and can be found here.  While the other one is by Brian Loesgen and can be found here.   
  • Using the Looping functoid and the Table looping functoid together does not always work so well.  Please see my sample post to follow shortly on why these two don’t work together.
  • BizBert has put together a great group of articles about the BizTalk Mapper.  Lots of information about functoids, performance, etc. You can find it here.

February 28, 2008 Posted by | BizTalk, Visual Studio 2005, What I Learned Yesterday | , , | Leave a comment

Finding out the mapper can do what I want it to do

This is a follow up post to my previous post talking about how I used some XSLT with an interim schema to get some mapping done.  In my travels around the internet since then I have found a post from Jeff Lynch about how to use the mapper to map “Small Loops“.  Well this was exactly what I was trying to get done in my previous post.  Simply put the looping functoid can get done what I need.  No more interim schema!

 here is the new and improved map

Notice how I linked the PONum, ShippingDate attributes to the Looping functoid seperately from the Item element.  Once I slowed down long enough to think about what the XSLT this would generate it made sense.  Each element or attribute plugged into the left side of the Looping functoid will have it’s own loop created for it.  That is the key here.   I have also updated the source code for this guy.

 I have to say, I like this solution much better!

February 19, 2008 Posted by | BizTalk | | 1 Comment

Some XSLT for when you just can’t get the mapper to do what you want it to do

Update – Please see my subequent post titled “Finding out the mapper can do what I want it to do” which makes this “hack” not necessary any longer 

Recently I have been spending some (maybe too much) time on the msdn forums to see if I can help out anyone that is having a problem with BizTalk.  I figure since most of my knowledge comes from reading blog posts, I need to do something that gives back to the world. 

So on friday a post showed up about needing to map from one format with data in a non repeating header and moving it to a repeating element in a destination document.  You can read actual post here.  An example of the source and destination would be as follows

The source schema looks like:


 And our destination schema looks like


So as you can see we need to take the PONum and ShippingDate attributes from the Header element of the source document and put them into two different MainLoop elements in our destination document.  Then take the repeating Item elements from the source and fill out the remaining MainLoop elements in the destination document. 

Getting the PONum and ShippingDate out proved to be a bit more challenging than I had orignally thought.   I did not want to write just straight XSLT and by pass the mapper.  That is always an option, but then I am not really using the tools available.    What I decided to do was use inline XSLT in a scripting functoid to copy the Source\Header\PONum and ShippingDate fields to the MainLoop.  So I came up first with a map that looked like this


But the Xml was not what I wanted… (not even close!)


So when looking at the xslt the map was creating I realized that  getting a MainLoop element for both my PONum and ShippingDate field would require me to have an “empty” spot in my destination schema that I could push the XSLT from my two scripting functoids into.  So I created a temporary destination schema that contained a “PlaceHolder” for me to overwrite.


Now my map looks like this…


Which produces the correct output…


Now I know, if this is a huge document, or I am doing lots of these throughout the day and performance is a very big consideration the straight XSLT approach would be more performant, but until I need to have the speed, I will take the maintainability.  You can download the project that demonstrates this here

 Mape1082 I hope this helps you somehow. 


February 19, 2008 Posted by | BizTalk | 1 Comment

What I Learned Yesterday 2

  • The BTSTask application that I spoke about yesterday really seems to get the job done quickly for me.  How it took me this long to actually spend some time with it is puzzling.  I really like it!  The biggest challenge in using it is that it is a command-line tool, so using it within Visual Studio is not something I am really skilled at just yet, so I am trying to make it a stand-alone tool for myself. 
  • When the Pre and Post ProcessingScripts for BizTalk Deployments execute are somewhat puzzling to me.  Not that it is that hard to understand the concept of a Pre or Post script on installation so much as it is when they run and what will be in place for me to utilize.  I have a few batch files I want to execute, but I cannot seem to figure out how to get them to execute from my custom script.  I suppose I could put them into the script, but that is not as flexible for me.  More on this to come…
  • I think that the ugliest trophy in all of college football (and maybe sports for that matter) is the Fiesta Bowl trophy.  I am not sure who designed it or what they were thinking, but what ever happened to a nice solid gold bowl of tostito’s corn chips? Tonite it is the Orange Bowl who’s trophy with oranges and without looks atleast tolerable…

January 3, 2008 Posted by | BizTalk, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 1

 – Beware the BizTalk 2006 R2 Trial edition.  Ran into a situation where the trial edition was installed and had expired.  I was unaware that it was the R2 trial edition, since in the the list of adapters none of the WCF adapters where present.  So I thought I would just reinstall the Enterprise version of 2006.  This created lots of problems with WMI and ruined my Virtual Image.  Good thing there was a backup of the image.  So uninstalled it first off the image the second time around and everything looks good so far.

– Installing a custom Biztalk application.  While I really like Scott Colestock’s Deployment Framework for BizTalk, not all of the places I do work at are quick to embrace it.  I think because sometimes it is like killing a fly with a sledgehammer for smaller applications.  So as an alternative I have been spending some time with BTSTask and just creating a batch file to execute which adds my additional resources and binding files.  This in conjunction with the Deploy command in visual studio creates a rather easy way to get a BizTalk application to export a valid .msi file.

–  I was out the other day and found out about a cool appliation from the MSDN Architecture Center for reading the Architecture Journal, which can be downloaded here.  While I could go into detail here about all of the interesting and handy features it has Richard Seroter has already done this on his blog here.

– Last piece is the about book that I am almost done reading Pro WF by Bruce Bukovics is really well put together in my opinion.  It is not really about building one application so much as it is about disecting all of the pieces that make up WF.  For a BizTalker such as myself I really enjoyed it because I can see how many of the components translate directly to things I deal with every day in BizTalk orchestrations such as compensation, dehydration, tracking, etc.  Now I just have to get my “I owe you lunch application” updated to use it. 

January 2, 2008 Posted by | BizTalk, What I Learned Yesterday | , | Leave a comment