Ben Runchey’s Integration Dev Blog

a.k.a Ben’s spot to put stuff he does not want to forget

What I Learned Yesterday 12

  • Orchestration Throttling within BizTalk – It appears that Scott Colestock has a nice approach to throttling.  Richard Seroter has it outlined on his blog here.  While I like this solution, I still wish that the good guys on the BizTalk Server dev team would provide some sort of functionality for us in this realm.  I realize it is maybe not the most important functionality, but it is probably the only feature from BizTalk 2000/2002 that I miss.
  • Archiving Pipeline Component – Ran into a situation where I wanted to archive outbound flat-files in a send pipeline.  So I looked into using the Archiving component written by Gilles Zunino a few years ago since it has worked so well for me on the inbound files.  Well after some testing it appears that there are some issues using this with the FF Assembler in the same Send Pipeline.  I tried implementing some of the solutions others have posted on their blogs, but none worked for me.  Not exactly sure what causes this, but to handle archiving my file, I used the CForwardOnlyEventingReadStream Class which is found in the Microsoft.BizTalk.Streaming namespace (and assembly of the same name).   I found a great post about using it at Mikael Hakanssons blog here about how he was using it in a component to archive messages to a TEXT field.  Basically what this class provides is the ability to have a callback for when BizTalk reads the stream to give you the opportunity at the same time.
  • Xobni Add in for Outlook – If you are using outlook you might consider trying the addin from Xobni.  It really makes “seeing across” your email intuitive.  It tracks all sorts of “social” information about your inbox. 
  • BizTalk Best Practices Analyzer – While this is not a new application, I have been trying to use it more religiously when setting up a configuration on one my client’s biztalk server group.  It just makes sure to catch things that I may miss that do not adhere to what the BizTalk team has designated as best practices.  Kind of nice to schedule running this on a regular basis (like nightly maybe) to catch when something might have gotten out of wack on your server.  It would be nice to be able to have the report be loaded into the MMC console.  Kind of a one stop shop to manage your BizTalk server. 

May 27, 2008 Posted by | BizTalk, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 11

  • Google and their “exodus” – Found an article on earlier today about “What’s next for Google?”.  Much of the article focused on why many of the most talented people are moving on from Google.  I don’t think that those writing these articles are always in touch with what makes many technical people tick.  Working for Google was probably something that allowed many of these engineers to work on advanced projects, learning new things from people they may not otherwise have run into.  The one down side to this for these engineers is that they are working FOR other people who are making the decisions.  Many of the best engineers would ultimately like to have some sucess on their OWN terms.  This in my opinion is why many of the best people are most likely moving on to start their own thing, or have more say in what goes on.
  • Blogical sFTP adapter for BizTalk – Found a really great sFTP adapter on Codeplex.  While I still love the adapters from /n Software, if you are only going to be using the sFTP adapter, it is kind of expensive to pay the 2500/proc to buy it.  Not to mention the guys that wrote it are very quick to respond to emails (thanks Mikael!).
  • Sql Server Integration Services – I am having to work with this quite a bit these days.  I like it but there are a few things I wish it had
    • The support for Checkpoints is nice, but I wish it were improved to support multiple checkpoints within one package.  I have to basically use a custom built state machine to make this happen for myself right now.
    • Ability to have some common scripts.  I would like to have to only write common code once and share it between all of my script tasks.  I realize that I can use a .NET component to do this, but for small blocks of code I would like to be able to use a module of some sort.

May 14, 2008 Posted by | BizTalk, SSIS, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 10

  • Found a great tool for exporting data from existing SQL Server Tables to a script – The SQL Scripter!  What a clever name.  You can find it at
  • Looking into Ruby on Rails these days for a potential project and I have to say WOW!  this environment is really productive.  Just in the short time that I have spent with it I can tell it was written with how a developer needs to work as it’s main goal.  The strict MVC support, ActiveRecord for the ORM layer, and the built in support for unit testing.  While I really like it, I still really thing .NET and the CLR/DLR is the best of both worlds.  I have not had a chance to see the MVC support in the most recent version of ASP.NET (3 or 3.5, can’t remember), but my advice to Microsoft is to start building your tools to utilize your frameworks with the developer in mind.  I think that is reason number 1 why people are raving about Ruby.  The productivity in this environment is sweet!
  • What is with the script task in SQL Server Integration Services?  I would like to find out why it is not possible to just access any global variable inside of my script task!  For some reason, very smart people like to make me declare what global variables I am going to need access to before I can use them in the code.  I hope they change this in the next version!
  • I have pretty much decided to go with XSL Transformations instead of the biztalk mapper for any maps which start to get complicated for two reasons.
    • First reason is development time.  I can code XSL much faster than I can trial and error my way through using the functoids available.  Also the lack of a simple if then else type functoid can sometimes frustrate me to no end.
    • Being able to debug an XSL is easier when it is written using variable names that have context. 

April 19, 2008 Posted by | BizTalk, Ruby on Rails, T-SQL, What I Learned Yesterday | 1 Comment

What I Learned Yesterday 8

  • In my past use of generics I largely used them to strongly type the parameters of a list class, or a utilitiy class.  Basically one generic type per class.  But I recently ran into a situation where I needed to use multiple generic types in one class.  I have to admit I think the coolest thing that was in the 2.0 verson of C# was the support for generics.  I love it.  here is a link to a hard core use of generics which helped me figure out how to use generics to solve my problem.
  • Don’t short change yourself when it comes to load testing your application.  There are so many things that you cannot simulate within your app in basic unit testing and functional testing.  Certain things only appear when an app is pushed.  I really wish you people out there who think you are “Business People” would shut up and listen to the developers.  You are paying us for our experience, why don’t you listen to it for a change!  Oh, but listening is a business decision, not something we want to leave up to the developers!

March 12, 2008 Posted by | C# Generics, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 7

  • It has been a while since I have had to really go at things using T-SQL.  Lots of my recent experiences have been around CRUD type operations where we would read the data into an object model work on it and then save it back to the database.  But I am working on a project where we are loading a temp table and then inserting/updating records into another table.  To get the identity columns of the newly inserted or newly updated records the OUTPUT keyword in T-SQL is a life saver.   This is only available in SQL Server 2005/2008 though.  This link here is a good overview of what it can do.
  • I spoke of how cool the use of Over is with rownumber, but here is an article where the guy is improving on top.  I have to admit the things that are in SQL 2005 and newer from just a T-SQL standpoint are pretty sweet.  They make writing some of the useless looping logic that permiated all aspects of my “business logic” a much easier task. Shows how long I have been doing XML and Integration that I have not spent any large amounts of time learning these new features (which are not so new at the time I am writing this)
  • Hiding the Results Pane (or should I say pain?) in the SQL 2005 Management Studio is something I have wanted for quite some time.  I have always liked being able to have a full screen for editing my T-SQL script.  I have finally figured out how to make this happen thanks to this guy here.
  • To output a null date to a FlatFile Schema in Biztalk set the format of the date field to yyyyMMdd.  Then use the pad character to zero fill if you need. Beautiful

March 7, 2008 Posted by | BizTalk, T-SQL, What I Learned Yesterday | Leave a comment

What I Learned Yesterday 6

  • To enable edit and contiue I had to uninstall the Windows SDK debugging tools.  Apparently this had a profiler running which did not allow for me to do the edit and continue in vs 2005.   Life is much better now.  I had actually never used this in 2005 previously.
  • BizTalk CrossReferencing API – looks like this is the way to solve those pesky file counters that some people like to use to avoid duplicate files or as a reference point. Again since I feel like all I do is regurgitate things that other people have already writted good blog posts on I will just do some simple link forwarding on this one.  The first one is by a guy named Eric (not sure about his last name) and can be found here.  While the other one is by Brian Loesgen and can be found here.   
  • Using the Looping functoid and the Table looping functoid together does not always work so well.  Please see my sample post to follow shortly on why these two don’t work together.
  • BizBert has put together a great group of articles about the BizTalk Mapper.  Lots of information about functoids, performance, etc. You can find it here.

February 28, 2008 Posted by | BizTalk, Visual Studio 2005, What I Learned Yesterday | , , | Leave a comment

What I learned yesterday 5

  • Free file sharing – I remember back in 2000 when everything was going to be on the internet but the bandwidth just wasn’t there to support using the internet for everything like file storage, pictures, etc.  But now that bandwidth is not a problem, I am really enjoying the free tools and applications available.  Right now I am using to store my files that I cannot easily upload for my blog or to just easily have access to things I use at various client sites, but might not have for one reason or another. 
  • More SharePoint web services fun (or rather confusion) – The FileRef field should not be encoded when you are trying to update/delete a file using the UpdateListItems web method of the List web service.  Thanks to Geoffry Samper for his post here .
  • Repeatable deployments in BizTalk.  – Looks like Brian Loesgen and Paul Petrov have each individually done a certain part of what I was planning to do with a bit of my free time using BTSTask.  Hopefully I can take these ideas and build on them for my own diabolical needs.
  • Pretty cool panoramic of the inside of an Airbus 380 cockpit
  • Rockford Lhotka has an article which I agree with here .  Actually he has many articles I agree with in his Lhotka’s Labyrinth column in Visual Studio Magazine.

January 31, 2008 Posted by | What I Learned Yesterday | , | Leave a comment

What I Learned Yesterday 4

Bit of knowledge for you terminal services users.  If a machine is just wacko and you cannot quite get things to clean up when it comes to the number of terminal server connections  use mstsc /console at the command line to run the remote desktop tool.  This switch will ignore the max number of administrative connections to a server.

A pretty cool Biztalk documentation utility for finding out how all of those really great posters map back to the help file can be found at the BizTalk server team blog here.  Check this one out!.

While the use of the WSS adapter within BizTalk makes life a lot easier, installing it in your environment when your SharePoint site is on another server is a different story.  It is not that this is hard, but it is not really clear.  So I had to put together a document help out a SharePoint admin at a client of mine so that we could get the adapter web service installed on the SharePoint server.  Find the document here

The solution I am currently working on uses a few document libraries within SharePoint to  keep track of what status a file is in.  We tried to use Views exclusively, but the problem here was getting the security right.  And since we wanted to allow for custom views still, it was too big of a pain.  So we created different document libraries and assigned rights at the document library level, it made things much easier.  The next need was to get all of the necessary Columns and views set up correctly across these three Document Libraries in an automated fashion.  So I created this little application which takes in an XML file to drive the process using the SharePoint object model.  You can download it here.  While this is not the world’s greatest utility, it works and can be used as a basis for future needs. 

Also while working with Sharepoint Object model to create a view within a Document Library that only includes the files set the “Scope” property of the SPView object you are working with to “SPViewScope.Recursive”.  While I think this is understandable, why is it that the API for doing things always seems to have a few areas where they just cannot name things the same (or even remotely the same in this case) as what you see through the UI.  In the UI this is the “Folders” area of the view

I have been doing software for a while now (last 11 years), and I am a big fan of breaking down bigger problems into its smaller core parts, solving each, and then reassembling them to create the solution.  This is for one main reason, smaller is usually simpler.  Smaller teams, smaller problem domain, etc.  While eventually we have to pay the piper and assemble our small parts into the grand solution, even this can be broken into a few core parts.  I found a good arguement for small teams here that I have to agree with.

Upgrading BizTalk 2006 to BizTalk 2006 R2 nuance:  to get the WCF adapters and EDI/AS2 options installed you have to run the installation AGAIN after upgrading.  The upgrade only upgrades your existing configuration.  The additional install will allow you to add the WCF and EDI/AS2 options to your configuration.  Again this makes sense, but is not really clear from reading the documentation.

January 25, 2008 Posted by | What I Learned Yesterday | , | Leave a comment

What I Learned Yesterday 3

  • I have never really spent any time trying to figure out the trusted vs. untrusted host concept in BizTalk.  Then finally the other day I was moving an application from my virtual machine to one that a client of mine was using.  In the process when trying to apply the binding file, my hosts were all marked as trusted, while his hosts were not marked as trusted.  Just to make sure we were not getting into something that would create a problem I did a little research and found this great article by Richard Seroter about when the “trusted” flag comes into play.  It appears it plays an important role when you want to pass through the credentials of the calling application/user to the destination system.
  • Been spending a lot of time with SharePoint recently mostly on the integration side in pushing documents to SharePoint using the BizTalk SharePoint services adapter.  But I also had a need to delete some files, create directories and sub directories within Document Libraries.  All of the things that I cannot easily do with the SharePoint Adapter.  Or if I can, it is not easily figured out. So I am using the WSS Web Servicesto do all of the things that the Adapter does allow me to do.  Doing a search on “sharepoint services web services” brings up a whole host of articles people have written in the past, so I won’t go into it.  The really interesting thing about these are how you can tell their age from the signatures.  Rather than having a good strongly-typed object to pass into a method, all of the methods take either a string, or an int, or just an xml document which must adhere to a structure. The documentation covers all of this, but it is definitely easier coding to a strongly-typed object than an xml document.  Misspellings can become a real pain.  Also, for queries CAML is used, which while understandable is really again quite archaic
  • Continuing to work with BTS Task and found out a gotcha about the “Overwrite” flag in the AddResource option.  In order for you to overwrite the files you need to have all of the hosts that are referencing them turned off, or at least not have the file loaded into memory just yet.  That killed about 30 minutes as I tried to figure out why it would not let me overwrite a file.
  • Windows Sharepoint Services Adapter for BizTalk.  Remember there are two parts to this guy.  One the adapter runtime which must be installed on the biztalk server itself.  The other is the WebService side.  If you look at the site you install the adapter into there is an application named BTSharePointAdapterWS within it which runs inside of a different app pool than the site this application is in.  The moral of the story is that any SharePoint server you want to send a message to using the adapter must have this web service installed.  I will put up a post on this a bit later describing what to do.

January 16, 2008 Posted by | What I Learned Yesterday | , , | Leave a comment

What I Learned Yesterday 2

  • The BTSTask application that I spoke about yesterday really seems to get the job done quickly for me.  How it took me this long to actually spend some time with it is puzzling.  I really like it!  The biggest challenge in using it is that it is a command-line tool, so using it within Visual Studio is not something I am really skilled at just yet, so I am trying to make it a stand-alone tool for myself. 
  • When the Pre and Post ProcessingScripts for BizTalk Deployments execute are somewhat puzzling to me.  Not that it is that hard to understand the concept of a Pre or Post script on installation so much as it is when they run and what will be in place for me to utilize.  I have a few batch files I want to execute, but I cannot seem to figure out how to get them to execute from my custom script.  I suppose I could put them into the script, but that is not as flexible for me.  More on this to come…
  • I think that the ugliest trophy in all of college football (and maybe sports for that matter) is the Fiesta Bowl trophy.  I am not sure who designed it or what they were thinking, but what ever happened to a nice solid gold bowl of tostito’s corn chips? Tonite it is the Orange Bowl who’s trophy with oranges and without looks atleast tolerable…

January 3, 2008 Posted by | BizTalk, What I Learned Yesterday | Leave a comment