Migrating BizTalk to Azure iPaaS – Part 1

By Bill Chesnut

This is the 1st of a multipart series of Blog Posts to help companies understand Migrating BizTalk to Azure Integration Platform as a Service (IPaaS).

Should I be migrating from BizTalk to Azure iPaaS? This is a question I am being ask more and more lately for a couple of different reasons.

BizTalk development has always been a very specialized skill set with a limited number of resources available in the market, so companies are now starting to look at Azure iPaaS as a viable alternative to BizTalk in terms of finding developers with the correct skill sets.

There are 3 other reasons many companies are looking at moving from BizTalk to Azure iPaaS:

  • cost, consumption pricing instead of product licensing;
  • location of data, many companies are dealing with their consumers over the internet, so having their integration resources in Azure makes more sense;
  • Infrastructure, companies are seeing the advantages of not having to maintain their own infrastructure and using Platform as a Service (PaaS) and Software as a Service (SaaS) offerings as a cost-effective alternative.

Unlike the BizTalk middleware products that covered most features required for all integration scenarios, Azure iPaaS is a collection of Azure services that may or may not be utilised for each scenario.

The base components that make up Azure iPaaS are Service Bus, Logic Apps, API Management and Event Grid. For some integration scenarios you may also use services like but not limited to: SQL Database, Storage, Function App, Web Site (hosing your WebAPI or WebJobs), Cosmos Db and Virtual Machines.  In this series of blog posts, I will look at the different scenarios and the Azure services required.

Instead of spending time here to explain each of the Azure iPaaS services, I have included links to Azure documentation on each product/service mentioned:

All Azure Product/Services can be found here: https://docs.microsoft.com/en-us/azure/#pivot=products

Before you jump full steam into migrating from your on-premises (or cloud hosted) BizTalk to Azure iPaaS, there are a few things that you need to look at to see if the migration makes sense:

  • Location of your data
  • Sensitivity of your data
  • Security Policies for accessing cloud-based resources
  • Location of consumers/partners

Let’s look at each of these items in detail:

Location of your data, if most of your data currently used by BizTalk is located inside of your data center it would be hard to justify moving all of that data to and from Azure to use Azure iPaaS.

If your data is from your consumers/partners and it is coming in and going out via the internet, iPaaS can be a great solution in terms of cost, scalability and availability.

Sensitivity of your data, again this will relate back to the first item, if the data is coming in and going out over the internet, your company has already accepted the risk, so iPaaS will be a viable solution, otherwise, a risk assessment will need to be undertaken.

Security Policies for accessing cloud based resources can sometimes be one of the hardest nuts to crack, it requires educating your management to the benefits and security safe guards of Azure. It also means designing your iPaaS solution to use some of the additional security features available in Azure.  The final point to look at is location of consumers/partners, again if everyone using the integration service of BizTalk are located inside of your data center, having them traverse to Azure and back for data does not make sense, but if many of your consumers/partners are outside of your offices or mobile users, Azure make perfect sense for availability and scale.

Once a company has decided that moving to Azure iPaaS makes sense, the next item to consider is the structure of their existing BizTalk integration solution, does the existing BizTalk solutions follow best practice:

  • having incoming data mapped to an internal/canonical format
  • processed in BizTalk using the internal/canonical format
  • then mapped back to the outbound format before leaving BizTalk

if so, this will make the migration to Azure iPaaS a less disruptive and staged process.

If you would like to speak more about this topic or other ways Azure can help to transform your business processes, I would be glad to come out and present or chat directly with you, please use the Contact Us page to reach out to me.

This series will continue with the following blog posts:

  • Migrating inbound messaging to Azure using Logic Apps, Event Grid, Service Bus and the BizTalk Claim Check Pipeline component that I have built.
  • Migrating outbound messaging to Azure, using the same tooling as inbound
  • Choices for message archiving in Azure
  • End to End Monitoring of Azure iPaaS
  • EDI (EDIFACT, X12 & AS2) Capabilities of Azure iPaaS
  • Azure iPaaS CI/CD

Cross Posted to https://www.biztalkbill.com

Publishing BizTalk SOAP Services as REST with Azure API Management

Currently BizTalk Server 2016 has support for REST, but the support is fairly limited and is missing some feature that most developer expect from REST services.

To overcome these missing feature for companies that are exposing these services to their consumers/partner over the internet, I will show you have to use Azure API Management to publish SOAP services from BizTalk as REST.

For this blog post I took the Hello World example from the BizTalk SDK samples and converted it to a Request/Response orchestration and used the WCF publishing wizard to publish

image

Publish schema as WCF service (this allows better control over the URL)

image

Rename the Service and Operation, Select Schemas

image

Select the location to publish to and I am allowing Anonymous for my example

Note: I ended up using BizTalkWcfService2 as the URL, because an issue I am working with the API Management group

image

Publish Service

image

Now you need to setup the App Pool and make sure you can get the WDSL, for this example, We also need to update the WSDL to have the internet name for the server, by default the WSDL is going to be generated with the local server name

image

I downloaded the WSDL file and changed the server name

Open your Azure API Management Instance and go to the Add a new API blade

image

Click Upload to upload the WSDL file, if it was not necessary to change the WSDL file you could use the URL instead

image

Update the highlighted fields with your values, Click “Create”

image

Wait for the create to complete, Click “Done”

image

Now before we can use our newly imported SOAP service exposed as REST, we need to add it to a Product to allow users to call it.

I am using a Product Named BizTalk, you can create and use any Product Name you like

image

Now I go back to the API Definition, Click on our “submit” operation

image

Then Click on the “Test” Tab

image

This is the Test blade, notice that API Management has supplied the API Management Subscription Key (necessary to call API Management, this is based on the product we put our API in), the Sample JSON Document and a “Send” button to test with.  Click the “Send” button

image

View the results of the call

image

You will notice that the send and receive bodies are JSON, but we are calling a SOAP Service, this is what the SOAP call would look like

image

Lets now examine how API Management published our SOAP Service as REST, on the API tab, Click the “View Code”

image

The API Management use a policy to do the inbound and outbound transformation, the policy uses the liquid language to do the translation from JSON to XML and them XML to JSON and include error handling

image

The process of importing our WSDL as REST to SOAP automatically created the policy that does the transformations and also created the inbound and outbound JSON schemas

image

In a later blog post I will talk about how you can modify the schemas and the transformation.

One of the main features that BizTalk is missing with its REST adapter is the ability to provide the definition of the API for the clients to use to generate the code to call our REST services, in the Developer Portal, API Management provides either Open API (swagger) or WADL for our clients to use.

image

I hope this blog post helped you understand how you can use Azure API Management to publish your BizTalk SOAP Services as REST

Creating BizTalk Server 2016 Developer from Azure Gallery Image (updated)

Creating BizTalk Server 2016 Developer from Azure Gallery Image (updated)

The process for creating a BizTalk 2016 Developer machine with 2016 is back to the way is was in the previous 2013R2 Azure Gallery Images, almost everything is install and all you need to do is some configuration. As an update from my

previous blog post, I will walk through the steps here:

Start with the BizTalk Server 2016 Developer Azure Gallery Image

image

Create a new Virtual Machine from the Azure Gallery Image, Logon to your newly created machine

image

I then joined my machine to my Azure AD Domain Services Domain and updated some of the machine settings

image

SQL 2016 is installed and configured, the only thing I found was around some of the enabled protocols, so open SQL Configuration manager and enable Named Pipes and TCP/IP, this requires a restart of SQL to become effective

image

Visual Studio 2015 Professional is installed, you will just need to Sign In with your MSDN linked email account to activate

image

The remainder of building a BizTalk Server 2016 Developer machines is the same as my previous blog post, Starting from Configuring BizTalk Server 2016 Developer Edition – https://www.biztalkbill.com/2017/03/21/creating-biztalk-server-2016-developer-from-azure-gallery-image/

BizTalk Server 2016 goes GA

Published By: Bill Chesnut

Today marks several milestones on the history of BizTalk, it was released originally around this time in 2000 and the latest version has gone into General Availability.

There is a long list of what is new in BizTalk Server 2016, the complete list can be found here, I will highlight the one that will be the most significant to most existing BizTalk Server customers

  • SQL Server 2016 AlwaysOn Availability Groups – no more SQL log shipping for DR
  • Support for Visual Studio 2015, Windows Server 2016, SQL Server 2016 and Office 2016
  • Re-engineered SFTP adapter which will support more SFTP servers
  • Shared Access Signature support for WCF Bindings and Service Bus Messaging Adapter

Of all these items in the What New list, the most significant is the SQL Server 2016 AlwaysOn Availability Group support, in my career with BizTalk the biggest challenge at most customers was setting up a proper DR environment and since the release of AlwaysOn Availability Group in SQL 2012, most customers DBA always asked why can’t we use AlwaysOn Availability Groups, and I had to explain that they did not support MSDTC transaction that BizTalk uses MSTDC between database (in both single and multiple SQL Servers) for transactional support.  So what has change to make it supported now, heaps; there have been changes to Windows Server 2016 and SQL Server 2016 that now allow MSDTC transactions to be supported by AlwaysOn Availability Groups.  There are a few very specific details that need to be follow when setting this up for BizTalk, the most notable is that BizTalk requires at least 4 SQL Instances for it’s databases, this is because the current implementation of MSDTC supported by AlwaysOn Availability Groups does not support transaction inside of the same SQL Instance, which BizTalk uses extensively.

If you are interested in talking about how the new features and changes in BizTalk Server 2016 can help your existing BizTalk Infrastructure, please contact us.