In my blog post and talks early december, I gave an extensive introduction on deploying and hosting your Sitecore solution on Azure Web Apps; right after the release of Sitecore 8.2 Update 1, which was the first Sitecore version to support Web Apps. Since then, a lot happened. Not only Sitecore released both Update 2 and Update 3, but the community and implementation partners are acquiring more and more experience with using ARM powered deployments. One of those is how to solve the quest for a true blue green deployment.

What’s a blue green deployment?

Blue Green is a “deployment strategy that relies on two identical production configurations that alternate between active and inactive. One environment is referred to as blue, and the duplicate environment is dubbed green”.
^{Source: http://searchitoperations.techtarget.com/definition/blue-green-deployment}

The staging slot feature of Azure Web Apps, together with the easy swapping and slot (configuration) settings, do make using this strategy child’s play. So how can you alter your ARM based Sitecore deployment to leverage the full potential of this feature? That is exactly what this article is about.

Continuous Delivery wish list

If I need to come up with a wish list of features for the ultimate Continuous Integration or Continuous Delivery setup, these would be my main requirements:

• A clean install per deployment, based on vanilla Sitecore install (no history or previous / old file build-up)
• Keep our database content and only deploy new or changed items
• Don’t disturb visitors of the live site with the deployment process
• An option to rollback the deployment upon failure
• A repeatable process, which is of course fully automated

With the example ARM templates and default SCWDPs (Sitecore Web Deploy Packages) from Sitecore, this wouldn’t be possible out-of-the-box. Because you cannot re-provision the environment (running the same ARM template gives you an error), it doesn’t provision staging slots for the different Web App instance roles, and it doesn’t tell you how to keep your Sitecore data while re-provisioning. So, we need to add functionality to the ARM templates. But first, let’s set up a simple, repeatable process, in four steps:

Required provisioning & deployment steps

1. Roll out your infrastructure first, like resource groups, databases, hosting plans and slots. This will be done only once per environment.
2. Then, provision a vanilla Sitecore install into your staging slot (this step includes installing databases and vanilla content). As with step one, this will be done only once.
3. Then, create a repeatable re-provisioning step, provisioning a vanilla Sitecore install into your staging slot, but without databases this time! (this actually wipes your staging slot each time your provision). We will use the same ARM template as with step 2, but use a different SCWDP as the source for our provisioning.
4. Deploy your own solution over the staging slot & swap when ready, for example using Visual Studio Team Services.

Of course, after this step, you can test your new deployment, warm up the new site and swap when ready. But that’s all considerably mainstream stuff. I want to focus on these first four steps, because they differ from the default scripts and are required to get to this final stage.

Customizing ARM templates

The first task into achieving this process is to split the infrastructure provisioning from the Sitecore application deployment (msdeploy) within the ARM templates, using output parameters to stitch these consecutive provisioning steps together. And to add staging slots to the infrastructure provisioning, which we will use as our only deployment target from now on (always provision a new green slot, and deploy to this, only to swap the slots when ready; meaning the blue slot will be never touched by either your ARM templates or your deployment once you’ve set it up).

If you want a more extensive demo of what I’m doing, please watch the recording of my SUG Belarus presentation on the bottom of this blog post. Or you can download the full scripts directly from my GitHub repository at https://github.com/robhabraken/Sitecore-Azure-Scripts.

Splitting infra & msdeploy

Extract the msdeploy part from the original Sitecore ARM template example into a separate ARM template:

Please note that each resource block in ARM contains a dependsOn field and make sure you edit those dependencies accordingly.

Output parameters

Add all the parameters required in this second template as output parameters to the bottom of your original template. Azure will store them for you as a part of your deployment (as long as you name your deployment specifically) and you can retrieve them via PowerShell to supply to your second template later on, which you will be able to see in the PowerShell script on my GitHub repository. For the output parameters, you can both directly copy variable values into them, or compute values based upon other references or variables as you would be doing within any other part of ARM:

Staging slots

Then, add a staging slot to both your first (infrastructure) template:

… and add the same slot to your second (msdeploy) template:

As you can see, my slot is called “single-staging”. That, together with the name of my Web App being “azure-rg-single”, would end up in the URL “azure-rg-single-single-staging.azurewebsites.net”, which may seem odd, but in a multiple role setup, having more than one Web App Hosting Plan, you would end up with multiple stage slots, and as they show up as resources in your resource overview as well, naming them “staging” only would make it hard to distinguish the different slots per role. So choose your staging slot names wisely :-).

By performing these tasks, you can run the infrastructure part of the ARM template separately from the msdeploy part, which is required because we need to be able to repeat the msdeploy part, leaving the infrastructure (and filled databases) in tact. Again, for a full reference of what I’ve done, please check out my GitHub repository, where you can download my fully working (and tested) splitted ARM templates.

Databaseless SCWDPs

After altering the ARM templates, the second task is to generate a second Sitecore Web Deploy package, not containing any database related stuff. Only the vanilla Sitecore web root. With these databaless SCWDPs, you can re-run the msdeploy ARM template, without touching the databases.

Stripping the database installation from the web deploy packages can be done using msdeploy.exe: deploy from the original SCWDP downloaded from Sitecore, to a new zip archive, using the dbFullSQL & skip dbDacFx parameters. The parameters.xml file within the original SCWDP will tell you which parameters are required for which package / Sitecore role instance.

By running this command, msdeploy will generate a new package for you with the webroot only. It also configures all parameter values according to your input parameters, but you actually do not need to put real values in there, because you want the package to be re-usable and because you will overwrite those values anyway from your ARM template.

Because adding all those parameters can be a tediuous taks, you can use this PowerShell script to generate and invoke it for you.

After the initial provisioning of the infrastructure and first msdeploy (step 1 and 2), you can use this new SCWDP to run your msdeploy ARM template over and over again. Each time, it will wipe your staging slot (the green one), installing a new vanilla Sitecore web app based on your current database state, which is a great clean canvas for your next deployment.

Take it to the next level!

We’re almost approaching the ultimate CI/CD setup. Almost, because there’s one important step missing: we did deploy our new set of files to the green slot, but still used the same database as the blue slot. Which can lead to issues when your content update isn’t backwards compatible (however, it mostly is due to only adding new stuff) and it would make it a little harder to roll back if required (might not be the case regularly, and only one Unicorn sync away, but still…). So, let’s look at the ultimate scenario of duplicating the web database.

I won’t go over the following schema into detail, but here you can see all the steps required to get all of the above done, including separate databases. The idea is that you duplicate your live web database after proclaiming a content stop. Then tie your new web database to the green slot, update your master database with the new content, and publish it to the new web database. This would allow you to keep two completely separated environments running next to each other, each with their own file set and database. When you swap, you swap the connection strings too, both bringing your new site live immediately, and having a full operational rollback environment at hand:

 

Cargo payloads

Do you have custom role specific configuration for the content management, content delivery, processing and reporting instances? Don’t transform them in your Solution, use cargo payloads! For more info on this topic, check out this article by Bas Lijten: http://blog.baslijten.com/sitecore-on-azure-create-custom-web-deploy-packages-using-the-sitecore-azure-toolkit/.

Security

Bear in mind that you always need to secure your deployments! Use SAS to secure your blob storage containing your SCWDP downloads, add IP security to all environments except your blue production slot, and consider using Azure Key Vault to store your keys and passwords in. For a little more in depth explanation of these topics, you can watch the recordings of my SUG talk below.

Pitfalls

There are some pitfalls that we have encountered while implementing this strategy in real life scenarios. They may or may not apply to you, and I do not have a solution for all of them, but in these early stages of running Sitecore on Web Apps I find it very useful in particular to share all the bumps and possible solutions.

EventQueue on shared databases

You could experience what I’ve called the ‘EventQueue issue’ when using only one set of databases: the staging slot creates an extra Content Management instance, which could introduce concurrency issues with your production slot CM, both adding events to the shared event queue. Check out my blog post on this: https://www.robhabraken.nl/index.php/2683/the-staging-slot-eventqueue-issue/.

Shared indexing

If you do blue green deployments using an extra web database, please note that your Azure Search already updates your indexes based on the new content of your master database! You could duplicate your indexes too, but this would be very costly and updating the second index would take a lot of time as well, so I’m not sure what the best approach on this would be at the moment.

Content stop

If you want to avoid conflicts, content editors should not publish during deployments (although they can continue working on the master database); if you want to deploy continuously, like multiple times a day, this can be a huge disturbance for editors. If anyone has a good suggestion on this one, I would be all ears :-).

Sitecore ARM update coming up

Sitecore will release a new set of ARM templates shortly, containing a few aspects we’ve seen today, and more:

• Nested templates
• Splitting infra & application
• Module support (for add-ons)

When released, you can use those as a boilerplate for your own templates, tailored to your specific needs. However, the staging slot provisioning and re-provisioning using databaseless SCWDPs will not be part of the next release, so if you want to learn more about those, you can also study the examples on my GitHub account.

User Group presentations

SUG Belarus recording

My Blue Green Deployment presentation live at the Sitecore User Group in Belarus:

SUG Jordan recording

My Blue Green Deployment presentation via GoToMeeting for the Sitecore User Group in Jordan:

[NOT YET PUBLISHED]

 

And the slides of my Blue Green Deployments presentation: