Before the training, I had been reading endless blog posts and articles on deploying Sitecore solutions on Azure and learning from the masters Rob Habraken and Bas Lijten. I had also familiarized myself with the Sitecore Helix principles, the Habitat example solution, Unicorn, Sitecore DevOps and the Sitecore Azure Toolkit.
Reflecting my findings and technology choices for our Sitecore Helix solution, I had a pretty clear idea what we needed to meet the requirements of our coming project:
- All credentials and secrets would need to be stored in Azure KeyVault to ensure best possible security for our Sitecore on Azure solution.
- Item serialization would be done using Unicorn. Therefore we would need to figure out a way to run Unicorn sync in Azure PaaS context.
- We would have to customize the Cargo Payloads a bit.
- Continuous delivery should be done using Web Deploy Packages.
- Website root should be cleaned in every deployment. Otherwise, we can’t rely on Unicorn sync to function as it should.
- Zero downtime deployments are a must. I needed a way to deploy to deployment slots.
- Web Forms for Marketers module would need to be applied on top of the Out-of-the-box Sitecore ARM templates.
- CI/CD would need to run solely on top of Visual Studio Team Services and Hosted Agents. I didn’t want to waste time on setting up and maintaining a custom build server.
With all these things on my wish list for the training, I wasn’t sure what to expect. I was very pleased to find out that the MS black belts Christof and Katrien running the show had answers to many of them.
The training covered all the missing pieces I had been going over in my mind. In particular, I want to point out the following:
- The Out-of-the-box Sitecore ARM templates are great, but they don’t cover the security aspect. Christof showed how to store the passwords and credentials into Azure KeyVault during the initial provisioning and also how to retrieve them for continuous delivery.
- We were presented with different ways to run Web Deploy from a command line using Web Deploy Packages into Deployment Slots, and also how to strip the database files from deploy packages. Amazing – couldn’t ask for more!
- The Sitecore on Azure architecture used in the demo scripts was for xM environment – the so called Sitecore CMSonly license. This is what we were going to have in our project so I could start playing with the scripts directly without having to change the ARM templates. Perfect – just what I needed.
- They presented us a way to change the physical root path in Unicorn config files point to App_Data using publish parameter. This is a must if you want to sync items with Unicorn in Azure WebApps.
- Grande finale was in the end: Blue Green deployments! Christof explained how to run full + incremental deployments using Azure Key Vault, Site Slots and Unicorn. All the pieces in the puzzle were covered.
Blue Green deployments.
As you can imagine, I was quite hyped after the training and couldn’t wait to get my hands on the demo scripts covered there.
But then the hard work began. After going through the demo scripts in more details I discovered the following:
- The Sitecore solution they were using in the examples was very simple and didn’t follow Helix principles.
- They were using MSBuild directly from command line. Our solution had been built from scratch but it was utilizing Gulp for this.
- For provisioning the initial infrastructure they were using custom ARM templates. Since we also needed the xM setup this was not initially a problem but for the long run it would be inconvenient to maintain when there are updates to the OOB Sitecore ARM templates.
- Storing the passwords and credentials to Azure Key Vault was tied to an Azure account. For the long run, this would need to support to be run as Service Principal.
The first thing I had to address in our custom Sitecore Helix solution was how to replace the MSBuild with Gulp. This was fairly straightforward, since one could just sync the prebuilt solution files after the Gulp task using web deploy.
The next thing I stumbled on was the fact that our Unicorn files were scattered all around the Visual Studio solution. For running the Unicorn sync successfully in the Azure WebApps we needed to collect them under one root folder in App_Data. This could be accomplished from Gulp with the following – basically flattening the parent directory but not the subdirectories:
var websiteRoot = "./Website";
var unicornRoot = websiteRoot + "/App_Data/unicorn"
gulp.task("CI-Copy-Items", function() {
return gulp.src("./src/**/serialization/**/*.yml")
.pipe(rename(function (path) {
path.dirname = path.dirname.replace(/^(.+[\\\/])?(serialization)/, '');
}))
.pipe(gulp.dest(unicornRoot));
});
After these changes to the Sitecore solution and DevOps scripts I was able to run Unicorn successfully in the Azure PaaS environment and I could start setting up test environment in Azure.
Next we needed to apply a couple of XSD transformations to the WebApps’ web.config to support custom fonts etc. Since I was doing everything manually at this point it turned out to be easiest to apply these using the Sitecore Azure Toolkit. Also a couple of Sitecore specific config values related to website root and hostname were easy to apply this way and we finally had a fully working test environment in Azure PaaS.
Then it was time to look into getting the scripts work in CI context. Running the code deployments manually to Azure test environment was becoming a pain and I needed to remove my own, human errors from the equation. I had already changed the scripts to be run as Service Principal but the permissions to the Azure KeyVault secrets had been added manually. This was not good enough. Also, it turned out that the Sitecore Azure Toolkit (SAT) could not be run in the Visual Studio Team Services (VSTS) context so I was in trouble with the current scripts – they were depending on SAT.
Enter the new Sitecore on DevOps approach
Maybe the most valuable thing I got from the Sitecore on Azure training, in the end, was personally meeting with Christof and Katrien. I had been sharing my experiences along the way with them and they had provided me with some additional tips and advice. When they asked me whether I’d like to be their guinea pig with the new Sitecore on Azure DevOps scripts they were working on I couldn’t say no to them! And it turned out, later on, they were a life saver as well.
This new Sitecore on Azure DevOps approach they had been working on addressed several issues I had encountered with the previous ones:
- Service Principal permissions to Azure KeyVault secrets: The new scripts now supported setting permissions correctly for the account used in provisioning. This account could now be either an Azure account or Service Principal account.
- ARM template modularity: The provisioning of deployment slots and key-vault had now been separated into additional add-ons. This way one can use any Sitecore setup from the OOB ARM templates and just apply these on top of them.
- VSTS compatibility: The new scripts have been built and tested in VSTS context. They do not rely on SAT anymore.
- Publish parameters per role: these are now generated dynamically per role and deleted automatically afterwards for enhanced security.
- Special mention goes to Katrien for providing very clear and complete steps for configuring the Builds and Releases in VSTS.
You can find the new Sitecore on Azure DevOps scripts in GitHub.
I was privileged to get a chance to try them out and fine tune them to our needs.
Wrapping things up
It hasn’t been an easy ride, I can tell you that. Although Sitecore is constantly improving its Azure PaaS compatibility there have been quite a few surprises along the way. Also running code deployments to deployment slots and swapping the slots with the live ones has included many lessons learned but we're getting there. Now it’s mostly just optimizing the CI/CD pipeline for reliability and performance. And since we can’t utilize SAT in the CI, for now, I had to figure out other solutions for the needed XSD transformations.
Release definition in VSTS for code deployments
Some benchmarks from our current CI/CD pipeline:
- Initial provisioning of Sitecore xM setup with WFFM module: about 50 minutes
- First code deployments to Azure Web Apps’ deployment slots (see picture 3): 9 minutes
- Initial Unicorn sync in CM role: about 13 minutes
I didn’t even address here all the effort I put into getting the WFFM module work with our xM setup but let’s just say it works now. It would require a blog post of its own. And if you haven’t upgraded your solution to use Unicorn 4.0.0 - do it now. It was a game changer for us.
There's also a follow-up article about going to production with Sitecore on Azure PaaS.
Written by
Masi Malmi