Recently I got the opportunity to set up a clean Sitecore environment for a new project. I love greenfield solutions. Not because I’m not able to build maintainable code, but because I can innovate quicker: I try to make significant steps pn each new project, improving the baseline of our development process.

A re-invented mascot

I have been a Unicorn fan for some years now (over the commercial TDS alternative), but my admiration for this utility reached new heights when 3.0 was released. It enabled us to remove two tools from our development pipeline, Courier and Ship, and made the process of deploying Sitecore content a lot quicker and less complex. In the past we synced all content except media items between developers, and upon release, we compared the new database state with the current one in production using Sitecore Courier. This comparison required a duplicate serialization folder structure, but enabled us to generate a diff package. This Sitecore update package it generated was then deployed using Sitecore Ship via a TeamCity build step. But because this Sitecore update package might contain actual content (opposed to layouts, templates and other dev stuff), we went over it and altered it whenever and wherever necessary, before starting the deployment. You can see this process consists of quite some steps, and it still needs manual intervention.

In the new world we use Unicorn. Period. It overrides the default and clunky Sitecore serialization file format with a format that’s aimed at syncing, comparing and merging, which makes it easier and quicker to sync and easier and quicker to merge branches. And we’ve introduced different Unicorn-config files for our environments (development, test, acceptance and production), so we can sync different sets of content environment-aware. Now we sync all templates, layouts, development stuff and content between developers, but we only sync the templates, layouts and development stuff to the test environment and beyond. Our process now is simple and more reliable.

What I want to show you now, following this, is how we have configured TeamCity. This gives a clear view of our development process and eliminates all manual steps around Sitecore content. Essentially making it feel like a ‘code only’ deployment. It is a minimalistic setup, only using PowerShell scripts and thirteen steps (automatic testing is left out for now). Meet the one-click Sitecore deployment how I like it best.

Backup first

Before we start the actual deployment, we need to backup our target environment, both files and databases, so we are able to restore everything in case of emergency. I did a vanilla TeamCity installation using a SQL Server Express 2014 instance as database type.

Database backup

Because TeamCity runs on SQL, my build server already knows about the Backup-SqlDatabase cmdlet in PowerShell, so creating database backups, even on remote machines, is a breeze:

I wrote this small PowerShell script that requires the database instance name and the database name as script arguments. It then generates a timestamp string, so I can safely backup to a file that doesn’t exist yet. Notice I hard coded the backup target file location. I could have made an argument for that too, but since it’s the same for all my build steps and all my build configuration, I kind of consider that overengineering. I like to keep configuration arguments to a minimum, and since I only have four PowerShell scripts in total, this is a very maintainable solution.

The Backup-SqlDatabase cmdlet doesn’t need much explanation, but make sure your build server can see your database (port 1433 should be open, or if you’ve hardened that, let’s just say your default SQL port). And also, the user that runs your Build Agent service on your build server, should be granted db_backupoperator rights.

Now you can create a build step for each database you want to backup. Default, that would be Sitecore_master, Sitecore_core and Sitecore_web. Make this build step a PowerShell runner type step and point it to your script file:

%teamcity.agent.tools.dir%\Scripts\backup-database.ps1

and enter the Script arguments in the corresponding textbox:

This should give you three simple build steps that deliver you a bit of reassurance already: it’s now safe to mess with your database using Unicorn :-).

File backup

Next to the database, I want to create a snapshot of the exact state of the webroot just before the deployment starts. Again, PowerShell to the rescue! All in all, it’s just a file copy. But if you would copy all files of a Sitecore webroot, you would notice it takes quite some time. Based on my experience, you do need (or want) to backup the \App_Data, \sitecore, \temp and \upload folders. So we need to add some exclusions, but the -Exclude parameter of Copy-Item only works for file masks, not recursively on folders. The solution is to iterate through all files using Get-ChildItems using a Where clause and piping the result into a Copy-Item command. We then re-join (or construct) the destination path, et viola!

You can see I used the timestamp trick to create a new unique folder (that I can sort on date) too. And, lastly, after securing the Sitecore webroot, I will also backup the serialization folder used by Unicorn, since I will mess with those files too a little later on in the process.

Again, creating a build step in TeamCity is easy, using PowerShell runner type step with the following script file:

%teamcity.agent.tools.dir%\Scripts\backup-files.ps1

and the UNC path of the Sitecore server we’re targeting as the script argument:

I’ve created two steps in this setup: one for the Sitecore delivery server and one for the Sitecore management server. Of course, you could have only one server, or even more than two, when you’re using a reporting and / or a processing server, but for now we’ll assume you are using the default management and delivery setup, which you can scale to your likings. That’s why I create separate steps for those tasks.

Restore dependencies

Like everybody else, I assume, we’re fond of NuGet packages too. And for this project we’ve agreed that we only install third party extensions via NuGet. Within Visual Studio, your NuGet package manager will restore or update any missing our outdated dependencies at build time. But your build server does not. So we need to add one small build step that restores all NuGet packages for us, before we can (re)build our solution. Make sure you’ve fetched the latest NuGet executable on your build server, which can be done on the administrator dashboard of TeamCity:

… and create build step of the runner type NuGet Installer. You now only need to point to your solution file (*.sln) within the repository, relative to the checkout folder, and pick a restore mode. Which would be “Restore”, most likely.

Compile your front-end code

This might be a stranger in our midst, but since we use SASS for our stylesheets, among other scaffolding, frameworks, libraries and utilities regarding our front-end setup, we run both Bower and Grunt to automate repetitive tasks and compile our stylesheets. These tools run on Node.js. And I don’t want to prepare these things manually before starting my deployment. And I don’t want generated code in my repository. So, consequently, I will need to install Node.js, Bower and Grunt on my build server. If you do that globally, you can run these commands from within PowerShell too, so you don’t need to use the Node.js command prompt. Installing and using these tools are beyond the scope of this blog, but it illustrates very well how you can take out any manual steps and automate these tasks surprisingly easy using TeamCity and PowerShell.

Create another PowerShell build step. This time, we’re not interested in pointing to an external .ps1 file, since the script is so easy, but we do need to switch our working directory:

/Solution/Project.Website/front-end

This time, we’ll populate the Script source textbox of TeamCity, adding these two lines (everything else is configured using Grunt, and these are the exact same commands our developers run when checking out another branch):

Web Deploy

So we’ve got our backups, dependencies are restored, our front-end code is generated… it’s time to deploy the new release. First, make sure you’ve met the following prerequisites:

• Your build server should be able to reach the Sitecore server via port 80, 443 and 8172.
• You need to install MSWebDeploy on your build server and on all of your targeted Sitecore servers. Checkout this article on iis.net.
• Make sure the Web Management Service and Remote Agent Service are running (and set on Automatic) on the target machine.
• If you’ve installed MSWebDeploy manually via the .msi package, you will need to create a Web Deployment Handler on the target machine as well. Read this guide, it’ll help understanding why and how it is done.
• You also need to install Microsoft Build Tools 2015 on your TeamCity server to be able to build 2015 projects, since this is no longer part of the framework, but part of Visual Studio.

If that’s all in place, you can check two things to be sure everything runs smoothly:

• Run this command on your build server:  telnet sc-cm-1 8172 to be sure you can reach the target Sitecore server. Of course “sc-cm-1” is the fictional Sitecore Content Management server in this case, but you have to replace that by the server name, IP address or domain name of your target server. Blank screen is good, time out not so good.
• Run this command on your Sitecore server: netstat -aon | findstr :8172  to check if there’s actually something listening on this port locally. You should see some TCP LISTENERS when doing this, not an empty response.

Now you’re all set to create a build step in TeamCity with runner type Visual Studio (sln) and fill in the Solution file path (relative to the checkout folder), your Visual Studio version, which was 2015 in my case, the target (Build will be the most common) and the Configuration (Publish Profile in Visual Studio). The latter is an important setup, because it translates all configs for you, making them applicable to the correct target machine you’re deploying to. Since we leave all config files untouched in our repository / Visual Studio project, we use SlowCheetah a lot. Every change, even in our development environment, is made with an XSL translation, and all translations are target-aware from the start.

I won’t go into this any further, could deserve a whole blog post on itself, but one more thing needs attention. You’d better use https for your publishing URL and add the allow untrusted certificate attribute to your Command line parameters textbox in TeamCity:

This prevents you from looking the solution to a seemingly unrelated error, saying “Could not complete the request to remote agent URL. The operation has timed out.”

Fire it up!

Okay, Web Deploy can open up my website after building and deploying, but I want to make sure it has started correctly, being fully operational, before continuing the deployment. If there would be a strange 500 error in this part of the process, we should stop immediately and roll back. And we want to make sure the website is started again before asking Unicorn to sync the new Sitecore item state.

We use do to this, by putting the Wget executable with its dll’s in the script folder of the Build Agent, so we can create Command Line runner type build step, the Command executable being:

%teamcity.agent.tools.dir%\Wget\wget.exe

with the following parameters:

This gives a nice robot like log (not recursive though) in your build log.

Rinse and repeat

We now deployed our new release to the content management server. The database is still untouched. If anything went wrong up until this point, no one would have noticed. We can bale out safely. That’s why we always deploy to the management server first, check, and then repeat the last two steps (Web Deploy and Wget) for the delivery server as well. Just copy the steps and change the Configuration (publishing profile) and URL of the target machine.

Publish Unicorn files

So now the new code base has been deployed, we can update the database and inject the new templates, layouts, et cetera. But first, we have to publish the new serialization state to the Sitecore management server of our environment. This folder is located in the data folder, which isn’t included in the project, so Web Deploy didn’t publish these files. But they’re in our repository.

This boils down to a very simple PowerShell script. Do not forget to clean out the entire Unicorn folder first, otherwise deleting items wouldn’t work from your development machine:

Once more create a PowerShell build step, pointing to your script file:

%teamcity.agent.tools.dir%\Scripts\publish-unicorn.ps1

and enter the Script arguments to complete the file paths in the script:

Sync Unicorn

And last, but certainly not least, we may ask Unicorn to synchronize the database with the new serialization state shipped with the latest release of our website.

If you are using the latest beta release of Unicorn 3.1.0, you can use CHAP authentication, for which Kam provided an example script:

https://github.com/kamsar/Unicorn/(…)/PowerShell%20Remote%20Scripting

But for all other people relying on the latest stable release, that isn’t possible yet, so you have to add the authentication token as an AppSetting in your web project. Browsing through the history of the README.md file on GitHub leads to the previous (and actually currently applicable) instructions for this task:

(…)/Unicorn/blob/590a682128c17f3d4c1edf0eee46840cbcefc670/README.md

Having done this, we can call start the synchronization process by calling this PowerShell script from the last build step in TeamCity:

By making the domain of the target machine into an argument, you can re-use this script for multiple environments like test, acceptance and production. It’s all too easy isn’t it? This eliminates almost all tools and dependencies from your project, enabling you to sync and deploy fully automatic. A real one-click Sitecore deployment is what we’ve created!

and on that bombshell…

Well, not really, because there’s one more thing you should know when using different Unicorn.Configs.Default.config files for your environments based on the same serialization file set. This is the include we’re using to sync all content among developers:

and this is what we’ve configured in our Sitecore management translation:

So we serialize all content items to disk, but we only sync back the Settings folder into the database on our test, acceptance and production environment. However, you should note that Unicorn saves its include starting at the last leave within a folder with the name mentioned in the include. This means that the first setting creates a content folder within the content folder. But the latter one would start with the Settings folder right away (you never can look up in the tree past Settings anyway). To compensate for the loss of the extra content folder here, you need to add it into the name.