Migrate to Ghost 1.x on Azure

You might remember that I am running this blog on Ghost self hosted on Azure. I've written about how I've migrated from Wordpress here.

It turned out that upgrading to newer versions of Ghost wasn't as easy as I thought. I've installed Ghost in an Azure App Service from this Github repository by Felix Rieseberg, and I was relying on his Ghost-Updater-Azure tool for the updates. The update tool worked only once for me. That took me to Ghost v0.78. For the next versions it didn't work anymore. It was just doing nothing without any error message. Each time I tried to manually upgrade to a newer version it took more than a couple of hours, so I've quit each time.

Last year, Ghost announced a major release, version 1.0, and I was still on v0.78, so I had to migrate. I started with three migration goals in mind:

1. easy to do
2. have an easy path for future upgrades
3. host on Azure using PaaS

I started from the official Migrating to 1.0.0 guide, which made it clear that I will need a new fresh installation to which I should copy and import my theme and the content.

Ghost CLI

Given my previous experiences with upgrading Ghost, I was happy to see that they now provide this Ghost-CLI tool that is supposed to install, configure and upgrade Ghost through simple command line commands. Unfortunately, it doesn't work well on an Azure Web App based on Windows, so I didn't succeed in using it.

Ghost v1.x on Azure App Service Linux

Trying to get to a more reliable way of upgrading in the future, I insisted to use the Ghost-CLI and I tried to install it on an Azure App Service running Linux. I have found this tutorial which should help me do it: https://ourwayoflyf.com/ghost-v1-0-on-app-service-linux/.

This may be a good path for someone who is familiar with NodeJS, MySQL, Linux stack. That's not me :( I got stuck at the Knex Migration error that is also mentioned in the above blog.

After the deployment which I did with the provided template, I got Ghost v1.22, so a good opportunity to use the ghost update command (part of the Ghost-CLI tool) to try an update. It gave the error:

root@a8fbdc1b2630:/var/lib/ghost# ghost update
Checking for latest Ghost version
Downloading and updating Ghost to v1.22.0 > Installing dependencies > info
...
Running database migrations
An error occurred.
Message: 'Command failed: knex-migrator-migrate --init --mgpath /var/lib/ghost/current
[2018-04-02 07:36:00] ERROR

NAME: MigrationScript
MESSAGE: task.execute is not a function

level:normal

Error occurred while executing the following migration: 1-add-webhooks-table.js
MigrationScript: task.execute is not a function
    at MigrationScript.KnexMigrateError (/usr/local/lib/node_modules/ghost-cli/node_modules/knex-migrator/lib/errors.js:7:26)
    at new MigrationScript (/usr/local/lib/node_modules/ghost-cli/node_modules/knex-migrator/lib/errors.js:26:26)
    at /usr/local/lib/node_modules/ghost-cli/node_modules/knex-migrator/lib/index.js:353:19
...

TypeError: task.execute is not a function
    at /usr/local/lib/node_modules/ghost-cli/node_modules/knex-migrator/lib/index.js:308:25
    at tryCatcher (/usr/local/lib/node_modules/ghost-cli/node_modules/bluebird/js/release/util.js:16:23)
...

A database migration error. Might be a bug in the Ghost-CLI or in this Ghost version or simply something wrong with the Docker image I was using. Anyhow, it didn't seem like a good or a simple path for me to continue on, so I've dropped it.

Deploy from GitHub to Azure App Service

Not having success with the Ghost-CLI, I've decided to make a deployment from GitHub to Azure App Service, using the Ghost-Azure repository made by Radoslav Gatev, which configures Ghost for a deployment in an Azure App Service. This worked, and the entire migration steps that I'll detail next are part of this approach.

First, I have made a fork, so I can do the customizations needed for my blog.

The Upgrading Plan

For upgrading to next version of Ghost, my plan is to keep this fork up to date and Sync it with the Azure App.

This means to get the latest version of Ghost from the upstream Ghost-Azure repo. (Here is how to sync a fork). In the case that this repo is not updated I could get the latest Ghost version from its original repo. Then, I could make a pull request back to Ghost-Azure repo.

Once the latest Ghost version is merged in my fork, the next step would be to test that it works for my blog. Having the repo cloned locally, I can easily run the blog on my machine and do the checks. If I want I could also copy the latest content (DB and images) from the server, through FTP, to see that the DB gets migrated to latest version and that everything works as it should.

Then, if all good, I should push the changes to the onCode branch, which is the branch connected with my Azure Web App. The only thing left would be to go in the Azure portal and click Sync in the Deployment Options

This is the plan :) I'll let you know how it works when I'll do the first upgrade.

The Deployment

I did the initial deployment using the Azure deployment template from the repository, following Radoslav Gatev blog post.

It did create all the resources, but it didn't set the application settings, which I had to configure afterwards in the portal.

This wasn't a problem except for the WEBISTE_NODE_DEFAULT_VERSION. After the deployment, it was the only setting, and was 6.9.1. This setting says which node version the web app should use.

When I went into the App Service Editor, (which is very cool by the way) and executed in the console node db.js everything went well and it created the database. However, when I executed the npm rebuild it ended with some errors:

D:\home\site\wwwroot\node_modules\dtrace-provider>if not defined npm_config_node_gyp (node "D:\Program Files (x86)\npm\3.10.8\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "" rebuild ) 
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
D:\home\site\wwwroot\node_modules\dtrace-provider\build\DTraceProviderStub.vcxproj(20,3): error MSB4019: The imported project "D:\Microsoft.Cpp.Default.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
gyp ERR! build error 
gyp ERR! stack Error: `D:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (D:\Program Files (x86)\npm\3.10.8\node_modules\npm\node_modules\node-gyp\lib\build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Windows_NT 10.0.14393
gyp ERR! command "D:\\Program Files (x86)\\nodejs\\6.9.1\\node.exe" "D:\\Program Files (x86)\\npm\\3.10.8\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\home\site\wwwroot\node_modules\dtrace-provider
gyp ERR! node -v v6.9.1
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok 

Also, the web site wasn't starting... Ghost console output was Error: Cannot find module at npm install sqlite3

After some google-ing, I've realized that the problem was the node version, so I've set the WEBISTE_NODE_DEFAULT_VERSION to 8.9.0, reran the npm rebuild, and then everything was working. I had a fresh installation of Ghost in Azure App Service.

Migrate the Content from Ghost v0.78

Following the migration guide this should have been an easy two steps thing:

1. Export the content from the old site
2. Import the content in the new one

My problem was that the export from the old site didn't work. It didn't give any error in the Settings page, but when I ran it locally with verbose logging, I've seen the error in the console:

ERROR: Cannot read property 'client' of undefined 

Somewhere was a bug in Ghost. Either when it migrated the DB from a previous version to 0.78, either in the export functionality.

The trick I did to get my content was the following:

• install a new Ghost site with version 0.11, locally
• copy the /content/data/ghost.db from the old site to this v0.11 installation
• run the v0.11 site
  • when it started it migrated the database to its version
• export the content from the Settings\Labs screen from the v0.11 site

Add the Theme as a Git Submodule

The next step was to add the onCodeDesign theme to my new Ghost 1.19 installation. There are usually two ways to do this:

• upload the theme using the Settings\Design screen as shown in the migration guide
• copy the theme in git, in the content\themes folder and redeploy from GitHub

I didn't choose any of these. The reason is that I already keep my theme in a git repository on visualstudio.com in VSTS, and I don't want to loose all the history on how it was developed and modified. None of the above were a good option to have an easy deployment of future changes on the theme, and also to keep the history in git.

My solution was to add the theme as a git submodule to the GitHub repository of my blog. The submodule went into the content\themes\oncode folder, and it pointed to the same remote in VSTS. This, not only that preserved the history, but it also allowed me to keep the same repo for the theme which saved me some work.

The difficulty here was to setup the authentication to the theme repository on VSTS, which was not a public repo. When, Azure fetches the changes from the GitHub repo, it also needs to update the submodule, so it needs to have access to it. One solution would have been to use a SSH git URL to the submodule to the VSTS. This wasn't an option for my case because I use git-lfs (the theme repo has many images) and VSTS does not support git-lfs over SSH. Therefore, I had to configure a Personal Access Token for a new user in VSTS, and keep a HTTPS git URL.

The biggest advantage of this setup is that now, in my local clone of the repo I can do and test any changes to the theme. When I'm done, I just push the changes to the onCode branch, I go in the Azure portal and make a Sync and the changes are up.

Upgrading the Theme

The next step was to upgrade the theme to work with Ghost 1.x. This was a straight forward process. When I've activated the theme, it gave me some errors that I had to fix.

I've just followed the guide here and having the environment locally everything went fast and smooth.

Copy the Images

The next step was to copy the images from the old blog to the new one. This again, went smoothly. I've downloaded the images through FTP from the old site, and them I've uploaded them through FTP on the new one. The folders structure does not differ from v0.x to v1.x, so no issues here.

Also, most of my images are on Cloudinary, so I didn't have too many to copy from one place to the other.

Final Settings

The last steps were to review the settings on the new installation. Things like title & description, publication logo, publication icon (much nicer than on v0.x), publication cover etc.

To make the email settings you could put them in the config.production.json as shown here in the docs. However I prefer to use the App Service - Application settings, so I don't put my Sendgrid credentials on GitHub in a public repo.

Redirects

I do my redirect in the web.config as I've detailed in my post about migrating from Wordpress here.

In the previous installation I didn't had a nice way to deal with these. I set them up using the App Service Editor and that was it. Luckily they didn't need changing. Now, with my new setup of deploying from GitHub, I've just copy them in the new web.config which is part of my repo and just redeploy.

Ghost v1.x has support for uploading the redirects in a json file (Labs screen), but I just prefer the above to have them in the git history.

Switch the DNS and Setup the SSL

I run my site through Cloudflare for all the benefits in security (SSL, DDoS, etc) and performance.

This first thing was to go in Cloudflare and change the DNS setting to point to the new Azure App Service. Then, I've removed the domains from the old App Service and added them to the new one.

The oncodedesign.com got validated and was responding very fast. Almost instantaneously. I've added the SSL Binding using the Cloudflare certificate I already had setup in Azure and it was all good.

The www.oncodedesign.com didn't get validated even if it was changed in the DNS. I waited a day, thinking that I should be patient with the DNS propagation, but it still didn't work. After some reading I've switched off the HTTP proxy leaving DNS only for the www record

This did the trick. The domain was validated by Azure, and I could setup the SSL Bindind with the same certificate. Afterwards, I've switched it back to DNS and HTTP proxy and everything went well. Probably something was cached, and the new DNS record didn't push through.

Summary

To summaries:

• I've migrated from Ghost v0.78 to v1.19 wanting to keep hosting in Azure
• I wanted to get a easy way to future updates, so I looked into the Ghost CLI, but it doesn't play nice on Azure
• I ended up using an automated deployment from GitHub on Azure App Service
• I keep my theme development in its own git repo on VSTS, which is a git submodule in the main one
• I ended up with a nice environment that allows me to dev & test in my local clone and then easily push to Azure
• From now on the upgrades to the new versions should be easier given that I can keep my repo in sync with the one of Ghost or Ghost-Azure.

If you find yourself doing such a migration and you get stuck don't hesitate to ask me, I might have gone through the same.