martes, 17 de noviembre de 2015

Automated git deployments from Bitbucket

Posted by daycry at 18:11 Labels: , , , ,
Important: Bitbucket have changed how webhooks function, and the technique described in this post will no longer work without modification. One of my readers has created an updated version, and I recommend trying that instead. I no longer use this deployment method and won’t be updating my tutorial or answering comments, but I have left the comment section open so that readers can post their tips and help each other out.
Git may not have been designed as a deployment tool, but for small projects it can do the job quite nicely. What makes Git deployments attractive is how frictionless the process is: make some changes to your project, merge them into your production branch, push the commit to a remote repository and like magic the changes are live! Git knows which files need to be changed or deleted, so you don’t have to think about it. If you’re already using git to version control your project then you probably won’t even need to modify your existing workflow, once the initial setup is done.
I use Bitbucket for hosting my private repositories, and have recently implemented a deployment process that integrates with Bitbucket’s POST hooks feature. There are basically three things you need to do to make this work:
  • Set up SSH keys so your server can talk to Bitbucket
  • Clone your Bitbucket repository on your web server
  • Setup a hook on Bitbucket and an associated deployment script on your server
Here’s what your deployment workflow will look like once we’re done:
  • Develop your website locally
  • When you’re ready to deploy, commit your changes and push them to Bitbucket
  • When Bitbucket receives the commit it will notify a deployment script on your server
  • The deployment script will fetch the changes into a cloned repository on your server, and checkout files to your public web directory

Prerequisites

Before we get started, check that your server meets the following requirements:
  • Git is installed
  • You have shell access
  • The PHP exec function is enabled
Most shared web hosting accounts will fail at least one of those requirements, but if you’ve got a VPS or dedicated server then you should be good to go.
For the purposes of this tutorial I’m going to assume that you have a git repo already set up on Bitbucket, and that the repository’s directory structure mirrors your production website. For instance if you want an index.html file deployed to the root level of your website’s public directory, that same file will exist in the repo’s root directory.
I’m also going to assume that you will be deploying from a branch named production. In practice you can deploy from any branch other than master, but it’s a good idea to deploy from a branch that is not used for active development, so that you have control over when a deployment occurs. In my workflow I develop in the master branch, then merge master into production when I’m ready to deploy. Before we get started, make sure you’ve made an initial commit to your repository’s production branch, and pushed to Bitbucket.
I’ve tested this process on Centos, but you might need to change directory paths to suit your own server environment. Whenever you see a variable inside angled brackets in my in my code samples, such as <repo-name> or <username>, that’s a placeholder that you will need to replace with a value specific to your own project.
Now that the preliminaries are out of the way, let’s get started.

Set up SSH keys

For your server to connect securely to Bitbucket without a password prompt, it needs to use anSSH key.
On your server navigate to the ~/.ssh directory of the user that PHP runs under. I’m running the Apache suPHP module on my server, so PHP runs as the user that owns the website. On your server the web user might be the apache, nobody or www-data user. You will need to create the user’s .ssh directory if it doesn’t exist. At a shell prompt type:
cd ~/.ssh
ssh-keygen -t rsa
When prompted either accept the default key name (id_rsa) or give your key a unique name – I chose bitbucket_rsa. Press enter when asked for a passphrase, which will generate a passwordless key. Usually this isn’t recommended, but we need our script to be able to connect to Bitbucket without a passphrase.
A public and private key pair will be generated. Copy your public key – the one with a .pub extension – to the clipboard. On the Bitbucket website navigate to Account > SSH Keys, and choose to add a new key. Paste in your public key and save it.
Back on your server, edit your ~/.ssh/config file to add bitbucket.org as a host. This ensures that the correct key is used when connecting by SSH to bitbucket.org. You’ll need to create theconfig file if it doesn’t exist:
Host bitbucket.org
 IdentityFile ~/.ssh/bitbucket_rsa
Whenever you do a git fetch Bitbucket will verify your identity automatically, without prompting you for a password.

Cloning your repository

Now that SSH is configured you can clone your Bitbucket repository on your server. You might be tempted to clone the repository directly into the public website directory (for the sake of brevity we’ll call it www), but that approach comes with significant security risks. It requires that there is a.git folder inside of www, from which a malicious attacker could extract your entire website source code, possibly including database credentials and other sensitive information. Sure, you could use an .htaccess directive to hide the .git directory, but if that .htaccess file were accidentally deleted or edited you’d be left wide open.
A better approach is to store your git repository outside the public website directory, where it is hidden from prying eyes. If you create a bare repository then you can still checkout files to a detached working tree in www.
Before we begin, navigate to your repository on the Bitbucket website and copy its SSH URL. This will be in the format git@bitbucket.org:<username>/<repo-name>.git
Navigate the location on your server where you want to clone the repository. A good spot is probably one level above the www directory, which on my system is the website user’s home directory. At a shell prompt, clone your Bitbucket repository:
cd ~
git clone --mirror git@bitbucket.org:<username>/<repo-name>.git
Notice the --mirror flag? As its name implies this flag creates an exact mirror of the source repository, including mapping it’s remote branches. It implies --bare, which means that our repository will not have a working copy.
Your repository will be cloned into a directory called <repo-name>.git, in the user’s home directory. It doesn’t really matter what you name the directory, but suffixing the directory name with .git implies a bare repo, so it’s a useful naming convention to follow.
Now let’s do an initial checkout:
cd ~/<repo-name>.git
GIT_WORK_TREE=/home/<username>/www git checkout -f production
If this is first time you’ve communicated with bitbucket.org over SSH you may be prompted to accept Bitbucket’s server fingerprint, but if SSH is correctly configured you won’t be asked for your Bitbucket password or a key passphrase.
We have specified a GIT_WORK_TREE that corresponds to your public web directory, and checked out the production branch to that location. This step is important so that in future when our deployment script does a checkout we’re already on the correct branch.
Check that your initial checkout completed as expected, and that files from your production branch have been created in your public web directory. If everything worked as expected then you’re ready to set up automated deployments.

Create a Bitbucket POST hook

In your www directory make a new directory named deploy, containing three files: index.html,bitbucket-hook.php and deploy.log.
Add the following to bitbucket-deploy.php, changing $repo_dir, $web_root_dir and$git_bin_path to suite your server environment:
<?php
$repo_dir = '/home/<username>/<repo-name>.git';
$web_root_dir = '/home/<username>/www';

// Full path to git binary is required if git is not in your PHP user's path. Otherwise just use 'git'.
$git_bin_path = 'git';

$update = false;

// Parse data from Bitbucket hook payload
$payload = json_decode($_POST['payload']);

if (empty($payload->commits)){
  // When merging and pushing to bitbucket, the commits array will be empty.
  // In this case there is no way to know what branch was pushed to, so we will do an update.
  $update = true;
} else {
  foreach ($payload->commits as $commit) {
    $branch = $commit->branch;
    if ($branch === 'production' || isset($commit->branches) && in_array('production', $commit->branches)) {
      $update = true;
      break;
    }
  }
}

if ($update) {
  // Do a git checkout to the web root
  exec('cd ' . $repo_dir . ' && ' . $git_bin_path  . ' fetch');
  exec('cd ' . $repo_dir . ' && GIT_WORK_TREE=' . $web_root_dir . ' ' . $git_bin_path  . ' checkout -f');

  // Log the deployment
  $commit_hash = shell_exec('cd ' . $repo_dir . ' && ' . $git_bin_path  . ' rev-parse --short HEAD');
  file_put_contents('deploy.log', date('m/d/Y h:i:s a') . " Deployed branch: " .  $branch . " Commit: " . $commit_hash . "\n", FILE_APPEND);
}
?>
This script iterates over the payload object sent by Bitbucket, looking for commits made to the production branch. If any are found, a git fetch and checkout are performed and the deployment details are logged.
For security through obscurity you might choose to give your deployment script a difficult to guess name – bitbucket-hook-a13jsur5kcidwe89z.php, for example. The index.php file you created early is also a simple security measure: it stops anyone from viewing the directory index.
On the Bitbucket website navigate to your repository’s Administration > Hooks screen and add a new POST hook, pointed at http:/<domain>/deploy/bitbucket-hook.php.

Deploying

Whenever you are ready to deploy to your web server, merge your development branch into your production branch, and push the production branch to Bitbucket. Your custom POST hook will be triggered, and your deployment script will fetch the repository to the server and checkout the production branch to your web root.
Hey presto! With one commit your changes have been automatically deployed to your production web server.
My instructions might look fairly complicated, but after you’ve followed the steps once or twice it actually becomes really fast to set up.

Troubleshooting

Here are a few things to check if deployments aren’t working as expected.

File permissions

You should make sure that the web user (the user that PHP runs as) has permission to write to your local git repository. The easiest way to ensure this is for that user to own the repo:
chown -R <user>:<group> <repo-name>.git
Make sure that the same user also owns its ~/.ssh directory and contents.

Git path

You may find that you are unable to perform git commands using exec since the git binary is not in your PHP user’s PATH. This can be solved by including the full path to the git binary in your deployment script, for example:
$git_bin_path = '/usr/local/bin/git';
To find where your git binary is located, run this shell command:
which git
Hat tip to Jonathan Johnson for this one. His article might help solve other issues you’re having, too.

Payload

If you need to examine the Bitbucket payload that’s being sent to your deployment script, add the following line to the top of bitbucket-hook.php:
file_put_contents('deploy.log', serialize($_POST['payload']), FILE_APPEND);

No hay comentarios :

Publicar un comentario

Suscribirse a: Enviar comentarios ( Atom )