Deploying to Acquia Cloud with Bitbucket Pipelines

Bitbucket Pipelines is a CI/CD service, built into Bitbucket and offers an easy solution for building and deploying to Acquia Cloud for project's whose repositories live in Bitbucket and who opt out of using Acquia's own Pipelines service. Configuration of Bitbucket Pipelines begins with the creation of a bitbucket-pipelines.yml file and adding that file to the root of your repository. This configuration file details how Bitbucket Pipelines will construct the CI/CD environment and what tasks it will perform given a state change in your repository. Let's walk through an example of this configuration file built for one of our clients.

bitbucket-pipelines.yml

image: geerlingguy/drupal-vm:4.8.1
clone:
  depth: full
pipelines:
  branches:
    develop:
      - step:
         script:
           - scripts/ci/build.sh
           - scripts/ci/test.sh
           - scripts/ci/deploy.sh
         services:
           - docker
           - mysql
         caches:
           - docker
           - node
           - composer
    test/*:
      - step:
         script:
           - scripts/ci/build.sh
           - scripts/ci/test.sh
         services:
           - docker
           - mysql
         caches:
           - docker
           - node
           - composer
  tags:
    release-*:
      - step:
          name: "Release deployment"
          script:
            - scripts/ci/build.sh
            - scripts/ci/test.sh
            - scripts/ci/deploy.sh
          services:
            - docker
            - mysql
          caches:
            - docker
            - node
            - composer
definitions:
  services:
    mysql:
      image: mysql:5.7
      environment:
        MYSQL_DATABASE: 'drupal'
        MYSQL_USER: 'drupal'
        MYSQL_ROOT_PASSWORD: 'root'
        MYSQL_PASSWORD: 'drupal'

The top section of bitbucket-pipelines.yml outlines the basic configuration for the CI/CD environment. Bitbucket Pipelines uses Docker at its foundation, so each pipeline will be built up from a Docker image and then your defined scripts will be executed in order, in that container.

image: geerlingguy/drupal-vm:4.8.1
clone:
  depth: full

This documents the image we'll use to build the container. Here we're using the Docker version of  Drupal VM. We use the original Vagrant version of Drupal VM in Acquia BLT for local development. Having the clone depth set to full ensures we pull the entire history of the repository. This was found to be necessary during the initial implementation. The "pipelines" section of the configuration defines all of the pipelines configured to run for your repository. Pipelines can be set to run on updates to branches, tags or pull-requests. For our purposes we've created three pipelines definitions.

pipelines:
  branches:
    develop:
      - step:
         script:
           - scripts/ci/build.sh
           - scripts/ci/test.sh
           - scripts/ci/deploy.sh
         services:
           - docker
           - mysql
         caches:
           - docker
           - node
           - composer
    test/*:
      - step:
         script:
           - scripts/ci/build.sh
           - scripts/ci/test.sh
         services:
           - docker
           - mysql
         caches:
           - docker
           - node
           - composer

Under branches we have two pipelines defined. The first, "develop", defines the pipeline configuration for updates to the develop branch of the repository. This pipeline is executed whenever a pull-request is merged into the develop branch. At the end of execution, the deploy.sh script builds an artifact and deploys that to the Acquia Cloud repository. That artifact is automatically deployed and integrated into the Dev instance on Acquia Cloud. The second definition, "test/*", provides a pipeline definition that can be used for testing updates to the repository. This pipeline is run whenever a branch named 'test/*' is pushed to the repository. This allows you to create local feature branches prefixed with "test/" and push them forward to verify how they will build in the CI environment. The 'test/*' definition will only execute the build.sh and test.sh scripts and will not deploy code to Acquia Cloud. This just gives us a handy way of doing additional testing for larger updates to ensure that they will build cleanly. The next section of the pipelines definition is set to execute when commits in the repository are tagged.

tags:
  release-*:
    - step:
        name: "Release deployment"
        script:
          - scripts/ci/build.sh
          - scripts/ci/test.sh
          - scripts/ci/deploy.sh
        services:
          - docker
          - mysql
        caches:
          - docker
          - node
          - composer

This pipeline is configured to be executed whenever a commit is tagged with the name pattern of "release-*". Tagging a commit for release will run the CI/CD process and push the tag out to the Acquia Cloud repository. That tag can then be selected for deployment to the Stage or Production environments. The final section of the pipelines configuration defines services built and added to the docker environment during execution.

definitions:
  services:
    mysql:
      image: mysql:5.7
      environment:
        MYSQL_DATABASE: 'drupal'
        MYSQL_USER: 'drupal'
        MYSQL_ROOT_PASSWORD: 'root'
        MYSQL_PASSWORD: 'drupal'

This section allows us to add a Mysql instance to Docker, allowing our test scripts to do a complete build and installation of the Drupal environment, as defined by the repository. Additional resources on Bitbucket Pipelines and bitbucket-pipelines.yml:

• Build, test, and deploy with Pipelines
• Get started with Bitbucket Pipelines
• Configure bitbucket-pipelines.yml

Scripts

The bitbucket-pipelines.yml file defines the pipelines that can be run, and in each definition it outlines scripts to run during the pipeline's execution. In our implementation we've split these scripts up into three parts:

1. build.sh - Sets up the environment and prepares us for the rest of the pipeline execution.
2. test.sh - Runs processes to test the codebase.
3. deploy.sh - Contains the code that builds the deployment artifact and pushes it to Acquia Cloud.

Let's review each of these scripts in more detail.

build.sh

#!/bin/bash
apt-get update && apt-get install -o Dpkg::Options::="--force-confold" -y php7.1-bz2 curl && apt-get autoremove
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
apt-get install -y nodejs
apt-get install -y npm
cd hive
npm install
npm install -g gulp
cd ..
composer install
mysql -u root -proot -h 127.0.0.1 -e "CREATE DATABASE IF NOT EXISTS drupal"
export PIPELINES_ENV=PIPELINES_ENV

This script takes our base container, built from our prescribed image, and starts to expand upon it. Here we make sure the container is up-to-date, install dependencies such as nodejs and npm, run npm in our frontend library to build our node_modules dependencies, and instantiate an empty database that will be used later when we perform a test install from our codebase.

test.sh

#!/bin/bash
vendor/acquia/blt/bin/blt validate:phpcs --no-interaction --ansi --define environment=ci
vendor/acquia/blt/bin/blt setup --yes  --define environment=ci --no-interaction --ansi -vvv

The test.sh file contains two simple commands. The first runs a PHP code sniffer to validate our custom code follows prescribed standards. This command also runs as a pre-commit hook during any code commit in our local environments, but we execute it again here as an additional safeguard. If code makes it into the repository that doesn't follow the prescribed standards, a failure will be generated and the pipeline will halt execution. The second command takes our codebase and does a complete Drupal installation from it, instantiating a copy of Drupal 8 and importing the configuration contained in our repository. If invalid or conflicting configuration makes it into the repository, it will be picked up here and the pipeline will exit with a failure. This script is also where additional testing could be added, such as running Behat or other test suites to verify our evolving codebase doesn't produce regressions.

deploy.sh

#!/bin/bash
set -x
set -e

if [ -n "${BITBUCKET_REPO_SLUG}" ] ; then

    git config user.email "bitbucket@example.com"
    git config user.name "Bitbucket Pipelines"

    git remote add deploy $DEPLOY_URL;

    # If the module is -dev, a .git file comes down.
    find docroot -name .git -print0 | xargs -0 rm -rf
    find vendor -name .git -print0 | xargs -0 rm -rf
    find vendor -name .gitignore -print0 | xargs -0 rm -rf

    SHA=$(git rev-parse HEAD)
    GIT_MESSAGE="Deploying ${SHA}: $(git log -1 --pretty=%B)"

    git add --force --all

    # Exclusions:
    git status
    git commit -qm "${GIT_MESSAGE}" --no-verify

    if [ $BITBUCKET_TAG ];
      then
        git tag --force -m "Deploying tag: ${BITBUCKET_TAG}" ${BITBUCKET_TAG}
        git push deploy refs/tags/${BITBUCKET_TAG}
    fi;

    if [ $BITBUCKET_BRANCH ];
      then
        git push deploy -v --force refs/heads/$BITBUCKET_BRANCH;
    fi;

    git reset --mixed $SHA;
fi;

The deploy.sh script takes the product of our repository and creates an artifact in the form of a separate, fully-merged Git repository. That temporary repository then adds the Acquia Cloud repository as a deploy origin and pushes the artifact to the appropriate branch or tag in Acquia Cloud. The use of environment variables allows us to use this script both to deploy the Develop branch to the Acquia Cloud repository as well as deploying any tags created on the Master branch so that those tags appear in our Acquia Cloud console for use in the final deployment to our live environments. For those using BLT for local development, this script could be re-worked to use BLT's internal artifact generation and deployment commands.

Configuring the cloud environments

The final piece of the puzzle is ensuring that everything is in-place for the pipelines to process successfully and deploy code. This includes ensuring that environment variables used by the deploy.sh script exist in Bitbucket and that a user with appropriate permissions and SSH keys exists in your Acquia Cloud environment, allowing the pipelines process to deploy the code artifact to Acquia Cloud.

Bitbucket configuration

DEPLOY_URL environment variable

Configure the DEPLOY_URL environment variable. This is the URL to your Acquia Cloud repository.

1. Log in to your Bitbucket repository.
2. In the left-hand menu, locate and click on "Settings."
3. In your repository settings, locate the "Pipelines section" and click on "Repository variables."

4. Add a Repository variable:

   1. Name: DEPLOY_URL
   2. Value: The URL to your Acquia Cloud repository. You'll find the correct value in your Acquia Cloud Dashboard.

    

SSH keys

Deploying to Acquia Cloud will also require giving your Bitbucket Pipelines processes access to your Acquia Cloud repository. This is done in the form of an SSH key. To configure an SSH key for the Pipelines process:

1. In the "Pipelines" section of your repository settings we navigated to in steps 1-3 above, locate the "SSH keys" option and click through.
2. On the SSH keys page click the "Generate keys" button.
3. The generated "public key" will be used to provide access to Bitbucket in the next section.

Acquia Cloud configuration

For deployment to work, your Bitbucket Pipelines process will need to be able to push to your Acquia Cloud Git repository. This means creating a user account in Acquia Cloud and adding the key generated in Bitbucket above. You can create a new user or use an existing user. You can find more information on adding SSH keys to your Acquia Cloud accounts here: Adding a public key to an Acquia profile. To finish the configuration, log back into your Bitbucket repository and retrieve the Known hosts fingerprint.