Catapult

CLI Tool to create, deploy, and manage releases.

Install

1. Create an AWS account
2. Set up AWS credentials using any of the recommended approches
3. Install libgit2

Linux:

Requires the development version of openssl to be installed.

• Debian based distros:

  apt install libssl-dev

• RedHat based distros:

  yum install openssl-devel

wget https://github.com/libgit2/libgit2/archive/v1.0.1.tar.gz
tar xzf v1.0.1.tar.gz
cd libgit2-1.0.1/
cmake .
make
sudo make install

# Refresh cache for shared libraries
# https://github.com/libgit2/pygit2/issues/603
sudo ldconfig

MacOS

brew install libgit2

4. Install catapult:

cd ./catapult
pip3 install -r requirements.txt
python3 setup.py develop

Configuration

Set these environment variables to configure catapult:

• CATAPULT_GIT_REPO: path to the git repository (default: ./). !!! If this is not set, catapult should be run inside the git repository !!!
• CATAPULT_AWS_PROFILE: use this profile from your credential file. Required.
• CATAPULT_AWS_MFA_DEVICE: ARN of the MFA device used to get the session credentials. You can find this on the "Security Credentials" tab of your user account in IAM.

Additional environment variables used for integration with GitHub and Shortcut (formerly Clubhouse):

• GITHUB_API_TOKEN: A GitHub personal access token with repo scope. Generate one at https://github.com/settings/tokens
• SHORTCUT_API_TOKEN: A Shortcut API token. Generate one at https://app.shortcut.com/settings/account/api-tokens

Description

The release process of an application, or project, is driven by a file stored in a S3 bucket. The file contains information describing the release and it's used by Concourse to create and deploy the build artifacts.

A release is identified by an integer version, this number increments every release. version_id is the S3 version ID which is unique a unique string assigned to a specific state of the object.

commit is the git reference to the code used for creating the image or any final artifact. Every release is expected to produce a final docker image which is identified by the image ID stored in image.

The same structure is used to represent a deploy, but in this case the version and the S3 version ID are used to identify a specific deploy. This is needed to allow multiple deploy of the same version, which is useful when rolling back a release to an older version.

The file contains other metadata regarding the release:

• author, who cut the release.
• changelog, all the changes introduced from the preceding release.
• timestamp, when the release was created.

Schema:

{
  "version": number (integer),
  "author": string (email address),
  "changelog": string,
  "commit": string (git sha),
  "image": string (docker image sha256),
  "timestamp": string (ISO8601),
  "version_id": string (S3 version)
}

Example:

{
  "version": 2,
  "author": "[email protected]",
  "changelog": "commit 42182bb85bebe8d7f...",
  "commit": "42182bb85bebe8d7f1ee515c13517be0dee8ada3",
  "image": "sha256:6ac40dec2b3af61e868954d641491d95a3fb74ad239d7584025b930d1f9997bd",
  "timestamp": "2018-07-03t17:14:18+00:00",
  "version_id": "qf.c4fqpt1wyaofxm_qeghylxsp_dogi"
}

Documentation

Run catapult --help or catapult --help=<task> to know more about all the command's options.

Why S3?

Storing and driving the release using a single S3 file allows us to:

• have a fine-grained control on the release and deploy permission using AWS IAM.
• keep a history of all the releases and deploys without relying on Concourse.
• store release's information in a single file using a custom format.

Actions

Create release

catapult release <name> creates a new release of the application based on the current git HEAD.

catapult release <name> --commit=<git-sha> allows to create a release using the specified commit.

IAM Permissions

A user needs the below permissions to be able to release an application.

Resource: arn:aws:s3:::<bucket>/<name>

• s3:PutObject
• s3:GetObject
• s3:ListBucketVersions
• s3:ListBucket
• s3:GetObjectVersion

Resource: arn:aws:s3:::<bucket>

• s3:ListAllMyBuckets
• s3:HeadBucket

Deploy release

catapult deploy <name> <environment> makes the latest release available for deployment.

catapult deploy <name> <environment> --version=<version> this is useful if the release to be deployed is not the latest (i.e. rollback).

IAM Permissions

A user needs the below permissions to be able to deploy an application.

Resource: arn:aws:s3:::<bucket>/<name>

• s3:PutObject
• s3:GetObject
• s3:ListBucketVersions
• s3:ListBucket
• s3:GetObjectVersion

Resource: arn:aws:s3:::<bucket>

• s3:ListAllMyBuckets
• s3:HeadBucket

Concourse Resource

Catapult commands are used to implement the resource type release.

Docker

# build an image with catapult installed
docker build --target=catapult -t release-resource -f Dockerfile .

# build an image for the concourse resource
docker build --target=release-resource -t release-resource -f Dockerfile .

Notes

Most of the code relies on the client sending the correct information, but more validation checks can be implemented in Concourse to avoid the creation of invalid releases.

i.e.

A client can upload two releases with the same number twice, but concourse will use only one of them and ignore the other one.