Automate Packer with GitHub Actions

Packer templates let you define machine images as code, so you can establish a version control repository as the source of truth for image contents. This lets you adopt code review for changes and use automated builds to keep your images up-to-date. HCP Packer further simplifies image creation and deployment by providing a central image registry for operations and development teams. By integrating Packer and HCP Packer into continuous integration (CI) pipelines, you can automatically trigger image builds on changes to your version control repository.

In this tutorial, you will set up a complete GitHub Actions workflow to automatically build and manage different versions of a machine image. Whenever you push changes to designated branches (main, development, staging) or create a new Git tag, the workflow will trigger a Packer build, send the build metadata to HCP Packer, and set the corresponding HCP Packer channel to the build iteration. In the process, you will learn how to automate and scale Packer builds across your organization.

Prerequisites

This tutorial assumes that you are familiar with the standard Packer and HCP Packer workflows. If you are new to Packer, complete the Get Started tutorials first. If you are new to HCP Packer, complete the Get Started HCP Packer tutorials first.

For this tutorial, you will need:

• A GitHub account
• An HCP account
• An HCP Packer Registry and HCP service principal
• An AWS account and AWS Access Credentials

Configure GitHub action

Fork the Learn Packer GitHub Actions repository. This repository contains files that define the Packer build and the GitHub Actions workflow.

In your forked repository, go to Settings, Secrets, then Actions. Create the following repository secrets and set them to their respective values. The GitHub Action workflow in this repository will use these credentials to provision your resources and push metadata to HCP Packer.

Once complete, the Actions secrets page displays the six secrets.

Navigate to the Actions tab and enable the pre-configured workflow defined in the repository's .github/workflows directory by clicking I understand my workflows, go ahead and enable them.

Then, clone your forked repository to your local machine. If using the CLI, replace USERNAME with your GitHub username.

$ git clone https://github.com/USERNAME/learn-packer-github-actions

Review configuration

In this section, you will review the Packer template file and the GitHub Actions workflow to understand how GitHub Actions will automate your image builds.

Review Packer template

Open build.pkr.hcl. This file contains Packer HCL code for generating a HashiCups machine image. HashiCups is a demo application that lets you view and order customized HashiCorp branded coffee. The HashiCups application consists of a frontend React application and multiple backend services.

The amazon-ebs.ubuntu-lts source block retrieves an Ubuntu 22.04 image from the us-west-1 region to use as the base image.

source "amazon-ebs" "ubuntu-lts" {
  region = "us-west-1"
  
  source_ami_filter {
    filters = {
      virtualization-type = "hvm"
      name                = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
      root-device-type    = "ebs"
    }
    owners      = ["099720109477"]
    most_recent = true
  }
  
  instance_type  = "t2.small"
  ssh_username   = "ubuntu"
  ssh_agent_auth = false

  ami_name    = "hashicups_{{timestamp}}"
  ami_regions = ["us-west-1"]
}

Within the build block, the hcp_packer_registry block configures packer to send image metadata to the HCP Packer registry.

build {
  hcp_packer_registry {
    bucket_name = "learn-packer-github-actions"
    description = <<EOT
This is an image for HashiCups.
    EOT

    bucket_labels = {
      "hashicorp-learn" = "learn-packer-github-actions",
    }
  }
  ## ...
}

The amazon-ebs.ubuntu-lts block retrieves the build source. The file provisioner copies a systemd unit file for the HashiCups service. The shell provisioner runs the setup-deps-hashicups.sh script to install and configure the service.

build {
  ## ...
  sources = [
    "source.amazon-ebs.ubuntu-lts",
  ]

  # systemd unit for HashiCups service
  provisioner "file" {
    source      = "hashicups.service"
    destination = "/tmp/hashicups.service"
  }

  # Set up HashiCups
  provisioner "shell" {
    scripts = [
      "setup-deps-hashicups.sh"
    ]
  }
  ## ...
}

After building the image, Packer stores the iteration ID in a file named packer_manifest.json. The GitHub Action retrieves the iteration ID from this file and updates the respective channel to reference it. You will review how the GitHub action uses this value in the next section.

build {
  ## ...
  post-processor "manifest" {
    output     = "packer_manifest.json"
    strip_path = true
    custom_data = {
      iteration_id = packer.iterationID
    }
  }
}

The image built by packer is now fully configured to run the HashiCups application. When a server using this image launches, systemd will start HashiCups.

Review Actions workflow

The .github directory contains the GitHub Actions workflow and a helper script.

Open .github/workflows/build_and_deploy.yaml to review the GitHub Actions workflow. The first line sets a name for the workflow.

name: Build and Deploy

Next, the on block restricts this workflow to new tags that comply with semantic versioning or pushes to the development, staging, or main branches.

on:
  push:
    tags: ["v[0-9].[0-9]+.[0-9]+"]
    branches:
      - "development"
      - "staging"
      - "main"

The env section defines environment variables, which all jobs and steps in the workflow will have access to. The workflow uses HCP_CLIENT_ID and HCP_CLIENT_SECRET to authenticate and send build metadata to HCP Packer. The helper script uses the remaining environment variables to create and update HCP Packer channels using the HCP Packer API.

env:
  HCP_CLIENT_ID: ${{ secrets.HCP_CLIENT_ID }}
  HCP_CLIENT_SECRET: ${{ secrets.HCP_CLIENT_SECRET }}
  HCP_PROJECT_ID: ${{ secrets.HCP_PROJECT_ID }}
  HCP_ORGANIZATION_ID: ${{ secrets.HCP_ORGANIZATION_ID }}
  HCP_BUCKET_NAME: "learn-packer-github-actions"

Finally, the configuration defines two jobs. The build-image job uses Packer to build the image, and the update-hcp-packer-channel job updates the HCP Packer channel to point to the newly built image.

Both jobs run on the ubuntu-latest image, which contains the Packer binary.

jobs:
  build-image:
    name: Build
    runs-on: ubuntu-latest
    ##...
  update-hcp-packer-channel:
    name: Update HCP Packer channel
    runs-on: ubuntu-latest
    ##...

The build-image job has the following steps:

1. Checkout clones the GitHub repository to retrieve the current configuration. The uses argument defines the Docker image to run for the step and uses GitHub's actions/checkout@v3 action.

   .github/workflows/build_and_deploy.yml

   - name: Checkout
     uses: actions/checkout@v3
   

2. Configure AWS Credentials creates environment variables using values retrieved from GitHub secrets and sets the AWS region to us-west-1.

   .github/workflows/build_and_deploy.yml

   - name: Configure AWS Credentials
         uses: aws-actions/configure-aws-credentials@v1
         with:
           aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
           aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           aws-region: us-west-1
   

3. Packer Init initializes the Packer template by installing all plugins referenced in the template.

   .github/workflows/build_and_deploy.yml

   - name: Packer Init
         run: packer init .
   

4. Packer Build - Branches builds the machine images defined in the root directory. The github.ref contains the branch or tag name that triggered the workflow, however, this step only runs when the workflow is triggered by a push to a branch.

   .github/workflows/build_and_deploy.yml

   - name: Packer Build - Branches
         if: "startsWith(github.ref, 'refs/heads/')"
         run: packer build .
   

5. Packer Build - Tags builds the machine images defined in the root directory for new semantically-versioned tags. Since HCP Packer build fingerprints must be unique, this job sets the build fingerprint to the execution's timestamp. This ensures that builds from tagged commits do not share the same fingerprint as builds in the main, development, and staging branches.

   .github/workflows/build_and_deploy.yml

   - name: Packer Build - Tags
         if: startsWith(github.ref, 'refs/tags/v')
         run: HCP_PACKER_BUILD_FINGERPRINT=$(date +'%m%d%YT%H%M%S') packer build .
   

6. Get HCP Packer Iteration ID from Packer Manifest retrieves the iteration ID from the packer_manifest.json file generated by the Packer build post-processor. This step exports the iteration ID so the update-hcp-packer-channel job can use it to update the HCP Packer channel.

   Tip

   GitHub generates a new commit SHA when you create a Pull Request (PR). If your version control system does not create new commit SHAs for PRs, you must create a unique fingerprint for the Packer build when merging PRs.

   .github/workflows/build_and_deploy.yml

   - name: Get HCP Packer Iteration ID from Packer Manifest
         id: hcp
         run: |
           last_run_uuid=$(jq -r '.last_run_uuid' "./packer_manifest.json")
           build=$(jq -r '.builds[] | select(.packer_run_uuid == "'"$last_run_uuid"'")' "./packer_manifest.json")
           iteration_id=$(echo "$build" | jq -r '.custom_data.iteration_id')
           echo "::set-output name=iteration_id::$iteration_id"
   

Once the build-image job completes, the Actions workflow triggers the update-hcp-packer-channel job. This job has two steps:

1. Checkout clones the current configuration.

   .github/workflows/build_and_deploy.yml

   - name: Checkout
     uses: actions/checkout@v3
   

2. Create and set channel updates the HCP Packer channel corresponding to the branch or tag to serve the new iteration. This step determines the channel name using the github.ref_name and replaces all dots in tags to dashes to create a valid channel name (for example: v1.0.0 becomes v1-0-0). Then, it runs the create_channel_iteration.sh script with the channel name and the iteration ID from the build-image job.

   .github/workflows/build_and_deploy.yml

   - name: Create and set channel
         working-directory: .github/scripts
         run: |
           channel_name=$( echo ${{github.ref_name}} | sed 's/\./-/g')
           ./create_channel_iteration.sh $HCP_BUCKET_NAME $channel_name "${{ needs.build-image.outputs.iteration_id }}"
   

Open the .github/scripts/create_channel_iteration.sh file. This script authenticates to HCP, creates the HCP Packer channel if it does not exist, and updates the channel to point to the given iteration. The main branch is mapped to the HCP Packer release channel to show how you can decouple branch names from channel names.

# If on main branch, set channel to release
if [ "$channel_name" == "main" ]; then
  channel_name="release"
Fi

Create development build

Create a new development branch in your forked repository.

$ git checkout -b 'development'

Change the HASHICUPS_VERSION variable in setup-deps-hashicups.sh to v1.0.0.

# Get HashiCups config
git clone https://github.com/hashicorp-demoapp/hashicups-setups
cd hashicups-setups/docker-compose-deployment
git checkout server

HASHICUPS_VERSION="v1.0.0"
sed -i 's/HashiCups/HashiCups - ${HASHICUPS_VERSION}/g' docker-compose.yaml

# Use `compose create` to fetch container data without running the container
sudo docker compose create

Stage your changes.

$ git add setup-deps-hashicups.sh

Commit the changes with a message.

$ git commit -m 'Update HashiCups footer message with version'
[development aba0bd5] Update HashiCups footer message with version
 1 file changed, 1 insertion(+), 1 deletion(-)

Push the changes.

$ git push --set-upstream origin development

Verify development build

Click on your forked repository's Actions tab, then click on the running workflow.

Click on the Build job. Notice that it builds the image using the Packer Build - Branches step.

Once the workflow completes, go to your HCP Packer registry. Click on the learn-packer-github-actions bucket, then click on Iterations. Packer displays the completed build, and the build fingerprint matches the Git commit's SHA.

Click on Channels. Notice the Update HCP Packer channel workflow step created a development channel and set it to the iteration triggered by the development branch.

Create another development build

Change the HASHICUPS_VERSION variable in setup-deps-hashicups.sh to v1.1.0.

# Get HashiCups config
git clone https://github.com/hashicorp-demoapp/hashicups-setups
cd hashicups-setups/docker-compose-deployment
git checkout server

HASHICUPS_VERSION="v1.1.0"
sed -i 's/HashiCups/HashiCups - ${HASHICUPS_VERSION}/g' docker-compose.yaml

# Use `compose create` to fetch container data without running the container
sudo docker compose create

Stage your changes.

$ git add setup-deps-hashicups.sh

Commit the changes with a message.

$ git commit -m 'Update HashiCups footer message with new version'
[development a9724f9] Update HashiCups footer message with new version
 1 file changed, 1 insertion(+), 1 deletion(-)

Push the changes.

$ git push

The push triggered a new workflow in your forked repository Actions.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page. Notice that the workflow updated the development channel to point to the second iteration, the latest build in the development branch. This workflow makes the development GitHub branch the source of truth for the development channel.

Create and merge a pull request

In your forked repository within GitHub, create a pull request from the development branch. In the base repository drop-down, choose your forked repository and the main branch. Then click Create pull request.

Merge the pull request. Do not delete the development branch.

The Actions tab shows that the merged pull request triggered a workflow for the main branch.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page. Notice that the workflow created a channel named release and set it to the build triggered by the merge to the main branch. This makes the main GitHub branch the source of truth for the release channel.

Create tagged build

Git tags let you version your changes. By tagging specific commits, you can support multiple image build versions. This lets downstream image consumers query specific build versions outside of the development, staging, and production channels.

Checkout the main branch.

$ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.

Pull the latest changes from main.

$ git pull

Create a new Git tag.

$ git tag v1.1.0

Push the tag.

$ git push --tags

Verify tagged build

Go to your forked repository's Actions tab to find the new workflow run. Notice the commit SHA for the main branch and the v1.1.0 tag are the same. To ensure HCP Packer build fingerprints are unique, the Build Image - Tags step set the fingerprint to the current timestamp instead of the commit SHA.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page in HCP Packer. Notice that the workflow created a channel named v1-1-0 and set it to the build triggered by the v1.1.0 tag. This makes the v1.1.0 Git tag the source of truth for the v1-1-0 channel.

Destroy resources

Before moving on, destroy the AMIs created during this tutorial and their snapshots to avoid incurring additional costs. In the AMI console for us-west-1 select the AMIs, then click on the Actions button and the Deregister option. Delete the snapshots by selecting the snapshots, then click on the Actions button and the Delete option.

Next steps

In this tutorial, you set up a GitHub Actions workflow to automatically build and manage different versions of a machine image. Then, you triggered builds by pushing commits to the development and main branches, and by creating a new Git tag. You can adapt this Actions workflow to automate and scale Packer builds across your organization.

For more information on topics covered in this tutorial, review the following resources.

• Refer to the Build Images in CI/CD Packer documentation for more guidance on automating Packer builds.
• Complete the Schedule Image Iteration Revocation for Compliance tutorial to learn how to revoke image iterations to ensure your images are compliant and secure.
• Read the Build a Golden Image Pipeline with HCP Packer tutorial to learn how create a build pipeline for base images and application images.
• Watch the Automating Image Pipelines with HCP Packer HashiTalks presentation to learn how HashiCorp leverages HCP Packer and GitHub Actions in production to automate our image build, test, and deployment pipeline.