Writing BuildSpec Files to Compile Lambdas

As we’ve seen, the CodePipeline service has pre-built actions for things like checking out source code from a repository, or creating a CloudFormation stack. For those pipeline tasks that can only be performed by executing commands on a build agent, you need CodeBuild projects. And you define the commands that are executed in a CodeBuild project by writing a buildspec file.

What we’ll do in this tutorial is step through one of the buildspec files from the sample code. This buildspec file includes commands to read secrets from the AWS secrets manager, install the Lambda build tools, set the Nuget or Maven package sources (including with Github credentials) and perform the Lambda build. Let’s take a look.

Buildspec File Basics

A buildspec file is a YAML-formatted set of instructions for a CodeBuild project. In the pipeline.yaml template from the sample code, the buildspec file is referenced in the CodeBuild resource, as shown on line 179 below:

The referenced “BuildFile” value in the resource shown above comes from a parameter in the template, which is set by the pipeline script during deployment. If you don’t set this property in the template, CodeBuild looks for the “buildspec.yml” file in the root of the source directory.

In the sample code you’ll see that there are two buildspec files, found in the “/buildspec” directory. These two files execute the build commands for either the OpenAPI or the SAM versions of the code. A command line argument in the pipeline.zsh deployment script specifies which of these two versions are used. There’s also a “test.buildspec.yml” file in the common repository which defines the commands for the automated API tests in the Test stage of the pipeline.

The Example Buildspec File

Here’s the full “sam.buildspec.yml” file from the .Net version of the sample code, which we’ll be stepping through for the rest of this tutorial. (We’ll note a couple of differences in the Java version along the way, as well):

Exporting Variables

Each action in the pipeline has the option to define a namespace to which environment variables can be added. This is a way to pass outputs to subsequent stages in the pipeline. Along these lines, when you look at lines 2-4 in the sam.buildspec.yaml file, you can see the declaration for the VERSION_NUMBER environment variable:

This variable gets a value assigned at runtime on line 16, which is then referenced in the pipeline.yaml template, as shown below on line 283 (scroll all the way to the right), where it passes the value through as a CloudFormation parameter:

Extracting Secrets

As you saw when you set up your workstation for this course, the projects in the “aws-connectedcar-dotnet-serverless” or “aws-connectedcar-java-serverless” repositories depend on shared libraries that are hosted as packages on Github. Although these are made available as public packages, Github requires the use of a personal access token to download them. To avoid putting credentials of any kind in the source repository, we’re using the AWS Secrets Manager service to make these GitHub credentials available during the build step.

In lines 5-6 below, the sam.buildspec.yaml file shows how the personal access token is extracted from the Secrets Manager and assigned to the “TOKEN” environment variable:

Line 17 in this code shows how this TOKEN environment variable is subsequently used in the dotnet command that adds the Github packages URL as a NuGet source location.

Setting Up the Build Agent

Following the “env” section of the buildspec file, there are a series of commands organized into groups under the “phases” section. The “install” group does what the name suggests: it sets the platform for the build agent and runs commands that install the software that will be used by the other commands that follow:

These are basically the steps that you took when setting up your development workstation. These steps have to be performed here for each build, because the build agents are torn down after each execution of the pipeline.

Packaging the Lambdas

Here we have the actual Lambda build step, which uses the dotnet tools installed in a previous command. This is the same command you can see in the deployment scripts, which compiles the Lambdas and packages them into a zip file ready for uploading:

Uploading Artifacts to S3

Having built the Lambda deployment package, the “post-build” commands now perform a series of artifact uploads to S3 so that they can be accessed by CloudFormation in the “deploy” stage of the pipeline:

Line 24 shows the S3 command that cleans up any existing files from the target S3 bucket and folders. Next, on line 25, the Lambda package file is uploaded. This file name incorporates the VERSION_NUMBER variable that we assigned previously. This variable is also passed as an input parameter to CloudFormation in the deploy stage, which then uses it to reference this file. Lastly, lines 26-31 show all the CloudFormation templates being uploaded to the S3 bucket.