Understanding CloudFormation

When you dig into the sample code for this course, you’ll see that it defines dozens of individual AWS resources for deployment. It’s not feasible to create these resources manually, one-by-one, which means you need a way to deploy these resources together, in an automated and repeatable fashion. Knowing how to do this is a key part of working with cloud generally, and serverless in particular. And that’s why we’re kicking off this course with a section on CloudFormation.

Writing Imperative Infrastructure-as-Code

To explain the benefits to using CloudFormation, we need to backtrack a little to the concept of immutable infrastructure. This is an idea that originates with setting up servers, and the core concept is that you shouldn’t tinker with a server once it’s deployed. Instead, you should have scripts that automatically perform all the steps required to configure and launch a server. Then, when you need to update that server, you update the scripts and run them again, replacing the existing server instance with a new one. The value of this approach lies in the way you’re always defining your infrastructure in code, which is both self-documenting and enables automated and repeatable deployment processes.

When we shift over to the serverless world, we still want to define our infrastructure as code, to reap these benefits of self-documentation, automation, and repeatability. The breadth of the AWS control plane gives us the capabilities we need to script the creation of any resources we might incorporate in our serverless solutions. However, there’s a fundamental problem with the imperative approach we’ve been describing when applied to long-lived services.

The imperative approach is built around the idea that to perform updates, you’re discarding resources (like servers) and rebuilding them from scratch. A lot of the services used in serverless solutions contain data that stops you from simply tearing them down and replacing them with every change. This data includes user accounts in a service like Cognito, or table data in a service like DynamoDB. You could theoretically apply scripts to perform updates in place for resources like this, but this approach gets unworkably complicated when you have to port these changes back to the scripts that perform the initial deployment.

Writing Declarative Infrastructure-as-Code

The solution to this problem can be found in the CloudFormation service, because instead of relying on imperative scripts to define resources, it lets you define them declaratively. To illustrate how the declarative approach is different, let’s look at a technology that we’re all familiar with. In the screenshot below, you can see the output of the PostgreSQL explain command, as applied to a sample query for a relational database:

As we all know, with SQL you don’t need to specify what actions are required to efficiently query a database. You just declare the structure of the query and the columns you want to output, and the database engine then does the hard work for you of figuring out the optimal combination of indexes and table scans that will return your data.

In a similar way, CloudFormation lets you declaratively define the target state for your resources. Then when you deploy those resources, CloudFormation builds and executes a list of the control plane actions needed to get to that state. Even better, when you update your resource definitions at a later time, CloudFormation infers the differences between the current and target states and executes the actions needed to make the transition.

Stepping Through the CloudFormation Deployment Process

To show how CloudFormation deployments work in more detail, let’s step through the concept diagram shown below. This will identify the different parts of CloudFormation deployments and show how they work together. Our starting point is a call to the CloudFormation “create-stack” command from the command line (upper-left), including as arguments the location of a local template file that contains all the resource definitions, a list of parameter name/value pairs, and a stack name. Once invoked with this command, CloudFormation reads the template, checks that it’s well-formed and otherwise error-free, and then generates the list of actions to be performed (upper-right).

Then, before creating anything else for the solution, CloudFormation creates a stack resource (lower-left). This resource will persist whether the deployment succeeds or fails. After creating this stack, CloudFormation deploys the resources for the solution, executing the list of actions one by one. When finished, the result is a stack resource that’s persistently associated with the other resources for the solution, as illustrated by the dashed box (bottom-right).

Working with Stack Commands

When working with CloudFormation to deploy or update solutions, you’re always working with stack commands. We’ve seen the “create-stack” command described above, and there’s also a corresponding “update-stack” command. To perform a stack update, you modify the resource definitions in the template, and/or the values for the template parameters. You then call the “update-stack” command, which takes the same arguments as the create-stack command.

There’s also a helpful “describe-stack” command that provides access to the values of the outputs defined in the CloudFormation template. You use outputs like these to get values for those properties that are generated at build-time, such as API Gateway IDs. There’s a query syntax for this command that makes it possible to get values for both individual and sets of stack outputs.

Rolling Back Failed Updates

One of the more powerful features built into CloudFormation is its ability to execute stack operations as a single unit-of-work, or transaction. This means that if a CloudFormation operation fails at any point while executing, the stack being operated upon is then automatically rolled back to its prior state. We’ll demonstrate this feature in the labs.

Additional Features

Here’s two additional features that are good to know about but aren’t covered in this course.

Change Sets

CloudFormation ChangeSets is a feature that will be familiar to Terraform users, because it’s equivalent to a Terraform plan. Instead of applying resource changes immediately with a stack command, you have to option to create a Change Set that itemizes the actions that will be performed. Once reviewed, you can choose whether to go ahead and apply these actions.

Drift Tracking

We’ve been emphasizing in this lesson the idea that you only ever update resources through CloudFormation and stack commands. But in the event that changes do get made to resources outside this process, the Drift feature makes it possible to track these and send notifications to interested parties.