autozane bash

Terraform Recipe: Continuous Integration on Amazon Web Services

Most Terraform use cases involve relatively static infrastructure. In this recipe I will explore a more dynamic use case. Here we will leverage Terraform to run a continuous code integration into Amazon Web Services (AWS). Below I will provide a visual of what is achieved, followed up with a description of each step in what I call the "Infrastructure Time Line".

  1. We start off with a code release that is already running in an ASG from a previous deployment. This is referenced as Amazon Machine Image-x (AMI-x).

  2. Provision your new code in a new AMI via PackerIO (I will cover this specific step in a future post, stay tuned). The resulting AMI is referred to as AMI-y. We then pass AMI-y as a variable to Terraform. Terraform is then leveraged to deploy a new Autoscaling Group (ASG) to run along side the current running ASG (Using AMI-y).

  3. Terraform waits for successful provisioning of the new ASG. The ASG is configured to use EC2 health checks since we are not leveraging an ELB in this use case. We will rely on a watchdog script to report to the instance's autoscaling group when its systemd process is running.

  4. This step shows the creation of the new ASG, while keeping the exiting ASG in place.

  5. Upon successful creation of the new ASG, the previous deploy's ASG will be deleted.

The watchdog script to report into the ASG:

Here is the supporting Systemd Unit. It ensures the application is started first via the "Require" before it checks.

The key Terraform component to this step is the create_before_destroy. It is applied to the aws_launch_configuration and aws_autoscaling_group resources.

Along side the launch config and autoscaling group resources are some other supporting resources such as Instance Profile, IAM roles + policies, and security group settings. These resources are specific to the service and environment deployed.

Here is a view into the directory layout:

└── terraform
    ├── dev
    │   ├── environment-args.tfvars
    │   └──
    ├── prod
    │   ├── environment-args.tfvars
    │   └──
    ├── stage
    │   ├── environment-args.tfvars
    │   └──
    ├── <--- **what we looked at in this post**

Driving this CI deployment is Jenkins. The Terraform state is managed on a per service / per environment basis and stored in s3. I will go into this in more detail in a future blog post.

In a nutshell, this Terraform recipe can come it pretty handy to manage a simple CI deployment with AWS Autoscaling Groups and stateless applications!

Use Bash Utilities to Update aws/credentials for AssumeRole

Some AWS assume roles bash foo that can come in handy.

aws sts assume-role --role-arn <ROLEARN> --role-session-name <ROLESESSIONNAME> |\
    tr '{}' ',,' |\
    awk -F:  '
                    BEGIN { RS = "," ; print "[PROFILENAME]"}
                    /:/{ gsub(/"/, "", $2) }
                    /AccessKeyId/{ print "aws_access_key_id = " $2 }
                    /SecretAccessKey/{ print "aws_secret_access_key = " $2 }
                    /SessionToken/{ print "aws_session_token = " $2 }
    '  >> ~/.aws/credentials

OR if you don't want to touch your .aws/credentials file

aws sts assume-role --role-arn arn:aws:iam::1111111111111:role/role-test --role-session-name "RoleSessionTest" |\ 
    grep -w 'AccessKeyId\|SecretAccessKey\|SessionToken' |\ 
    awk  '{print $2}' | sed  's/\"//g;s/\,//' > awscre
    export AWS_ACCESS_KEY_ID=`sed -n '3p' awscre`
    export AWS_SECRET_ACCESS_KEY=`sed -n '1p' awscre`
    export AWS_SECURITY_TOKEN=`sed -n '2p' awscre`