Brandon Nicoll

Your policy will look something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "ec2:RunInstances",
            "Effect": "Allow",
            "Resource": "arn:aws:ec2"
        },
        {
            "Action": "ec2:DescribeInstances",
            "Effect": "Allow",
            "Resource": "arn:aws:ec2"
        },
        {
            "Action": "ec2:CreateImage",
            "Effect": "Allow",
            "Resource": "arn:aws:ec2"
        },
        {
            "Action": "ec2:DescribeImages",
            "Effect": "Allow",
            "Resource": "arn:aws:ec2"
        },
        {
            "Action": "ec2:TerminateInstances",
            "Effect": "Allow",
            "Resource": "arn:aws:ec2"
        }
    ]
}

Making Endpoints Meet

I set out to make a simple system in which users would be able to automate deployments of their projects to an Auto Scaled group. The first step in this process would be to allow users to expose an endpoint that, when fired, will do the following:

This endpoint can be utilized as a GitHub webhook, Git hook, or even as part of a larger deployment pipeline after a number of build and test steps have successfully executed. I decided to call this system Quadraxis; I created a new Laravel project, added the AWS SDK for PHP to the composer.json file, then added the keys to the .env file. If you’re interested, Quadraxis is on GitHub, but the actual implementation of the system is less important than the concepts behind automating AMI creation.

Using User Data

A crucial piece of this puzzle is solved with utilization of the User Data feature when creating an instance. User data can be passed to the API when launching an instance to provide an initialization script that will run once the instance is ready. I saved the exact script I used for this example in this Gist for reference. Note that when trying to execute a bash script in User Data, it is important to include “#!/bin/sh” (or wherever shell might reside on your flavor of OS) as the first line. Also note this value needs to be base 64 encoded when submitting to the API. Of course, this step is made easier in this example by having the project in a public GitHub repository and written in an interpreted language. The endpoint created by Quadraxis will find the saved initialization script for the specific project and apply it when launching the instance.

    public function webhook(Request $request, $name)
    {
        //Find the project
        $project = Project::where('name', $name)
                        ->get()
                        ->first();
        if (is_null($project)){
            abort(404);
        }
        
        //Instantiate the EC2 client
        $ec2Client = Ec2Client::factory(array(
            'region'  => getenv('AWS_REGION'),
            'version' => '2012-10-17'
        ));

        //Launch the instance from a pre-determined AMI with saved
        //user data and into a pre-configured security group
        $result = $ec2Client->runInstances(array(
            'ImageId'        => $project->base_ami_id,
            'MinCount'       => 1,
            'MaxCount'       => 1,
            'InstanceType'   => 't2.medium',
            'UserData'       => base64_encode(
                                $project->init_script),
            'SecurityGroups' => array('torvus-sec-group')
        ));
        
        //Save the new instance ID for later
        $instanceId = $result->search('Instances[0].InstanceId');
    }

Waiter, There’s a Fly in My Soup

Before trying to create the image, the instance needs to be in a “running” state. We have a couple of options, such as polling DescribeInstances until the instance is ready, or we could try using a “waiter”. Waiters are a useful feature in many of the AWS SDKs that will allow you to wait for a resource to be in a specific state. We can extend the endpoint code further to wait until the instance is in a running state with a waiter.

        $ec2Client->waitUntil('InstanceRunning', [
            'InstanceIds' => array($instanceId)
        ]);

A Positive Self-Image

Once the instance is in a running state, we can issue the command to create an image.

        $result = $ec2Client->createImage(array(
            'InstanceId' => $instanceId,
            'Name' => $project->name . time()
        ));

        $imageId = $result->search('ImageId');

Custom “Waiter”

Now we need to wait for the image creation to finish so we can terminate the instance, but there isn’t a “wait for this image to be available” waiter out-of-the-box for the AWS SDK for PHP(at least I don’t think there is). I simply wrote a loop that checks every thirty seconds for the image to be available.

        while (1) { 
            $result = $ec2Client->describeImages(array(
                'ImageIds' => array($imageId)
            ));
            $imageState = $result->search('Images[0].State');
            if ($imageState == 'available') {
                break;
            }
            sleep(30);
        }

Terminate the Instance

We have our deployable artifact, so we can terminate the instance used to create the AMI with the TerminateInstances command:

        $result = $ec2Client->terminateInstances(array(
            'InstanceIds' => array($instanceId)
        ));

Wrapping Up

With this AMI, we now have what we need to start changes to an application down a deployment pipeline. You may be wondering about how to solve problems like environment configuration, managing DNS, or cycling out old instances in an automated fashion. I’m planning on continuing writing more functionality in Quadraxis to address these problems and using it as a backdrop for more posts like this one. Stay tuned! In the meantime, let me know what you think on Twitter, @bgnicoll.