Brooklyn + Terraform Integration

In this article, we’ll take a look at Apache Brooklyn’s integration with Hashicorp Terraform, a tool for building, changing, and versioning infrastructure safely and efficiently.

 

Brief Overview of Terraform

The key component of a Terraform system is the configuration file. Configuration files describe to Terraform the components needed to run anything from a single application to an entire datacenter.

Terraform generates an execution plan describing what it will do to reach the desired end state. Upon application of the configuration, Terraform builds all of the infrastructure that is specified in the configuration. Once infrastructure has been created, the user may modify the configuration freely. Terraform is able to determine what has changed in the configuration and create incremental execution plans that can be applied to change the existing infrastructure.

Terraform can manage low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.

 

Brooklyn + Terraform Integration

The io.cloudsoft.terraform.TerraformConfiguration Brooklyn entity adds deployment and lifecycle management of Terraform configurations to Brooklyn.

Users can now author YAML Brooklyn blueprints which specify the location on which the Terraform CLI is to be installed, and provide a Terraform configuration file to be applied.

The Terraform entity includes sensors which provide the status of the configuration (whether it has been applied), the current execution plan, and the state of the created infrastructure. It also includes effectors which can apply or destroy the configuration.

Support for more advanced monitoring and management of the actual infrastructure created by Terraform (as opposed to just managing the configuration via automated Terraform CLI calls) is on the roadmap.

 

Getting Set Up

The easiest way to get Brooklyn + Terraform up and running is to download the standalone distribution here. You’ll need Java 7+ installed.

$ tar xzf brooklyn-terraform-dist.tar.gz
$ cd brooklyn-terraform
$ ./start.sh launch

At this point, Brooklyn is ready to deploy and manage Terraform configurations.

 

Applying an Example Configuration

Now that Brooklyn is running, open localhost:8081 in your browser to view the Brooklyn GUI. When the GUI first loads, you will be immediately prompted to create an application. Select the YAML tab to add your own blueprint.

The following Brooklyn blueprint will install the Terraform CLI on your local machine where Brooklyn is running and use the CLI to both provision a VM on Amazon EC2 and assign it an elastic IP.

Note that the location where the Terraform CLI is installed and the location(s) where the Terraform CLI actually creates infrastructure are specified independently of each other. The CLI installation location is specified by the location: value in the blueprint, and the location(s) where Terraform creates infrastructure is specified in the Terraform configuration file that you supply in the blueprint via tf.configuration.contents.

The location: value must be an ssh-able location, as described in the Brooklyn documentation. If localhost is chosen, some extra steps must be taken described here.

location: localhost
name: Brooklyn Terraform Example
services:
- type: io.cloudsoft.terraform.TerraformConfiguration
  name: Amazon EC2 VM with Elastic IP
  brooklyn.config:
    tf.configuration.contents: |        
        provider "aws" {
            access_key = "YOUR_ACCESS_KEY"
            secret_key = "YOUR_SECRET_KEY"
            region = "us-east-1"
        }

        resource "aws_instance" "example" {
            ami = "ami-408c7f28"
            instance_type = "t1.micro"
            tags {
                Name = "brooklyn-terraform-example"
            }
        }

        resource "aws_eip" "ip" {
            instance = "${aws_instance.example.id}"
        }

Paste this blueprint into the Brooklyn YAML window, add your EC2 access key and secret key, and click “Finish” to apply the configuration!

Provisioning will last for a few minutes — while you wait for its completion, have a look at your AWS console to see your VM starting up, as well as the Brooklyn “Applications” tab for detail on all your deployments, across all your infrastructure.

Once the configuration has been fully applied and provisioning is complete, all of the Terraform-specific sensors (beginning with tf.*) should look similar to this:

Using these sensors, you can see exactly which resources have been created, the state of those resources, and of course whether the application of the configuration was successful overall.

Integrating with other tools is simple thanks to Brooklyn’s REST API. The creation was just a POST to /v1/applications, and the sensor data can be retrieved with GET /v1/applications/:appId/entities/:entityId/sensors/tf.state.

Destroying infrastructure is just as easy. Once you’d like to terminate the VM that you created, simply invoke the “destroy” effector under the “Effectors” tab. Brooklyn will invoke the Terraform destroy command which will clean up anything that was created when the configuration was applied.

Conclusion

Apache Brooklyn is now able to deploy and manage the lifecycle of Hashicorp Terraform configurations, and deeper management of the infrastructure created by those configurations is on the roadmap. This integration turns an already a powerful and easy-to-use tool for provisioning infrastructure into a fully managed and automated solution.

What would you like to see next? Should we automatically insert cloud credentials? Should we use Brooklyn to add monitoring and logging to these nodes? Let us know at our mailing list: dev@incubator.brooklyn.apache.org. Thanks!

Video

Here is a video of the Brooklyn + Terraform integration in action.