Adopt Cloud and DevOps

AWS CloudFormation Goes YAML. DevOps Engineers Rejoice!

AWS’s new support for YAML format is a big improvement for the developer experience. In this post, we introduce some of the new YAML features.

by Louie Corbo

Last month Amazon Web Services (AWS) introduced support for YAML (Yet Another Markup Language OR YAML Ain't a Markup Language) for CloudFormation. CloudFormation is AWS’s method for defining infrastructure code that provisions cloud resources on demand. We use CloudFormation daily when automating our customers' environments in AWS as part of continuous delivery pipelines.

In this post we’ll highlight some of the newly introduced YAML features and explain how they work in practice.

YAML or JSON?

Is it worth switching to YAML from the original option JavaScript Object Notation (JSON)?  The basic contrast between the two formats is JSON is easier for computers to parse and generate, whereas YAML is easier for humans to read. Ultimately it boils down to personal preference, but we’re definitely in the YAML camp for obvious reasons (after all, we’re a customer experience firm).

There are two types of users: Those who have existing JSON CloudFormation code and those who do not. If you do not have an existing JSON code base, then I’d recommend starting with YAML because it is much easier to read and write, especially for novices. 

If you have an existing JSON CloudFormation code base, the good news is that YAML is a superset of JSON, meaning any existing JSON can easily be converted to YAML with the right tools.  While there are a number of free online resources that will convert JSON to YAML for you, we'd suggest against using them, primarily because posting your code on a public website is not a sound security practice.  The easier solution requires Ruby and can be done with this single command:

ruby –ryaml –rjson –e ‘puts YAML.dump(JSON.parse(STDIN.read))’ < Example.json >> Example.yaml
 

This command reads the Example.json as input and outputs Example.yaml.  Our example uses Ruby 2.3.1 but I’m sure you can do the same thing in Python or another language.

It is important to note that since YAML is a superset of JSON, conversions are more or less a one-way process.  Once you convert to YAML and take advantage of some of its enhanced features, converting back will not be as straightforward of a process.

JSON to YAML

Let's start out with a template snippet available on AWS example site.

{
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Resources" : {
        "myStack" : {
       "Type" : "AWS::CloudFormation::Stack",
       "Properties" : {
          "TemplateURL" : "https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template",
              "TimeoutInMinutes" : "60"
       }
        }
    },
    "Outputs": {
       "StackRef": {"Value": { "Ref" : "myStack"}},
       "OutputFromNestedStack" : {
             "Value" : { "Fn::GetAtt" : [ "myStack", "Outputs.BucketName" ] }
       }
    }
}
 

Save this command as Example.json and run the above ruby command against it.  The newly created file should look like:  

---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  myStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template
      TimeoutInMinutes: '60'
Outputs:
  StackRef:
    Value:
      Ref: myStack
  OutputFromNestedStack:
    Value:
      Fn::GetAtt:
      - myStack
      - Outputs.BucketName
Run this in CloudFormation and it will work just like the JSON file does.  You'll also note that while it’s only slightly fewer lines of code than the JSON version it is much more readable.

 

!GetAtt Function

One of the features in YAML is the ability to use a new abbreviated syntax to refer to CloudFormation functions. The former “Fn::GetAtt” can be written as !GetAtt, for example. This coupled with some other YAML syntax rules allows us to shorten up the code while still maintaining readability.  Applying these changes to our example YAML results in the following: 

---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  myStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template
      TimeoutInMinutes: '60'
Outputs:
  StackRef:
    Value: !Ref myStack
  OutputFromNestedStack:
    Value: !GetAtt myStack.Outputs.BucketName
 

!Sub Function

Looking at AWS's documentation, it shows us how to do two separate things with the simplified substitution, or the !Sub function. 

The first is to use a literal block to specify the user data script.

UserData:
  "Fn::Base64":
    !Sub |
      #!/bin/bash -xe
      yum update -y aws-cfn-bootstrap
      /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource LaunchConfig 
 --configsets wordpress_install --region ${AWS::Region}
      /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} 
 --resource WebServerGroup --region ${AWS::Region}

This example shows that using the “|” character after the !Sub function results in all the following lines being read as part of the Userdata section of the CloudFormation template, line breaks and all. Substitution comes in where all of the variables enveloped in a ${} will be converted into their proper values at runtime.  For example ${AWS::Region} may be read as “us-east-1” or the respective region where this CloudFormation template is running. This is a great new feature as previously the !Join function was required for these kinds of complex string manipulations, which makes it very difficult to read the code.

YAML Scalars

Yaml scalars allow content to be written in block notation using the character “|”.  This is a feature used in the above example and not available in Json.

The second AWS example uses mapping functions as part of the substitution.

Name: !Sub
  - www.${Domain}
  - { Domain: !Ref RootDomainName }

In this example, a mapping function like !Ref (or !FindinMap) adds an additional step to the syntax where we define substitution variable “Domain” using an Intrinsic Function Reference.  What if we have a slightly more complex example that combines a mapping function in a literal block?  Here’s one approach to solve it using Yaml’s block sequences:

"Fn::Base64": !Sub
 - #!/bin/bash -xe &&
   yum update -y aws-cfn-bootstrap &&
   echo “My domain is www.${Domain}”  &&
   /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} 
 --resource LaunchConfig --configsets wordpress_install --region ${AWS::Region}  &&
   /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} 
 --resource WebServerGroup --region ${AWS::Region}
 - {Domain: !Ref RootDomainName}

Finally, if you want to use multiple variables that use mapping, they can all be defined in the bottom line separated by a comma.

- {Domain: !Ref RootDomainName, AnotherVariable: AnotherVariableDefined}
 

Scalars and block sequences are features not available in Json, so this solution is only applicable when using Yaml.  If you are using Json these solutions don’t apply and you would still need to use the intrinsic function Join as part of a solution.

Summary

AWS’s new support for YAML format is a big improvement for the developer experience. In this post we introduced some of the new YAML features including !GetAtt and !Sub. We also showed how to convert your existing JSON templates to YAML with one line of Ruby code. If you’re like us, YAML support for CloudFormation is a long overdue feature and we’re glad AWS has rolled it out. Within our DevOps team, there is much rejoicing!

Learn more about our DevOps and Cloud service offering.

Louie Corbo
Louie Corbo
DevOps Engineer
Contact Louie

Related Articles