* Add user_file function to functions.py

[coreone]

I added a function to functions.py that I'm hoping will be valuable to others using this library. Most of the code for it was lifted from:

https://github.com/devstructure/python-cloudformation/blob/master/cloudformation/__init__.py#L66-L89

which I have used previously. Basically, this allows you to read in a file which you would like to include in a CloudFormation config and have it automatically placed in the CF template using the correct "Fn::Join" process. For example, let's say you have a simple user_data script like:

#!/bin/bash
INSTANCEID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`

function error_exit {
  cfn-signal -e 1 '____'
  exit 1
}

/opt/aws/bin/cfn-init --region ____ -s ____ -r SeqProd || error_exit 'Failed to bootstrap'

You can then do something like the following in a template.py file:

user_data_script = user_file(
    "userdata.sh",
    [
        ref("TheInstanceWaitHandle"),
        ref("AWS::Region"),
        ref("AWS::StackName")
    ]
)

You can then just add this to an EC2 instance's properties using:

"UserData": base64(user_data_script)

What you get in the output is:

"UserData": {
  "Fn::Base64": {
    "Fn::Join": [
      "\n",
      [
        "#!/bin/bash",
        "INSTANCEID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`",
        "",
        "function error_exit {",
        {
          "Fn::Join": [
            "",
            [
              "  cfn-signal -e 1 '",
              {
                "Ref": "TheInstanceWaitHandle"
              },
              "'"
            ]
          ]
        },
        "  exit 1",
        "}",
        "",
        {
          "Fn::Join": [
            "",
            [
              "/opt/aws/bin/cfn-init --region ",
              {
                "Ref": "AWS::Region"
              },
              " -s ",
              {
                "Ref": "AWS::StackName"
              },
              " -r SeqProd || error_exit 'Failed to bootstrap'"
            ]
          ]
        }
      ]
    ]
  }
},

This will then take care of reading in the file, replacing the "____" with the referenced Ref values, and adding it to the template correctly. You can even pass normal files that do not require reference value substitution as an easy way to just include files.

[tehsmyers]

Yay I totally missed the notification for this! Sorry about that.

Test coverage is needed, as well as linting.

Using template replacements with **kwargs also (I think?) solves the issue of having the same number of args as interpolation markers with the added benefit of not having to write your own parser.

[coreone]

Sorry it has taken a while to get back to this...

So, I got linting working (flake8) and will look into writing some tests. For the template replacements, however, I ran into a snag. The template piece of string looks great, however the way this has to work for CF is to split the variables out into their own join per line. For example, let's say I have the following in a script file:

function error_exit {
  /usr/bin/cfn-signal -e 1 '____'
  exit 1
}

which I want to replace with a "Ref": "TheInstanceWaitHandle". For this to work in JSON for CF, it would need to look like:

"function error_exit {",
{
  "Fn::Join": [
    "",
    [
      "  /usr/bin/cfn-signal -e 1 '",
      {
        "Ref": "TheInstanceWaitHandle"
      },
      "'"
    ]
  ]
},
"  exit 1",
"}",

so, simply solving the variable replacement issue would only put a string in place of the variable, where what I really need to do is split out the ____ (or some other matching pattern) so that it can be made into a multi-part line. I have updated the code to do this using surrounding %% delimiters in this new commit, so we don't need positional parameters anymore at least.

So now, a scriptlet like this:

#!/bin/bash
function error_exit {
  /usr/bin/cfn-signal -e 1 '%%$INSTANCEWAITHANDLE%%'
  exit 1
}

with data like this:

user_data_script = user_file("userdata.sh",{"$INSTANCEWAITHANDLE": ref("TheInstanceWaitHandle"),})
cft = CloudFormationTemplate(description="This template is designed to create VMs in EC2")
properties = {
    "UserData": base64(user_data_script),
}

will render this:

"UserData": {
  "Fn::Base64": {
    "Fn::Join": [
      "\n",
      [
        "#!/bin/bash",
        "function error_exit {",
        {
          "Fn::Join": [
            "",
            [
              "  /usr/bin/cfn-signal -e 1 '",
              {
                "Ref": "TheInstanceWaitHandle"
              },
              "'"
            ]
          ]
        },
        "  exit 1",
        "}"
      ]
    ]
  }
},