Using AWS Parameter Store in Jenkins Pipeline

Sometimes you need to be able to use Dynamic parameters or Secrets in you Jenkins pipelines which isn’t trivial with Jenkins Credential Store so AWS Parameter can help.

What is Parameter Store

AWS Parameter Store is part of the AWS Systems Manager group of tools. It allows you to store parameters as either plain text or a secret string within your AWS Account. These parameters can be namespaced by a path. e.g.

/dev/jenkins/name
/dev/jenkins/token

Secrets can be added and updated in Parameter Store through various APIs and other tooling like Terraform in a dynamic fashion which makes it a very flexible offering and far cheaper than Secrets Manager.

Using Parameters from Parameter Store

IAM Policy

First thing that you need is for your Jenkins to have an IAM policy and role that allows it to access the parameters.

This is a minimum example but adapt it to work with your security requirements.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ssm:*",
            "Resource": "arn:aws:ssm:<region>:<account>:parameter/*"
        }
    ]
}

Plugins

Next step is to install the aws-parameter-store plugin which provides the required step to be able to grab the parameters in a pipeline.

If you do not already have it installed it would be advisable to install the mask-passwords plugin too to stop any secrets being displayed in the Jenkins output

Code

Now that the Controller has IAM permissions to get secrets and the plugin to add the step is available we can grab parameters from the Parameter Store

withAWSParameterStore(naming: 'basename', path: '/dev/jenkins/', regionName: 'eu-west-1') {
  sh("make USER=${NAME} TOKEN=${TOKEN} release")
}

The withAWSParameterStore step wraps any nested commands allowing them to use the parameters according to the arguments passed in. For this example it makes any parameter under the path /dev/jenkins/ available and the basename says that just the last element on the path forms the variable name. e.g. /dev/jenkins/name is accessed through ${NAME}

As well as basename you can also specify relative or absolute the descriptions for which can be found in the plugin documentation

Masking the Output

As some of these values are sensitive they should not be sent to the Jenkins log in clear text. Using the mask-passwords plugin obfuscates the text.

withAWSParameterStore(naming: 'basename', path: '/dev/jenkins/', regionName: 'eu-west-1') {
  wrap([$class: 'MaskPasswordsBuildWrapper', varPasswordPairs: [[password: TOKEN, var: 'SECRET']]]) {
    sh("make USER=${NAME} TOKEN=${TOKEN} release")
  }
}

Wrapping the nested commands with the MaskPasswordsBuildWrapper will stop any of those commands from printing the given list of variables to the console

Jenkins not in AWS?

If Jenkins is not in AWS you can still use this technique. But rather than assigning a Role to Jenkins you will need to create an IAM user with the correct policy attached and then store the credentials in the Jenkins Credential Store using the AWS Credentials Plugin.

Then you can pass in the Credentials ID to the withAWSParameterStore call.

withAWSParameterStore(credentialsId: 'AWS_USER', naming: 'basename', path: '/dev/jenkins/', regionName: 'eu-west-1')

Bonus

However the main limitation is that most secrets get into Jenkins by being manually added in by administrators. This means that it is a manual job to rotate and update credentials when they need changing.

You can pull Parameters out in you JCasC files too. Install the configuration-as-code-secret-ssm Secret Source extension plugin.

As above ensure your Jenkins has the correct IAM policy attached to get parameters.

Set the environment variable to the prefix required

---
credentials:
  system:
    domainCredentials:
      - credentials:
        - string:
            scope: GLOBAL
            id: opsgenie-api-key
            secret: "ABVD12342jfj"
            description: OpsGenie API Access Key
        - usernamePassword:
            scope: GLOBAL
            id: "artifactory-api-user-token"
            username: "admin"
            password: "apikey123"
            description: "Service account user and token for artifactory"

but of course you do not want to be hard coding in our secret values and committing them you want to template them out and use AWS Parameter Store to house them.

To do this a new plugin needs to be installed called configuration-as-code-secret-ssm this is the Secret Source extension plugin that does the work. Once installed you will need to set an environment variable with the prefix for your parameters. In the case of the examples above it would be /product1/

CASC_SSM_PREFIX=/product1/

Once this is set and Jenkins has AWS permissions to read the required parameters (see the plugin help configuration-as-code-secret-ssm), you can template out the variables in your JCasC

---
credentials:
  system:
    domainCredentials:
      - credentials:
        - string:
            scope: GLOBAL
            id: opsgenie-api-key
            secret: "${opsgenie_api_key}"
            description: OpsGenie API Access Key
        - usernamePassword:
            scope: GLOBAL
            id: "artifactory-api-user-token"
            username: ${service-account-name}
            password: ${service-account-artifactory-token}
            description: "Service account user and token for artifactory"