Hey!
Background
To give an overview of what I’m doing - I’m working on a project in tf 0.12. The goal is to stand up some databases and data migration infrastructure. I’ve put together a couple of modules to handle that and a calling module to supply input.
In my test environment I have 3 RDS instances, 3 DMS replication instances (1 per RDS), and 6 DMS endpoints (3 source, 3 target). What I want to do is allow some database, dev types into the AWS console so they can configure the endpoints, and replication tasks manually and then once happy I can port that into the module.
Issue
I want to create a few sets of aws_iam_policy, aws_iam_group and aws_iam_group_policy_attachment resources so that per environment all users in the group will have certain permissions to explicitly defined DMS resources.
The IAM policy that is being rendered contains syntax errors, and as far as I can tell terraform is adding a trailing comma to list objects in the policy definition.
Versions
Terraform: 0.12.19
provider.aws: version = “2.46”
provider.onepassword: version = “0.5”
provider.random: version = “2.2”
What I’m doing
-
I’m using outputs/data sources to source 3 tuples of dms ARN’s from a separate child module - one ‘list’ each for source endpoints, target endpoints and repl instances - some example data below.
-
I’m merging these in a local var into a single list
-
I’m formatting this var with jsonencode and then attempting to use it as a templating var in a json iam policy template
- This didn’t work, so I’ve attempted the same using the data source aws_iam_policy_document, but ran into the same error
terraform console
local.all_dms_merged
[
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
]
jsonencode(local.all_dms_merged)
[“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”]
At this point, it looks as though jsonencode has done the trick and I now have a properly formatted string to inject into my template, so I run this:
resource "aws_iam_policy" "dms_policy" {
name = "tf-dms-permissions-${terraform.workspace}"
description = "Policy allowing console users to access dms resources created in terraform workspace."
path = "/"
policy = templatefile("${path.module}/policies/dms_permissions.json", { dms_resources = jsonencode(local.all_dms_merged) })
}
The json policy template saved in ${path.module}/policies/dms_permissions.json
{
“Version”: “2012-10-17”,
“Sid”: “DMSAllowedOperations”,
“Statement”: [
{
“Effect”: “Allow”,
“Resource”: [
${dms_resources}
],
“Action”: [
“dms:DescribeSchemas”,
“dms:DescribeRefreshSchemasStatus”,
“dms:ModifyReplicationTask”,
“dms:StartReplicationTask”,
“dms:DescribeEventSubscriptions”,
“dms:DescribeEndpointTypes”,
“dms:DescribeEventCategories”,
“dms:StartReplicationTaskAssessment”,
“dms:DescribeOrderableReplicationInstances”,
“dms:ListTagsForResource”,
“dms:DescribeConnections”,
“dms:DescribeReplicationInstances”,
“dms:DeleteReplicationTask”,
“dms:TestConnection”,
“dms:DescribeEndpoints”
]
}
]
}
Expected
The rendered policy file is formatted as specified in either the template supplied to the templatefile function or in the data.aws_iam_policy_document renderer.
Actual
The aws iam/CreatePolicy API rejects the rendered policy with: MalformedPolicyDocument
2020/02/27 11:25:38 [DEBUG] aws_iam_policy.dms_policy: apply errored, but we’re indicating that via the Error pointer rather than returning it: Error creating IAM policy example: MalformedPolicyDocument: Syntax errors in policy.
The terraform plan looks like this:
aws_iam_policy.dms_policy will be created
- resource “aws_iam_policy” “dms_policy” {
- arn = (known after apply)
2020/02/27 11:25:34 [DEBUG] command: asking for input: “Do you want to perform these actions in workspace “dev”?”
- description = “Policy allowing console users to access dms resources created in terraform workspace.”
- id = (known after apply)
- name = “tf-dms-permissions-dev”
- path = “/”
- policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ “dms:DescribeSchemas”,
+ “dms:DescribeRefreshSchemasStatus”,
+ “dms:ModifyReplicationTask”,
+ “dms:StartReplicationTask”,
+ “dms:DescribeEventSubscriptions”,
+ “dms:DescribeEndpointTypes”,
+ “dms:DescribeEventCategories”,
+ “dms:StartReplicationTaskAssessment”,
+ “dms:DescribeOrderableReplicationInstances”,
+ “dms:ListTagsForResource”,
+ “dms:DescribeConnections”,
+ “dms:DescribeReplicationInstances”,
+ “dms:DeleteReplicationTask”,
+ “dms:TestConnection”,
+ “dms:DescribeEndpoints”,
]
+ Effect = “Allow”
+ Resource =
+ [
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
],
+ Sid = “DMSAllowedOperations”
},
]
+ Version = “2012-10-17”
}
)
}
The problem with this policy it seems is the trailing comma in both the Action list and Resource list.
If I drop this policy into policy sim and remove these trailing commas, the json is valid and I’d expect the call to AWS would succeed.
If this is a known bug - is there a workaround?