Introduction

AWS Security Groups act as a virtual firewall for your instance to control inbound and outbound traffic. When you have a large number of rules, managing them can become complex and error-prone. This blog post introduces a method to simplify this process using Terraform and CSV files.

Prerequisites

This post assumes you have basic familiarity with Terraform and AWS Security Groups. If you are using Visual Studio Code then you may find the extensions for Hashicorp Terraform and Edit CSV helpful, though they are not strictly required.

The Power of Terraform and CSV

Terraform is a powerful tool that allows us to define and provide virtual infrastructure using a declarative configuration language. By using CSV files in conjunction with Terraform, we can manage our security group rules in a more organized and maintainable way. This approach adheres to the DRY (Don't Repeat Yourself) principle, reducing redundancy and potential errors.

Setting Up Your CSV File

The first step is to set up our CSV file. This file will contain all the necessary information for our security group rules. Here's an example of what this might look like:

id,name,description,from_port,to_port,protocol_type,cidr_block,self,src_sg_id,type
1,web,Allow HTTP,80,80,TCP,10.123.0.0/16,,,ingress
2,web,Allow HTTPS,443,443,TCP,10.123.0.0/16,,,ingress
3,web,Allow all egress,0,65535,-1,0.0.0.0/0,,,egress
4,db,Allow MySQL/MariaDB,3306,3306,TCP,10.0.0.0/16,,,ingress
5,db,Allow PostgreSQL,5432,5432,TCP,10.0.0.0/16,,,ingress
6,db,Allow all egress,0,65535,-1,0.0.0.0/0,,,egress
7,web,Allow self,80,80,TCP,,true,,ingress
8,db,Allow web,3306,3306,TCP,,,web,ingress

Each row represents a rule with its details:

• id: A unique identifier for each rule. This value must be unique for each row.
• name: The name of the security group. [1]
• description: A description for the rule.
• from_port and to_port: The range of ports that the rule applies to.
• protocol_type: The protocol of the rule.
• cidr_block: The CIDR block for the rule. [2]
• self: Set to true to indicate that this security group will trust itself. [2]
• src_sg_id: The name from column 2 of another security group to trust. [2]
• type: Whether the rule is for ingress or egress traffic.

[1] Warning: changing the name property in the CSV will trigger Terraform to recreate the existing security group matching the original name. Take care when changing this property in a production setting.
[2] Note: the cidr_block, self, and src_sg_id are mutually exclusive, as per the Terraform AWS Provider docs. If you specify one of these values in a row then the others should be empty.

Writing the Terraform Code

Next, we'll walk through the Terraform code that reads this CSV file and creates the corresponding AWS resources.

# Decode the CSV file into a list of maps
locals {
  sg_rules = csvdecode(file("./data/sg_rules.csv"))
  sg_names = toset([for rule in local.sg_rules : rule.name])
}

# Create a security group for each unique name
resource "aws_security_group" "sample_sg" {
  for_each = { for name in local.sg_names : name => name }

  name = each.value
}

# Create security group rules based on the CSV file
resource "aws_security_group_rule" "cidr_block_rules" {
  for_each = { for rule in local.sg_rules : rule.id => rule if rule.cidr_block != "" }

  type              = each.value.type
  from_port         = each.value.from_port != "" ? tonumber(each.value.from_port) : 0
  to_port           = each.value.to_port != "" ? tonumber(each.value.to_port) : 65535
  protocol          = each.value.protocol_type
  cidr_blocks       = [each.value.cidr_block]
  security_group_id = aws_security_group.sample_sg[each.value.name].id
  description       = each.value.description
}

resource "aws_security_group_rule" "self_rules" {
  for_each = { for rule in local.sg_rules : rule.id => rule if rule.self != "" }

  type              = each.value.type
  from_port         = each.value.from_port != "" ? tonumber(each.value.from_port) : 0
  to_port           = each.value.to_port != "" ? tonumber(each.value.to_port) : 65535
  protocol          = each.value.protocol_type
  self              = each.value.self == "true" ? true : false
  security_group_id = aws_security_group.sample_sg[each.value.name].id
  description       = each.value.description
}

resource "aws_security_group_rule" "src_sg_rules" {
  for_each = { for rule in local.sg_rules : rule.id => rule if rule.src_sg_id != "" }

  type                     = each.value.type
  from_port                = each.value.from_port != "" ? tonumber(each.value.from_port) : 0
  to_port                  = each.value.to_port != "" ? tonumber(each.value.to_port) : 65535
  protocol                 = each.value.protocol_type
  source_security_group_id = each.value.src_sg_id != "" ? aws_security_group.sample_sg[each.value.src_sg_id].id : ""
  security_group_id        = aws_security_group.sample_sg[each.value.name].id
  description              = each.value.description
}

This code reads a CSV file containing security group rules using the csvdecode function, creates a security group for each unique name using the aws_security_group resource, then creates the corresponding security group rules using the aws_security_group_rule resources and associates them to the appropriate security group.

You will notice there are three separate resource blocks for aws_security_group_rule, each corresponding to one of the aforementioned mutually exclusive source types. This way we can use a CSV with a mix of these rule types, giving us greater flexibility and the ability to make truly comprehensive security groups to reflect a variety of use cases.

Running Your Terraform Code

To run your Terraform code:

1. Initialize your Terraform workspace with terraform init.
2. Check what changes will be made with terraform plan.
3. If everything looks good, apply the changes with terraform apply.

You should see output indicating that the resources were created successfully.

Conclusion

Managing AWS Security Group rules can be complex when dealing with large lists of rules. By using Terraform in conjunction with CSV files, we can simplify this process and make our infrastructure more maintainable.

This approach is just one way to manage AWS resources with Terraform. There are many other possibilities and improvements you could make based on your specific needs.

I hope you found this tutorial helpful! If you have any questions or feedback, feel free to leave a comment below.

Useful References

• Terraform - csvdecode function
• Terraform - aws_security_group resource
• Terraform - aws_security_group_rule resource