Importing AWS RDS Resources with Terraform
Written by:
Igor Gorovyy
DevOps Engineer Lead & Senior Solutions Architect
LinkedIn
Overview
This article explores the process of importing existing AWS RDS databases into Terraform infrastructure code. This is a crucial step in ensuring an Infrastructure as Code (IaC) approach for managing cloud resources.
When a company aims to save costs while planning a Disaster Recovery Plan (DRP), the simplest approach is backup and resource restoration. In cloud infrastructure, resources are typically restored in a different region.
This article does not cover other strategies like stand-by, active-active DRP approaches, as this is not the main topic. The key intent is to demonstrate a "cost-effective and simple..." Terraform import mechanism and its use for importing RDS resources.
When restoring resources, it becomes necessary to manage them with Terraform code. The Terraform import mechanism enables rapid and secure resource recovery.
Before Terraform 1.6, import required manual state management and careful tracking of resource locations, which could lead to potential errors. From version 1.6 onwards, Terraform offers a simpler and more reliable import system, significantly streamlining infrastructure recovery in DRP scenarios.
Import Process Diagram
graph TD;
A[Existing AWS RDS Resource] -->|Import Command| B[Import into Terraform]
B -->|Generate Configuration| C[Create main.tf]
C -->|Check Changes| D[terraform plan]
D -->|Apply Changes| E[terraform apply]
E -->|Resource Under Terraform Management| F[Finalized RDS]Why is Importing Necessary?
Importing existing resources allows:
- Centralized infrastructure management using Terraform
- Tracking configuration changes through version control
- Avoiding manual resource setup and reducing error risks
- Simplified deployment and scaling
- Efficient recovery after incidents or failures
Key Steps for Importing RDS Resources
1. Change Tracking
- Infrastructure State Preservation
- Terraform uses the
terraform.tfstatefile to store the current state of resources. -
This helps avoid discrepancies between the actual state in the cloud and the configuration in code.
-
Conflict Prevention
- Using locking mechanisms (
terraform plan) prevents simultaneous conflicting updates. -
This is particularly useful in collaborative environments.
-
Modification History
- Using Git or other version control systems helps track changes and quickly revert to previous configurations.
- Terraform logging enables tracking of the reasons and consequences of modifications.
2. Process Automation
- CI/CD Integration
- Terraform can be integrated with automation systems like GitHub Actions, GitLab CI/CD, Jenkins, or AWS CodePipeline.
-
This enables automatic change application after testing.
-
Automated Testing
terraform validatechecks the correctness of configuration syntax.terraform planevaluates upcoming changes before applying them.-
terratestis used for infrastructure unit testing. -
Scheduled Updates
- Using automated infrastructure update processes ensures timely updates without risk of failures.
-
Applying
terraform apply -auto-approvein controlled environments simplifies configuration updates. -
Automated Deployment
- Terraform modules allow standardized environment deployment without manual intervention.
- Automated deployment reduces time-to-market for products.
3. Scalability
- Flexible Deployment
- Terraform supports parameterized variables, making it easy to adapt configurations for different environments (dev, staging, production).
-
Using modules helps avoid code duplication.
-
Multi-Region Deployment
- Terraform allows managing infrastructure in multiple AWS regions from a single configuration.
- Using
providerwithaliassimplifies multi-region deployments. -
In our case, we move resources to the "Blue" region (
provider = aws.Blue) and import them into the code. -
Environment Cloning
- Terraform supports
terraform workspace, enabling the creation of isolated environments for testing without impacting production. - The ability to quickly copy configurations makes it easy to create new infrastructure instances.
Code Examples
Import an Existing RDS into Terraform
terraform plan -generate-config-out=generated-resources.tf
terraform import aws_db_instance.example_db my-existing-db
Declaring an RDS Resource in Terraform
import {
provider = aws.Blue
to = aws_db_instance.product1-dev-service1-migrated
id = "product1-dev-service1-migrated"
}
import {
provider = aws.Blue
to = aws_db_instance.product1-dev-service2-migrated
id = "product1-dev-service2-migrated"
}
import {
provider = aws.Blue
to = aws_db_instance.product1-dev-service3-migrated
id = "product1-dev-service3-migrated"
}
import {
provider = aws.Blue
to = aws_db_instance.product1-dev-service4-migrated
id = "product1-dev-service4-migrated"
}
import {
provider = aws.Blue
to = aws_db_instance.product1-service5-migrated
id = "product1-service5-migrated"
}
# __generated__ by Terraform
# Please review these resources and move them into your main configuration files.
# __generated__ by Terraform
resource "aws_db_instance" "product1-dev-service1-migrated" {
provider = aws.Blue
allocated_storage = 100
allow_major_version_upgrade = null
apply_immediately = null
auto_minor_version_upgrade = false
availability_zone = "eu-west-1a"
backup_retention_period = 5
backup_target = "region"
backup_window = "20:31-21:01"
ca_cert_identifier = "rds-ca-rsa2048-g1"
character_set_name = null
copy_tags_to_snapshot = false
custom_iam_instance_profile = null
customer_owned_ip_enabled = false
db_name = "owner"
db_subnet_group_name = "multiregion-dev-vpc-blue"
dedicated_log_volume = false
delete_automated_backups = true
deletion_protection = true
domain = null
domain_auth_secret_arn = null
domain_fqdn = null
domain_iam_role_name = null
domain_ou = null
enabled_cloudwatch_logs_exports = []
engine = "postgres"
engine_lifecycle_support = "open-source-rds-extended-support"
engine_version = jsonencode(16.6)
final_snapshot_identifier = null
iam_database_authentication_enabled = false
identifier = "product1-dev-service4-migrated"
identifier_prefix = null
instance_class = "db.t4g.large"
iops = 3000
kms_key_id = "arn:aws:kms:eu-west-1:294***********8:key/1*********-8**f-4**0-8**1-**********b"
license_model = "postgresql-license"
maintenance_window = "wed:01:55-wed:02:25"
manage_master_user_password = null
master_user_secret_kms_key_id = null
max_allocated_storage = 0
monitoring_interval = 0
monitoring_role_arn = null
multi_az = false
nchar_character_set_name = null
network_type = "IPV4"
option_group_name = "default:postgres-16"
parameter_group_name = "multiregion-rds-service4-pg-16"
password = null # sensitive
performance_insights_enabled = false
performance_insights_kms_key_id = null
performance_insights_retention_period = 0
port = 5432
publicly_accessible = false
replica_mode = null
replicate_source_db = null
skip_final_snapshot = true
snapshot_identifier = null
storage_encrypted = true
storage_throughput = 125
storage_type = "gp3"
tags = {}
tags_all = {}
timezone = null
upgrade_storage_config = null
username = "service1"
vpc_security_group_ids = ["sg-0fe****************6"]
}
# __generated__ by Terraform
resource "aws_db_instance" "product1-dev-service2-migrated" {
provider = aws.Blue
allocated_storage = 605
allow_major_version_upgrade = null
apply_immediately = null
auto_minor_version_upgrade = false
availability_zone = "eu-west-1a"
backup_retention_period = 5
backup_target = "region"
backup_window = "21:14-21:44"
ca_cert_identifier = "rds-ca-rsa2048-g1"
character_set_name = null
copy_tags_to_snapshot = false
custom_iam_instance_profile = null
customer_owned_ip_enabled = false
db_name = "service2"
db_subnet_group_name = "multiregion-dev-vpc-blue"
dedicated_log_volume = false
delete_automated_backups = true
deletion_protection = true
domain = null
domain_auth_secret_arn = null
domain_fqdn = null
domain_iam_role_name = null
domain_ou = null
enabled_cloudwatch_logs_exports = []
engine = "postgres"
engine_lifecycle_support = "open-source-rds-extended-support"
engine_version = jsonencode(16.6)
final_snapshot_identifier = null
iam_database_authentication_enabled = false
identifier = "product1-dev-service2-migrated"
identifier_prefix = null
instance_class = "db.m6g.large"
iops = 12000
kms_key_id = "arn:aws:kms:eu-west-1:294***********8:key/1*********-8**f-4**0-8**1-**********b"
license_model = "postgresql-license"
maintenance_window = "sun:00:07-sun:00:37"
manage_master_user_password = null
master_user_secret_kms_key_id = null
max_allocated_storage = 0
monitoring_interval = 0
monitoring_role_arn = null
multi_az = false
nchar_character_set_name = null
network_type = "IPV4"
option_group_name = "default:postgres-16"
parameter_group_name = "multiregion-rds-pg-logical-16"
password = null # sensitive
performance_insights_enabled = false
performance_insights_kms_key_id = null
performance_insights_retention_period = 0
port = 5432
publicly_accessible = false
replica_mode = null
replicate_source_db = null
skip_final_snapshot = true
snapshot_identifier = null
storage_encrypted = true
storage_throughput = 500
storage_type = "gp3"
tags = {}
tags_all = {}
timezone = null
upgrade_storage_config = null
username = "service2"
vpc_security_group_ids = ["sg-0fe****************6"]
}
# __generated__ by Terraform
resource "aws_db_instance" "product1-service3-migrated" {
provider = aws.Blue
allocated_storage = 83
allow_major_version_upgrade = null
apply_immediately = null
auto_minor_version_upgrade = false
availability_zone = "eu-west-1a"
backup_retention_period = 1
backup_target = "region"
backup_window = "21:12-21:42"
ca_cert_identifier = "rds-ca-rsa2048-g1"
character_set_name = null
copy_tags_to_snapshot = false
custom_iam_instance_profile = null
customer_owned_ip_enabled = false
db_name = "service5"
db_subnet_group_name = "multiregion-dev-vpc-blue"
dedicated_log_volume = false
delete_automated_backups = true
deletion_protection = true
domain = null
domain_auth_secret_arn = null
domain_fqdn = null
domain_iam_role_name = null
domain_ou = null
enabled_cloudwatch_logs_exports = []
engine = "postgres"
engine_lifecycle_support = "open-source-rds-extended-support"
engine_version = jsonencode(16.6)
final_snapshot_identifier = null
iam_database_authentication_enabled = false
identifier = "product1-service5-migrated"
identifier_prefix = null
instance_class = "db.t4g.large"
iops = 3000
kms_key_id = "arn:aws:kms:eu-west-1:294***********8:key/1*********-8**f-4**0-8**1-**********b"
license_model = "postgresql-license"
maintenance_window = "thu:01:30-thu:02:00"
manage_master_user_password = null
master_user_secret_kms_key_id = null
max_allocated_storage = 0
monitoring_interval = 0
monitoring_role_arn = null
multi_az = false
nchar_character_set_name = null
network_type = "IPV4"
option_group_name = "default:postgres-16"
parameter_group_name = "multiregion-rds-dev-athena-16"
password = null # sensitive
performance_insights_enabled = false
performance_insights_kms_key_id = null
performance_insights_retention_period = 0
port = 5432
publicly_accessible = false
replica_mode = null
replicate_source_db = null
skip_final_snapshot = true
snapshot_identifier = null
storage_encrypted = true
storage_throughput = 125
storage_type = "gp3"
tags = {}
tags_all = {}
timezone = null
upgrade_storage_config = null
username = "service3"
vpc_security_group_ids = ["sg-0fe****************6"]
}
# __generated__ by Terraform
resource "aws_db_instance" "product1-dev-service4-migrated" {
provider = aws.Blue
allocated_storage = 330
allow_major_version_upgrade = null
apply_immediately = null
auto_minor_version_upgrade = false
availability_zone = "eu-west-1a"
backup_retention_period = 5
backup_target = "region"
backup_window = "00:59-01:29"
ca_cert_identifier = "rds-ca-rsa2048-g1"
character_set_name = null
copy_tags_to_snapshot = false
custom_iam_instance_profile = null
customer_owned_ip_enabled = false
db_name = "managementaccounting"multiregion-rds-pg-audit-16
db_subnet_group_name = "multiregion-dev-vpc-blue"
dedicated_log_volume = false
delete_automated_backups = true
deletion_protection = true
domain = null
domain_auth_secret_arn = null
domain_fqdn = null
domain_iam_role_name = null
domain_ou = null
enabled_cloudwatch_logs_exports = []
engine = "postgres"
engine_lifecycle_support = "open-source-rds-extended-support"
engine_version = jsonencode(16.6)
final_snapshot_identifier = null
iam_database_authentication_enabled = false
identifier = "product1-dev-service3-migrated"
identifier_prefix = null
instance_class = "db.t4g.large"
iops = 3000
kms_key_id = "arn:aws:kms:eu-west-1:294***********8:key/1*********-8**f-4**0-8**1-**********b"
license_model = "postgresql-license"
maintenance_window = "wed:23:11-wed:23:41"
manage_master_user_password = null
master_user_secret_kms_key_id = null
max_allocated_storage = 0
monitoring_interval = 0
monitoring_role_arn = null
multi_az = false
nchar_character_set_name = null
network_type = "IPV4"
option_group_name = "default:postgres-16"
parameter_group_name = "multiregion-rds-pg-audit-16"
password = null # sensitive
performance_insights_enabled = false
performance_insights_kms_key_id = null
performance_insights_retention_period = 0
port = 5432
publicly_accessible = false
replica_mode = null
replicate_source_db = null
skip_final_snapshot = true
snapshot_identifier = null
storage_encrypted = true
storage_throughput = 125
storage_type = "gp3"
tags = {}
tags_all = {}
timezone = null
upgrade_storage_config = null
username = "service4"
vpc_security_group_ids = ["sg-0fe****************6"]
}
# __generated__ by Terraform
resource "aws_db_instance" "product1-dev-service5-migrated" {
provider = aws.Blue
allocated_storage = 64
allow_major_version_upgrade = null
apply_immediately = null
auto_minor_version_upgrade = false
availability_zone = "eu-west-1a"
backup_retention_period = 5
backup_target = "region"
backup_window = "01:22-01:52"
ca_cert_identifier = "rds-ca-rsa2048-g1"
character_set_name = null
copy_tags_to_snapshot = false
custom_iam_instance_profile = null
customer_owned_ip_enabled = false
db_name = "service1"
db_subnet_group_name = "multiregion-dev-vpc-blue"
dedicated_log_volume = false
delete_automated_backups = true
deletion_protection = true
domain = null
domain_auth_secret_arn = null
domain_fqdn = null
domain_iam_role_name = null
domain_ou = null
enabled_cloudwatch_logs_exports = []
engine = "postgres"
engine_lifecycle_support = "open-source-rds-extended-support"
engine_version = jsonencode(16.6)
final_snapshot_identifier = null
iam_database_authentication_enabled = false
identifier = "product1-dev-service1-migrated"
identifier_prefix = null
instance_class = "db.t4g.large"
iops = 3000
kms_key_id = "arn:aws:kms:eu-west-1:294***********8:key/1*********-8**f-4**0-8**1-**********b"
license_model = "postgresql-license"
maintenance_window = "tue:22:05-tue:22:35"
manage_master_user_password = null
master_user_secret_kms_key_id = null
max_allocated_storage = 0
monitoring_interval = 0
monitoring_role_arn = null
multi_az = false
nchar_character_set_name = null
network_type = "IPV4"
option_group_name = "default:postgres-16"
parameter_group_name = "multiregion-rds-pg-logical-16"
password = null # sensitive
performance_insights_enabled = false
performance_insights_kms_key_id = null
performance_insights_retention_period = 0
port = 5432
publicly_accessible = false
replica_mode = null
replicate_source_db = null
skip_final_snapshot = true
snapshot_identifier = null
storage_encrypted = true
storage_throughput = 125
storage_type = "gp3"
tags = {}
tags_all = {}
timezone = null
upgrade_storage_config = null
username = "service5"
vpc_security_group_ids = ["sg-0fe****************6"]
}
Planning and applying changes
terraform plan
terraform apply -auto-approve
Conclusion
Strategic Benefits
- Improved Manageability
- Centralized management of all resources through a single configuration.
- Ease of tracking and controlling changes.
-
Automated updates and dependency management.
-
Resource Optimization
- Cost reduction through proper infrastructure planning.
- Automated scaling and efficient resource distribution.
- Minimization of human error through automated deployments.
Long-Term Benefits
- Investment in the Future
- Simplified infrastructure expansion with IaC.
- Ensuring deployment stability and repeatability.
-
Compliance with security and regulatory standards.
-
Operational Improvements
- Reduced infrastructure management time.
- Automation of deployment and update processes.
- Enhanced overall reliability and service availability.
References: Hashicorp