This article intends to provide insight into the native approach of doing Terraform to import the cloud resources into the terraform state file which might reside in your local system / remote back-end.
Before get started, Let’s go through some of the concepts,
What is the purpose of the state file?
Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures. This state is stored by default in a local file named “terraform.tfstate”, but it can also be stored remotely, which works better in a team environment.
How does terraform state management work in most cases?
1. The developers could update the desired infrastructure changes in the terraform configuration files that reside under any SCM.
2. As soon as the new changes are committed to the repository, there could be an automated pipeline that will be triggered to perform a set of terraform operations.
3. Those terraform operations could be read/write state files to make the desired changes that were mentioned in the SCM after applying those changes in the infrastructure.
4. In the end, all will be in sync including the configuration files in SCM, terraform state files, and the infrastructure changes.
All are in Sync — Terraform State
This is Awesome, but how about your use-case is slightly different like below:
1. There is a correct / missing terraform configuration files in SCM.
2. The infrastructure changes are already done manually due to some reasons.
3. Both of the above pieces of information is not even present in the terraform state files.
Out of Sync — Terraform State
Here, the result will be out of sync, and some level of manual intervention is required to make everything in sync together. You could leverage these fixes using terraform import and the state management by going through the below concept:
The primary purpose of Terraform state is to store bindings between objects in a remote system and resource instances declared in your configuration. When Terraform creates a remote object in response to a change of configuration, it will record the identity of that remote object against a particular resource instance, and then potentially update or delete that object in response to future configuration changes.
Okay, that’s all the theory part — Following are the tool sets that we are using for the sake of quick demonstration, these could be leveraged to specific versions according to your use cases which doesn’t matter whether the state file is configured as local/remote back-end.
“Take a note that, in real-life scenarios- it isn’t just a bunch of updates/changes while running terraform plan phase, instead it returns something that you have to destroy resources first before making any changes/updates. This is where you have to be more cautious and update the state file with most care by importing each object. You can solve those problems with the given approach.”
Let’s dive into the practical way of doing this.
1.Assume that, we have two people — Sam & Sandy.
2. Sam created a Resource group in Azure by following tags using Terraform.
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "this" {
name = "tf-rg-1"
location = "East US"
tags = {
created_by = "sam"
is_billable = "no"
}
}
3. After the terraform operation such as ‘init, plan and apply’ — the resource group has been successfully created in Azure.
4. Sandy noticed that whatever the resources comes under this Resource group should be billable. But Sandy wasn’t aware that the change has to be done through Terraform. Instead, Sandy updated these changes manually from Azure portal and added extra 7 more tags as well.
created_by = "sam"
is_billable = "yes"
ab = "cd"
ef = "gh"
ij = "kl"
mn = "op"
qr = "st"
uv = "wx"
wx = "yz"
5. Sandy informed these changes to Sam.
6. Sam run the ‘terraform plan’ phase with the current configuration file (main.tf) to understand Sandy’s changes and requested Sandy to make any further changes only through Terraform going forward.
$ terraform plan
~ update in-place
Terraform will perform the following actions:
# azurerm_resource_group.this will be updated in-place
~ resource "azurerm_resource_group" "this" {
id = "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"
name = "tf-rg-1"
~ tags = {
- "ab" = "cd" -> null
- "ef" = "gh" -> null
- "ij" = "kl" -> null
~ "is_billable" = "yes" -> "no"
- "mn" = "op" -> null
- "qr" = "st" -> null
- "uv" = "wx" -> null
- "wx" = "yz" -> null
# (1 unchanged element hidden)
}
# (1 unchanged attribute hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
7. Sam decided to import this resource group to make everything in sync together (including the configuration files in SCM, terraform state files, and the infrastructure changes).
8. Sam run the following terraform actions before start importing,
# List the object in the current state file
$ terraform state list -state=terraform.tfstate
azurerm_resource_group.this
# Show the state information of the object `azurerm_resource_group.this` in the current state file
$ terraform state show azurerm_resource_group.this
# azurerm_resource_group.this:
resource "azurerm_resource_group" "this" {
id = "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"
location = "eastus"
name = "tf-rg-1"
tags = {
"created_by" = "terraform"
"is_billable" = "yes"
}
}
# Renamed the current main.tf file as Backup
$ mv main.tf main.tf-backup
# Renamed the current state file as Backup
$ mv terraform.tfstate terraform.tfstate-backup
9. Sam understands that the object (azurerm_resource_group.this) that needs to be imported is using the resource module ‘azurerm_resource_group’. Hence, Sam looked at the documentation and analyzed that this object can be imported via the following dummy configuration file which doesn’t require to pass any arguments (such as ‘name, location, etc).
provider "azurerm" {
features {}
}
# Here the resource id is 'this' for the resource module 'azurerm_resource_group'
resource "azurerm_resource_group" "this" {
}
10. Let’s import via the following terraform actions and verify those information's,
# Initialize the terraform
$ terraform init
Initializing the backend...
...blah...blah....blah...
Terraform has been successfully initialized!
# Import the remote object and it will create a new state file (terraform.tfstate) for the object 'azurerm_resource_group.this'
$ terraform import azurerm_resource_group.this /subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1
azurerm_resource_group.this: Importing from ID "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"...
azurerm_resource_group.this: Import prepared!
Prepared azurerm_resource_group for import
azurerm_resource_group.this: Refreshing state... [id=/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1]
Import successful!
# List the objectID in the new state file
$ terraform state list -state=terraform.tfstate
azurerm_resource_group.this
$ terraform state show azurerm_resource_group.this
# azurerm_resource_group.this:
resource "azurerm_resource_group" "this" {
id = "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"
location = "eastus"
name = "tf-rg-1"
tags = {
"created_by" = "sam"
"is_billable" = "yes"
"ab" = "cd"
"ef" = "gh"
"ij" = "kl"
"mn" = "op"
"qr" = "st"
"uv" = "wx"
"wx" = "yz"
}
timeouts {}
}
11. Perfect !!! Now, we have the old and new information's of the state file. Sam is going to overwrite the imported state information's (terraform.tfstate) with the old one (terraform.tfstate-backup) which was renamed earlier.
# Remove the object 'azurerm_resource_group.this' which is outdated from the old state file 'terraform.tfstate-backup'
$ terraform state rm -state=terraform.tfstate-backup azurerm_resource_group.this
Removed azurerm_resource_group.this
Successfully removed 1 resource instance(s).
# Overwrite the updated information of object 'azurerm_resource_group.this' to the old state file 'terraform.tfstate-backup'
# The format is - terraform state mv [options] SOURCE DESTINATION
$ terraform state mv -state=terraform.tfstate -state-out=terraform.tfstate-backup azurerm_resource_group.this azurerm_resource_group.this
Move "azurerm_resource_group.this" to "azurerm_resource_group.this"
Successfully moved 1 object(s).
# Removed the imported state file and renamed the old state file to the current one
$ rm terraform.tfstate && mv terraform.tfstate-backup terraform.tfstate
# List and show the objectID in the renamed state file
$ terraform state list -state=terraform.tfstate
azurerm_resource_group.this
$ terraform state show azurerm_resource_group.this
# azurerm_resource_group.this:
resource "azurerm_resource_group" "this" {
id = "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"
location = "eastus"
name = "tf-rg-1"
tags = {
"created_by" = "sam"
"is_billable" = "yes"
"ab" = "cd"
"ef" = "gh"
"ij" = "kl"
"mn" = "op"
"qr" = "st"
"uv" = "wx"
"wx" = "yz"
}
timeouts {}
}
12. At this point of time, Sam is successfully synced the state file.
13. Sam removed the dummy configuration file (main-dummy.tf) and renamed the old configuration file (main.tf-backup) how it was before. After that, Sam runs the ‘terraform plan’ phase, but the result is still showing there are changes/updates.
# Removed 'main-dummy.tf' file and renamed the file 'main.tf-backup' how it was before.
$ rm main-dummy.tf && mv main.tf-backup main.tf
$ terraform plan
azurerm_resource_group.this: Refreshing state... [id=/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# azurerm_resource_group.this will be updated in-place
~ resource "azurerm_resource_group" "this" {
id = "/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1"
name = "tf-rg-1"
~ tags = {
- "ab" = "cd" -> null
- "ef" = "gh" -> null
- "ij" = "kl" -> null
~ "is_billable" = "yes" -> "no"
- "mn" = "op" -> null
- "qr" = "st" -> null
- "uv" = "wx" -> null
- "wx" = "yz" -> null
# (1 unchanged element hidden)
}
# (1 unchanged attribute hidden)
# (1 unchanged block hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Why Because? Even though you sync the state file, terraform still looks for your configuration file and returns the results of what needs to be changed / not by comparing it with the existing information defined in the state file. Hence, it is important that you have to update the same information in both the state file (terraform.tfstate) and the configuration files (main.tf) to keep everything in sync.
14. Sam updated the respective changes in the configuration file (main.tf) as well.
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "this" {
name = "tf-rg-1"
location = "East US"
tags = {
created_by = "sam"
is_billable = "yes"
ab = "bc"
ab = "cd"
ef = "gh"
ij = "kl"
mn = "op"
qr = "st"
uv = "wx"
wx = "yz"
}
}
15. Finally, Sam run the plan phase and make sure everything is in sync.
$ terraform plan
azurerm_resource_group.this: Refreshing state... [id=/subscriptions/XX-XX-XX-XX/resourceGroups/tf-rg-1]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
So, that’s it. If steps are known, there is plenty of ways to automate, I am leaving that to YOU !!! Happy Terraforming !!!
If you like this piece of information, please support me with your gesture as a followers and the claps.
Share with your friends and followers
Start blogging about your favorite technologies, reach more readers and earn rewards!
Join other developers and claim your FAUN account now!
User Popularity
16
Influence
795
Total Hits
1
Posts
Read, Learn, Know, Teach
Hand curated newsletters for Developers, private Slack with like minded people, podcasts, job offers, news and more!
Only registered users can post comments. Please, login or signup.