How to manage existing Azure Resource Groups using Terraform

This post has been republished via RSS; it originally appeared at: ITOps Talk Blog articles.

Introduction

Many Ops teams are looking at adopting Infrastructure as Code (IaC) but are encountering the dilemma of not being able to start from a green field perspective. In most cases organizations have existing resources deployed into Azure, and IaC adoption has become organic. Lack of available time and resources creates a potential "technical debt" scenario where the team can create new resources with IaC, but does not have a way to retrace the steps for existing deployments.

 

HashiCorp Terraform has a little used featured known as "import" which allows existing Resource Groups to be added to the state file, allowing Terraform to become aware of them. However, there are some restrictions around this which we are going to cover off.

 

Before We Begin

I'm going to walk us through a typical example of how "terraform import" can be used, but I want to share a few useful links first that I believe are worth looking at. The first is the HashiCorp Docs page on "terraform import". This covers the basics, but it worth noting the information on configuration.

 

"The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration."

 

The second link is the Microsoft Docs tutorial on Storing Terraform State in Azure Storage, as we will use this option in the example.

 

Example

Ok, so let's get to the fun stuff now! In this example I have an existing Resource Group in Azure called "legacy-resource-group". Inside that I have an existing VNet called "legacy-vnet" (10.0.0.0/16 CIDR) and a default subnet (10.0.0.0/24 CIDR).

 

VNet

image1.jpg

Subnet

image2.jpg

 

If I try to create a new Terraform deployment that adds something to the Resource Group it will be unsuccessful as Terraform did not create the group to start with, so it has no reference in its state file. It will show an output like this:

 

 

 

 

 

 

 

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

 

 

 

 

 

 

 

What I want to do is import the resource group into an existing Terraform State file I have located in Azure Storage so that I can then manage the resource located within. 

 

Let's Start

In the example I am going to use the Azure Cloud Shell simply because it already has Terraform available, but you can obviously do this from your local machine using AZ CLI, Terraform or even VSCode. As we are going to use Azure Cloud Shell we will be using Vim to create our TF files, so if you are not fully up to speed on Vim you can find a great reference sheet here.

 

Step 1

We need to gather the resourceid of a legacy-resource-group, to do this we can gather the information from the properties section of the Resource Group blade, or we can type into the shell the following command:

 

 

 

 

 

 

az group show --name legacy-resource-group --query id --output tsv

 

 

 

 

 

 

 

This will output the resourceid in the following format:

 

 

 

 

 

 

/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group

 

 

 

 

 

 

 

Take a note of the resourceid as we will use it in a few steps.

 

Step 2

Now, we need to create a new Terraform file called import.tf. In a none shared state situation, we would only need to add a single line shown below:

 

 

 

 

 

 

resource "azurerm_resource_group" "legacy-resource-group" {}

 

 

 

 

 

 

 

However, as we are using a shared state, we need to add a few things. Lets first create our new file using the following command:

 

 

 

 

 

 

vi import.tf

 

 

 

 

 

 

 

We can then (using our commands guide sheet if you need it) add the following lines of code:

 

 

 

 

 

 

provider "azurerm" { version = "~>2.0" features {} } # This will be specific to your own Terraform State in Azure storage terraform { backend "azurerm" { resource_group_name = "tstate" storage_account_name = "tstateXXXXX" container_name = "tstate" key = "terraform.tfstate" } } resource "azurerm_resource_group" "legacy-resource-group" {}

 

 

 

 

 

 

 

Now we need to initiate Terraform in our working directory.

 

 

 

 

 

 

terraform init

 

 

 

 

 

 

 

If successful, Terraform will be configured to use the backend "azurerm" and present the following response:

 

 

 

 

 

 

Initializing the backend... Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins... - Checking for available provider plugins... - Downloading plugin for provider "azurerm" (hashicorp/azurerm) 2.25.0... Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

 

 

 

 

 

 

 

Now that Terraform has been initiated successfully, and the backend set to "azurerm" we can now run the following command to import the legacy-resource-group into the state file:

 

 

 

 

 

 

terraform import azurerm_resource_group.legacy-resource-group /subscriptions/<SUBSCRIPTONID>/resourceGroups/legacy-resource-group

 

 

 

 

 

 

 

If successful, the output should be something like this:

 

 

 

 

 

 

Acquiring state lock. This may take a few moments... azurerm_resource_group.legacy-resource-group: Importing from ID "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group"... azurerm_resource_group.legacy-resource-group: Import prepared! Prepared azurerm_resource_group for import azurerm_resource_group.legacy-resource-group: Refreshing state... [id=/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform. Releasing state lock. This may take a few moments...

 

 

 

 

 

 

 

We can now look at the state file to see how this has been added:

 

 

 

 

 

 

{ "version": 4, "terraform_version": "0.12.25", "serial": 1, "lineage": "cb9a7387-b81b-b83f-af53-fa1855ef63b4", "outputs": {}, "resources": [ { "mode": "managed", "type": "azurerm_resource_group", "name": "legacy-resource-group", "provider": "provider.azurerm", "instances": [ { "schema_version": 0, "attributes": { "id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group", "location": "eastus2", "name": "legacy-resource-group", "tags": {}, "timeouts": { "create": null, "delete": null, "read": null, "update": null } }, "private": "eyJlMmJmYjczMC1lY2FhLTExZTYtOGY4OC0zNDM2M2JjN2M0YzAiOnsiY3JlYXRlIjo1NDAwMDAwMDAwMDAwLCJkZWxldGUiOjU0MDAwMDAwMDAwMDAsInJlYWQiOjMwMDAwMDAwMDAwMCwidXBkYXRlIjo1NDAwMDAwMDAwMDAwfSwic2NoZW1hX3ZlcnNpb24iOiIwIn0=" } ] } ] }

 

 

 

 

 

 

 

As you can see, the state file now has a reference for the legacy-resource-group, but as mentioned in the HashiCorp docs, it does not have the entire configuration.

 

Step 3.

What we are going to do now is to add two new subnets to our legacy-vnet. In this example I have created a new main.tf and renamed my import.tf as import.tfold just for clarity.

 

 

 

 

 

 

provider "azurerm" { version = "~>2.0" features {} } terraform { backend "azurerm" { resource_group_name = "tstate" storage_account_name = "tstateXXXXX" container_name = "tstate" key = "terraform.tfstate" } } resource "azurerm_resource_group" "legacy-resource-group" { name = "legacy-resource-group" location = "East US 2" } resource "azurerm_subnet" "new-subnet1" { name = "new-subnet1" virtual_network_name = "legacy-vnet" resource_group_name = "legacy-resource-group" address_prefixes = ["10.0.1.0/24"] } resource "azurerm_subnet" "new-subnet2" { name = "new-subnet2" virtual_network_name = "legacy-vnet" resource_group_name = "legacy-resource-group" address_prefixes = ["10.0.2.0/24"] }

 

 

 

 

 

 

 

We now use this file with "terraform plan" which should result in an output like this:

 

 

 

 

 

 

Acquiring state lock. This may take a few moments... Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. azurerm_resource_group.legacy-resource-group: Refreshing state... [id=/subscriptions/(SUBSCRIPTIONID>/resourceGroups/legacy-resource-group] ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # azurerm_subnet.new-subnet1 will be created + resource "azurerm_subnet" "new-subnet1" { + address_prefix = (known after apply) + address_prefixes = [ + "10.0.1.0/24", ] + enforce_private_link_endpoint_network_policies = false + enforce_private_link_service_network_policies = false + id = (known after apply) + name = "new-subnet1" + resource_group_name = "legacy-resource-group" + virtual_network_name = "legacy-vnet" } # azurerm_subnet.new-subnet2 will be created + resource "azurerm_subnet" "new-subnet2" { + address_prefix = (known after apply) + address_prefixes = [ + "10.0.2.0/24", ] + enforce_private_link_endpoint_network_policies = false + enforce_private_link_service_network_policies = false + id = (known after apply) + name = "new-subnet2" + resource_group_name = "legacy-resource-group" + virtual_network_name = "legacy-vnet" } Plan: 2 to add, 0 to change, 0 to destroy. ------------------------------------------------------------------------ Note: You didn't specify an "-out" parameter to save this plan, so Terraform can't guarantee that exactly these actions will be performed if "terraform apply" is subsequently run.

 

 

 

 

 

 

 

We can see that the plan is aware that only 2 items are to be added, which is our 2 new subnets. We can now run the following to deploy the configuration:

 

 

 

 

 

 

terraform apply -auto-approve

 

 

 

 

 

 

 

The final result should be similar to the below:

 

 

 

 

 

 

Acquiring state lock. This may take a few moments... azurerm_resource_group.legacy-resource-group: Refreshing state... [id=/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group] azurerm_subnet.new-subnet2: Creating... azurerm_subnet.new-subnet1: Creating... azurerm_subnet.new-subnet1: Creation complete after 1s [id=/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group/providers/Microsoft.Network/virtualNetworks/legacy-vnet/subnets/new-subnet1] azurerm_subnet.new-subnet2: Creation complete after 3s [id=/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group/providers/Microsoft.Network/virtualNetworks/legacy-vnet/subnets/new-subnet2] Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

 

 

 

 

 

 

 

If we now look in our portal, we can see the new subnets have been created.

 

Subnets

image3.jpg

 

If we take a final look at our state file, we can see the new resources have been added:

 

 

 

 

 

 

{ "version": 4, "terraform_version": "0.12.25", "serial": 2, "lineage": "cb9a7387-b81b-b83f-af53-fa1855ef63b4", "outputs": {}, "resources": [ { "mode": "managed", "type": "azurerm_resource_group", "name": "legacy-resource-group", "provider": "provider.azurerm", "instances": [ { "schema_version": 0, "attributes": { "id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group", "location": "eastus2", "name": "legacy-resource-group", "tags": {}, "timeouts": { "create": null, "delete": null, "read": null, "update": null } }, "private": "eyJlMmJmYjczMC1lY2FhLTExZTYtOGY4OC0zNDM2M2JjN2M0YzAiOnsiY3JlYXRlIjo1NDAwMDAwMDAwMDAwLCJkZWxldGUiOjU0MDAwMDAwMDAwMDAsInJlYWQiOjMwMDAwMDAwMDAwMCwidXBkYXRlIjo1NDAwMDAwMDAwMDAwfSwic2NoZW1hX3ZlcnNpb24iOiIwIn0=" } ] }, { "mode": "managed", "type": "azurerm_subnet", "name": "new-subnet1", "provider": "provider.azurerm", "instances": [ { "schema_version": 0, "attributes": { "address_prefix": "10.0.1.0/24", "address_prefixes": [ "10.0.1.0/24" ], "delegation": [], "enforce_private_link_endpoint_network_policies": false, "enforce_private_link_service_network_policies": false, "id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group/providers/Microsoft.Network/virtualNetworks/legacy-vnet/subnets/new-subnet1", "name": "new-subnet1", "resource_group_name": "legacy-resource-group", "service_endpoints": null, "timeouts": null, "virtual_network_name": "legacy-vnet" }, "private": "eyJlMmJmYjczMC1lY2FhLTExZTYtOGY4OC0zNDM2M2JjN2M0YzAiOnsiY3JlYXRlIjoxODAwMDAwMDAwMDAwLCJkZWxldGUiOjE4MDAwMDAwMDAwMDAsInJlYWQiOjMwMDAwMDAwMDAwMCwidXBkYXRlIjoxODAwMDAwMDAwMDAwfX0=" } ] }, { "mode": "managed", "type": "azurerm_subnet", "name": "new-subnet2", "provider": "provider.azurerm", "instances": [ { "schema_version": 0, "attributes": { "address_prefix": "10.0.2.0/24", "address_prefixes": [ "10.0.2.0/24" ], "delegation": [], "enforce_private_link_endpoint_network_policies": false, "enforce_private_link_service_network_policies": false, "id": "/subscriptions/<SUBSCRIPTIONID>/resourceGroups/legacy-resource-group/providers/Microsoft.Network/virtualNetworks/legacy-vnet/subnets/new-subnet2", "name": "new-subnet2", "resource_group_name": "legacy-resource-group", "service_endpoints": null, "timeouts": null, "virtual_network_name": "legacy-vnet" }, "private": "eyJlMmJmYjczMC1lY2FhLTExZTYtOGY4OC0zNDM2M2JjN2M0YzAiOnsiY3JlYXRlIjoxODAwMDAwMDAwMDAwLCJkZWxldGUiOjE4MDAwMDAwMDAwMDAsInJlYWQiOjMwMDAwMDAwMDAwMCwidXBkYXRlIjoxODAwMDAwMDAwMDAwfX0=" } ] } ] }

 

 

 

 

 

 

 

Wrap Up

Let's look at what we have done:

  1. We've taken an existing resource group that was unmanaged in Terraform and added to our state file.
  2. We have then added two new subnets to the VNet, without destroying any existing legacy resources.
  3. We have then confirmed these have been added to the state file.

Ideally, we all want to be able to use "terraform import" to drag the entire config into the state file so that all resources will exist within the configuration, and from what HashiCorp have stated on the docs site this is on the roadmap, but for now this at least allows us to manage legacy resource groups moving forward.

 

 

 

 

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.