google_compute_router - Provider Version >=5.32.0 causes permadiff for advertised_ip_ranges

[bahag-klickst]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.8.5
on darwin_arm64 | amd64

• provider registry.terraform.io/hashicorp/google v5.32.0 (and greater)
• provider registry.terraform.io/hashicorp/google-beta v5.32.0 (and greater)

Affected Resource(s)

google_compute_router

Terraform Configuration

variable "cloud_router" {
  description = "List of cloud router"
  default     = []
  type = list(object({
    name               = string
    project_id         = string
    vpc_name           = string
    region             = optional(string, "europe-west1")
    asn                = optional(number)
    advertise_mode     = optional(string, "DEFAULT")
    advertised_groups  = optional(list(string), [])
    keepalive_interval = optional(number, 20) # must be equal or greater than 20
    advertised_ip_ranges = optional(list(object({
      range       = string
      description = string
    })), [])
  }))
}

cloud_router = [
  {
    name              = "xxxx"
    project_id        = "xxxx"
    vpc_name          = "xxxx"
    asn               = 64999
    advertise_mode    = "CUSTOM"
    advertised_ip_ranges = [
      {
        range       = "10.122.7.0/24"
        description = "Network 1"
      },
      {
        range       = "10.122.8.0/24"
        description = "Network 2"
      },
      {
        range       = "10.22.25.96/28"
        description = "Network 3"
      },
      {
        range       = "10.22.18.120/29"
        description = "Network 4"
      },
    ]
  },
]

resource "google_compute_router" "gcp_router" {
  for_each   = { for k, v in var.cloud_router : v.name => v }
  name       = each.key
  network    = google_compute_network.vpc_network[each.value.vpc_name].name
  region     = each.value.region
  project    = xxxx
  dynamic "bgp" {
    for_each = each.value.asn != null ? { 1 = 1 } : {}
    content {
      asn                = each.value.asn
      advertise_mode     = each.value.advertise_mode
      advertised_groups  = each.value.advertise_mode == "CUSTOM" ? each.value.advertised_groups : null
      keepalive_interval = each.value.keepalive_interval
      dynamic "advertised_ip_ranges" {
        for_each = { for k, v in each.value.advertised_ip_ranges : k => v if each.value.advertise_mode == "CUSTOM" }
        content {
          range       = advertised_ip_ranges.value.range
          description = advertised_ip_ranges.value.description
        }
      }
    }
  }
}

Debug Output

# google_compute_router.gcp_router["xxxx"] will be updated in-place
  ~ resource "google_compute_router" "gcp_router" {
        id                            = "projects/xxxx/regions/europe-west1/routers/xxxx"
        name                          = "xxxx"
        # (7 unchanged attributes hidden)

      ~ bgp {
            # (5 unchanged attributes hidden)

          ~ advertised_ip_ranges {
              ~ description = "Network 4" -> "Network 3"
              ~ range       = "10.22.18.120/29" -> "10.22.25.96/28"
            }
          ~ advertised_ip_ranges {
              ~ description = "Network 3" -> "Network 4"
              ~ range       = "10.22.25.96/28" -> "10.22.18.120/29"
            }

            # (2 unchanged blocks hidden)
        }
    }

Expected Behavior

Preserve the order, as it was before v5.32.0 no diff was shown as the order was preserved

Actual Behavior

Permadiff is shown even after appy, maybe due to the fix introduced as part of #9353 / #18228

Steps to reproduce

1. terraform apply

Important Factoids

No response

References

No response

b/348509494

[LucaPrete]

Managed to reproduce the issue. This is a simplified configuration that triggers the perma diff:

resource "google_compute_router" "gcp_router" {
  name    = "router"
  project = "PROJECT"
  network = "https://www.googleapis.com/compute/v1/projects/PROJECT/global/networks/VPC_NAME"
  region  = "REGION"

  bgp {
    asn                = 64512
    advertise_mode     = "CUSTOM"
    keepalive_interval = 30

    advertised_ip_ranges {
      range       = "192.168.2.0/24"
      description = "desc1"
    }

    advertised_ip_ranges {
      range       = "192.168.1.0/24"
      description = "desc1"
    }
  }
}

FYI perma diff doesn't happen if advertised IP ranges are put in order.

@SarahFrench @bahag-klickst I'll need to look again at hashicorp/terraform-provider-google/pull/18228.
I'll be OOO starting tomorrow for a week. If nobody has time to look this before, I'll take it as soon as I'll be back.

[bahag-klickst]

Hej @LucaPrete, @SarahFrench
Thanks for taking care and reproducing.
Just a small hint from my side.
as you can see in my code example, we are building the dynamic block based on

for_each = { for k, v in each.value.advertised_ip_ranges : k => v if each.value.advertise_mode == "CUSTOM" }

which means we use a list as input

advertised_ip_ranges = optional(list(object({
  range       = string
  description = string
})), [])

and convert it to a map, so basically our assumption was, that the order in the list defined in the tfvars which we converted to a map would be kept.

Best Regards,
Tim

[SarahFrench]

It may be that the approach used in that that PR (reordering to match config) isn't compatible with dynamic blocks- writing an acceptance test with dynamic blocks would be a good starting point for solving this

[ggtisc]

Confirmed issue!

As @LucaPrete describes the permadiff can only be replicated if advertised_ip_ranges are put in disorder. You could take the example code he shared to confirm the issue.

[WilliamDuncanson]

Can reproduce.

[SarahFrench]

Closed by GoogleCloudPlatform/magic-modules#11262

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.