Azurestack ASDK - Error: determining environment: unable to locate metadata for environment "stack" from custom metadata host "management.local.azurestack.external"

[matthorgan]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureStack Provider) Version

Terraform v1.2.6
azurestack v1.0.0

Affected Resource(s)

• azurestack_resource_group

Terraform Configuration Files

provider "azurestack" {
  metadata_host = "management.local.azurestack.external"
  features {}
}

# Create a resource group
resource "azurestack_resource_group" "test" {
  name     = "test-rg"
  location = "local"
}

Debug Output

https://gist.github.com/matthorgan/27fecca7ee03b3e4ce8efaaf85479658

Expected Behaviour

A resource group should be created

Actual Behaviour

Error before RG is created

Steps to Reproduce

1. terraform apply

Important Factoids

Attempting to run Terraform on my ASDK environment produces the error Error: determining environment: unable to locate metadata for environment "stack" from custom metadata host "management.local.azurestack.external". I am using the az cli for authentication and have confirmed I can create resources via the az cli so the permissions are correct.

[TheBlackMini]

The provider seems to be ignoring "environment" / "ARM_ENVIRONMENT"

[TheOnlyWei]

@matthorgan I was able to get a successful TestAccResourceGroup_basic test (from this repository) using Terraform 1.3.6 and hashicorp/azurestack 1.0.0.

go test -run TestAccResourceGroup_basic
PASS
ok      github.com/hashicorp/terraform-provider-azurestack/internal/services/resource   63.813s

Make sure to follow the README and verify the environment variables are set up correctly. The ARM_METADATA_HOSTNAME environment variable was changed to ARM_METADATA_HOST in this commit in July:
378e115

I set these environment variables in PowerShell (values are left empty for demonstration):

[System.Environment]::SetEnvironmentVariable('ARM_CLIENT_ID','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_CLIENT_SECRET','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_SUBSCRIPTION_ID','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_TENANT_ID','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_TEST_LOCATION','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_TEST_LOCATION_ALT','', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_TEST_LOCATION_ALT2','', 'Machine')
[System.Environment]::SetEnvironmentVariable('TF_ACC','1', 'Machine')
[System.Environment]::SetEnvironmentVariable('ARM_METADATA_HOST','', 'Machine') // REMOVE HTTPS/HTTP
[System.Environment]::SetEnvironmentVariable('ARM_ENVIRONMENT','', 'Machine')

[sebastus]

Was there ever a solution to this issue? I'm attempting to use the Terraform provider for Azure Stack just now for the first time and ran into this. I have curled that endpoint and there's nothing about environment in it. Is the message giving the wrong impression?

[sebastus]

Solution:
The metadata host endpoint is queried:
"https://%s/metadata/endpoints?api-version=2020-06-01"

It yields output similar to this:

[
 {
  "portal":"https://portal.local.azurestack.external/",
  "authentication":{
   "loginEndpoint":"https://login.microsoftonline.com/",
   "audiences":[
    "https://management.azurestacktenant.onmicrosoft.com/1896984d-3500-435a-9eb3-5f4580e945c3"
   ]
  },
  "graphAudience":"https://graph.windows.net/",
  "graph":"https://graph.windows.net/",
  "name":"AzureStack-User-1896984d-3500-435a-9eb3-5f4580e945c3",
  "suffixes":{
   "keyVaultDns":"vault.local.azurestack.external",
   "storage":"local.azurestack.external"
  },
  "gallery":"https://providers.azurestack.local:30016/"
 }
]

In some version of the api, it most likely returned a name like "stack" rather than the "AzureStack-User-xxx" name. It's this name that's needed in the ARM_ENVIRONMENT variable.

[matthorgan]

Solution: The metadata host endpoint is queried: "https://%s/metadata/endpoints?api-version=2020-06-01"

It yields output similar to this:

[
 {
  "portal":"https://portal.local.azurestack.external/",
  "authentication":{
   "loginEndpoint":"https://login.microsoftonline.com/",
   "audiences":[
    "https://management.azurestacktenant.onmicrosoft.com/1896984d-3500-435a-9eb3-5f4580e945c3"
   ]
  },
  "graphAudience":"https://graph.windows.net/",
  "graph":"https://graph.windows.net/",
  "name":"AzureStack-User-1896984d-3500-435a-9eb3-5f4580e945c3",
  "suffixes":{
   "keyVaultDns":"vault.local.azurestack.external",
   "storage":"local.azurestack.external"
  },
  "gallery":"https://providers.azurestack.local:30016/"
 }
]

In some version of the api, it most likely returned a name like "stack" rather than the "AzureStack-User-xxx" name. It's this name that's needed in the ARM_ENVIRONMENT variable.

Nice one @sebastus. I didn't get a chance to come back to this as I'm not using Azure Stack anymore but this is useful to know, thanks 👍

[mwadman]

I ran into this issue today as well, using Terraform version 1.4.0, AzureStack provider version 1.0.0 and Azure CLI version 2.29.2.

I tried setting environment or ARM_ENVIRONMENT but this seems to be ignored as @TheBlackMini noted.
If these aren't set then this provider seems to take the name of the region/stack as configured when running az cloud register, but then trimming the word "Azure" from the front if it is present.
For example, if you register the region with az cloud register --name "AzureStack" (as the provider documentation suggests) and then set this to be used, this provider will attempt to connect to the environment named "Stack", which doesn't exist according to the metadata endpoint.
Because of this behaviour, we were able to successfully connect with this provider by registering the region with something similar to az cloud register --name "AzureAzureStack-User-1896984d-3500-435a-9eb3-5f4580e945c3", using the ID from the above example and prepending with an additional "Azure".