-
Notifications
You must be signed in to change notification settings - Fork 34
Description
I'm trying to setup the AKS extension in Azure Gov. I installed the extension as outlined in the document. I started going through the example to test functionality. Can't get passed the online-endpoint. Digging through logs I found a bunch of auth errors and the configMap for amlarc-identity-config has an audience listed as https://management.azure.com/ which will give auth errors. I tried manually changing the configMap and deleting the pod but the same audience was used.
I tried using audience=https://management.usgovcloudapi.net/ in the extension installation. no luck
I asked Microsoft Support and they told me to use --configuration-settings identity.audience=https://management.usgovcloudapi.net/
Either way the config is making to the extensionconfigs object in the cluster but do not seem to apply to the helm chart for deployment.
Command used.
az k8s-extension create -n azureml -g resourcegroup --cluster-name mycluster -t managedClusters --extension-type Microsoft.AzureML.Kubernetes
--configuration-settings enableTraining=True enableInference=True inferenceRouterServiceType=LoadBalancer allowInsecureConnections=True InferenceRouterHA=False internalLoadBalancerProvider=azure nodeSelector.AppSet=App1 identity.audience=https://management.usgovcloudapi.net/
This is the yaml for the identity config map. kind: ConfigMap
apiVersion: v1
metadata:
name: amlarc-identity-config
namespace: azureml
resourceVersion: '78613296'
creationTimestamp: '2025-06-10T15:16:21Z'
labels:
amlarc-component: identity
app.kubernetes.io/managed-by: Helm
annotations:
meta.helm.sh/release-name: azureml
meta.helm.sh/release-namespace: azureml
data:
config: |-
sidecar:
image: mcr.microsoft.com/azureml/amlarc/docker/identity-sidecar:1.1.66
listenPort: "9999"
dialAddr: https://amlarc-identity-proxy-service.azureml.svc.cluster.local
tokenTTL: "24h"
servePath: "/token"
controller:
maxConcurrentReconciles: 1
resyncPeriod: 0
webhookLabelKey: "ml.azure.com/identity"
automountServiceAccountToken: true
tokenExchangeRetrySleepDuration: "30s"
tokenExchangeTotalWaitingDuration: "10m"
cache:
ttl: "10m"
tls:
ca: "/etc/identity/certs/ca.crt"
certificates:
- keyPath: "/etc/identity/certs/tls.key"
certPath: "/etc/identity/certs/tls.crt"
proxy:
port: 9999
servePath: "/token"
remoteHost: https://usgovvirginia.api.ml.azure.us
extensionIdentity:
namespace: "azureml"
name: "azureml"
audience: https://management.azure.com/
resourceID: "azureml"
secretName: "amlarc-arc-extension-token-secret" <= this is not populating in the cluster.
secretKey: "token"
mountDir: "/etc/identity/arc"
enabled: true
clusterType: "AKS"
clientID: "4fb95352-xxxxxxxxxxxxx"