The Secret That Vanished: The Auth and Key Vault Lab

Identity failures are masters of disguise. A managed identity loses access to Key Vault, or a secret gets rotated and the old version disappears, and the application starts throwing 500s that look exactly like a code bug. The team digs through application logs for a defect that does not exist, because the real problem is an authorization change two layers down. The auth lab makes that failure reproducible so you can learn to recognize its fingerprint.

What the lab does

It runs an App Service with a managed identity that reads a secret from a dedicated Key Vault. The Logic App controller breaks the secret path three ways: it points the app at a secret name that does not exist, points it at a missing version, or removes the identity’s permission to read the vault at all. Each one stops the app from resolving its secret, and each one fails in a slightly different place.

The inject contract

{ "scenario": "missing-secret", "durationSeconds": 300 }

The scenarios are missing-secret (the app looks for a name that is not there), missing-version (the name exists but the requested version does not), rbac-remove (the identity loses Key Vault read permission), and recover. The first two are application-config failures. The third is an infrastructure-grade permission failure, and it is the one most likely to send a team down the wrong path, because nothing in the app changed.

From POST to symptom

sequenceDiagram participant Op as Operator participant LA as Logic App participant APP as App Service participant KV as Key Vault participant M as Azure Monitor Op->>LA: POST rbac-remove LA->>KV: Remove identity read permission APP->>KV: Read secret KV-->>APP: Authorization failed APP-->>M: App returns 500 Note over APP: No deployment, no code change Op->>LA: POST recover LA->>KV: Restore permission and app config

The signal that proves it

The app health collapses to 500, caught by the standard 5xx scheduled query.

AppServiceHTTPLogs
| where TimeGenerated > ago(5m)
| where _ResourceId =~ "<auth-webapp-resource-id>"
| where ScStatus >= 500
| summarize FailureCount = count() by _ResourceId
| where FailureCount > 5

The 5xx alert tells you the app is unhealthy. The thing that separates a fast recovery from a slow one is reading the application logs alongside it, where the Key Vault lookup or authorization failure is named explicitly. That log line is the fingerprint. Once you have seen it in a drill, you recognize it instantly in a real incident instead of suspecting your own code first.

Recovery

recover restores both the app’s secret configuration and the identity’s Key Vault access in one call. The app resolves its secret again and returns 200. Baseline is a healthy app and a managed identity that can read the vault.

The lesson

When an app starts failing and nothing was deployed, suspect identity and configuration before you suspect code. Secret and permission failures present as generic 500s, which is why they waste so much incident time: the symptom points at the application, but the cause is in the access plane. The defense is to make the underlying failure legible, log the Key Vault error clearly, and rehearse the scenario so the shape of an auth failure is already familiar when it shows up unannounced.

Next: The Database Said No, the PostgreSQL lab.