Throttled by Design: The Quota and Rate-Limit Lab

Rate limiting is a failure mode you usually cannot summon. To test how your system handles 429 responses, you traditionally had to actually exhaust some external service’s quota, which is slow, expensive, and rude to whoever owns that quota. The quota lab sidesteps all of that. It builds an API that throttles itself, emitting deterministic 429 responses on command, with no external dependency to abuse.

What the lab does

It runs an App Service API with internal throttling controls. The Logic App controller can tighten the quota so requests start getting 429, or switch on a burst mode that drives a wave of traffic to trip the limit naturally. Because the throttling is internal and deterministic, you get the same behavior every run, which is exactly what you want when teaching a team to recognize and handle rate limits.

The inject contract

{ "scenario": "throttle", "durationSeconds": 300 }

The scenarios are throttle (lower the quota so callers hit 429), burst (drive a wave of traffic into the limit), and recover. The throttle mode shows you the steady-state of a throttled service. The burst mode shows you what a real traffic spike against a tight quota looks like.

From POST to symptom

sequenceDiagram participant Op as Operator participant LA as Logic App participant APP as Quota API participant C as Caller participant M as Azure Monitor Op->>LA: POST throttle LA->>APP: Reduce quota C->>APP: Requests APP-->>C: 429 Too Many Requests APP-->>M: 429 rate climbs M->>M: Scheduled query 429 count over threshold Note over C: Retries can inflate response time Op->>LA: POST recover LA->>APP: Restore normal quota

The signal that proves it

A scheduled query counts 429 responses specifically, not generic 4xx, so the alert points straight at throttling.

AppServiceHTTPLogs
| where TimeGenerated > ago(5m)
| where _ResourceId =~ "<quota-app-resource-id>"
| where ScStatus == 429
| summarize FailureCount = count() by _ResourceId
| where FailureCount > 5

There is a second-order effect worth watching alongside the 429 count: response time can climb when callers retry aggressively. A naive retry loop turns a throttling event into a load event, which is its own small lesson about how clients can amplify the very problem they are reacting to.

Recovery

recover restores normal quota, the 429s stop, and any retry-driven latency settles. Baseline is callers getting 200, no 429 in the logs, and response time back to normal.

The lesson

429 is not an error in the way a 500 is, and treating it like one leads to bad client behavior. A 500 means something broke. A 429 means slow down. The right response is backoff, not a tight retry, and the quota lab makes it cheap to see what happens when a client gets that wrong: the retries pile on, the latency climbs, and a temporary limit becomes a self-sustaining load problem. Alert on 429 distinctly from other 4xx so you can recognize throttling on sight, and use the lab to confirm your clients back off like they should.

Next: The Certificate You Stopped Trusting, the TLS lab.