Azure governance becomes a very different problem at fleet scale.
Managing a few subscriptions is usually a policy and process problem. Managing thousands of subscriptions across delegated customer environments becomes a platform engineering problem. At that level, the challenge is not simply creating policies, alerts, dashboards, or remediation scripts. The real challenge is keeping every subscription aligned to the same approved governance baseline, detecting drift quickly, and giving operators enough visibility to act with confidence.
This article walks through the architecture and design principles behind a multi-tenant Azure Governance Platform built to manage:
- 5,000 Azure subscriptions
- 29 million Azure resources
- 63 standard governance policies per subscription
- 120 standard alerts per subscription
- 315,000 policy baselines across the fleet
- 600,000 alert baselines across the fleet
- 100 standard users
- 6 platform administrators
- Azure Lighthouse delegated customer environments
- Event-driven resource scanning
- Full policy and alert drift remediation
The platform was designed as both a governance control plane and an automation engine. Its purpose is to maintain a consistent, version-controlled governance state across a large Azure estate while giving operators the ability to detect, understand, and automatically correct drift.
The Governance Problem at Scale
In small Azure environments, governance is often managed through a combination of Azure Policy, Azure Monitor alerts, dashboards, scripts, and manual reviews. That approach works when the number of subscriptions is manageable.
At 5,000 subscriptions, the problem changes.
Each subscription needs a consistent baseline. In this platform, that baseline includes 63 standard policies and 120 standard alerts per subscription. That means the platform is responsible for maintaining roughly 315,000 policy baselines and 600,000 alert baselines across the fleet.
The hard part is not deploying those controls once. The hard part is ensuring they remain consistent over time.
Subscriptions change. Resources are created, updated, renamed, deleted, moved, or reconfigured. Customers operate in their own environments. Administrators make emergency changes. Policy definitions evolve. Alert thresholds change. Remediation logic improves. New standards are introduced.
Without a centralized governance platform, each of those changes can create drift.
At fleet scale, drift is not an edge case. It is the normal operating condition unless the platform is designed to detect and correct it continuously.
Design Goal: Governance as Versioned Fleet State
The core design principle behind the platform is simple:
Governance should be treated as a versioned fleet state, not a one-time deployment activity.
Each subscription is expected to match an approved platform baseline. That baseline includes policy assignments, alert rules, automation configuration, monitored resource metadata, and remediation state.
The platform continuously compares actual state against expected state. When it finds drift, it can report it, track it, and automatically correct it.
This creates a control loop:
- Define the approved governance baseline.
- Discover actual state across subscriptions.
- Compare actual state to expected state.
- Detect drift.
- Queue remediation.
- Apply corrections.
- Record the result.
- Repeat continuously.
This approach turns governance into an operational system rather than a static compliance checklist.
Reference Architecture
The platform uses a layered Azure architecture designed for security, scale, isolation, and operational control.
At a high level, the architecture includes:
- React, Vite, and Tailwind CSS frontend
- Azure Application Gateway with Web Application Firewall
- Azure API Management as the API policy layer
- Azure App Service Environment v3 for isolated hosting
- Azure Functions for backend APIs and automation
- Durable Functions for orchestration
- Queue-triggered Functions for asynchronous work
- Timer-triggered Functions for scheduled reconciliation
- PostgreSQL Flexible Server as the metadata and state store
- PgBouncer for connection pooling
- Azure Service Bus for automation fan-out
- Azure Key Vault for secrets and connection strings
- Application Insights and Log Analytics for telemetry
- GitHub Actions with self-hosted runners for CI/CD
The platform is separated into development and production environments, with the same architectural pattern used across both. This allows changes to be validated before being promoted to production.
Front Door and Security Layer
User traffic enters through public DNS and routes into Azure Application Gateway with WAF enabled.
The WAF provides the first layer of protection for the platform. It enforces managed rules, custom rules, and front-door protections before requests ever reach the application layer.
From there, traffic is routed to either the frontend application or API layer depending on the request path.
The frontend is hosted on Azure App Service Environment v3, providing an isolated hosting boundary for the React application. API traffic is routed through Azure API Management, which acts as the centralized policy enforcement point for backend services.
API Management as the Policy Enforcement Layer
Azure API Management plays a critical role in the architecture.
Rather than allowing clients to call backend Functions directly, all API traffic flows through APIM. This creates a single place to enforce API-level controls such as:
- JWT validation
- Authorization checks
- RBAC enforcement
- CORS policy
- Header validation
- Route-level policy enforcement
- Backend routing
- Request shaping
- API versioning
This is especially important in a multi-tenant governance platform because access control must be consistent and centralized.
APIM acts as the contract boundary between the frontend and backend orchestration layer. The frontend does not need to understand the details of backend workflows, queueing, remediation, or orchestration. It calls governed APIs, and APIM ensures those requests meet the required security and policy conditions.
Backend Orchestration with Azure Functions
The backend is built with Azure Functions using multiple trigger patterns:
- HTTP-triggered Functions for API requests
- Durable Functions for long-running orchestration
- Queue-triggered Functions for asynchronous processing
- Timer-triggered Functions for scheduled reconciliation
This combination gives the platform flexibility.
HTTP-triggered Functions handle direct API interactions from the portal. Durable Functions coordinate workflows that require multiple steps, retries, state transitions, or fan-out/fan-in patterns. Queue-triggered Functions process background jobs without tying execution to a user request. Timer-triggered Functions ensure periodic governance reconciliation happens even when no user action occurs.
This is important because governance workflows are often long-running. A single user action may require the platform to inspect subscriptions, query resource state, evaluate policy drift, update metadata, enqueue remediation, and track completion.
Those workflows should not run synchronously behind a browser request.
By separating request handling from orchestration and execution, the platform remains responsive while still supporting large-scale automation.
PostgreSQL as the Governance State Store
PostgreSQL Flexible Server acts as the central metadata and state store for the platform.
The database stores the information needed to understand and operate the fleet, including:
- Customer metadata
- Subscription metadata
- Monitored Azure resources
- Policy versions
- Alert baselines
- Deployed governance state
- Resource change state
- Scan results
- Drift records
- Remediation jobs
- Automation status
- Operational history
- Reporting data
This rich dataset is what allows the platform to move beyond simple deployment automation.
Because the platform stores both expected state and observed state, it can answer important operational questions:
- Which subscriptions are out of compliance?
- Which policy versions are deployed?
- Which alert rules are missing?
- Which resources changed recently?
- Which remediation jobs succeeded?
- Which remediation jobs failed?
- Which customer environments are drifting most often?
- Which resources are generating the most operational noise?
At 29 million Azure resources, the database is not just a persistence layer. It is the analytical foundation for governance operations.
PgBouncer is used to protect PostgreSQL from connection pressure created by serverless workloads. This matters because Azure Functions can scale quickly, and without connection pooling, bursty workloads can overwhelm the database with too many concurrent connections.
Service Bus for Automation Fan-Out
Azure Service Bus is used as the asynchronous automation backbone.
This allows the platform to decouple user actions and scheduled scans from the actual remediation work. Instead of forcing a user request to wait for every downstream operation to complete, the backend can publish work to Service Bus and allow workers to process jobs independently.
This pattern provides several advantages:
- Better reliability
- Safer retry handling
- Improved workload isolation
- Reduced frontend and API timeout risk
- Controlled fan-out across thousands of subscriptions
- Support for dead-letter handling
- Better operational visibility into queued work
For a governance platform managing thousands of subscriptions, queue-based processing is essential. It allows the platform to absorb bursts of change events, distribute work safely, and prevent long-running automation from blocking the user experience.
Event-Driven Resource Scanning
The platform scans resources when Azure resources change.
This event-driven model allows the governance system to react closer to real time. When a resource is created, changed, or removed, the platform can update its resource metadata and determine whether that change affects policy, alerting, or remediation state.
However, event-driven systems should not be the only control mechanism.
Events can be delayed. Events can be missed. External systems can fail. Permissions can change. Subscriptions can be onboarded or modified outside the normal process.
For that reason, the platform also performs scheduled policy reconciliation every three days. This gives the architecture both speed and correctness.
The event-driven path allows the system to respond quickly. The scheduled reconciliation path ensures the fleet eventually returns to the approved baseline even if individual changes are missed.
This hybrid model is one of the most important design decisions in the platform.
Drift Detection and Auto-Remediation
The platform was designed not only to detect drift, but to automatically fix it.
Drift can occur when a subscription no longer matches the approved governance baseline. Examples include:
- Missing policy assignments
- Incorrect policy versions
- Missing alert rules
- Modified alert thresholds
- Disabled monitoring configuration
- Missing automation wiring
- Resources that should be monitored but are not
- Subscriptions that are partially onboarded
- Failed or incomplete remediation jobs
When drift is detected, the platform records the state, determines the required correction, and can queue remediation automatically.
This turns governance into a closed-loop system.
The goal is not just to notify an operator that something is wrong. The goal is to restore the environment to the approved state with as little manual intervention as possible.
Admins still need visibility and control, but they should not have to manually repair the same categories of drift across thousands of subscriptions.
User and Admin Model
The platform supports a focused operational user base:
- 6 platform administrators
- 100 standard users
Admins are responsible for managing platform configuration, governance baselines, remediation workflows, and operational oversight.
Standard users interact with the platform through controlled views and workflows. They can review governance state, inspect resources, evaluate drift, and understand remediation outcomes based on their access level.
This separation is important because large-scale governance platforms need both usability and control. Not every user should be able to modify baselines or trigger remediation across customer environments.
Role-based access helps ensure that users can access the information they need without exposing sensitive operational controls unnecessarily.
Multi-Tenant Azure Lighthouse Model
The platform is designed for Azure Lighthouse-based delegated customer environments.
Azure Lighthouse enables centralized management across customer tenants while allowing customers to retain ownership of their subscriptions. This model is powerful, but it also introduces architectural requirements.
A platform operating across delegated environments must handle:
- Tenant boundaries
- Subscription-level authorization
- Customer isolation
- Scalable resource discovery
- Consistent onboarding
- Reliable offboarding
- Delegated access changes
- Cross-tenant operational visibility
The governance platform abstracts much of this complexity from users. Operators interact with a centralized portal and workflow engine, while the backend handles the details of delegated subscription access, metadata synchronization, policy state, and remediation execution.
CI/CD and Environment Consistency
The platform uses GitHub Actions with self-hosted runners to deploy infrastructure, backend services, frontend assets, API Management configuration, and full environment updates.
This is important because the governance platform itself must follow the same principles it enforces.
If the platform is responsible for maintaining consistency across thousands of subscriptions, then its own deployment process also needs to be consistent, repeatable, and auditable.
The CI/CD model supports:
- Infrastructure deployment
- Backend deployment
- Frontend deployment
- API Management deployment
- Full environment promotion
- Dev-to-prod validation
- Repeatable release workflows
This gives the team a controlled way to evolve the platform while reducing the risk of configuration drift between environments.
Observability and Operational Telemetry
Application Insights and Log Analytics provide the telemetry layer for the platform.
Observability is not an optional feature in a governance system. It is part of the control plane.
The platform needs to know:
- Which APIs are being called
- Which workflows are running
- Which scans are succeeding
- Which scans are failing
- Which remediation jobs are queued
- Which remediation jobs are stuck
- Which subscriptions are drifting
- Which errors are recurring
- Which integrations are unhealthy
- Which deployments changed platform behavior
Without telemetry, the platform would be blind to its own operational state.
For a fleet-scale governance system, observability must be designed into the architecture from the beginning.
Why This Architecture Works
This architecture works because each layer has a clear responsibility.
The frontend provides the user experience.
Application Gateway and WAF protect the ingress path.
API Management enforces API policy and access control.
Azure Functions handle backend execution.
Durable Functions coordinate long-running workflows.
Service Bus decouples automation from user requests.
PostgreSQL stores the governance state and operational history.
PgBouncer protects the database from bursty serverless connection patterns.
Key Vault secures secrets.
Application Insights and Log Analytics provide visibility.
GitHub Actions provides repeatable delivery.
The result is a platform that can operate across thousands of subscriptions without relying on manual processes, one-off scripts, or disconnected operational views.
Lessons Learned
The biggest lesson is that governance at scale is a state-management problem.
Policies and alerts are only part of the solution. The real challenge is knowing what should exist, what actually exists, what version is deployed, what changed, what drifted, and what needs to be corrected.
Another lesson is that event-driven scanning must be paired with scheduled reconciliation. Events provide speed, but reconciliation provides confidence.
API Management also proved valuable as a centralized enforcement layer. In a multi-tenant platform, backend APIs should not be exposed without a consistent security, authorization, and policy boundary.
The database design matters more than many teams expect. Once a platform manages millions of resources, metadata becomes a strategic asset. A rich operational dataset allows the platform to support reporting, remediation, trend analysis, and future intelligence.
Finally, auto-remediation must be designed carefully. The goal is not reckless automation. The goal is controlled, observable, repeatable correction of known drift patterns.
Thought Leadership: The Future of Cloud Governance Is Autonomous Operations
Cloud governance is often treated as a compliance function. But at scale, governance becomes an operations problem.
The future of governance is not a static dashboard showing what is wrong. The future is an autonomous control plane that can detect changes, understand expected state, identify drift, and safely repair known issues.
That does not remove the need for human operators. It changes their role.
Instead of manually checking policy assignments or rebuilding alerts, operators can focus on exceptions, standards, platform evolution, and risk decisions. The platform handles repetitive enforcement. Humans handle judgment.
This is where cloud governance is heading:
- From manual reviews to continuous evaluation
- From static policies to versioned baselines
- From disconnected scripts to orchestrated workflows
- From alert fatigue to actionable drift detection
- From reactive operations to automated remediation
- From subscription-by-subscription management to fleet-level control
At small scale, governance can be a checklist.
At large scale, governance must be a platform.
Conclusion
Building a governance platform for 5,000 subscriptions and 29 million Azure resources requires more than policy deployment. It requires a secure architecture, a reliable state model, event-driven scanning, scheduled reconciliation, asynchronous automation, and controlled remediation.
The platform described here was built around a simple idea: every subscription should continuously converge back to the approved governance baseline.
That means every policy, alert, resource state, drift record, and remediation action becomes part of a larger operational control loop.
This is the difference between deploying governance and operating governance.
At fleet scale, that difference matters.