Azure deployment of the fakes estate
This repository is an azd project. It deploys the fakes workload into an existing application landing zone. It does not create resource groups.
Landing-zone source of truth: calab-ai/azure-platform-landing-zone bicep/80-landingzones (ADR 0011, PLA-67). Canonical names for this estate: infra/lz-defaults.json.
Parent issue: MOC-4. Children: MOC-12 (HTTP / Container Apps), MOC-14 (Azure SQL + CI seed).
Terms this page uses
| Term | What it means here | Longer explanation |
|---|---|---|
| azd | Azure Developer CLI. An operator runs azd up once to bootstrap: provision, then SQL grants and seed, then the QDox deploy. Later CD runs azd provision, then azd deploy, then SQL grants and seed. A docs-only push does not. |
Install azd |
| Application landing zone | Resource groups that already exist, created by azure-platform-landing-zone. This repo deploys the fakes into them. It does not create the groups. |
Azure landing zones |
| Azure Container Apps | Hosts the QDox HTTP fake. Consumption plan, scale-to-zero. The first mention below abbreviates this to ACA. | Container Apps overview |
| Federated credential | A trust on the GitHub Organization app so Actions can sign in to Azure with OIDC. No client secret. | Workload identity federation |
Azure Container Apps (ACA) dynamic sessions are explicitly out of scope: they have no stable hostname, so consumers cannot pin connection config.
Landing-zone resource groups
80-landingzones names RGs with ADR 0011 D2. instance is not part of RG names; it is used for the spoke VNet and for workload resource names this repo creates.
rg-<workloadAbbr>-<envAbbr>-<regionAbbr>-<purpose>
purpose β {core, network, mgmt, data}
spoke VNet: vnet-<workloadAbbr>-<envAbbr>-<regionAbbr>-<instance>
Mocks landing zone parameters:
| Param | Value |
|---|---|
workloadAbbr |
mock |
workloadName |
app-mocks |
envAbbr |
all |
envName |
shared |
location |
australiaeast |
regionAbbr |
aue |
instance |
01 |
| subscription | c4ccdf9e-39c3-4fe0-ba13-51cca77439fd |
Exact names this estate looks up:
| Purpose | Name | App-LZ-team RBAC | Who owns contents | This repo |
|---|---|---|---|---|
| core | rg-mock-all-aue-core |
Contributor | Workload | Deploys CAE, QDox Container App, ACR, Key Vault (app secrets), UAMIs, Azure SQL sql-mock-all-aue-01 + edw, and a local Log Analytics workspace for CAE logs (log-mock-all-aue-01). |
| network | rg-mock-all-aue-network |
Reader | Landing zone | Looked up only (spoke vnet-mock-all-aue-01). Do not recreate VNet/NSGs/PEs. CAE is Consumption, not VNet-injected. |
| mgmt | rg-mock-all-aue-mgmt |
Reader | LZ / platform | Not written. App team is Reader; do not create LAW here. |
| data | rg-mock-all-aue-data |
Contributor | LZ contract | Looked up as existing. Empty for now. Reserved for Cosmos / AI Search / Storage (allowedNamespaces=Microsoft.DocumentDB|Microsoft.Search|Microsoft.Storage). Do not put Azure SQL here β Microsoft.Sql is not in that list. |
azd / GitHub defaults for this estate:
| Variable | Value |
|---|---|
AZURE_SUBSCRIPTION_ID |
c4ccdf9e-39c3-4fe0-ba13-51cca77439fd (CD maps this from org var AZURE_INTERNAL_SUBSCRIPTION_ID) |
AZURE_LOCATION |
australiaeast (hardcoded in CD; LZ location) |
AZURE_ENV_NAME |
shared (hardcoded in CD; not a GitHub variable) |
AZURE_RESOURCE_GROUP |
rg-mock-all-aue-core |
SQL_RESOURCE_GROUP |
rg-mock-all-aue-core (same RG; SQL is not in data) |
Workload resource names (CAF-ish, namePrefix = mock-all-aue-01):
| Resource | Name |
|---|---|
| Container Apps Environment | cae-mock-all-aue-01 |
| QDox Container App | ca-qdox-mock-all-aue-01 |
| Key Vault | kv-mock-all-aue-01 |
| ACR | acrmockallaue01 |
| SQL logical server | sql-mock-all-aue-01 |
| QDox UAMI | id-qdox-mock-all-aue-01 |
| CI UAMI | id-ci-mock-all-aue-01 |
| Local Log Analytics (CAE logs) | log-mock-all-aue-01 |
The LZ does not create CAE, Key Vault, ACR, SQL, or identities. Those are workload-owned and live in this repo, all in core. CAE, KV, ACR, and SQL stay public for now (no private-endpoint redesign in this estate).
Override the LZ identity of this estate (unusual) with azd env vars LZ_WORKLOAD_ABBR, LZ_WORKLOAD_NAME, LZ_ENV_ABBR, LZ_ENV_NAME, LZ_REGION_ABBR, LZ_INSTANCE. Defaults match the table above.
What an operator must supply
Azure credentials must not be committed.
Dotted arrows and highlighted nodes are what you do. Dashed arrows are what the tools do.
flowchart TD
classDef you fill:#fff3b0,stroke:#333,stroke-width:2px
lz(("You: confirm the four landing-zone resource groups and the spoke VNet")):::you
miss["azd up fails if those lookups miss"]
env(("You: create the shared azd environment on the landing-zone subscription")):::you
up["azd up provisions workload resources, grants and seeds SQL, then deploys QDox"]
lz -.-> miss
miss --> env
env -.-> up
linkStyle 1 stroke:#666,stroke-dasharray:6 4
1. Confirm the application landing zone exists
PLA-67 is Deployed. In calab-ai/azure-platform-landing-zone, bicep/80-landingzones should already be on subscription c4ccdf9e-39c3-4fe0-ba13-51cca77439fd with the params in the table above. That deploy also needed:
appTeamGroupObjectIdβ Entra group for the mocks app team (Contributor on core + data, Reader on network + mgmt)- Hub outputs from
50-connectivity(hubVnetId,hubVnetName,routeTableId,firewallPrivateIp) centralLogAnalyticsWorkspaceIdfrom40-managementvnetAddressPrefixβ a/22for the four-subnet spoke layout
Confirm these still exist before azd up:
az group show --name rg-mock-all-aue-core
az group show --name rg-mock-all-aue-network
az group show --name rg-mock-all-aue-mgmt
az group show --name rg-mock-all-aue-data
az network vnet show --resource-group rg-mock-all-aue-network --name vnet-mock-all-aue-01
azd up fails if those RGs or the spoke VNet are missing (existing lookups).
2. azd environment pointing at the LZ subscription
az login
azd auth login
azd env new shared --location australiaeast --subscription c4ccdf9e-39c3-4fe0-ba13-51cca77439fd
# or, if the env already exists:
azd env set AZURE_SUBSCRIPTION_ID c4ccdf9e-39c3-4fe0-ba13-51cca77439fd
azd env set AZURE_LOCATION australiaeast
The subscription id is parameterized (AZURE_SUBSCRIPTION_ID). The value above is the documented default for this estate, not the only possible target.
Container Apps logs always use a small Log Analytics workspace in core (log-mock-all-aue-01). App-LZ-team is Reader on the mgmt RG and Reader cannot listKeys on the platform LAW. Do not set CENTRAL_LAW_ID β the azd env value is accepted so leftover operator config does not fail ARM, but it is ignored. CAE never ingests from the 40-management workspace.
3. Entra admin for Azure SQL
Recommended: a security group that contains the people who will run azd plus (after first deploy) the CI identity.
azd env set SQL_ENTRA_ADMIN_OBJECT_ID <entra-object-id>
azd env set SQL_ENTRA_ADMIN_LOGIN "Calab Mocks SQL Admins"
azd env set SQL_ENTRA_ADMIN_TYPE Group
If you skip these, preprovision resolves the signed-in user (or service principal) from Microsoft Graph and sets both the object id and the real UPN / display name. A placeholder login such as azd-deployer is not valid β Azure SQL requires login and sid to identify the same Entra principal.
4. Default Entra access
Every mock endpoint that offers Entra authentication must grant these principals by default:
| Principal | Entra identity | Default access |
|---|---|---|
| Calab technical team | Security group sg-calab-technical-team (object ID 22dbaf10-c856-45a9-aaa9-550a382ea76e) |
Full |
| Automated smoke tests | User test@calab.ai (object ID b91ff458-164a-495e-ab94-e415afdc2274; MFA disabled) |
Reader |
For the current Azure SQL estate, Bicep/azd, postprovision, and CD express that rule as:
SQL_FULL_ACCESS_IDENTITIES=sg-calab-technical-team:22dbaf10-c856-45a9-aaa9-550a382ea76e:X
CONSUMER_SQL_IDENTITIES=cal-nlps-demo-app:049274d4-1650-4b95-b927-3ab8f61be95e,test@calab.ai:b91ff458-164a-495e-ab94-e415afdc2274:E
SQL_FULL_ACCESS_IDENTITIES receives db_owner; CONSUMER_SQL_IDENTITIES receives db_datareader. A descriptor is Name:guid:E|X: E identifies an Entra user or application (a user object ID or application/client ID), while X identifies an Entra group (group object ID). The legacy Name:client-id form still defaults to E. For an application or managed identity, E must use the client id as the SID (CAST(sid AS uniqueidentifier) matches the token's application id). The directory object id is a different identifier and is only for FROM EXTERNAL PROVIDER WITH OBJECT_ID, which still needs Graph. This lets postprovision/CI run Graph-free CREATE USER [...] WITH SID = ..., TYPE = ... and verify the principal type, SID, and role idempotently.
The NeuralOps demo app remains a reader because SAIR uses it for connection_url authentication. Defaults apply only when each variable is unset; an explicit empty string opts out. A name-only descriptor falls back to FROM EXTERNAL PROVIDER, which needs the SQL server's system-assigned identity granted Directory Readers in Entra.
ARM cannot CREATE USER. The grant is T-SQL in infra/scripts/grant-sql-identities.sh, invoked from azd postprovision and from CD after merge to main (or workflow_dispatch on main). A PR does not grant.
5. GitHub Organization Actions variables (after the first azd up)
See CD after merge to main. First azd up is still the operator bootstrap (Owner/UAA roleAssignments). CD then logs in as the organization app (CALAB_AI_AZURE_CLIENT_ID) via OIDC. No Azure client secret is created. Do not assume that org app is the Bicep id-ci-* UAMI.
QDox Basic-auth password is generated at first provision (openssl rand in infra/scripts/preprovision.sh) and stored only in Key Vault (qdox-password). The SQL authentication admin password uses the same generation path: preprovision generates it when Key Vault confirms sql-admin-password is missing (fail-closed on lookup errors), Bicep writes it to kv-mock-all-aue-01, and the login name goes to sql-admin-username. Neither password is derived from uniqueString, written to git, or emitted as a Bicep output. Preprovision may put either value in the azd env for that ARM run only; postprovision (including CALAB_MOCKS_SKIP_SQL_SEED=1) and a watcher that fires when azd exits strip FAKE_HOST_PASSWORD and SQL_ADMIN_PASSWORD from .azure/<env>/.env so a failed provision cannot leave plaintext on disk. Subsequent deploys resolve the vault as kv-<workloadAbbr>-<envAbbr>-<regionAbbr>-<instance> (or AZURE_KEY_VAULT_NAME) and leave existing secrets in place β they never pick "the first vault in the RG". To rotate QDox, set a new qdox-password version and pass it as FAKE_HOST_PASSWORD on the next azd up. To rotate SQL auth, set a new sql-admin-password version in the vault and run azd provision (or let CD run): main.bicep reads the secret through a Key Vault reference and applies it to the server on every provision, so the vault is the single source of truth and a partial failure converges on the next run. infra/scripts/provision-core-rg.sh does not touch SQL, so it never writes SQL credentials. Do not put passwords in infra/main.parameters.json.
Preprovision checks whether qdox-password / sql-admin-password exist through the ARM control plane (secret metadata, never the value), so CD's org app β which has no data-plane role β can still tell "missing" from "exists" and generate a password on the first merge after this change. Any other lookup error fails closed.
The estate Key Vault uses Azure RBAC, not access policies. The sg-calab-technical-team group (KEY_VAULT_ADMIN_GROUP_OBJECT_ID, default 22dbaf10-c856-45a9-aaa9-550a382ea76e) gets Key Vault Secrets Officer on every provision, including CD, so the vault always keeps a human admin. The human operator (KEY_VAULT_OPERATOR_PRINCIPAL_ID, which preprovision fills from the signed-in principal when it is unset and the run is not GitHub Actions) gets Key Vault Secrets Officer on the vault. It is deliberately not AZURE_PRINCIPAL_ID: CD provisions as the org app, and binding the role to the deploying principal would give that app vault-wide secret access. The QDox Container App identity gets Key Vault Secrets User on qdox-username and qdox-password only, so the internet-facing QDox revision cannot read sql-admin-password. Those role assignments can exist before Entra will actually allow the read. A deployment script (qdox-kv-ready) signs in as the QDox identity and retries both secret reads for up to 10 minutes, and the Container App is created only after that succeeds. The CI UAMI and the CD org app get no data-plane access; ARM resolves the SQL password through enabledForTemplateDeployment plus Microsoft.KeyVault/vaults/deploy/action.
The first provision after this change (often CD on merge) switches an existing vault from access policies to RBAC in one PUT, and role assignments can take a few minutes to propagate. If that provision fails between the switch and the Secrets Officer assignments, an Owner restores access with:
az role assignment create \
--role "Key Vault Secrets Officer" \
--assignee "<operator-object-id>" \
--scope "$(az keyvault show --name kv-mock-all-aue-01 --query id -o tsv)"
First-time azd up
Install azd and (for a fully seeded SQL estate on the laptop) sqlcmd.
Landing-zone RGs must already exist (step 1 above).
az login
azd auth login # if you are not already using azd's az-cli auth
azd env new shared --location australiaeast --subscription c4ccdf9e-39c3-4fe0-ba13-51cca77439fd
# optional Entra admin / consumer MIs as above (do not set CENTRAL_LAW_ID)
azd up
azd up = preprovision (resolve SQL Entra admin login + generate QDox and SQL admin passwords if missing) + provision (Bicep into the existing LZ core RG for workload resources; network/mgmt/data are looked up only) + postprovision (SQL grants + seed; fails if sqlcmd is missing unless CALAB_MOCKS_SKIP_SQL_SEED=1) + deploy (build fake-host/, push to ACR, update the QDox Container App). Postprovision runs before the QDox deploy.
Postprovision runs as the SQL Entra admin. The CI UAMI cannot CREATE USER for itself. Install sqlcmd before the first azd up, or skip explicitly and grant later as that admin. continueOnError is false β a failed seed is a failed azd up.
Useful outputs (also stored in the azd environment):
| Output | Use |
|---|---|
AZURE_RESOURCE_GROUP |
rg-mock-all-aue-core (Container App and Azure SQL) |
SQL_RESOURCE_GROUP |
Same as AZURE_RESOURCE_GROUP (core). Firewall / sqlcmd. |
AZURE_SUBSCRIPTION_ID |
c4ccdf9e-39c3-4fe0-ba13-51cca77439fd |
AZURE_LOCATION |
australiaeast |
LZ_NETWORK_RG_NAME / LZ_MGMT_RG_NAME / LZ_DATA_RG_NAME |
Looked-up LZ RGs (this repo does not deploy into them) |
LZ_SPOKE_VNET_NAME |
vnet-mock-all-aue-01 |
QDOX_BASE_URL |
Consumer connection config (stable hostname) |
QDOX_USERNAME |
Basic-auth username (qdox-fake by default) |
AZURE_KEY_VAULT_NAME |
Where to read qdox-password and sql-admin-password |
SQL_SERVER_FQDN |
Azure SQL host for AAD-token or SQL-auth clients |
SQL_ADMIN_LOGIN |
SQL authentication login (sqladmin by default) |
SQL_DATABASE_NAMES |
Databases Bicep created (edw by default) |
CI_MANAGED_IDENTITY_CLIENT_ID |
Bicep id-ci-* UAMI client id. Not automatically the CD login principal. CD uses org var CALAB_AI_AZURE_CLIENT_ID; copy this output there only if an operator has confirmed that UAMI is the org app. |
Fetch the QDox password (this is the only secret consumers of the HTTP fake need):
az keyvault secret show \
--vault-name "$(azd env get-value AZURE_KEY_VAULT_NAME)" \
--name qdox-password \
--query value -o tsv
Fetch the SQL authentication password (Key Vault sql-admin-password; login is sqladmin / SQL_ADMIN_LOGIN):
az keyvault secret show \
--vault-name "$(azd env get-value AZURE_KEY_VAULT_NAME)" \
--name sql-admin-password \
--query value -o tsv
Smoke-test QDox (the first request after scale-to-zero may take 15β60s β see cold start):
./infra/scripts/smoke-qdox.sh
Adding your laptop to the SQL firewall
Bicep allows Azure services (0.0.0.0 / AllowAllWindowsAzureIps) so NeuralOps running in Azure can connect. GitHub-hosted runners and laptops are not Azure services. The postprovision hook opens a run-scoped firewall rule (azd-<timestamp>-<pid>) against the core RG and deletes it when the hook exits. CI does the same with gha-${GITHUB_RUN_ID}.
To keep a laptop rule across runs (opt-in persistent access):
azd env set SQL_PERSIST_CLIENT_FIREWALL 1
That reuses the name azd-client and does not delete it. Existing leftover azd-client rules from earlier deploys are not removed automatically:
az sql server firewall-rule delete \
--resource-group "$(azd env get-value SQL_RESOURCE_GROUP)" \
--server "$(azd env get-value SQL_SERVER_NAME)" \
--name azd-client \
--yes
If you connect with SSMS/sqlcmd from a new network without persisting:
az sql server firewall-rule create \
--resource-group "$(azd env get-value SQL_RESOURCE_GROUP)" \
--server "$(azd env get-value SQL_SERVER_NAME)" \
--name laptop \
--start-ip-address <your-public-ip> \
--end-ip-address <your-public-ip>
Scale-to-zero and cold start
The QDox Container App is configured with min replicas = 0, Consumption workload profile. After ~5 minutes idle (cooldownPeriod 300s) the replica is removed and you pay nothing for compute.
Smoke-test authors should treat the first request after idle as slow:
| Situation | Typical wait |
|---|---|
| Replica already running | < 1s |
| Scale from zero (image already in the environment) | 15β40 seconds |
| First revision / image pull on a new revision | 30β90 seconds |
GET /health is unauthenticated and is what ACA uses for startup/liveness probes. A 401 on /Record without Authorization is the app rejecting the caller β not a cold-start failure.
infra/scripts/smoke-qdox.sh uses a 120s timeout on /health specifically to absorb a cold start.
This latency is acceptable for post-deployment smoke tests and live demos. It is not appropriate for a latency-sensitive production integration; those callers should keep their own retry/timeout budget at β₯ 60s on first connect.
Azure SQL SKU: free serverless vs Basic DTU
Parameter SQL_SKU (azd env SQL_SKU), default FreeServerless:
| Mode | How to set | What it is | Auto-pause | When to use |
|---|---|---|---|---|
| FreeServerless (default) | azd env set SQL_SKU FreeServerless |
GP_S_Gen5_1, useFreeLimit: true, 32 GB, ~100k vCore-seconds/month |
Yes β 60 minutes idle (autoPauseDelay; Azure rejects any other delay on free-limit DBs) |
Demos and smoke tests. Stay on this unless it hurts. |
| Basic | azd env set SQL_SKU Basic |
5 DTU, 2 GB, always on | No | Fallback if the free-offer monthly cap, subscription free-DB limit, or auto-pause resume latency is disruptive. |
Free-offer exhaustion (SQL_FREE_LIMIT_EXHAUSTION, default AutoPause): when the monthly free vCore-seconds are gone the database pauses until next month. BillOverUsage keeps it online and bills the overage β only switch if a demo month would otherwise go dark.
Auto-pause vs first-connection latency
On FreeServerless, an idle edw database auto-pauses after 60 minutes (the Azure free-limit default; 15 minutes is rejected with ProvisioningDisabled). The next AAD-token connection resumes it. Resume is typically 30β90 seconds (occasionally longer). Client libraries should:
- set a connection timeout β₯ 2 minutes for the fake EDW
- retry the first query on timeout / transient errors
Azure-hosted consumers (Container Apps, App Service, Functions in the same subscription) are already allowed through the server firewall. They still wait on resume.
The Entra administrator stays set; Microsoft Entra authentication only is explicitly disabled (azureADOnlyAuthentication: false on the server and on Microsoft.Sql/servers/azureADOnlyAuthentications named Default) so SQL authentication remains allowed alongside Entra (portal checkbox unchecked). Redeploys must not flip that checkbox on. SQL auth uses login sqladmin (sql-admin-username in Key Vault); administratorLogin cannot change after create, so on an existing server preprovision reuses the live login name. The password lives in Key Vault sql-admin-password and is re-applied to the server on every provision; it is never a Bicep output. Azure rejects SQL credential changes while Entra-only auth is on, so preprovision disables it on an existing server before the Bicep server PUT, and Bicep then keeps it disabled. The logical server has a system-assigned managed identity so CREATE USER ... FROM EXTERNAL PROVIDER can resolve names via Graph if that identity is granted Directory Readers; the grant script prefers Graph-free CREATE USER ... WITH SID, TYPE = E|X using the configured user/group object ID or application client ID. Local docker-compose continues to use SQL auth against SQL Server on Linux, which cannot do Entra β that split is intentional (see the root README). Azure seed/CI still use Entra (SQLCMD_AUTH=aad).
AAD sqlcmd does not pass the access token on argv. go-sqlcmd uses --authentication-method ActiveDirectoryAzCli (the az login session). ODBC sqlcmd 17.8+ writes the token to a 0600 UTF-16LE file and passes -G -P <file> (the path is on argv; the bearer token is not). If a client cannot do either, set SQLCMD_ACCESS_TOKEN and expect that limitation to be documented rather than silently putting the token in -P.
CI seeding
.github/workflows/azure-estate.yml validate job on pull requests and on push to main; deploy job only after merge to main (see CD after merge to main):
- Compiles Bicep (also on PRs) and asserts the template looks up
rg-mock-all-aue-*instead of creating RGs. The validate job does not requestid-token: writeand does notazure/login. azd provision, then hydrate azd env from core-RG resource GETs (infra/scripts/hydrate-azd-env.sh), thenazd deployβ rebuilds/pushesfake-hostand updates the QDox Container App. The workflow logs in as the org app (CALAB_AI_AZURE_CLIENT_ID) and treats that app as subscription Owner, so provision can apply role assignments.main.bicepis subscription-scoped, so CI must not runazd env refresh(that reads the subscription deployment). CI does not runazd upas a single command.- Opens a temporary firewall rule for the GitHub-hosted runner IP on the core RG (SQL) and deletes it afterwards.
- Grants the CI managed identity and
SQL_FULL_ACCESS_IDENTITIESdb_owner, and grantsCONSUMER_SQL_IDENTITIESdb_datareader. Defaults include the Calab technical-team group as owner plus the NeuralOps demo app andtest@calab.aismoke-test user as readers. Each Graph-free grant checks the configured Entra type (Euser/app orXgroup) and SID beforeALTER ROLE, then CD runs the grant script a second time to prove theIF NOT EXISTSpath. PRs runGRANT_SQL_SELFTEST=1(SID byte order only; no Azure). - Runs the same
sql-estate/seed/apply.shused by docker-compose, withSQLCMD_AUTH=aadandSQLCMD_SKIP_CREATE_DATABASE=1(Bicep owns SKU/CREATE DATABASE). Non-create scripts are applied withsqlcmd -d <database>because Azure SQL Database rejectsUSE. Scripts are idempotent (IF NOT EXISTS/CREATE OR ALTER/MERGE), so re-running resets rows to the git baseline.
The first azd up is the operator bootstrap. CD after that, on an estate-path push or workflow_dispatch on main, runs azd provision, then azd deploy, then SQL grants and seed.
If a previous deploy already granted CI User Access Administrator, remove it:
az role assignment delete \
--assignee "$(azd env get-value CI_MANAGED_IDENTITY_PRINCIPAL_ID)" \
--role "User Access Administrator" \
--scope "$(az group show -n rg-mock-all-aue-core --query id -o tsv)"
01_create_database.sql is skipped against Azure SQL so a T-SQL CREATE DATABASE cannot accidentally provision a full-price SKU beside the Bicep database.
sql-estate/databases/edw/ holds the fake EDW DDL + seed (the 7 SAIR views). The Azure estate workflow applies those scripts when it runs: an estate-path push to main (infra/**, azure.yaml, sql-estate/**, .github/workflows/azure-estate.yml, fake-host/**) or workflow_dispatch on main (SQLCMD_SKIP_CREATE_DATABASE=1 so Bicep keeps the SKU). A docs-only merge does not seed. Schema, keys, and probe queries: docs/projects/sair/edw-schema.md. Consuming managed identities can SELECT as soon as they are granted and that workflow has reseeded.
Connecting as a consuming managed identity
From a process that has the consumer MI (or az login as that identity):
# token for Azure SQL
az account get-access-token --resource https://database.windows.net/ --query accessToken -o tsv
Then open a TDS connection to SQL_SERVER_FQDN, database edw, authentication ActiveDirectoryDefault / ActiveDirectoryManagedIdentity:
SELECT TOP (5) * FROM dbo.vw_WorkOrder_Enriched;
SELECT TOP (5) * FROM dbo.vw_Site_Equipment;
SELECT TOP (5) * FROM dbo.vw_WorkOrder_Cost_Totals;
SELECT TOP (5) * FROM dbo.vw_Maintenance_Schedule;
SELECT TOP (5) * FROM dbo.vw_Failure_Codes;
SELECT TOP (5) * FROM dbo.vw_CrossAsset_Peer_Metrics;
SELECT TOP (5) * FROM dbo.vw_Asset_Attributes;
CD after merge to main
.github/workflows/azure-estate.yml is the CD pipeline. It does not deploy from pull requests or forks.
| Event | Jobs |
|---|---|
pull_request touching infra/**, fake-host/**, sql-estate/**, azure.yaml, or the workflow file |
validate only (Bicep compile + script syntax). No Azure login. No id-token. |
push to main on those same paths |
validate, then deploy (azure/login OIDC β azd provision β hydrate-azd-env.sh β azd deploy β SQL grants and seed) |
workflow_dispatch on main |
Same as push to main. The OIDC subject is the numeric claim repo:calab-ai@86651652/calab-mocks@1333873497:ref:refs/heads/main. Dispatch from any other ref is skipped. |
Path filters mean a docs-only merge does not start CD. A merge that changes the estate paths above does.
Operator bootstrap vs incremental CD
First deploy is still a human azd up on a principal that can create roleAssignments (step First-time azd up). The documented azd env is shared (azd env new shared). After that, an estate-path push to main (or workflow_dispatch on main) runs OIDC login, azd provision, hydrate, azd deploy, then SQL grants and seed. A docs-only push does not.
The CD login principal is the GitHub Organization app (CALAB_AI_AZURE_CLIENT_ID). The workflow treats that app as subscription Owner, so azd provision can finish remaining Bicep (role assignments, resource-provider registration, Key Vault, Log Analytics, the Container Apps environment, and QDox). main.bicep is subscription-scoped. CD must not azd env refresh (that GETs the subscription deployment).
Bicep still creates id-ci-mock-all-aue-01 (CI_MANAGED_IDENTITY_* outputs) for SQL grants and two federated credentials on that UAMI (classic + numeric main-ref subjects). CD does not log in as that UAMI unless CALAB_AI_AZURE_CLIENT_ID happens to be the same client id. Treat the org app as the CD principal; do not copy CI_MANAGED_IDENTITY_CLIENT_ID into the org var unless you have confirmed they are the same identity. The org app still needs the numeric subject added by an operator β this repo cannot Graph-patch it.
GitHub Organization variables (not secrets, not repo variables)
CD reads organization-level Actions variables (org Settings β Secrets and variables β Actions β Variables). These identities/subscription are shared across Calab repos. There is no client secret and no AZURE_CLIENT_SECRET. Do not add a repo or org GitHub variable named AZURE_ENV_NAME β the workflow hard-codes shared because there is one mocks landing zone (envName: shared).
| GitHub Organization variable | Maps to (OIDC / azd) | Notes |
|---|---|---|
CALAB_AI_AZURE_CLIENT_ID |
AZURE_CLIENT_ID / azure/login client-id |
Required. Org app that CD logs in as. |
CALAB_AI_AZURE_TENANT_ID |
AZURE_TENANT_ID / azure/login tenant-id |
Required. |
AZURE_INTERNAL_SUBSCRIPTION_ID |
AZURE_SUBSCRIPTION_ID / azure/login subscription-id |
Required. Mocks LZ is c4ccdf9e-39c3-4fe0-ba13-51cca77439fd. |
| (not a GitHub variable) | AZURE_ENV_NAME=shared |
Hardcoded in the workflow. |
| (not a GitHub variable) | AZURE_LOCATION=australiaeast |
Hardcoded in the workflow (LZ location). |
If CALAB_AI_AZURE_CLIENT_ID, CALAB_AI_AZURE_TENANT_ID, or AZURE_INTERNAL_SUBSCRIPTION_ID is missing, the deploy job fails on main (it does not skip). That is intentional: a silent skip would hide that OIDC is not configured. Pull-request validate still passes without those variables.
The org app needs:
- Owner on the mocks subscription, which is what the workflow uses for
azd provision - A federated credential whose subject is the immutable claim GitHub actually mints for this repo:
repo:calab-ai@86651652/calab-mocks@1333873497:ref:refs/heads/main
The classic subject repo:calab-ai/calab-mocks:ref:refs/heads/main does not match (AADSTS7002131). This repo already emits repo:ORG@ORGID/REPO@REPOID:ref:β¦ (GitHub unique-id subject). You cannot strip the numeric ids from the token.
An operator must add that federated credential on the organization app (CALAB_AI_AZURE_CLIENT_ID = 3129cbc0-243d-41b9-b730-ac88986c61f8). CD logs in as that app. This repository cannot Graph-patch that app from CI. Main ref only. Do not add a pull_request federated credential (repo:β¦:pull_request or repo:β¦@β¦:pull_request).
flowchart TD
classDef you fill:#fff3b0,stroke:#333,stroke-width:2px
add(("You: add the numeric-subject federated credential on the organization app")):::you
stored["Entra stores that subject on the organization app"]
cd["Estate-path push to main: OIDC login, azd provision, azd deploy, SQL grants and seed"]
add -.-> stored
stored --> cd
linkStyle 1 stroke:#666,stroke-dasharray:6 4
Prefer Graph / az rest / az ad app federated-credential with the full subject string so Azure stores exactly what GitHub mints. Do not set GITHUB_DEFAULT_BRANCH (or Bicep githubDefaultBranch) to refs/heads/main β Bicep already prefixes ref:refs/heads/, and a doubled path becomes β¦:ref:refs/heads/refs/heads/main (AADSTS7002131). The existing org-app credential is named calab-mocks-main.
{
"name": "calab-mocks-main",
"issuer": "https://token.actions.githubusercontent.com",
"subject": "repo:calab-ai@86651652/calab-mocks@1333873497:ref:refs/heads/main",
"audiences": ["api://AzureADTokenExchange"]
}
az ad app federated-credential create \
--id 3129cbc0-243d-41b9-b730-ac88986c61f8 \
--parameters ./github-calab-mocks-main-numeric.json
Portal: Entra ID β App registrations β that org app β Certificates & secrets β Federated credentials β Add credential.
- Graph / CLI / βOther issuerβ subject identifier: paste the full string
repo:calab-ai@86651652/calab-mocks@1333873497:ref:refs/heads/main. - GitHub Actions deploying Azure resources wizard: the UI already prefixes
ref:. The Entity field must berefs/heads/mainβ notrefs/heads/refs/heads/main, and not the fullrepo:β¦subject. If the wizard instead asks for a Branch name and already prefixesrefs/heads/, enter onlymain. Organization / repository fields for the numeric claim arecalab-ai@86651652andcalab-mocks@1333873497.
A doubled refs/heads/ is the bug that produced AADSTS7002131 on CD. Issuer https://token.actions.githubusercontent.com, audience api://AzureADTokenExchange. The org-wide wildcard FIC (repo:calab-ai/*) does not match numeric subjects.
Keep a classic-subject credential as a sibling only if other Calab repos still mint repo:org/name:ref:β¦.
Push to main and workflow_dispatch of this workflow while the selected ref is main both mint the numeric subject. A pull request must not be able to azure/login as this privileged identity.
Bicep still creates two federated credentials on the workload id-ci-* UAMI (githubMainFedCred classic + githubMainNumericFedCred numeric). First azd up with the defaults (GITHUB_REPOSITORY=calab-ai/calab-mocks, orgId 86651652, repoId 1333873497) provisions both. Owner and repo names are split from GITHUB_REPOSITORY; the only extra inputs are the numeric ids. That UAMI is not the CD login principal unless CALAB_AI_AZURE_CLIENT_ID happens to be the same client id β so dual FICs on the UAMI do not unblock azure/login until the org app has the numeric subject too.
After OIDC login works, Copilot's later issue is still outstanding: the org app vs id-ci-* UAMI for SQL db_owner. CD seeds as the org app; grants target the UAMI. A green azure/login is not a green seed.
If a previous Bicep deploy created a github-pull-request federated credential on the id-ci-* UAMI, delete it (that UAMI is not the CD login principal unless the client ids match):
az identity federated-credential delete \
--resource-group rg-mock-all-aue-core \
--identity-name "$(azd env get-value CI_MANAGED_IDENTITY_NAME)" \
--name github-pull-request --yes
Override GITHUB_REPOSITORY / GITHUB_DEFAULT_BRANCH / GITHUB_ORG_ID / GITHUB_REPO_ID in the azd env if this repo is renamed or the default branch is not main. Owner and repo names come from GITHUB_REPOSITORY (owner/repo); do not set separate GITHUB_ORG / GITHUB_REPO strings. If you change GITHUB_REPOSITORY, also set the numeric ids for that owner/repo β otherwise the numeric FIC subject will not match any GitHub token. Those overrides affect the Bicep UAMI federated credentials only; the org app's federated credential is configured on the org identity itself.
Linear Releases (continuous, not scheduled)
.github/workflows/linear-release.yml records what shipped to main. It is CD-style: one completed Linear Release per merge to main, not a nightly/cron train. Azure estate deploy is a separate pipeline; when that deploy succeeds we attach the GitHub Actions run to the same release.
| Event | What happens |
|---|---|
push to main |
linear/linear-release-action sync with version = git SHA. Pipeline: Mocks (continuous). |
Successful Azure fakes estate CD on main (push or workflow_dispatch) |
Same action syncs that SHA again and attaches the Azure CD Actions run. Skipped if CD failed or was a pull-request validate. |
pull_request |
Does not create production Linear releases. |
schedule / cron |
Not used. |
Repo secret (Settings β Secrets and variables β Actions): LINEAR_RELEASE_KEY β the Linear pipeline access key (not a personal API key). Pass it only as access_key: ${{ secrets.LINEAR_RELEASE_KEY }}. Never echo it. Official docs call the secret LINEAR_ACCESS_KEY; this repo stores it as LINEAR_RELEASE_KEY.
If Azure OIDC CD is still red, the push-to-main job still creates the SHA release. A later successful CD run attaches the deploy.
Adding a second HTTP fake (additive Bicep)
The fake-host image is built once (azure.yaml service qdox). Each personality is a Container App with FAKE_HOST_PERSONALITIES=<name>.
- Implement and register the personality in
fake-host/(seefake-host/README.md). - Add Key Vault secrets
<name>-username/<name>-passwordnext to the existingqdox-*secrets ininfra/modules/core-workload.bicep(core RG). - Instantiate
infra/modules/fake-host-app.bicepagain. Copy the commented example at the bottom ofinfra/modules/core-workload.bicep(fooApp). LeaveazdServiceNameempty so azd does not try to build a second image. - Output the new FQDN. Consumers get a new stable hostname.
No new azure.yaml service is required.
Destroy
Do not run azd down against this estate. The predown hook refuses by default because azd would delete AZURE_RESOURCE_GROUP (rg-mock-all-aue-core), which the landing zone owns.
To remove workload resources without deleting LZ RGs:
az resource list --resource-group rg-mock-all-aue-core --query "[].id" -o tsv | xargs -r az resource delete --ids
az keyvault purge --name "$(azd env get-value AZURE_KEY_VAULT_NAME)"
Do not delete resources in rg-mock-all-aue-data as part of tearing down this estate β this repo does not put anything there.
Override (destroys LZ RGs): CALAB_MOCKS_ALLOW_AZD_DOWN=1 azd down --purge --force.
File map
azure.yaml azd project (qdox service = fake-host image)
infra/lz-defaults.json canonical LZ subscription / RG / resource names
infra/main.bicep subscription scope: existing LZ RGs + modules
infra/modules/core-workload.bicep CAE, ACR, KV, identities, QDox app β core RG
infra/modules/sql-estate.bicep Azure SQL + edw β core RG (not data)
infra/modules/fake-host-app.bicep reusable scale-to-zero Container App
infra/modules/sql-database.bicep FreeServerless / Basic per database
infra/scripts/preprovision.sh SQL Entra admin login + QDox/SQL passwords
infra/scripts/lib-qdox-password-env.sh strip FAKE_HOST_PASSWORD / SQL_ADMIN_PASSWORD from azd env
infra/scripts/hydrate-azd-env.sh CI: populate azd env without env refresh
infra/scripts/postprovision.sh grants + seed after azd provision
infra/scripts/sqlcmd-aad.sh AAD sqlcmd without putting the token on argv
infra/scripts/predown.sh blocks azd down from deleting LZ RGs
infra/scripts/grant-sql-identities.sh
infra/scripts/smoke-qdox.sh
sql-estate/seed/apply.sh local SQL auth *and* Azure AAD
sql-estate/databases/edw/ fake EDW DDL + seed (7 SAIR views)
docs/projects/sair/edw-schema.md SAIR schema reference + probe queries
.github/workflows/azure-estate.yml Bicep validate on PR; provision, deploy, and seed on an estate-path push to main
.github/workflows/linear-release.yml Linear Release on merge to main (SHA); attach Azure CD run on success