Known issues

Issues you may hit, with workarounds. Each entry says when it was found and when it was fixed. For fixes and changes, see What’s new.

Bootstrap: new role needed for policy guardrails

Projects bootstrapped before policy_mode existed lack the Resource Policy Contributor role on the deploy identity. Contributor cannot write policy assignments, so the first deploy fails with an authorization error.

Workaround: re-run the bootstrap (it adds the role), or set policy_mode: none.

Policy: public-access policy is skipped in public scenarios

The “ML workspaces should disable public network access” policy is only assigned when the scenario is private. In public scenarios it would always be non-compliant, so it is left out on purpose.

Policy: compliance results are delayed

New assignments can take up to 30 minutes to report compliance. See the Policy FAQ.

Azure DevOps: pipelines are not registered

Bootstrap creates the repo, service connection and variables, but does not register the pipelines. Create them from the generated YAML files in the repository.

Bootstrap: no required reviewers on environments

Environments are created without approval reviewers. Add reviewers manually for production.

Dependency holds

Some pins are held on purpose, and are recorded in versions.lock.json:

• provider:azurerm is held below 5.0, because the Azure Verified Modules require it.
• pypi:pyarrow is held below 26.
• pypi:azureml-mlflow is held at 1.60.0.

Cost estimates are list price

scripts/Get-ScenarioCostEstimates.ps1 uses the Azure Retail Prices API. The figures are list price and order of magnitude only.

No Responsible AI automation

Fairness, explainability and content-safety checks are not generated yet. See Responsible AI.

Feature store scenario is infrastructure only

Scenario 7 creates the feature store workspace but no feature set, offline store or materialization, and the training pipeline does not use it. See Managed feature store.

Live deployment coverage

The e2e.yml workflow applies a scenario, checks the workspace exists, then destroys everything. Results for the dev environment (workload IaC only; the bootstrap is not covered):

Live runs found and fixed these problems: Terraform zone sets that were unknown at plan time, duplicate storage private endpoint names, compute created before the workspace private endpoint (now explicit depends_on), Standard ACR rejecting the default Deny network action, storage and Key Vault needing explicit Allow ACLs for public scenarios, a Terraform conditional with mismatched object types for the managed network, and the OIDC token expiring during long applies.

The GitHub bootstrap (Terraform, 46 resources) was applied live on October 15, 2026, and the generated Bicep project then deployed basic-public to the dev and prod environments through the infra pipeline using the bootstrap identity and federated credentials. The same project with Terraform stopped at terraform init with the 403 described below, because the test subscription’s policy turns state storage public access off. The Azure DevOps bootstrap (43 resources) was also applied live on the same date, and its Bicep infra pipeline passed validate, deploy_dev and deploy_prod. Not verified live: the Azure DevOps Terraform pipeline, the Terraform pipeline against state storage with public access off, private runners and private state storage, separate plan and apply identities, Managed DevOps Pools, the rai option, the scenario 7 feature-set pack and endpoint deployment.

Azure DevOps notes

• The bootstrap sets the Azure Repos default branch to main (a new repository defaults to master) and authorizes every pipeline to use the generated environments and service connections, so the first run does not wait for a manual Permit.
• A push to main starts the infra pipeline. Starting a manual run at the same time collides on the deployment name (DeploymentActive); wait for the push run to finish.
• Hosted agents print a notice that ubuntu-latest moves to Ubuntu 26 on October 19, 2026.

Bicep security scan

The Bicep pipelines scan with PSRule for Azure, not Checkov. Checkov cannot parse the AVM based Bicep (it reported zero checks and a parsing error, then passed) and crashes on the compiled template. PSRule expands each .bicepparam file and reports findings as warnings; the step fails only if it cannot scan (no results or an error). Findings such as storage firewall, Key Vault logging and ACR geo-replication are informational for the default scenario. The Terraform pipelines still use Checkov.

Soft-deleted names and managed-network rules

• Key Vaults (purge protection) and AML workspaces stay soft-deleted after a destroy, which blocks redeploying the same names. Key Vault names use the suffix, but workspace names use only the project name, so change both suffix and project_name (or purge the soft-deleted workspace) before redeploying.
• Managed-network private endpoint outbound rules for storage, Key Vault, ACR and the workspace are created by the service. The templates define only FQDN rules.
• Every private scenario (2, 3, 4 and 11) is verified live on Terraform and Bicep. The private runners and private state storage from the bootstrap are not.

Training pipeline needs reachable workload storage

ml-train uploads data and component code to the workspace storage account from the runner. If your subscription policy forces storage public network access to Disabled and shared keys off, which overrides what the templates set, the upload fails: az ml data create reports that key based authentication is not permitted, and az ml component create reports AuthorizationFailure. This was seen on October 15, 2026 with the basic-public Bicep deployment, so the training pipeline and the new components have not run live. Use a self-hosted runner that can reach the storage account (see below), and set the workspace systemDatastoresAuthMode to identity so uploads use Entra ID instead of the account key. The templates do not set that property yet.

Terraform state storage and runner networking

The bootstrap state storage account is Entra-only (shared keys off) but uses a public endpoint so GitHub-hosted runners can reach it. If your tenant policy forces storage public access off, terraform init in the pipeline fails with 403. Set runner_label to a self-hosted runner label (for example an ARC scale set on a private AKS cluster in a VNet with access to the account). The reference pattern is the private AKS and ARC bootstrap in retail-demand-forecasting-mlops/runner-bootstrap; it is not generated by this accelerator yet.

GitHub OIDC subjects

New GitHub repositories issue ID-based subjects (repo:owner@ownerId/repo@repoId:environment:env). The bootstrap builds the federated credentials with these IDs.

Docs site (GitHub Pages)

pages.yml only runs when the repository variable PAGES_ENABLED is true. This repository is public, Pages is enabled (source: GitHub Actions) and the variable is set. For a fork or a private copy, enable Pages in Settings (private repositories need a paid plan), then set the variable.

End-to-end workflow

.github/workflows/e2e.yml plans (and optionally applies, verifies and destroys) the scenarios you list, for Terraform and Bicep in a dedicated test subscription.

• PRs: add the safe-to-test label to run plan or what-if only.
• Apply: run it manually with apply: true and a scenario list. Use scenarios: all for the full matrix (scenarios 1 to 12, Terraform and Bicep, 24 jobs, 4 at a time). A full run has not been completed yet, and there is no scheduled run, because the gate needs a manual approval and a full run costs real money.

Policy: run only the affected scenarios when a change touches them, and run the full set before a release tag and after changes to shared IaC modules. Each job has its own resource group, suffix and deployment name, and failures do not stop the other jobs.

• Setup: an e2e environment with AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID (OIDC identity with Contributor and RBAC Administrator, plus a custom role allowing Microsoft.Authorization/locks/* for the governance scenario).
• Each run uses a unique name_suffix, so soft-deleted Key Vaults and workspaces never collide. Purge can be blocked by policy; the vaults then expire on their own.
• It tests the workload IaC only. The GitHub and Azure DevOps bootstrap is tested with tests/Invoke-E2E.ps1.

Private runners and private state storage

Set use_private_networking: true (GitHub) to deploy self-hosted ACI runners in a new VNet with a NAT gateway (AVM cicd-agents-and-runners, as in the ALZ Accelerator). The state storage then gets a private endpoint and public access is disabled. runner_label defaults to self-hosted.

• The runner registration uses a GitHub PAT with repo admin rights. -Deploy reuses GITHUB_TOKEN (or gh auth token); set TF_VAR_github_runners_token to override.
• Not yet verified live. Azure DevOps (Managed DevOps Pools) is not generated yet.

ADLS data is not an MLTable

The bundled taxi-csv, taxi-parquet and nyc-taxi-open samples register as MLTable assets. data_source: adls registers a single uri_file, because src/prep.py parses MLTable by hand and reads only local paths. MLTable for ADLS needs prep.py support and a live run.

Ubuntu 26.04 and ubuntu-latest

GitHub moves the ubuntu-latest label to Ubuntu 26.04 between October 19 and November 19, 2026. Generated workflows use the runner_label input (default ubuntu-latest), so they move with it. This repository’s CI runs the tests on ubuntu-26.04 in a non-blocking canary job first. If a generated workflow breaks after the move, set runner_label: ubuntu-24.04 to pin the old image and open an issue.