APIOps (azure/apiops)

APIOps

https://github.com/azure/apiops
Admin
APIOps is a DevOps tool that applies GitOps principles to Azure API Management, enabling...

Tokens:26,039
Snippets:219
Trust Score:8.9
Update:1 month ago
Show doc for...
Context Summary (auto-generated)
Raw
# APIOps

APIOps applies GitOps and DevOps principles to Azure API Management (APIM), enabling version-controlled infrastructure automation for API design, development, and deployment. The tool provides two core binaries—an extractor that exports APIM configurations to file-based artifacts, and a publisher that deploys those artifacts across environments. This approach puts your entire API Management infrastructure under source control, allowing changes to be reviewed, audited, and promoted through CI/CD pipelines.

The framework supports two primary workflows: portal-first development where APIs are created in the Azure portal and extracted to Git, and code-first development where API artifacts are authored in an IDE and pushed to APIM. APIOps handles APIs, backends, diagnostics, loggers, named values, policies, products, gateways, tags, version sets, and policy fragments. It integrates with both Azure DevOps and GitHub Actions, providing ready-to-use pipeline templates.

## Extractor Configuration

The extractor exports APIM resources to a structured folder hierarchy. You can filter which resources to extract using a YAML configuration file.

```yaml
# configuration.extractor.yaml - Filter resources to extract
apiNames:
  - demo-conference-api
  - payment-api

backendNames:
  - helloworldfromfuncapp

diagnosticNames:
  - applicationinsights

loggerNames:
  - apim-lab-insights

namedValueNames:
  - environment
  - api-key

productNames:
  - starter
  - unlimited

tagNames:
  - production
  - internal

policyFragmentNames:
  - ForwardContext
```

## Publisher Configuration

The publisher deploys artifacts to target APIM instances with environment-specific overrides. Configuration supports YAML, environment variables, and JSON.

```yaml
# configuration.prod.yaml - Override values for production environment
apimServiceName: apim-prod-instance

namedValues:
  - name: environment
    properties:
      displayName: environment
      value: "https://api.production.com"
  - name: mysecretvalue
    properties:
      displayName: mysecretvalue
      keyVault:
        identityClientId: "your-identity-client-id"
        secretIdentifier: "https://your-keyvault.vault.azure.net/secrets/mysecret"
  - name: api-key
    properties:
      displayName: api-key
      value: "{#apiKeySecret#}"  # Token replaced at runtime

loggers:
  - name: appinsights
    properties:
      loggerType: applicationInsights
      description: Production Application Insights
      resourceId: "/subscriptions/sub-id/resourceGroups/rg-prod/providers/microsoft.insights/components/appinsights-prod"
      credentials:
        instrumentationKey: "{{appinsights-instrumentation-key}}"
      isBuffered: true

diagnostics:
  - name: applicationinsights
    properties:
      verbosity: Error
      loggerId: "/subscriptions/sub-id/resourceGroups/rg-prod/providers/Microsoft.ApiManagement/service/apim-prod/loggers/appinsights"

backends:
  - name: helloworldfromfuncapp
    properties:
      url: "https://prod-funcapp.azurewebsites.net/api"
      resourceId: "https://management.azure.com/subscriptions/sub-id/resourceGroups/rg-prod/providers/Microsoft.Web/sites/prod-funcapp"
      credentials:
        header:
          x-functions-key:
            - "{{funcapp-key}}"

apis:
  - name: demo-conference-api
    diagnostics:
      - name: applicationinsights
        properties:
          verbosity: Information
          loggerId: "/subscriptions/sub-id/resourceGroups/rg-prod/providers/Microsoft.ApiManagement/service/apim-prod/loggers/appinsights"
```

## API Artifact Structure

APIs are stored in `artifacts/apis/apiName/` with their specifications, policies, and metadata.

```json
// artifacts/apis/demo-conference-api/apiInformation.json
{
  "properties": {
    "apiRevision": "1",
    "authenticationSettings": {},
    "description": "A sample API with conference resources including Speakers, Sessions, and Topics.",
    "displayName": "Demo Conference API",
    "isCurrent": true,
    "path": "conference",
    "protocols": ["https"],
    "serviceUrl": "https://conferenceapi.azurewebsites.net",
    "subscriptionKeyParameterNames": {
      "header": "Ocp-Apim-Subscription-Key",
      "query": "subscription-key"
    },
    "subscriptionRequired": true
  }
}
```

```yaml
# artifacts/apis/demo-conference-api/specification.yaml
openapi: 3.0.1
info:
  title: Demo Conference API
  description: Conference management API with speakers, sessions, and topics
  version: '1.0'
servers:
  - url: https://api.example.com
paths:
  /sessions:
    get:
      summary: GetSessions
      operationId: GetSessions
      parameters:
        - name: speakername
          in: query
          schema:
            type: string
        - name: dayno
          in: query
          schema:
            type: integer
      responses:
        '200':
          description: OK
          content:
            application/json: {}
  /session/{id}:
    get:
      summary: GetSession
      operationId: GetSession
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: integer
      responses:
        '200':
          description: OK
components:
  securitySchemes:
    apiKeyHeader:
      type: apiKey
      name: Ocp-Apim-Subscription-Key
      in: header
security:
  - apiKeyHeader: []
```

## Policy Configuration

Policies are stored as XML files at global, product, API, and operation levels.

```xml
<!-- artifacts/policy.xml - Global policy -->
<policies>
  <inbound>
    <cors allow-credentials="false">
      <allowed-origins>
        <origin>{{environment}}</origin>
      </allowed-origins>
      <allowed-methods>
        <method>GET</method>
        <method>POST</method>
      </allowed-methods>
    </cors>
  </inbound>
  <backend>
    <forward-request />
  </backend>
  <outbound />
  <on-error />
</policies>
```

```xml
<!-- artifacts/products/starter/policy.xml - Product-level rate limiting -->
<policies>
  <inbound>
    <rate-limit calls="5" renewal-period="60" />
    <quota calls="100" renewal-period="604800" />
    <base />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
  </outbound>
</policies>
```

## Product Artifact Structure

Products define subscription tiers and associate APIs with access policies.

```json
// artifacts/products/starter/productInformation.json
{
  "properties": {
    "approvalRequired": false,
    "description": "Subscribers will be able to run 5 calls/minute up to a maximum of 100 calls/week.",
    "displayName": "Starter",
    "state": "published",
    "subscriptionRequired": true,
    "subscriptionsLimit": 1,
    "terms": ""
  }
}
```

```json
// artifacts/products/starter/apis.json - APIs included in product
["demo-conference-api", "payment-api"]
```

```json
// artifacts/products/starter/groups.json - Groups with access
["developers", "guests"]
```

## Backend Configuration

Backends define upstream service connections with authentication credentials.

```json
// artifacts/backends/helloworldfromfuncapp/backendInformation.json
{
  "properties": {
    "credentials": {
      "header": {
        "x-functions-key": ["{{helloworldfromfuncapp-key}}"]
      }
    },
    "description": "Azure Function backend",
    "protocol": "http",
    "resourceId": "https://management.azure.com/subscriptions/sub-id/resourceGroups/rg-apim/providers/Microsoft.Web/sites/helloworldfromfuncapp",
    "url": "https://helloworldfromfuncapp.azurewebsites.net/api"
  }
}
```

## Named Values Configuration

Named values store reusable configuration that can be referenced in policies using double-brace syntax.

```json
// artifacts/named values/environment/namedValueInformation.json
{
  "properties": {
    "displayName": "environment",
    "secret": false,
    "tags": [],
    "value": "https://www.example.com"
  }
}
```

## Logger Configuration

Loggers connect APIM to monitoring services like Application Insights.

```json
// artifacts/loggers/apim-lab-insights/loggerInformation.json
{
  "properties": {
    "credentials": {
      "instrumentationKey": "{{Logger-Credentials-instrumentation-key}}"
    },
    "isBuffered": true,
    "loggerType": "applicationInsights",
    "resourceId": "/subscriptions/sub-id/resourceGroups/rg-apim/providers/microsoft.insights/components/apim-insights"
  }
}
```

## Azure DevOps Extractor Pipeline

Manually trigger the extractor to export APIM configuration and create a pull request.

```yaml
# tools/azdo_pipelines/run-extractor.yaml
parameters:
  - name: APIM_INSTANCE_NAME
    displayName: APIM instance name
    type: string
  - name: RESOURCE_GROUP_NAME
    displayName: APIM instance resource group name
    type: string
  - name: APIM_REPOSITORY_NAME
    type: string
    displayName: APIM repository for pull request
  - name: API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH
    type: string
    displayName: Folder where you want to extract the artifacts
  - name: TARGET_BRANCH_NAME
    type: string
    displayName: Target branch for pull request
    default: main
  - name: CONFIGURATION_YAML_PATH
    type: string
    displayName: Optional configuration file
    values:
      - Extract All
      - configuration.extractor.yaml
  - name: API_SPECIFICATION_FORMAT
    type: string
    displayName: API Specification Format
    values:
      - OpenAPIV3Yaml
      - OpenAPIV3Json
      - OpenAPIV2Yaml
      - OpenAPIV2Json

trigger: none

variables:
  - group: apim-automation

stages:
  - stage: create_artifact_from_portal
    displayName: Create artifact from portal
    jobs:
      - job: create_artifact_from_portal
        pool:
          vmImage: ubuntu-latest
        steps:
          - task: AzureCLI@2
            displayName: Set extraction variables
            inputs:
              azureSubscription: "$(SERVICE_CONNECTION_NAME)"
              scriptType: pscore
              scriptLocation: inlineScript
              inlineScript: |
                Write-Host "##vso[task.setvariable issecret=true;variable=AZURE_BEARER_TOKEN]$(az account get-access-token --query accessToken --output tsv)"
                Write-Host "##vso[task.setvariable issecret=true;variable=AZURE_SUBSCRIPTION_ID]$(az account show --query id --output tsv)"
              addSpnToEnvironment: true
          - task: PowerShell@2
            displayName: Run extractor
            env:
              AZURE_BEARER_TOKEN: $(AZURE_BEARER_TOKEN)
              AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
              AZURE_RESOURCE_GROUP_NAME: ${{ parameters.RESOURCE_GROUP_NAME }}
              API_MANAGEMENT_SERVICE_NAME: ${{ parameters.APIM_INSTANCE_NAME }}
              API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: $(Build.ArtifactStagingDirectory)/${{ parameters.API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH }}
              API_SPECIFICATION_FORMAT: ${{ parameters.API_SPECIFICATION_FORMAT }}
```

## Azure DevOps Publisher Pipeline

Automatically triggered on merge to main, publishes artifacts to APIM instances.

```yaml
# tools/azdo_pipelines/run-publisher.yaml
trigger:
  branches:
    include:
      - main
  paths:
    exclude:
      - tools/*

parameters:
  - name: API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH
    type: string
    displayName: Folder where the artifacts reside
    default: "artifacts"
  - name: COMMIT_ID
    type: string
    displayName: Publish scope
    default: publish-artifacts-in-last-commit
    values:
      - publish-artifacts-in-last-commit
      - publish-all-artifacts-in-repo

variables:
  - group: apim-automation

stages:
  - stage: push_changes_to_Dev_APIM
    displayName: Push changes to Dev APIM
    jobs:
      - job: push_changes_to_Dev_APIM
        pool:
          vmImage: ubuntu-latest
        steps:
          - template: run-publisher-with-env.yaml
            parameters:
              API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: ${{ parameters.API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH }}
              RESOURCE_GROUP_NAME: $(RESOURCE_GROUP_NAME)
              API_MANAGEMENT_SERVICE_NAME: $(APIM_NAME)
              ENVIRONMENT: "Dev"
              COMMIT_ID: ${{ parameters.COMMIT_ID }}

  - stage: push_changes_to_Prod_APIM
    displayName: Push changes to Prod APIM
    jobs:
      - deployment: push_changes_to_Prod_APIM
        pool:
          vmImage: ubuntu-latest
        environment: 'Prod'  # Requires approval
        strategy:
          runOnce:
            deploy:
              steps:
                - template: run-publisher-with-env.yaml
                  parameters:
                    API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: ${{ parameters.API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH }}
                    RESOURCE_GROUP_NAME: $(RESOURCE_GROUP_NAME_Prod)
                    CONFIGURATION_YAML_PATH: $(Build.SourcesDirectory)/configuration.prod.yaml
                    ENVIRONMENT: "Prod"
                    COMMIT_ID: ${{ parameters.COMMIT_ID }}
```

## GitHub Actions Extractor Workflow

Manually trigger extraction from APIM portal and create a pull request with changes.

```yaml
# .github/workflows/run-extractor.yaml
name: Run - Extractor

on:
  workflow_dispatch:
    inputs:
      CONFIGURATION_YAML_PATH:
        description: 'Extract configuration'
        required: true
        type: choice
        options:
          - Extract All APIs
          - configuration.extractor.yaml
      API_SPECIFICATION_FORMAT:
        description: 'API Specification Format'
        required: true
        type: choice
        options:
          - OpenAPIV3Yaml
          - OpenAPIV3Json

env:
  apiops_release_version: v6.0.0

jobs:
  extract:
    runs-on: ubuntu-latest
    environment: dev
    steps:
      - uses: actions/checkout@v4

      - name: Run extractor
        env:
          AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
          AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
          AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
          AZURE_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          AZURE_RESOURCE_GROUP_NAME: ${{ secrets.AZURE_RESOURCE_GROUP_NAME }}
          API_MANAGEMENT_SERVICE_NAME: ${{ secrets.API_MANAGEMENT_SERVICE_NAME }}
          API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: ${{ github.workspace }}/apimartifacts
          API_SPECIFICATION_FORMAT: ${{ github.event.inputs.API_SPECIFICATION_FORMAT }}
        run: |
          $uri = "https://github.com/Azure/apiops/releases/download/${{ env.apiops_release_version }}/extractor-linux-x64.zip"
          Invoke-WebRequest -Uri "$uri" -OutFile "extractor.zip"
          Expand-Archive -Path "extractor.zip" -DestinationPath "./extractor"
          chmod +x "./extractor/extractor"
          & "./extractor/extractor"
        shell: pwsh

      - name: Create pull request
        uses: peter-evans/create-pull-request@v6
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-message: "Extract APIM artifacts"
          title: "APIM Extraction - ${{ github.run_id }}"
          labels: extract, automated pr
```

## GitHub Actions Publisher Workflow

Automatically publish APIM artifacts on push to main branch.

```yaml
# .github/workflows/run-publisher.yaml
name: Run - Publisher

on:
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      COMMIT_ID_CHOICE:
        description: 'Publish scope'
        required: true
        type: choice
        default: "publish-artifacts-in-last-commit"
        options:
          - "publish-artifacts-in-last-commit"
          - "publish-all-artifacts-in-repo"

jobs:
  get-commit:
    runs-on: ubuntu-latest
    outputs:
      commit_id: ${{ steps.commit.outputs.commit_id }}
    steps:
      - name: Set Commit Id
        id: commit
        run: echo "commit_id=${GITHUB_SHA}" >> $GITHUB_OUTPUT

  Push-Changes-To-APIM-Dev:
    needs: get-commit
    uses: ./.github/workflows/run-publisher-with-env.yaml
    with:
      API_MANAGEMENT_ENVIRONMENT: dev
      COMMIT_ID: ${{ needs.get-commit.outputs.commit_id }}
      API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: apimartifacts
    secrets: inherit

  Push-Changes-To-APIM-Prod:
    needs: [get-commit, Push-Changes-To-APIM-Dev]
    uses: ./.github/workflows/run-publisher-with-env.yaml
    with:
      API_MANAGEMENT_ENVIRONMENT: prod
      CONFIGURATION_YAML_PATH: configuration.prod.yaml
      API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH: apimartifacts
      COMMIT_ID: ${{ needs.get-commit.outputs.commit_id }}
    secrets: inherit
```

## Environment Variables Reference

Required environment variables for extractor and publisher tools.

```bash
# Authentication - Required for both extractor and publisher
export AZURE_CLIENT_ID="your-service-principal-client-id"
export AZURE_CLIENT_SECRET="your-service-principal-secret"
export AZURE_TENANT_ID="your-azure-tenant-id"
export AZURE_SUBSCRIPTION_ID="your-subscription-id"

# OR use bearer token directly
export AZURE_BEARER_TOKEN="eyJhbGciOiJIUzI1Ni..."

# APIM instance location - Required for both
export AZURE_RESOURCE_GROUP_NAME="rg-apim"
export API_MANAGEMENT_SERVICE_NAME="my-apim-instance"

# Artifact path - Required for both
export API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH="/path/to/artifacts"

# Extractor-specific options
export API_SPECIFICATION_FORMAT="OpenAPIV3Yaml"  # OpenAPIV3Json, OpenAPIV2Yaml, OpenAPIV2Json
export CONFIGURATION_YAML_PATH="/path/to/configuration.extractor.yaml"

# Publisher-specific options
export CONFIGURATION_YAML_PATH="/path/to/configuration.prod.yaml"
export COMMIT_ID="ca82a6dff817ec66f44342007202690a93763949"  # Only publish changes from this commit

# Azure cloud environment (optional)
export AZURE_CLOUD_ENVIRONMENT="AzureGlobalCloud"  # AzureUSGovernment, AzureChinaCloud
```

## Service Principal Setup

Create a service principal with contributor access for the APIM resource group.

```bash
# Create service principal with required permissions
az ad sp create-for-rbac \
  --name "apiops-automation" \
  --role contributor \
  --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group-name} \
  --sdk-auth

# Output JSON contains required credentials:
# {
#   "clientId": "...",
#   "clientSecret": "...",
#   "subscriptionId": "...",
#   "tenantId": "..."
# }

# Store these as secrets in Azure DevOps or GitHub:
# - AZURE_CLIENT_ID
# - AZURE_CLIENT_SECRET
# - AZURE_SUBSCRIPTION_ID
# - AZURE_TENANT_ID
```

## Local Development with Dev Container

Debug the extractor and publisher locally using VS Code Dev Containers.

```bash
# 1. Clone the repository and open in VS Code
git clone https://github.com/Azure/apiops.git
cd apiops
code .

# 2. Reopen in Dev Container (VS Code will prompt)

# 3. Authenticate with Azure
az login --use-device-code

# 4. Create environment files from templates
cp ./tools/code/.env.extractor.template ./tools/code/.env.extractor
cp ./tools/code/.env.publisher.template ./tools/code/.env.publisher

# 5. Edit .env.extractor with your values:
# AZURE_SUBSCRIPTION_ID=your-sub-id
# AZURE_RESOURCE_GROUP_NAME=your-rg
# API_MANAGEMENT_SERVICE_NAME=your-apim
# API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH=/workspaces/apiops/artifacts

# 6. Run from VS Code debugger:
# - "Launch Extractor" to extract from APIM
# - "Launch Publisher" to publish to APIM
```

## Version Set Configuration

Configure API versioning with version sets linking multiple API revisions.

```json
// artifacts/version sets/myapi-versions/versionSetInformation.json
{
  "properties": {
    "displayName": "My API Versions",
    "versioningScheme": "Segment",
    "description": "Version set for My API"
  }
}
```

```yaml
# Link API to version set in configuration
apis:
  - name: myapi-v1
    properties:
      apiVersionSetId: "/subscriptions/sub-id/resourceGroups/rg/providers/Microsoft.ApiManagement/service/apim/apiVersionSets/myapi-versions"
      apiVersion: "v1"
  - name: myapi-v2
    properties:
      apiVersionSetId: "/subscriptions/sub-id/resourceGroups/rg/providers/Microsoft.ApiManagement/service/apim/apiVersionSets/myapi-versions"
      apiVersion: "v2"
```

## Gateway Configuration

Configure self-hosted gateways with associated APIs.

```json
// artifacts/gateways/my-gateway/gatewayInformation.json
{
  "properties": {
    "description": "On-premises gateway",
    "locationData": {
      "name": "New York datacenter"
    }
  }
}
```

```json
// artifacts/gateways/my-gateway/apis.json
["demo-conference-api", "internal-api"]
```

```yaml
# Override gateway APIs per environment
gateways:
  - name: my-gateway
    apis:
      - name: demo-conference-api
      - name: payment-api
```

APIOps is designed for teams managing Azure API Management across multiple environments (development, staging, production). The typical workflow involves extracting configurations from a lower environment, reviewing changes through pull requests, and automatically promoting approved changes to higher environments. This enables infrastructure-as-code practices for API governance, ensuring all changes are version-controlled and auditable.

Integration patterns include connecting to Azure Functions, Logic Apps, and other backend services through backend configurations with credential management via named values. The configuration override system allows environment-specific values (endpoints, secrets, resource IDs) to be substituted during deployment while maintaining a single source of truth in the artifact repository. Teams can choose between portal-first development for rapid prototyping or code-first development for strict change control, with both approaches converging on the same Git-based artifact structure.