CircleCI Docs (circleci/circleci-docs)

CircleCI Docs

https://github.com/circleci/circleci-docs
Admin
CircleCI Docs is a comprehensive technical documentation site for CircleCI, built with Antora for...

Tokens:522,027
Snippets:2,996
Trust Score:8.8
Update:2 weeks ago
Show doc for...
Context Summary (auto-generated)
Raw
# CircleCI Docs

CircleCI is a continuous integration and continuous delivery (CI/CD) platform that automates the build, test, and deployment of software. CircleCI enables development teams to rapidly release code with confidence by automating the software development process using smart, efficient pipelines. The platform supports multiple execution environments including Docker containers, Linux VMs, macOS, Windows, and GPU machines.

CircleCI's configuration-as-code approach uses YAML configuration files stored in `.circleci/config.yml` to define pipelines that orchestrate jobs, workflows, and deployment steps. The platform integrates with GitHub, GitLab, and Bitbucket, and provides features like orbs (reusable configuration packages), caching, parallelism, and approval workflows. CircleCI Server is also available for self-hosted enterprise deployments.

## Configuration Reference

### Basic Job Configuration

Defines a job with steps to execute in a specified execution environment.

```yaml
version: 2.1

jobs:
  build:
    docker:
      - image: cimg/base:2024.12
    resource_class: xlarge
    working_directory: ~/project
    environment:
      NODE_ENV: production
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm ci
      - run:
          name: Run tests
          command: npm test
```

### Docker Executor with Multiple Containers

Configures a job with primary and service containers for database testing.

```yaml
version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.18.1
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD
        environment:
          DATABASE_URL: postgresql://user@localhost/test_db
      - image: postgres:14.12
        environment:
          POSTGRES_USER: user
          POSTGRES_DB: test_db
      - image: redis:7.2
    steps:
      - checkout
      - run:
          name: Wait for services
          command: dockerize -wait tcp://localhost:5432 -wait tcp://localhost:6379 -timeout 60s
      - run:
          name: Run integration tests
          command: npm run test:integration
```

### Machine Executor (Linux VM)

Runs jobs in a dedicated Linux virtual machine for full system access.

```yaml
version: 2.1

jobs:
  build-docker:
    machine:
      image: ubuntu-2204:current
      docker_layer_caching: true
    resource_class: large
    steps:
      - checkout
      - run:
          name: Build Docker image
          command: |
            docker build -t myapp:$CIRCLE_SHA1 .
            docker push myapp:$CIRCLE_SHA1
```

### macOS Executor

Configures a job to build and test iOS/macOS applications.

```yaml
version: 2.1

jobs:
  build-ios:
    macos:
      xcode: 15.4.0
    resource_class: macos.m1.medium.gen1
    steps:
      - checkout
      - run:
          name: Install CocoaPods
          command: pod install
      - run:
          name: Build and test
          command: |
            xcodebuild -workspace MyApp.xcworkspace \
              -scheme MyApp \
              -destination 'platform=iOS Simulator,name=iPhone 15' \
              test
```

### Windows Executor

Configures a job to run on Windows for .NET or Windows-specific builds.

```yaml
version: 2.1

jobs:
  build-windows:
    machine:
      image: windows-server-2022-gui:current
      shell: powershell.exe -ExecutionPolicy Bypass
    resource_class: windows.medium
    steps:
      - checkout
      - run:
          name: Build .NET application
          command: dotnet build -c Release
      - run:
          name: Run tests
          command: dotnet test --no-build -c Release
```

## Workflows

### Sequential Workflow with Dependencies

Defines a workflow where jobs run in sequence based on dependencies.

```yaml
version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - checkout
      - run: npm ci
      - persist_to_workspace:
          root: .
          paths:
            - node_modules

  test:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run: npm test

  deploy:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run: npm run deploy

workflows:
  build-test-deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test
          filters:
            branches:
              only: main
```

### Fan-Out/Fan-In Workflow

Runs multiple test jobs in parallel then fans in for deployment.

```yaml
version: 2.1

workflows:
  build-and-deploy:
    jobs:
      - build
      - unit-tests:
          requires:
            - build
      - integration-tests:
          requires:
            - build
      - e2e-tests:
          requires:
            - build
      - security-scan:
          requires:
            - build
      - deploy:
          requires:
            - unit-tests
            - integration-tests
            - e2e-tests
            - security-scan
          filters:
            branches:
              only: main
```

### Manual Approval Workflow

Holds a workflow for manual approval before deploying to production.

```yaml
version: 2.1

workflows:
  deploy-with-approval:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy-staging:
          requires:
            - test
      - hold-for-approval:
          type: approval
          requires:
            - deploy-staging
      - deploy-production:
          requires:
            - hold-for-approval
          context: production-credentials
```

### Scheduled Workflow

Configures a workflow to run on a cron schedule.

```yaml
version: 2.1

workflows:
  nightly-build:
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - main
    jobs:
      - build
      - run-full-test-suite
      - generate-reports

  commit-workflow:
    jobs:
      - build
      - quick-tests
```

### Matrix Jobs

Runs a job multiple times with different parameter combinations.

```yaml
version: 2.1

jobs:
  test:
    parameters:
      node-version:
        type: string
      os:
        type: executor
    executor: << parameters.os >>
    steps:
      - checkout
      - run:
          name: Install Node.js << parameters.node-version >>
          command: nvm install << parameters.node-version >> && nvm use << parameters.node-version >>
      - run: npm ci
      - run: npm test

executors:
  linux:
    docker:
      - image: cimg/base:current
  macos:
    macos:
      xcode: 15.4.0

workflows:
  test-matrix:
    jobs:
      - test:
          matrix:
            parameters:
              node-version: ["18", "20", "22"]
              os: [linux, macos]
            exclude:
              - node-version: "18"
                os: macos
```

## Orbs

### Using Registry Orbs

Imports and uses pre-built orbs from the CircleCI Orb Registry.

```yaml
version: 2.1

orbs:
  node: circleci/node@5.2.0
  aws-cli: circleci/aws-cli@4.1.3
  slack: circleci/slack@4.13.3

jobs:
  build-and-deploy:
    executor: node/default
    steps:
      - checkout
      - node/install-packages:
          pkg-manager: npm
      - run: npm run build
      - aws-cli/setup:
          role_arn: arn:aws:iam::123456789:role/deploy-role
      - run:
          name: Deploy to S3
          command: aws s3 sync ./dist s3://my-bucket/
      - slack/notify:
          event: pass
          template: success_tagged_deploy_1

workflows:
  build-deploy:
    jobs:
      - build-and-deploy:
          context: aws-credentials
```

### Creating Inline Orbs

Defines reusable commands and executors within your configuration.

```yaml
version: 2.1

orbs:
  my-orb:
    commands:
      install-deps:
        parameters:
          cache-key:
            type: string
            default: v1-deps
        steps:
          - restore_cache:
              keys:
                - << parameters.cache-key >>-{{ checksum "package-lock.json" }}
          - run: npm ci
          - save_cache:
              key: << parameters.cache-key >>-{{ checksum "package-lock.json" }}
              paths:
                - node_modules

      notify:
        parameters:
          message:
            type: string
        steps:
          - run:
              name: Send notification
              command: |
                curl -X POST $WEBHOOK_URL \
                  -H "Content-Type: application/json" \
                  -d '{"text": "<< parameters.message >>"}'

    executors:
      node-browser:
        docker:
          - image: cimg/node:20.18.1-browsers

jobs:
  test:
    executor: my-orb/node-browser
    steps:
      - checkout
      - my-orb/install-deps:
          cache-key: v2-deps
      - run: npm test
      - my-orb/notify:
          message: "Tests completed successfully!"

workflows:
  main:
    jobs:
      - test
```

## Caching and Workspaces

### Dependency Caching

Saves and restores dependencies to speed up builds.

```yaml
version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-deps-{{ checksum "package-lock.json" }}
            - v1-deps-
      - run: npm ci
      - save_cache:
          key: v1-deps-{{ checksum "package-lock.json" }}
          paths:
            - node_modules
            - ~/.npm
      - run: npm run build
      - store_artifacts:
          path: dist
          destination: build-output
```

### Workspace Persistence

Shares data between jobs in a workflow using workspaces.

```yaml
version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - checkout
      - run: npm ci
      - run: npm run build
      - persist_to_workspace:
          root: .
          paths:
            - dist
            - node_modules
            - package.json

  deploy:
    docker:
      - image: cimg/node:20.18.1
    steps:
      - attach_workspace:
          at: /tmp/workspace
      - run:
          name: Deploy application
          command: |
            cd /tmp/workspace
            npm run deploy

workflows:
  build-and-deploy:
    jobs:
      - build
      - deploy:
          requires:
            - build
```

## Parallelism and Test Splitting

### Parallel Test Execution

Splits tests across multiple containers to reduce execution time.

```yaml
version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.18.1
    parallelism: 4
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-deps-{{ checksum "package-lock.json" }}
      - run: npm ci
      - run:
          name: Run tests with splitting
          command: |
            TESTFILES=$(circleci tests glob "src/**/*.test.ts" | circleci tests split --split-by=timings)
            npm test -- $TESTFILES
      - store_test_results:
          path: test-results
      - store_artifacts:
          path: coverage
```

## Environment Variables and Contexts

### Using Environment Variables and Contexts

Configures secure environment variables at various levels.

```yaml
version: 2.1

jobs:
  deploy:
    docker:
      - image: cimg/base:current
    environment:
      DEPLOY_ENV: production
    steps:
      - checkout
      - run:
          name: Deploy with secrets
          environment:
            LOG_LEVEL: debug
          command: |
            echo "Deploying to $DEPLOY_ENV"
            echo "Using AWS credentials from context"
            aws s3 sync ./dist s3://$S3_BUCKET/
          no_output_timeout: 30m

workflows:
  deploy:
    jobs:
      - deploy:
          context:
            - aws-production
            - slack-notifications
          filters:
            branches:
              only: main
```

## Docker Layer Caching and Remote Docker

### Building Docker Images

Sets up remote Docker for building and pushing container images.

```yaml
version: 2.1

jobs:
  build-and-push:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - setup_remote_docker:
          version: default
          docker_layer_caching: true
      - run:
          name: Build and push Docker image
          command: |
            echo $DOCKER_PASSWORD | docker login -u $DOCKER_USER --password-stdin
            docker build -t myorg/myapp:$CIRCLE_SHA1 .
            docker build -t myorg/myapp:latest .
            docker push myorg/myapp:$CIRCLE_SHA1
            docker push myorg/myapp:latest
```

## Pipeline Parameters and Dynamic Configuration

### Pipeline Parameters

Defines parameters that can be passed when triggering a pipeline.

```yaml
version: 2.1

parameters:
  run-integration-tests:
    type: boolean
    default: false
  deploy-environment:
    type: enum
    enum: ["staging", "production"]
    default: "staging"
  node-version:
    type: string
    default: "20"

jobs:
  build:
    docker:
      - image: cimg/node:<< pipeline.parameters.node-version >>
    steps:
      - checkout
      - run: npm ci
      - run: npm run build

  integration-tests:
    docker:
      - image: cimg/node:<< pipeline.parameters.node-version >>
    steps:
      - checkout
      - run: npm run test:integration

  deploy:
    docker:
      - image: cimg/base:current
    steps:
      - run:
          name: Deploy to << pipeline.parameters.deploy-environment >>
          command: ./deploy.sh << pipeline.parameters.deploy-environment >>

workflows:
  build-and-deploy:
    jobs:
      - build
      - integration-tests:
          requires:
            - build
          filters: << pipeline.parameters.run-integration-tests >>
      - deploy:
          requires:
            - build
```

### Dynamic Configuration with Setup Workflow

Uses a setup configuration to generate the actual pipeline configuration.

```yaml
version: 2.1

setup: true

orbs:
  continuation: circleci/continuation@1.0.0
  path-filtering: circleci/path-filtering@1.0.0

jobs:
  generate-config:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - path-filtering/set-parameters:
          mapping: |
            src/api/.* run-api-tests true
            src/web/.* run-web-tests true
            infrastructure/.* run-infra-tests true
      - continuation/continue:
          configuration_path: .circleci/generated-config.yml
          parameters: /tmp/pipeline-parameters.json

workflows:
  setup:
    jobs:
      - generate-config
```

## Reusable Commands and Executors

### Defining Reusable Configuration Elements

Creates parameterized commands and executors for reuse across jobs.

```yaml
version: 2.1

executors:
  node-executor:
    parameters:
      version:
        type: string
        default: "20"
    docker:
      - image: cimg/node:<< parameters.version >>
    working_directory: ~/project
    environment:
      NODE_ENV: test

commands:
  setup-project:
    description: "Checkout and install dependencies"
    parameters:
      install-browsers:
        type: boolean
        default: false
    steps:
      - checkout
      - restore_cache:
          keys:
            - deps-{{ checksum "package-lock.json" }}
      - run: npm ci
      - when:
          condition: << parameters.install-browsers >>
          steps:
            - run: npx playwright install
      - save_cache:
          key: deps-{{ checksum "package-lock.json" }}
          paths:
            - node_modules

  run-with-retry:
    description: "Run a command with automatic retry"
    parameters:
      command:
        type: string
      max-retries:
        type: integer
        default: 3
    steps:
      - run:
          name: Execute with retry
          command: << parameters.command >>
          max_auto_reruns: << parameters.max-retries >>
          auto_rerun_delay: 10s

jobs:
  unit-tests:
    executor:
      name: node-executor
      version: "20"
    steps:
      - setup-project
      - run-with-retry:
          command: npm test
      - store_test_results:
          path: test-results

  e2e-tests:
    executor:
      name: node-executor
      version: "20"
    steps:
      - setup-project:
          install-browsers: true
      - run-with-retry:
          command: npm run test:e2e
          max-retries: 2
```

## Conditional Steps and Logic

### Conditional Execution Based on Parameters

Uses conditional logic to run different steps based on parameters.

```yaml
version: 2.1

parameters:
  environment:
    type: enum
    enum: ["dev", "staging", "prod"]
    default: "dev"

jobs:
  deploy:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - when:
          condition:
            equal: [prod, << pipeline.parameters.environment >>]
          steps:
            - run:
                name: Production deployment checks
                command: |
                  echo "Running production safety checks..."
                  ./scripts/check-prod-readiness.sh
      - run:
          name: Deploy to << pipeline.parameters.environment >>
          command: ./deploy.sh << pipeline.parameters.environment >>
      - unless:
          condition:
            equal: [dev, << pipeline.parameters.environment >>]
          steps:
            - run:
                name: Notify Slack
                command: ./notify-slack.sh "Deployed to << pipeline.parameters.environment >>"
                when: always

workflows:
  deploy:
    when:
      or:
        - equal: [main, << pipeline.git.branch >>]
        - << pipeline.parameters.environment >> == "dev"
    jobs:
      - deploy
```

## Summary

CircleCI provides a comprehensive CI/CD platform that enables teams to automate their entire software delivery pipeline through declarative YAML configuration. The platform's key strengths include flexible execution environments (Docker, Linux VM, macOS, Windows, GPU), powerful workflow orchestration with job dependencies, parallelism, and manual approvals, and extensive reusability through orbs, commands, and executors.

For integration, CircleCI connects with major version control platforms (GitHub, GitLab, Bitbucket), supports pipeline parameters for dynamic configuration, and provides robust caching and workspace features for efficient builds. Teams can leverage the Orb Registry for pre-built integrations with tools like AWS, Slack, Docker, and Kubernetes, or create custom orbs for organization-specific workflows. CircleCI Server offers self-hosted deployment options for enterprises with specific compliance or security requirements.