### Install cargo-mutants from Source Source: https://mutants.rs/installation.html Use this command to install cargo-mutants with the latest stable features. Ensure your Rust toolchain is up-to-date. ```bash cargo install --locked cargo-mutants ``` -------------------------------- ### Example Mutant Name Source: https://mutants.rs/print.html An example of a full mutant name, including file, line number, function signature, and the mutation. Regex can match any substring within this name. ```text src/outcome.rs:157: replace ::serialize -> Result with Ok(Default::default()) ``` -------------------------------- ### Cargo-Mutants Output Example Source: https://mutants.rs/print.html This output shows a typical run of cargo-mutants, detailing the number of mutants tested, build and test times, specific mutations that were not caught, and a summary of the results (missed, caught, unviable). ```text ; cargo mutants Found 14 mutants to test Copy source to scratch directory ... 0 MB in 0.0s Unmutated baseline ... ok in 1.6s build + 0.3s test Auto-set test timeout to 20.0s src/lib.rs:386: replace ::source -> Option<&(dyn std::error::Error + 'static)> with Default::default() ... NOT CAUGHT in 0.6s build + 0.3s test src/lib.rs:485: replace copy_symlink -> Result<()> with Ok(Default::default()) ... NOT CAUGHT in 0.5s build + 0.3s test 14 mutants tested in 0:08: 2 missed, 9 caught, 3 unviable ``` -------------------------------- ### Configuration File Filtering Example Source: https://mutants.rs/filter_mutants.html Shows how to specify mutant filtering rules within the `.cargo/mutants.toml` configuration file. The `exclude_re` key accepts a list of regex strings. ```toml exclude_re = ["impl Debug"] # same as -E ``` -------------------------------- ### Command-line Filtering Examples Source: https://mutants.rs/filter_mutants.html Demonstrates how to use command-line options to filter mutants. Use `-E` to exclude mutants matching a regex and `-F` to include mutants matching a regex. ```bash -E 'impl Debug' # don’t test `impl Debug` methods, because coverage of them might be considered unimportant. ``` ```bash -F 'impl Serialize' -F 'impl Deserialize' # test implementations of these two traits. ``` -------------------------------- ### Generate Fish Shell Completions for Cargo Mutants Source: https://mutants.rs/integrations.html Emits completion scripts for the Fish shell. Install these scripts to `conf.d` for your shell. ```bash cargo mutants --completions fish >~/.config/fish/conf.d/cargo-mutants-completions.fish ``` -------------------------------- ### GitHub Actions Workflow for cargo-mutants Source: https://mutants.rs/ci.html This workflow runs `cargo mutants` on pushes to main and pull requests. It installs `cargo-mutants` using `install-action`, executes the mutation tests, and uploads results as an artifact. Configure the `paths` to match your project structure. ```yaml name: cargo-mutants env: CARGO_TERM_COLOR: always on: push: branches: - main pull_request: paths: - ".cargo/mutants.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "src/**" - "testdata/**" - "tests/**" jobs: cargo-mutants: runs-on: ubuntu-latest steps: - uses: actions/checkout@v6 - uses: taiki-e/install-action@v2 with: tool: cargo-mutants - run: cargo mutants -vV --in-place - uses: actions/upload-artifact@v7 if: always() with: name: mutants-out path: mutants.out ``` -------------------------------- ### GitHub Actions Matrix for Sharding Source: https://mutants.rs/print.html Example of using a GitHub Actions matrix to shard the mutants run into 8 parts. This configuration is for a CI environment to distribute mutation testing tasks. ```yaml # Example of using a GitHub Actions matrix to shard the mutants run into 8 parts. # See https://github.com/sourcefrog/cargo-mutants/blob/main/.github/workflows/tests.yml for a full example. # Only run this on PRs or main branch commits that could affect the results, # so we don't waste time on doc-only changes. Adjust these paths and branch names ``` -------------------------------- ### GitHub Actions Matrix for Sharding Mutants Source: https://mutants.rs/shards.html This example configures a GitHub Actions workflow to run cargo-mutants across 8 shards using a matrix strategy. Ensure that the `--shard` argument's denominator matches the number of shards defined in the matrix. ```yaml on: pull_request: paths: - ".cargo/*.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "mutants_attrs/**" - "src/**" - "testdata/**" - "tests/**" push: branches: - main # Actions doesn't support YAML references, so it's repeated here paths: - ".cargo/*.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "mutants_attrs/**" - "src/**" - "testdata/**" - "tests/**" jobs: # Before testing mutants, run the build and tests on all platforms. # You probably already have CI configuration like this, so don't duplicate it, # merge cargo-mutants into your existing workflow. test: strategy: matrix: os: [macOS-latest, ubuntu-latest, windows-latest] version: [stable, nightly] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v6 - uses: dtolnay/rust-toolchain@master with: toolchain: ${{ matrix.version }} components: rustfmt - uses: swatinem/rust-cache@v2 - name: rustfmt run: cargo fmt --all -- --check - name: Build run: cargo build --all-targets - name: Test run: cargo test --workspace cargo-mutants: runs-on: ubuntu-latest # Often you'll want to only run this after the build is known to pass its basic tests, # to avoid wasting time, and to allow using --baseline=skip. needs: [test] strategy: fail-fast: false # Collect all mutants even if some are missed matrix: shard: [0, 1, 2, 3, 4, 5, 6, 7] steps: - uses: actions/checkout@v6 - uses: dtolnay/rust-toolchain@master - uses: Swatinem/rust-cache@v2 - uses: taiki-e/install-action@v2 name: Install cargo-mutants using install-action with: tool: cargo-mutants # Set an appropriate timeout for your tree here. # The denominator of the shard count must be the number of shards. - name: Mutants run: | cargo mutants --no-shuffle -vV --shard ${{ matrix.shard }}/8 --baseline=skip --timeout 300 --in-place - name: Archive mutants.out uses: actions/upload-artifact@v7 if: always() with: path: mutants.out name: mutants-shard${{matrix.shard}}.out ``` -------------------------------- ### Run 1% of Mutants with Sharding Source: https://mutants.rs/print.html Use `--shard` to test a fraction of mutants for quick checks. This is useful for testing cargo-mutants itself or getting a performance baseline. ```bash cargo mutants --shard 1/100 ``` -------------------------------- ### Filter Mutants by Package and Diff Source: https://mutants.rs/in-diff.html The --in-diff option can be combined with other filters like --package. This example tests mutants only in the 'foo' package that overlap with the specified diff. ```bash cargo mutants --in-diff --package foo ``` -------------------------------- ### Filter Mutants by Trait Implementation (Command Line) Source: https://mutants.rs/print.html Use the `-F` flag to include mutants from specific trait implementations. This example includes mutants from 'impl Serialize' and 'impl Deserialize'. ```bash cargo mutants -F 'impl Serialize' -F 'impl Deserialize' ``` -------------------------------- ### Exclude Mutants by Regex (Configuration File) Source: https://mutants.rs/print.html Configure mutant exclusion using regular expressions in the `.cargo/mutants.toml` file. This example excludes mutants matching 'impl Debug'. ```toml exclude_re = ["impl Debug"] ``` -------------------------------- ### GitHub Actions Workflow for Cargo Mutants Source: https://mutants.rs/print.html This workflow configures GitHub Actions to run mutation tests on every push to the main branch and every pull request. It installs cargo-mutants using install-action and uploads the results as an artifact. The workflow fails if any uncaught mutants are found. ```yaml # Example of how to configure a GitHub Actions workflow to run `cargo mutants` # on every push to main and every pull request that changes the code. # You could run this standalone or merge it into a workflow that runs other tests. name: cargo-mutants env: CARGO_TERM_COLOR: always on: push: branches: - main pull_request: # Only test PR if it changes something that's likely to affect the results, because # mutant tests can take a long time. Adjust these paths to suit your project. paths: - ".cargo/mutants.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "src/**" - "testdata/**" - "tests/**" jobs: cargo-mutants: runs-on: ubuntu-latest steps: - uses: actions/checkout@v6 - uses: taiki-e/install-action@v2 with: tool: cargo-mutants - run: cargo mutants -vV --in-place - uses: actions/upload-artifact@v7 if: always() with: name: mutants-out path: mutants.out ``` -------------------------------- ### Example Mutant Name Format Source: https://mutants.rs/filter_mutants.html This shows the typical format of a mutant's full name, which is used for filtering. It includes the file path, line number, function signature, and the specific mutation. ```text __ src/outcome.rs:157: replace ::serialize -> Result with Ok(Default::default()) ``` -------------------------------- ### Specify Manifest Path Source: https://mutants.rs/print.html Use the `--manifest-path` option to select the tree to test by providing a path to the `Cargo.toml` file. ```bash cargo mutants --manifest-path /path/to/Cargo.toml ``` -------------------------------- ### Run cargo-mutants with a Specific Rust Toolchain Source: https://mutants.rs/installation.html After installation, you can use cargo-mutants with older Rust toolchains by specifying the toolchain with the `+` option. This is useful for testing compatibility. ```bash cargo +1.48 mutants ``` -------------------------------- ### Specify Custom Configuration File Source: https://mutants.rs/print.html Use the `--config FILE` option to specify a custom path for the configuration file, instead of the default `.cargo/mutants.toml`. ```bash cargo mutants --config custom_mutants.toml ``` -------------------------------- ### Exclude Mutants by Trait Implementation (Command Line) Source: https://mutants.rs/print.html Use the `-E` flag to exclude mutants from specific trait implementations. This example excludes mutants from 'impl Debug'. ```bash cargo mutants -E 'impl Debug' ``` -------------------------------- ### Configure File Filters in .cargo/mutants.toml Source: https://mutants.rs/skip_files.html Define default file inclusion and exclusion patterns in the `.cargo/mutants.toml` configuration file. Command-line options override these settings. ```toml __ exclude_globs = ["src/main.rs", "src/cache/*.rs"] # like -e examine_globs = ["src/important/*.rs"] # like -f: test *only* these files ``` -------------------------------- ### Copy Version Control Directories Source: https://mutants.rs/print.html If your build or tests require version control directories, you can explicitly copy them using the --copy-vcs=true flag or by setting copy_vcs = true in .cargo/mutants.toml. ```bash cargo mutants --copy-vcs=true ``` -------------------------------- ### Rust Code Diff Example Source: https://mutants.rs/in-diff.html This diff shows the addition of a new function `two` to `src/lib.rs`. When used with `--in-diff`, cargo-mutants will focus on testing mutants affecting this new function. ```diff --- a/src/lib.rs 2023-11-12 13:05:25.774658230 -0800 +++ b/src/lib.rs 2023-11-12 12:54:04.373806696 -0800 @@ -2,6 +2,10 @@ "one".to_owned() } +pub fn two() -> String { + format!("{}", 2) +} + #[cfg(test)] mod test_super { use super::*; @@ -10,4 +14,9 @@ fn test_one() { assert_eq!(one(), "one"); } + + #[test] + fn test_two() { + assert_eq!(two(), "2"); + } } ``` -------------------------------- ### Pass Options to Test Binaries Source: https://mutants.rs/cargo-args.html Use a second double-dash (`-- --`) to pass options through to the test binaries executed by `cargo test`. ```bash cargo mutants -- -- --test-threads 1 --nocapture ``` -------------------------------- ### Combine --cargo-test-arg and -- Source: https://mutants.rs/cargo-args.html You can use both `--cargo-test-arg` and the `--` delimiter to pass different sets of arguments to `cargo test`. ```bash cargo mutants --cargo-test-arg=--lib -- --all-targets ``` -------------------------------- ### Configure File Filtering in .cargo/mutants.toml Source: https://mutants.rs/print.html Configure file filtering using `exclude_globs` and `examine_globs` in `.cargo/mutants.toml`. Command-line options override configuration file options. ```toml exclude_globs = ["src/main.rs", "src/cache/*.rs"] # like -e examine_globs = ["src/important/*.rs"] # like -f: test *only* these files ``` -------------------------------- ### Example of a mutation causing a test suite hang Source: https://mutants.rs/timeouts.html This code snippet demonstrates a scenario where a mutation to `should_stop` could cause an infinite loop, leading to a test suite hang. It's recommended to skip such functions. ```rust #![allow(unused)] fn main() { while !should_stop() { // something } } ``` -------------------------------- ### Example of skipping `with_capacity` calls Source: https://mutants.rs/skip_calls.html Demonstrates how cargo-mutants might mutate the argument of `Vec::with_capacity`. By default, calls to `with_capacity` are skipped to avoid mutations that could alter pre-allocation logic, which is often considered unnecessary to test explicitly. ```rust #![allow(unused)] fn main() { let mut v = Vec::with_capacity(4 * n); } ``` -------------------------------- ### Configure Feature Flags in .cargo/mutants.toml Source: https://mutants.rs/cargo-args.html Define feature flags in the `.cargo/mutants.toml` configuration file. Command-line flags take precedence. ```toml features = ["fail/failpoints", "other-feature"] all_features = true no_default_features = true ``` -------------------------------- ### GitHub Actions Workflow for Testing Source: https://mutants.rs/print.html This workflow defines jobs for building, testing, and running cargo-mutants across different platforms and Rust toolchains. ```yaml on: pull_request: paths: - ".cargo/*.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "mutants_attrs/**" - "src/**" - "testdata/**" - "tests/**" push: branches: - main # Actions doesn't support YAML references, so it's repeated here paths: - ".cargo/*.toml" - ".github/workflows/tests.yml" - "Cargo.*" - "mutants_attrs/**" - "src/**" - "testdata/**" - "tests/**" jobs: # Before testing mutants, run the build and tests on all platforms. # You probably already have CI configuration like this, so don't duplicate it, # merge cargo-mutants into your existing workflow. test: strategy: matrix: os: [macOS-latest, ubuntu-latest, windows-latest] version: [stable, nightly] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v6 - uses: dtolnay/rust-toolchain@master with: toolchain: ${{ matrix.version }} components: rustfmt - uses: swatinem/rust-cache@v2 - name: rustfmt run: cargo fmt --all -- --check - name: Build run: cargo build --all-targets - name: Test run: cargo test --workspace cargo-mutants: runs-on: ubuntu-latest # Often you'll want to only run this after the build is known to pass its basic tests, # to avoid wasting time, and to allow using --baseline=skip. needs: [test] strategy: fail-fast: false # Collect all mutants even if some are missed matrix: shard: [0, 1, 2, 3, 4, 5, 6, 7] steps: - uses: actions/checkout@v6 - uses: dtolnay/rust-toolchain@master - uses: Swatinem/rust-cache@v2 - uses: taiki-e/install-action@v2 name: Install cargo-mutants using install-action with: tool: cargo-mutants # Set an appropriate timeout for your tree here. # The denominator of the shard count must be the number of shards. - name: Mutants run: | cargo mutants --no-shuffle -vV --shard ${{ matrix.shard }}/8 --baseline=skip --timeout 300 --in-place - name: Archive mutants.out uses: actions/upload-artifact@v7 if: always() with: path: mutants.out name: mutants-shard${{matrix.shard}}.out ``` -------------------------------- ### Sharding Cost Model (Expanded) Source: https://mutants.rs/shards.html This formula models the total CPU seconds for running mutants across multiple shards, emphasizing the impact of the number of shards on the overall cost. ```plaintext N_SHARDS * (SHARD_STARTUP + CLEAN_BUILD + TEST) + N_MUTANTS * (INCREMENTAL_BUILD + TEST) ``` -------------------------------- ### GitHub Actions Workflow for Incremental Mutants Testing Source: https://mutants.rs/pr-diff.html This GitHub Actions workflow runs `cargo mutants` only on code changed in a pull request using the `--in-diff` flag. It fetches the full git history, generates a diff, installs `cargo-mutants`, and executes the test. Artifacts are uploaded for review. ```yaml name: Tests permissions: contents: read env: CARGO_TERM_COLOR: always on: push: branches: - main pull_request: jobs: incremental-mutants: runs-on: ubuntu-latest if: github.event_name == 'pull_request' steps: - uses: actions/checkout@v6 with: fetch-depth: 0 - name: Relative diff run: | git branch -av git diff origin/${{ github.base_ref }}.. | tee git.diff - uses: Swatinem/rust-cache@v2 - uses: taiki-e/install-action@v2 name: Install cargo-mutants using install-action with: tool: cargo-mutants - name: Mutants run: | cargo mutants --no-shuffle -vV --in-diff git.diff - name: Archive mutants.out uses: actions/upload-artifact@v7 if: always() with: name: mutants-incremental.out path: mutants.out ``` -------------------------------- ### GitHub Actions Workflow for Incremental Mutation Testing Source: https://mutants.rs/print.html This workflow demonstrates how to use the `--in-diff` feature of cargo-mutants to test only the code that has changed in a pull request. It fetches the diff, installs cargo-mutants, and runs the tests on the changed sections. This provides faster feedback for pull requests but may miss some issues found by full codebase testing. ```yaml # An example of how to run cargo-mutants on only the sections of code that have changed in a pull request, # using the `--in-diff` feature of cargo-mutants. # # This can give much faster feedback on pull requests, but can miss some problems that # would be found by running mutants on the whole codebase. name: Tests permissions: contents: read env: CARGO_TERM_COLOR: always on: push: branches: - main pull_request: jobs: incremental-mutants: runs-on: ubuntu-latest if: github.event_name == 'pull_request' steps: - uses: actions/checkout@v6 with: fetch-depth: 0 - name: Relative diff run: | git branch -av git diff origin/`${{ github.base_ref }}`.. | tee git.diff - uses: Swatinem/rust-cache@v2 - uses: taiki-e/install-action@v2 name: Install cargo-mutants using install-action with: tool: cargo-mutants - name: Mutants run: | cargo mutants --no-shuffle -vV --in-diff git.diff - name: Archive mutants.out uses: actions/upload-artifact@v7 if: always() with: name: mutants-incremental.out path: mutants.out ``` -------------------------------- ### Run Cargo-Mutants with Error Value Configuration Source: https://mutants.rs/error-values.html Execute cargo-mutants with specific flags, including `-F anyhow` and `-vV`, to utilize the configured error values for mutation testing. ```bash cargo r mutants -F anyhow -vV -j4 ```