cargo-mutants

https://mutants.rs
Admin
A mutation testing tool for Rust that helps improve program quality by finding places where bugs can...
Tokens:30,436
Snippets:468
Trust Score:7
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install and Run cargo-mutants

Source: https://mutants.rs/print.html

The standard command sequence to install the tool and execute it against a Rust project.

```bash
cargo install cargo-mutants && cargo mutants
```

--------------------------------

### Install cargo-mutants

Source: https://mutants.rs/print.html

Installs the cargo-mutants tool using the cargo package manager.

```bash
cargo install --locked cargo-mutants
```

--------------------------------

### Install Fish Shell Completions

Source: https://mutants.rs/print.html

Command to generate and install completion scripts for the Fish shell.

```bash
cargo mutants --completions fish >~/.config/fish/conf.d/cargo-mutants-completions.fish
```

--------------------------------

### Command-line file filtering examples

Source: https://mutants.rs/print.html

Examples of using command-line flags to restrict mutation testing to specific files or exclude them.

```bash
cargo mutants -f visit.rs -f change.rs
```

```bash
cargo mutants -e console.rs
```

```bash
cargo mutants -f src/db/*.rs
```

--------------------------------

### Install cargo-mutants from Source

Source: https://mutants.rs/installation.html

Use this command to install cargo-mutants with the latest stable features. Ensure your Rust toolchain is up-to-date.

```bash
cargo install --locked cargo-mutants
```

--------------------------------

### Example cargo-mutants output

Source: https://mutants.rs/print.html

Displays the console output after running cargo-mutants, showing mutation testing results.

```text
; cargo mutants
Found 14 mutants to test
Copy source to scratch directory ... 0 MB in 0.0s
Unmutated baseline ... ok in 1.6s build + 0.3s test
Auto-set test timeout to 20.0s
src/lib.rs:386: replace <impl Error for Error>::source -> Option<&(dyn std::error::Error + 'static)>
 with Default::default() ... NOT CAUGHT in 0.6s build + 0.3s test
src/lib.rs:485: replace copy_symlink -> Result<()> with Ok(Default::default()) ...
 NOT CAUGHT in 0.5s build + 0.3s test
14 mutants tested in 0:08: 2 missed, 9 caught, 3 unviable
```

--------------------------------

### Command Line Filtering Examples

Source: https://mutants.rs/print.html

Use -E to exclude specific patterns and -F to include only those matching the provided regex.

```bash
-E 'impl Debug'
```

```bash
-F 'impl Serialize' -F 'impl Deserialize'
```

--------------------------------

### Command-line Filtering Examples

Source: https://mutants.rs/filter_mutants.html

Demonstrates how to use command-line options to filter mutants. Use `-E` to exclude mutants matching a regex and `-F` to include mutants matching a regex.

```bash
-E 'impl Debug' # don’t test `impl Debug` methods, because coverage of them might be considered unimportant.
```

```bash
-F 'impl Serialize' -F 'impl Deserialize' # test implementations of these two traits.
```

--------------------------------

### Configuration File Filtering Example

Source: https://mutants.rs/filter_mutants.html

Shows how to specify mutant filtering rules within the `.cargo/mutants.toml` configuration file. The `exclude_re` key accepts a list of regex strings.

```toml
exclude_re = ["impl Debug"] # same as -E
```

--------------------------------

### Example Diff for Incremental Testing

Source: https://mutants.rs/print.html

A sample git diff showing the addition of a function and its corresponding test, used to demonstrate how --in-diff filters mutants.

```diff
--- a/src/lib.rs    2023-11-12 13:05:25.774658230 -0800
+++ b/src/lib.rs    2023-11-12 12:54:04.373806696 -0800
@@ -2,6 +2,10 @@
     "one".to_owned()
 }

+pub fn two() -> String {
+    format!("{}", 2)
+}
+
 #[cfg(test)]
 mod test_super {
     use super::*;
@@ -10,4 +14,9 @@
     fn test_one() {
         assert_eq!(one(), "one");
     }
+
+    #[test]
+    fn test_two() {
+        assert_eq!(two(), "2");
+    }
 }
```

--------------------------------

### GitHub Actions Workflow for cargo-mutants

Source: https://mutants.rs/ci.html

This workflow runs `cargo mutants` on pushes to main and pull requests. It installs `cargo-mutants` using `install-action`, executes the mutation tests, and uploads results as an artifact. Configure the `paths` to match your project structure.

```yaml
name: cargo-mutants

env:
  CARGO_TERM_COLOR: always

on:
  push:
    branches:
      - main
  pull_request:
    paths:
      - ".cargo/mutants.toml"
      - ".github/workflows/tests.yml"
      - "Cargo.*"
      - "src/**"
      - "testdata/**"
      - "tests/**"

jobs:
  cargo-mutants:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: taiki-e/install-action@v2
        with:
          tool: cargo-mutants
      - run: cargo mutants -vV --in-place
      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: mutants-out
          path: mutants.out

```

--------------------------------

### Generate Fish Shell Completions for Cargo Mutants

Source: https://mutants.rs/integrations.html

Emits completion scripts for the Fish shell. Install these scripts to `conf.d` for your shell.

```bash
cargo mutants --completions fish >~/.config/fish/conf.d/cargo-mutants-completions.fish
```

--------------------------------

### Example of with_capacity usage

Source: https://mutants.rs/print.html

A standard usage of Vec::with_capacity that cargo-mutants would typically attempt to mutate.

```rust
#![allow(unused)]
fn main() {
    let mut v = Vec::with_capacity(4 * n);
}
```

--------------------------------

### GitHub Actions Matrix for Sharding Mutants

Source: https://mutants.rs/shards.html

This example configures a GitHub Actions workflow to run cargo-mutants across 8 shards using a matrix strategy. Ensure that the `--shard` argument's denominator matches the number of shards defined in the matrix.

```yaml
on:
  pull_request:
    paths:
      - ".cargo/*.toml"
      - ".github/workflows/tests.yml"
      - "Cargo.*"
      - "mutants_attrs/**"
      - "src/**"
      - "testdata/**"
      - "tests/**"
  push:
    branches:
      - main
    # Actions doesn't support YAML references, so it's repeated here
    paths:
      - ".cargo/*.toml"
      - ".github/workflows/tests.yml"
      - "Cargo.*"
      - "mutants_attrs/**"
      - "src/**"
      - "testdata/**"
      - "tests/**"

jobs:
  # Before testing mutants, run the build and tests on all platforms.
  # You probably already have CI configuration like this, so don't duplicate it,
  # merge cargo-mutants into your existing workflow.
  test:
    strategy:
      matrix:
        os: [macOS-latest, ubuntu-latest, windows-latest]
        version: [stable, nightly]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v6
      - uses: dtolnay/rust-toolchain@master
        with:
          toolchain: ${{ matrix.version }}
          components: rustfmt
      - uses: swatinem/rust-cache@v2
      - name: rustfmt
        run: cargo fmt --all -- --check
      - name: Build
        run: cargo build --all-targets
      - name: Test
        run: cargo test --workspace
  cargo-mutants:
    runs-on: ubuntu-latest
    # Often you'll want to only run this after the build is known to pass its basic tests,
    # to avoid wasting time, and to allow using --baseline=skip.
    needs: [test]
    strategy:
      fail-fast: false # Collect all mutants even if some are missed
      matrix:
        shard: [0, 1, 2, 3, 4, 5, 6, 7]
    steps:
      - uses: actions/checkout@v6
      - uses: dtolnay/rust-toolchain@master
      - uses: Swatinem/rust-cache@v2
      - uses: taiki-e/install-action@v2
        name: Install cargo-mutants using install-action
        with:
          tool: cargo-mutants
      # Set an appropriate timeout for your tree here.
      # The denominator of the shard count must be the number of shards.
      - name: Mutants
        run: |
          cargo mutants --no-shuffle -vV --shard ${{ matrix.shard }}/8 --baseline=skip --timeout 300 --in-place
      - name: Archive mutants.out
        uses: actions/upload-artifact@v7
        if: always()
        with:
          path: mutants.out
          name: mutants-shard${{matrix.shard}}.out

```

--------------------------------

### Example Mutant Name Format

Source: https://mutants.rs/print.html

The full name of a mutant includes the file path, line number, and the specific code change applied.

```text
src/outcome.rs:157: replace <impl Serialize for ScenarioOutcome>::serialize -> Result<S::Ok, S::Error> with Ok(Default::default())
```

--------------------------------

### Filter Mutants by Package and Diff

Source: https://mutants.rs/in-diff.html

The --in-diff option can be combined with other filters like --package. This example tests mutants only in the 'foo' package that overlap with the specified diff.

```bash
cargo mutants --in-diff --package foo
```

--------------------------------

### Example Mutant Name Format

Source: https://mutants.rs/filter_mutants.html

This shows the typical format of a mutant's full name, which is used for filtering. It includes the file path, line number, function signature, and the specific mutation.

```text
__
src/outcome.rs:157: replace <impl Serialize for ScenarioOutcome>::serialize -> Result<S::Ok, S::Error> with Ok(Default::default())
```

--------------------------------

### Pass arguments to test binaries

Source: https://mutants.rs/print.html

Use a second double-dash to pass options directly to the test binaries.

```bash
cargo mutants -- -- --test-threads 1 --nocapture
```

--------------------------------

### Configure a ramdisk for temporary files on Linux

Source: https://mutants.rs/print.html

Mount a tmpfs directory and set the TMPDIR environment variable to improve I/O performance during mutation testing.

```bash
sudo mkdir /ram
sudo mount -t tmpfs /ram /ram
sudo chmod 1777 /ram
env TMPDIR=/ram cargo mutants
```

--------------------------------

### Struct Literal Mutation Example

Source: https://mutants.rs/print.html

Demonstrates a struct literal with a base expression that cargo-mutants can mutate by deleting individual fields.

```rust
#![allow(unused)]
fn main() {
let cat = Cat {
    name: "Felix",
    coat: Coat::Tuxedo,
    ..Default::default()
};
}
```

--------------------------------

### Example of a loop that may hang when mutated

Source: https://mutants.rs/print.html

A code structure where mutating the loop condition can lead to an infinite loop, causing the test suite to hang.

```rust
#![allow(unused)]
fn main() {
    while !should_stop() {
      // something
    }
}
```

--------------------------------

### Run cargo-mutants with a Specific Rust Toolchain

Source: https://mutants.rs/installation.html

After installation, you can use cargo-mutants with older Rust toolchains by specifying the toolchain with the `+` option. This is useful for testing compatibility.

```bash
cargo +1.48 mutants
```

--------------------------------

### Configure File Filters in .cargo/mutants.toml

Source: https://mutants.rs/skip_files.html

Define default file inclusion and exclusion patterns in the `.cargo/mutants.toml` configuration file. Command-line options override these settings.

```toml
__
exclude_globs = ["src/main.rs", "src/cache/*.rs"] # like -e
examine_globs = ["src/important/*.rs"] # like -f: test *only* these files

```

--------------------------------

### CPU Cost Model

Source: https://mutants.rs/print.html

A model for estimating the total CPU seconds consumed by all shards.

```text
N_SHARDS * (SHARD_STARTUP + CLEAN_BUILD + TEST) + N_MUTANTS * (INCREMENTAL_BUILD + TEST)
```

--------------------------------

### Pass Options to Test Binaries

Source: https://mutants.rs/cargo-args.html

Use a second double-dash (`-- --`) to pass options through to the test binaries executed by `cargo test`.

```bash
cargo mutants -- -- --test-threads 1 --nocapture
```

--------------------------------

### Rust Code Diff Example

Source: https://mutants.rs/in-diff.html

This diff shows the addition of a new function `two` to `src/lib.rs`. When used with `--in-diff`, cargo-mutants will focus on testing mutants affecting this new function.

```diff
--- a/src/lib.rs    2023-11-12 13:05:25.774658230 -0800
+++ b/src/lib.rs    2023-11-12 12:54:04.373806696 -0800
@@ -2,6 +2,10 @@
     "one".to_owned()
 }

+pub fn two() -> String {
+    format!("{}", 2)
+}
+
 #[cfg(test)]
 mod test_super {
     use super::*;
@@ -10,4 +14,9 @@
     fn test_one() {
         assert_eq!(one(), "one");
     }
+
+    #[test]
+    fn test_two() {
+        assert_eq!(two(), "2");
+    }
 }

```

--------------------------------

### Combine --cargo-test-arg and --

Source: https://mutants.rs/cargo-args.html

You can use both `--cargo-test-arg` and the `--` delimiter to pass different sets of arguments to `cargo test`.

```bash
cargo mutants --cargo-test-arg=--lib -- --all-targets
```

--------------------------------

### Configure skipped calls via CLI and config file

Source: https://mutants.rs/print.html

Specify functions to skip during mutation using command-line arguments or the mutants.toml configuration file.

```bash
cargo mutants --skip-calls=skip_this,and_this
```

```toml
skip_calls = ["skip_this", "and_this"]
```

--------------------------------

### Configure Feature Flags in .cargo/mutants.toml

Source: https://mutants.rs/cargo-args.html

Define feature flags in the `.cargo/mutants.toml` configuration file. Command-line flags take precedence.

```toml
features = ["fail/failpoints", "other-feature"]
all_features = true
no_default_features = true
```

--------------------------------

### Example of a mutation causing a test suite hang

Source: https://mutants.rs/timeouts.html

This code snippet demonstrates a scenario where a mutation to `should_stop` could cause an infinite loop, leading to a test suite hang. It's recommended to skip such functions.

```rust
#![allow(unused)]
fn main() {
    while !should_stop() {
      // something
    }
}
```

--------------------------------

### Example of skipping `with_capacity` calls

Source: https://mutants.rs/skip_calls.html

Demonstrates how cargo-mutants might mutate the argument of `Vec::with_capacity`. By default, calls to `with_capacity` are skipped to avoid mutations that could alter pre-allocation logic, which is often considered unnecessary to test explicitly.

```rust
#![allow(unused)]
fn main() {
    let mut v = Vec::with_capacity(4 * n);
}
```

--------------------------------

### Configure GitHub Actions for sharded mutation testing

Source: https://mutants.rs/print.html

Uses a matrix strategy to split the mutation testing workload into 8 parts. Ensure the shard denominator in the --shard argument matches the total number of shards in the matrix.

```yaml
__
# Example of using a GitHub Actions matrix to shard the mutants run into 8 parts.

# See https://github.com/sourcefrog/cargo-mutants/blob/main/.github/workflows/tests.yml for a full example.

# Only run this on PRs or main branch commits that could affect the results,
# so we don't waste time on doc-only changes. Adjust these paths and branch names
# to suit your project.
on:
  pull_request:
    paths:
      - ".cargo/*.toml"
      - ".github/workflows/tests.yml"
      - "Cargo.*"
      - "mutants_attrs/**"
      - "src/**"
      - "testdata/**"
      - "tests/**"
  push:
    branches:
      - main
    # Actions doesn't support YAML references, so it's repeated here
    paths:
      - ".cargo/*.toml"
      - ".github/workflows/tests.yml"
      - "Cargo.*"
      - "mutants_attrs/**"
      - "src/**"
      - "testdata/**"
      - "tests/**"

jobs:
  # Before testing mutants, run the build and tests on all platforms.
  # You probably already have CI configuration like this, so don't duplicate it,
  # merge cargo-mutants into your existing workflow.
  test:
    strategy:
      matrix:
        os: [macOS-latest, ubuntu-latest, windows-latest]
        version: [stable, nightly]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v6
      - uses: dtolnay/rust-toolchain@master
        with:
          toolchain: ${{ matrix.version }}
          components: rustfmt
      - uses: swatinem/rust-cache@v2
      - name: rustfmt
        run: cargo fmt --all -- --check
      - name: Build
        run: cargo build --all-targets
      - name: Test
        run: cargo test --workspace
  cargo-mutants:
    runs-on: ubuntu-latest
    # Often you'll want to only run this after the build is known to pass its basic tests,
    # to avoid wasting time, and to allow using --baseline=skip.
    needs: [test]
    strategy:
      fail-fast: false # Collect all mutants even if some are missed
      matrix:
        shard: [0, 1, 2, 3, 4, 5, 6, 7]
    steps:
      - uses: actions/checkout@v6
      - uses: dtolnay/rust-toolchain@master
      - uses: Swatinem/rust-cache@v2
      - uses: taiki-e/install-action@v2
        name: Install cargo-mutants using install-action
        with:
          tool: cargo-mutants
      # Set an appropriate timeout for your tree here.
      # The denominator of the shard count must be the number of shards.
      - name: Mutants
        run: |
          cargo mutants --no-shuffle -vV --shard ${{ matrix.shard }}/8 --baseline=skip --timeout 300 --in-place
      - name: Archive mutants.out
        uses: actions/upload-artifact@v7
        if: always()
        with:
          path: mutants.out
          name: mutants-shard${{matrix.shard}}.out
```

--------------------------------

### GitHub Actions Workflow for Incremental Mutants Testing

Source: https://mutants.rs/pr-diff.html

This GitHub Actions workflow runs `cargo mutants` only on code changed in a pull request using the `--in-diff` flag. It fetches the full git history, generates a diff, installs `cargo-mutants`, and executes the test. Artifacts are uploaded for review.

```yaml
name: Tests

permissions:
  contents: read

env:
  CARGO_TERM_COLOR: always

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  incremental-mutants:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0
      - name: Relative diff
        run: |
          git branch -av
          git diff origin/${{ github.base_ref }}.. | tee git.diff
      - uses: Swatinem/rust-cache@v2
      - uses: taiki-e/install-action@v2
        name: Install cargo-mutants using install-action
        with:
          tool: cargo-mutants
      - name: Mutants
        run: |
          cargo mutants --no-shuffle -vV --in-diff git.diff
      - name: Archive mutants.out
        uses: actions/upload-artifact@v7
        if: always()
        with:
          name: mutants-incremental.out
          path: mutants.out

```

--------------------------------

### Sharding Cost Model (Expanded)

Source: https://mutants.rs/shards.html

This formula models the total CPU seconds for running mutants across multiple shards, emphasizing the impact of the number of shards on the overall cost.

```plaintext
N_SHARDS * (SHARD_STARTUP + CLEAN_BUILD + TEST) + N_MUTANTS * (INCREMENTAL_BUILD + TEST)
```

--------------------------------

### Execution Time Model

Source: https://mutants.rs/print.html

A model for estimating the total execution time across all shards running in parallel.

```text
SHARD_STARTUP + (CLEAN_BUILD + TEST) + (N_TOTAL_MUTANTS / N_SHARDS) * (INCREMENTAL_BUILD + TEST)
```

--------------------------------

### Default Excluded VCS Directories

Source: https://mutants.rs/print.html

List of version control directories that are excluded from the copied build tree by default.

```text
__
.git
.hg
.jj
.bzr
.svn
_darcs
.pijul
```

--------------------------------

### Run cargo-mutants with specific filters

Source: https://mutants.rs/print.html

Execute cargo-mutants with specific filters and verbosity settings to isolate generated mutants.

```bash
cargo r mutants -F anyhow -vV -j4
```
Tokens:
Raw