TestivAI

https://testiv.ai/docs
Admin
TestivAI is an open-source visual regression testing tool that pairs screenshots with DOM snapshots...
Tokens:54,261
Snippets:447
Trust Score:3.6
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Install @testivai/witness CLI

Source: https://testiv.ai/docs/intro

Install the framework-agnostic CLI for the experimental sidecar mode.

```bash
npm install -D @testivai/witness  
```

--------------------------------

### Install dependencies with bundle install

Source: https://testiv.ai/docs/frameworks/ruby

Installs the gem dependencies.

```bash
bundle install  

```

--------------------------------

### Run setup wizard

Source: https://testiv.ai/docs/intro

Run the setup wizard to detect your framework and generate helper files plus a testivai.config.ts.

```bash
npx testivai init  
```

--------------------------------

### Install Playwright adapter and Chromium

Source: https://testiv.ai/docs/intro

Installs the Playwright adapter and the Playwright test runner, then installs the Chromium browser. Required before using the reporter and capture call.

```bash
npm install -D @testivai/witness-playwright @playwright/test  
npx playwright install chromium  
```

--------------------------------

### Install @testivai/witness-webdriverio

Source: https://testiv.ai/docs/frameworks/webdriverio

Installs the TestivAI Witness WebdriverIO adapter and its core package as dev dependencies. WebdriverIO is a peer dependency and should already be present from an existing setup.

```bash
npm install -D @testivai/witness @testivai/witness-webdriverio  
```

--------------------------------

### testivai run examples for different test frameworks

Source: https://testiv.ai/docs/cli/run

Examples of running testivai with various test frameworks: Cypress, pytest, Jest + Selenium, Maven/JUnit, Robot Framework, and RSpec.

```bash
# Cypress  
testivai run "cypress run --browser chrome"  
  
# pytest  
testivai run "pytest tests/ -v"  
  
# Jest + Selenium  
testivai run "npx jest tests/"  
  
# Maven / JUnit  
testivai run "mvn test"  
  
# Robot Framework  
testivai run "robot tests/"  
  
# RSpec  
testivai run "bundle exec rspec"  
```

--------------------------------

### Install TestivAI Python Selenium adapter

Source: https://testiv.ai/docs/frameworks/selenium

Install the TestivAI Python Selenium adapter. Use this before writing Python Selenium tests with TestivAI.

```bash
pip install "testivai[selenium]"  
```

--------------------------------

### Complete .testivai/config.json example

Source: https://testiv.ai/docs/guides/stable-baselines

Complete example of the `.testivai/config.json` file with all optional keys (`reportDir`, `ignoreSelectors`, `mask`, `diffRegions`) added by hand. Every field is optional and falls back to a safe default; delete any field you don't need. `npx testivai init` scaffolds a smaller version with only the defaults.

```json
{  
  "mode": "local",  
  "threshold": 0.1,  
  "reportDir": "visual-report",  
  "autoOpen": true,  
  "failOnDiff": false,  
  "stabilize": true,  
  "maxDiffPercent": 0,  
  "noiseAutoPass": false,  
  "noiseMaxDiffPercent": 1,  
  "ignoreSelectors": [  
    "#live-chat",  
    ".timestamp",  
    { "selector": "#dynamic-footer", "mode": "collapse" }  
  ],  
  "mask": [  
    "[data-testid='user-avatar']"  
  ],  
  "diffRegions": {  
    "minSize": 10,  
    "mergeDistance": 12  
  }  
}  

```

--------------------------------

### Install testivai for Playwright or Selenium

Source: https://testiv.ai/docs/frameworks/python

Install the Python adapter from PyPI. Use the playwright extra for playwright-python or the selenium extra for Selenium. Node.js is required for compare/report and is already bundled with Playwright for Python.

```bash
pip install testivai[playwright]   # or testivai[selenium]  

```

--------------------------------

### Playwright setup output

Source: https://testiv.ai/docs/cli/init

Output shown after scaffolding a Playwright project, including the reporter snippet, capture example, and commands to run and approve.

```text
✓ TestivAI is set up for Playwright (local mode).  
✓ .testivai/config.json  
✓ .testivai/baselines/  
✓ .gitignore  
  
1. Add the reporter to playwright.config.ts:  
    reporter: [['list'], ['@testivai/witness-playwright/reporter']],  
  
2. Capture a snapshot in a test:  
    import { witness } from '@testivai/witness-playwright';  
    await witness(page, testInfo, 'homepage');  
  
3. Run, review, approve:  
    npx playwright test  
    open visual-report/index.html  
    npx testivai approve --all   # then commit .testivai/baselines/  
```

--------------------------------

### Install TestivAI Selenium JavaScript adapter

Source: https://testiv.ai/docs/frameworks/selenium

Install the TestivAI Selenium adapter and its dependencies for JavaScript. Use this before writing any JavaScript Selenium tests with TestivAI.

```bash
npm i -D @testivai/witness-selenium @testivai/witness selenium-webdriver  
```

--------------------------------

### Install @testivai/witness-playwright

Source: https://testiv.ai/docs

Install the TestivAI Witness Playwright package as a dev dependency. This is the primary installation command for the open-source visual regression testing tool.

```bash
npm i -D @testivai/witness-playwright
```

--------------------------------

### Java Selenium witness example

Source: https://testiv.ai/docs/frameworks/selenium

Java example using the TestivAI Selenium adapter. It creates a ChromeDriver, navigates to a local page, and calls `SeleniumWitness.witness(driver, "homepage")` to capture a visual snapshot.

```java
import ai.testiv.testivai.SeleniumWitness;  
  
WebDriver driver = new ChromeDriver();  
driver.get("http://localhost:3000");  
SeleniumWitness.witness(driver, "homepage");  
```

--------------------------------

### Install @playwright/test

Source: https://testiv.ai/docs/frameworks/playwright

Installs the Playwright test framework, a prerequisite for using the TestivAI Playwright SDK.

```bash
npm install @playwright/test  
```

--------------------------------

### Start Chrome in GitHub Actions CI

Source: https://testiv.ai/docs/guides/troubleshooting

GitHub Actions step to start Chrome in headless mode with remote debugging, no sandbox, and disabled dev-shm usage to fix Chrome not starting in CI.

```yaml
- name: Start Chrome  
  run: |  
    google-chrome \
      --remote-debugging-port=9222 \
      --no-sandbox \
      --disable-dev-shm-usage \
      --headless &  
```

--------------------------------

### Python Selenium witness example

Source: https://testiv.ai/docs/frameworks/selenium

Python example using the TestivAI Selenium adapter. It uses a driver fixture, navigates to a local page, and calls `witness(driver, "homepage")` to capture a visual snapshot.

```python
from selenium import webdriver  
from testivai.selenium import witness  
  
def test_homepage(driver):          # your own driver fixture  
    driver.get("http://localhost:3000")  
    witness(driver, "homepage")  
```

--------------------------------

### JavaScript Selenium witness example

Source: https://testiv.ai/docs/frameworks/selenium

JavaScript example using the TestivAI Selenium adapter. It builds a Chrome driver, navigates to a local page, and calls `testivai.witness(driver, 'homepage')` to capture a visual snapshot.

```javascript
import { Builder } from 'selenium-webdriver';  
import { testivai } from '@testivai/witness-selenium';  
  
const driver = await new Builder().forBrowser('chrome').build();  
await driver.get('http://localhost:3000');  
await testivai.witness(driver, 'homepage');  
await driver.quit();  
```

--------------------------------

### Per-platform baselines with {platform} token

Source: https://testiv.ai/docs/guides/stable-baselines

Configure per-platform baseline directories using the {platform} token. Each OS gets its own baseline set, so Mac devs gate against baselines-darwin/ and CI gates against baselines-linux/. Baselines for each platform must be approved from a run on that platform.

```json
{
  "baselinesDir": ".testivai/baselines-{platform}"
}
```

--------------------------------

### Interactive wizard prompts

Source: https://testiv.ai/docs/cli/init

Example prompts shown when running the wizard without flags in a non-Playwright project. The wizard asks for mode, language, framework, and test directory.

```text
? Select mode:                 › Local — visual diffs on your machine, HTML reports  
? Select your language:        › JavaScript / TypeScript  
? Select your test framework:  › Cypress  
? Where are your test files?   › cypress/e2e  
```

--------------------------------

### No results found message for sharded CI

Source: https://testiv.ai/docs/mcp

Example message the server returns when no report exists yet, explaining why rather than assuming tests never ran. Shown for a capture-only shard in a sharded CI setup.

```text
No results found at visual-report/results.json because this is capture-only  
shard 3/8. 1 capture(s) are in .testivai/temp/, but no comparison ran here — a  
shard sees only its slice of the suite. The report is produced by the job that  
merges every shard.  
```

--------------------------------

### Initialize and witness locally with npx testivai

Source: https://testiv.ai/docs/guides/vibe-coded-apps

Initializes TestivAI in local mode and captures the running app at localhost:3000. The first command creates .testivai/config.json; the second opens headless Chrome, follows same-origin links, and captures screenshots and DOM for each page.

```bash
npx testivai init            # choose local mode — creates .testivai/config.json  
npx testivai witness http://localhost:3000  
```

--------------------------------

### Usage: npx testivai init [options]

Source: https://testiv.ai/docs/cli/init

Basic command to scaffold TestivAI in a project. Options include `-f, --force`, `-y, --yes`, and `--json`.

```bash
npx testivai init [options]  
```

--------------------------------

### Typical approval loop

Source: https://testiv.ai/docs/cli/approve

Standard workflow: run tests, review report, approve all, and commit baselines.

```bash
npx playwright test              # capture + report  
open visual-report/index.html    # review the diffs  
npx testivai approve --all       # accept the new look  
git add .testivai/baselines/     # commit the baselines  
```

--------------------------------

### Approve baselines and commit

Source: https://testiv.ai/docs/cli/capture

First run creates baselines. Approve them and commit to version control.

```bash
npx testivai approve --all  
git add .testivai/baselines/  
```

--------------------------------

### Start Chrome with remote debugging

Source: https://testiv.ai/docs/guides/troubleshooting

Start Chrome with remote debugging enabled on port 9222 to resolve the 'Browser debugging endpoint not found' error in the Witness SDK.

```bash
chrome --remote-debugging-port=9222  
```

--------------------------------

### Configure Testivai.witness options

Source: https://testiv.ai/docs/frameworks/ruby

Shows all available options for Testivai.witness. Anything omitted falls back to .testivai/config.json.

```ruby
Testivai.witness(page, "checkout",  
  ignore_selectors: [".live-chat", "#clock"],  # hidden from pixels AND the DOM signal  
  stabilize: true,        # freeze animations, hide caret, await web fonts  
  skip_dom: false,        # the DOM snapshot powers the noise hint  
  skip_element_map: false,# the element map powers selector attribution  
  max_elements: 3000,     # cap for very large pages  
  project_root: nil)      # defaults to the nearest ancestor holding .testivai/  

```

--------------------------------

### Run suite and generate visual report

Source: https://testiv.ai/docs/frameworks/ruby

Runs the test suite and then generates a visual report. The first run creates baselines under .testivai/baselines/; later runs diff against them and write visual-report/index.html.

```bash
bundle exec rspec     # your suite, unchanged — captures land in .testivai/temp/  
npx testivai report   # diff against baselines + write the HTML report  

```

--------------------------------

### Maven dependency for TestivAI Java adapter

Source: https://testiv.ai/docs/frameworks/selenium

Maven dependency for the TestivAI Java adapter. Note that the artifact is not on Maven Central yet; you must build and install it locally with `cd java && mvn install`.

```xml
<dependency>  
  <groupId>ai.testiv</groupId>  
  <artifactId>testivai</artifactId>  
  <version>0.1.0-SNAPSHOT</version>  
</dependency>  
<!-- you already have org.seleniumhq.selenium:selenium-java -->  
```

--------------------------------

### Jenkins pipeline for visual tests (Jenkinsfile)

Source: https://testiv.ai/docs/guides/ci-cd

Jenkins pipeline for visual testing with Playwright and Testivai. Installs dependencies, installs Chromium with dependencies, runs tests, and gates on visual diffs. Archives visual-report artifacts on failure.

```groovy
pipeline {  
    agent any  
    stages {  
        stage('Visual Tests') {  
            steps {  
                sh 'npm ci'  
                sh 'npx playwright install --with-deps chromium'  
                sh 'npx playwright test'  
                sh 'npx testivai report --fail-on-diff'  
            }  
        }  
    }  
    post {  
        failure {  
            archiveArtifacts artifacts: 'visual-report/**'  
        }  
    }  
}  

```

--------------------------------

### On-disk layout under .testivai/

Source: https://testiv.ai/docs/extension-api

The on-disk contract for TestIVAI: a `.testivai/` directory with `config.json`, `baselines/<name>/` (screenshot.png, metadata.json, optional dom.html, and a `.previous/` snapshot), and `temp/<name>/` for captures. Adapters must write a full-page screenshot to `temp/<name>/screenshot.png` and optionally `dom.html`.

```text
<projectRoot>/
└── .testivai/
    ├── config.json             # { "mode": "local", "threshold": 0.1, ... }
    ├── baselines/
    │   └── <name>/
    │       ├── screenshot.png  # required (PNG bytes)
    │       ├── metadata.json   # required ({ name, createdAt, updatedAt, ... })
    │       ├── dom.html        # optional — enables the noise-hint signal
    │       └── .previous/      # snapshot of the baseline before the last approve
    │           ├── screenshot.png
    │           ├── metadata.json
    │           └── dom.html
    └── temp/
        └── <name>/
            ├── screenshot.png  # required
            └── dom.html        # optional
```

--------------------------------

### GitHub Actions workflow for visual regression tests

Source: https://testiv.ai/docs/guides/ci-cd

Defines a CI job that installs dependencies, installs Chromium, builds the project, captures screenshots with TESTIVAI_CAPTURE_ONLY, runs the visual diff gate with --fail-on-diff, and uploads the report artifact on failure. Baselines are committed to git.

```yaml
name: Visual Regression Tests  
  
on:  
  push:  
    branches: [main]  
  pull_request:  
    branches: [main]  
  
jobs:  
  visual-tests:  
    runs-on: ubuntu-latest  
    steps:  
      - uses: actions/checkout@v4  
  
      - uses: actions/setup-node@v4  
        with:  
          node-version: 20  
          cache: 'npm'  
  
      - run: npm ci  
  
      - run: npx playwright install chromium --with-deps  
  
      - run: npm run build  
  
      # Capture only: the reporter writes .testivai/temp/ and skips comparing.  
      # In CI the gate below is what decides pass/fail, so comparing twice  
      # only produces a second, non-authoritative summary.  
      - run: npm test  
        env:  
          TESTIVAI_CAPTURE_ONLY: '1'  
  
      - name: Visual diff gate  
        run: npx testivai report --fail-on-diff  
  
      - name: Upload visual report  
        if: failure()  
        uses: actions/upload-artifact@v4  
        with:  
          name: visual-report  
          path: visual-report/  
```

--------------------------------

### Configure local mode and pages in .testivai/config.json

Source: https://testiv.ai/docs/guides/vibe-coded-apps

Configures TestivAI in local mode with a fixed list of pages to witness. Place this in .testivai/config.json to avoid passing --pages on the command line.

```json
{ "mode": "local", "pages": ["/", "/pricing", "/dashboard"] }  
```

--------------------------------

### Run TestivAI witness on localhost

Source: https://testiv.ai/docs/guides/ai-agents

Run `npx testivai witness` against a local server to capture every page (crawled or using `--pages`), diff against baselines, and write the report plus `results.json`. Use for apps without a test framework.

```bash
npx testivai witness http://localhost:3000  

```

--------------------------------

### PR comment example with TestivAI visual report

Source: https://testiv.ai/docs/github-action

Example of the PR comment posted by the Action. It includes a summary line, changed snapshots with diff percentages, DOM noise hints, and approval CLI commands. The comment is upserted per PR and identified by the marker <!-- testivai-visual-report -->.

```markdown
### 🔍 TestivAI Visual Report  
  
✅ 12 passed · ⚠️ 3 changed · 🆕 2 new  
  
#### Changed Snapshots  
<details>  
<summary>checkout-page — 0.5% different</summary>  
  
> 💡 DOM unchanged — pixel diff is likely render noise (anti-aliasing, font hinting).  
  
/testivai approve checkout-page  
</details>  
  
<details>  
<summary>nav-redesign — 8.5% different</summary>  
  
> 🧱 DOM changed — 2 added, 1 attribute change.  
  
/testivai approve nav-redesign  
</details>  

```

--------------------------------

### Approve and commit baselines

Source: https://testiv.ai/docs/frameworks/python

On the first run, all snapshots are marked 'new'. Approve them with the CLI and commit the baselines directory to version control.

```bash
npx testivai approve --all  
git add .testivai/baselines/  

```

--------------------------------

### Configure local mode in .testivai/config.json

Source: https://testiv.ai/docs/frameworks/webdriverio

Creates the local-mode marker file at the project root. Without this file, the adapter logs a warning and skips report generation, though pixels are still captured.

```json
{  
  "mode": "local",  
  "threshold": 0.1,  
  "reportDir": "visual-report",  
  "autoOpen": false,  
  "maxDiffPercent": 0,  
  "noiseAutoPass": false,  
  "stabilize": true,  
  "ignoreSelectors": []  
}  
```

--------------------------------

### merge-captures missing shard error output

Source: https://testiv.ai/docs/guides/ci-cd

Example output from 'merge-captures' when a shard is missing its manifest. It shows the failure message and the explanation that comparing would check only part of the suite.

```text
✗ expected 8 shard(s), received 6 (missing: 4, 7)  
  A shard that crashed or was cancelled reports no manifest.  
  Comparing now would check only part of the suite.  

```

--------------------------------

### Assert delayed content before capture

Source: https://testiv.ai/docs/guides/stable-baselines

Use this snippet when content arrives on a delay and the settle wait cannot detect it. It asserts that the element is visible before capturing the snapshot, addressing the limitation that the observer only waits for work the page has already started.

```javascript
await expect(page.getByTestId('chart')).toBeVisible();  
await witness(page, testInfo, 'dashboard');  

```
Tokens:
Raw