### Install @testivai/witness CLI Source: https://testiv.ai/docs/intro Install the framework-agnostic CLI for the experimental sidecar mode. ```bash npm install -D @testivai/witness ``` -------------------------------- ### Install dependencies with bundle install Source: https://testiv.ai/docs/frameworks/ruby Installs the gem dependencies. ```bash bundle install ``` -------------------------------- ### Run setup wizard Source: https://testiv.ai/docs/intro Run the setup wizard to detect your framework and generate helper files plus a testivai.config.ts. ```bash npx testivai init ``` -------------------------------- ### Install Playwright adapter and Chromium Source: https://testiv.ai/docs/intro Installs the Playwright adapter and the Playwright test runner, then installs the Chromium browser. Required before using the reporter and capture call. ```bash npm install -D @testivai/witness-playwright @playwright/test npx playwright install chromium ``` -------------------------------- ### Install @testivai/witness-webdriverio Source: https://testiv.ai/docs/frameworks/webdriverio Installs the TestivAI Witness WebdriverIO adapter and its core package as dev dependencies. WebdriverIO is a peer dependency and should already be present from an existing setup. ```bash npm install -D @testivai/witness @testivai/witness-webdriverio ``` -------------------------------- ### testivai run examples for different test frameworks Source: https://testiv.ai/docs/cli/run Examples of running testivai with various test frameworks: Cypress, pytest, Jest + Selenium, Maven/JUnit, Robot Framework, and RSpec. ```bash # Cypress testivai run "cypress run --browser chrome" # pytest testivai run "pytest tests/ -v" # Jest + Selenium testivai run "npx jest tests/" # Maven / JUnit testivai run "mvn test" # Robot Framework testivai run "robot tests/" # RSpec testivai run "bundle exec rspec" ``` -------------------------------- ### Install TestivAI Python Selenium adapter Source: https://testiv.ai/docs/frameworks/selenium Install the TestivAI Python Selenium adapter. Use this before writing Python Selenium tests with TestivAI. ```bash pip install "testivai[selenium]" ``` -------------------------------- ### Complete .testivai/config.json example Source: https://testiv.ai/docs/guides/stable-baselines Complete example of the `.testivai/config.json` file with all optional keys (`reportDir`, `ignoreSelectors`, `mask`, `diffRegions`) added by hand. Every field is optional and falls back to a safe default; delete any field you don't need. `npx testivai init` scaffolds a smaller version with only the defaults. ```json { "mode": "local", "threshold": 0.1, "reportDir": "visual-report", "autoOpen": true, "failOnDiff": false, "stabilize": true, "maxDiffPercent": 0, "noiseAutoPass": false, "noiseMaxDiffPercent": 1, "ignoreSelectors": [ "#live-chat", ".timestamp", { "selector": "#dynamic-footer", "mode": "collapse" } ], "mask": [ "[data-testid='user-avatar']" ], "diffRegions": { "minSize": 10, "mergeDistance": 12 } } ``` -------------------------------- ### Install testivai for Playwright or Selenium Source: https://testiv.ai/docs/frameworks/python Install the Python adapter from PyPI. Use the playwright extra for playwright-python or the selenium extra for Selenium. Node.js is required for compare/report and is already bundled with Playwright for Python. ```bash pip install testivai[playwright] # or testivai[selenium] ``` -------------------------------- ### Playwright setup output Source: https://testiv.ai/docs/cli/init Output shown after scaffolding a Playwright project, including the reporter snippet, capture example, and commands to run and approve. ```text ✓ TestivAI is set up for Playwright (local mode). ✓ .testivai/config.json ✓ .testivai/baselines/ ✓ .gitignore 1. Add the reporter to playwright.config.ts: reporter: [['list'], ['@testivai/witness-playwright/reporter']], 2. Capture a snapshot in a test: import { witness } from '@testivai/witness-playwright'; await witness(page, testInfo, 'homepage'); 3. Run, review, approve: npx playwright test open visual-report/index.html npx testivai approve --all # then commit .testivai/baselines/ ``` -------------------------------- ### Install TestivAI Selenium JavaScript adapter Source: https://testiv.ai/docs/frameworks/selenium Install the TestivAI Selenium adapter and its dependencies for JavaScript. Use this before writing any JavaScript Selenium tests with TestivAI. ```bash npm i -D @testivai/witness-selenium @testivai/witness selenium-webdriver ``` -------------------------------- ### Install @testivai/witness-playwright Source: https://testiv.ai/docs Install the TestivAI Witness Playwright package as a dev dependency. This is the primary installation command for the open-source visual regression testing tool. ```bash npm i -D @testivai/witness-playwright ``` -------------------------------- ### Java Selenium witness example Source: https://testiv.ai/docs/frameworks/selenium Java example using the TestivAI Selenium adapter. It creates a ChromeDriver, navigates to a local page, and calls `SeleniumWitness.witness(driver, "homepage")` to capture a visual snapshot. ```java import ai.testiv.testivai.SeleniumWitness; WebDriver driver = new ChromeDriver(); driver.get("http://localhost:3000"); SeleniumWitness.witness(driver, "homepage"); ``` -------------------------------- ### Install @playwright/test Source: https://testiv.ai/docs/frameworks/playwright Installs the Playwright test framework, a prerequisite for using the TestivAI Playwright SDK. ```bash npm install @playwright/test ``` -------------------------------- ### Start Chrome in GitHub Actions CI Source: https://testiv.ai/docs/guides/troubleshooting GitHub Actions step to start Chrome in headless mode with remote debugging, no sandbox, and disabled dev-shm usage to fix Chrome not starting in CI. ```yaml - name: Start Chrome run: | google-chrome \ --remote-debugging-port=9222 \ --no-sandbox \ --disable-dev-shm-usage \ --headless & ``` -------------------------------- ### Python Selenium witness example Source: https://testiv.ai/docs/frameworks/selenium Python example using the TestivAI Selenium adapter. It uses a driver fixture, navigates to a local page, and calls `witness(driver, "homepage")` to capture a visual snapshot. ```python from selenium import webdriver from testivai.selenium import witness def test_homepage(driver): # your own driver fixture driver.get("http://localhost:3000") witness(driver, "homepage") ``` -------------------------------- ### JavaScript Selenium witness example Source: https://testiv.ai/docs/frameworks/selenium JavaScript example using the TestivAI Selenium adapter. It builds a Chrome driver, navigates to a local page, and calls `testivai.witness(driver, 'homepage')` to capture a visual snapshot. ```javascript import { Builder } from 'selenium-webdriver'; import { testivai } from '@testivai/witness-selenium'; const driver = await new Builder().forBrowser('chrome').build(); await driver.get('http://localhost:3000'); await testivai.witness(driver, 'homepage'); await driver.quit(); ``` -------------------------------- ### Per-platform baselines with {platform} token Source: https://testiv.ai/docs/guides/stable-baselines Configure per-platform baseline directories using the {platform} token. Each OS gets its own baseline set, so Mac devs gate against baselines-darwin/ and CI gates against baselines-linux/. Baselines for each platform must be approved from a run on that platform. ```json { "baselinesDir": ".testivai/baselines-{platform}" } ``` -------------------------------- ### Interactive wizard prompts Source: https://testiv.ai/docs/cli/init Example prompts shown when running the wizard without flags in a non-Playwright project. The wizard asks for mode, language, framework, and test directory. ```text ? Select mode: › Local — visual diffs on your machine, HTML reports ? Select your language: › JavaScript / TypeScript ? Select your test framework: › Cypress ? Where are your test files? › cypress/e2e ``` -------------------------------- ### No results found message for sharded CI Source: https://testiv.ai/docs/mcp Example message the server returns when no report exists yet, explaining why rather than assuming tests never ran. Shown for a capture-only shard in a sharded CI setup. ```text No results found at visual-report/results.json because this is capture-only shard 3/8. 1 capture(s) are in .testivai/temp/, but no comparison ran here — a shard sees only its slice of the suite. The report is produced by the job that merges every shard. ``` -------------------------------- ### Initialize and witness locally with npx testivai Source: https://testiv.ai/docs/guides/vibe-coded-apps Initializes TestivAI in local mode and captures the running app at localhost:3000. The first command creates .testivai/config.json; the second opens headless Chrome, follows same-origin links, and captures screenshots and DOM for each page. ```bash npx testivai init # choose local mode — creates .testivai/config.json npx testivai witness http://localhost:3000 ``` -------------------------------- ### Usage: npx testivai init [options] Source: https://testiv.ai/docs/cli/init Basic command to scaffold TestivAI in a project. Options include `-f, --force`, `-y, --yes`, and `--json`. ```bash npx testivai init [options] ``` -------------------------------- ### Typical approval loop Source: https://testiv.ai/docs/cli/approve Standard workflow: run tests, review report, approve all, and commit baselines. ```bash npx playwright test # capture + report open visual-report/index.html # review the diffs npx testivai approve --all # accept the new look git add .testivai/baselines/ # commit the baselines ``` -------------------------------- ### Approve baselines and commit Source: https://testiv.ai/docs/cli/capture First run creates baselines. Approve them and commit to version control. ```bash npx testivai approve --all git add .testivai/baselines/ ``` -------------------------------- ### Start Chrome with remote debugging Source: https://testiv.ai/docs/guides/troubleshooting Start Chrome with remote debugging enabled on port 9222 to resolve the 'Browser debugging endpoint not found' error in the Witness SDK. ```bash chrome --remote-debugging-port=9222 ``` -------------------------------- ### Configure Testivai.witness options Source: https://testiv.ai/docs/frameworks/ruby Shows all available options for Testivai.witness. Anything omitted falls back to .testivai/config.json. ```ruby Testivai.witness(page, "checkout", ignore_selectors: [".live-chat", "#clock"], # hidden from pixels AND the DOM signal stabilize: true, # freeze animations, hide caret, await web fonts skip_dom: false, # the DOM snapshot powers the noise hint skip_element_map: false,# the element map powers selector attribution max_elements: 3000, # cap for very large pages project_root: nil) # defaults to the nearest ancestor holding .testivai/ ``` -------------------------------- ### Run suite and generate visual report Source: https://testiv.ai/docs/frameworks/ruby Runs the test suite and then generates a visual report. The first run creates baselines under .testivai/baselines/; later runs diff against them and write visual-report/index.html. ```bash bundle exec rspec # your suite, unchanged — captures land in .testivai/temp/ npx testivai report # diff against baselines + write the HTML report ``` -------------------------------- ### Maven dependency for TestivAI Java adapter Source: https://testiv.ai/docs/frameworks/selenium Maven dependency for the TestivAI Java adapter. Note that the artifact is not on Maven Central yet; you must build and install it locally with `cd java && mvn install`. ```xml ai.testiv testivai 0.1.0-SNAPSHOT ``` -------------------------------- ### Jenkins pipeline for visual tests (Jenkinsfile) Source: https://testiv.ai/docs/guides/ci-cd Jenkins pipeline for visual testing with Playwright and Testivai. Installs dependencies, installs Chromium with dependencies, runs tests, and gates on visual diffs. Archives visual-report artifacts on failure. ```groovy pipeline { agent any stages { stage('Visual Tests') { steps { sh 'npm ci' sh 'npx playwright install --with-deps chromium' sh 'npx playwright test' sh 'npx testivai report --fail-on-diff' } } } post { failure { archiveArtifacts artifacts: 'visual-report/**' } } } ``` -------------------------------- ### On-disk layout under .testivai/ Source: https://testiv.ai/docs/extension-api The on-disk contract for TestIVAI: a `.testivai/` directory with `config.json`, `baselines//` (screenshot.png, metadata.json, optional dom.html, and a `.previous/` snapshot), and `temp//` for captures. Adapters must write a full-page screenshot to `temp//screenshot.png` and optionally `dom.html`. ```text / └── .testivai/ ├── config.json # { "mode": "local", "threshold": 0.1, ... } ├── baselines/ │ └── / │ ├── screenshot.png # required (PNG bytes) │ ├── metadata.json # required ({ name, createdAt, updatedAt, ... }) │ ├── dom.html # optional — enables the noise-hint signal │ └── .previous/ # snapshot of the baseline before the last approve │ ├── screenshot.png │ ├── metadata.json │ └── dom.html └── temp/ └── / ├── screenshot.png # required └── dom.html # optional ``` -------------------------------- ### GitHub Actions workflow for visual regression tests Source: https://testiv.ai/docs/guides/ci-cd Defines a CI job that installs dependencies, installs Chromium, builds the project, captures screenshots with TESTIVAI_CAPTURE_ONLY, runs the visual diff gate with --fail-on-diff, and uploads the report artifact on failure. Baselines are committed to git. ```yaml name: Visual Regression Tests on: push: branches: [main] pull_request: branches: [main] jobs: visual-tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 cache: 'npm' - run: npm ci - run: npx playwright install chromium --with-deps - run: npm run build # Capture only: the reporter writes .testivai/temp/ and skips comparing. # In CI the gate below is what decides pass/fail, so comparing twice # only produces a second, non-authoritative summary. - run: npm test env: TESTIVAI_CAPTURE_ONLY: '1' - name: Visual diff gate run: npx testivai report --fail-on-diff - name: Upload visual report if: failure() uses: actions/upload-artifact@v4 with: name: visual-report path: visual-report/ ``` -------------------------------- ### Configure local mode and pages in .testivai/config.json Source: https://testiv.ai/docs/guides/vibe-coded-apps Configures TestivAI in local mode with a fixed list of pages to witness. Place this in .testivai/config.json to avoid passing --pages on the command line. ```json { "mode": "local", "pages": ["/", "/pricing", "/dashboard"] } ``` -------------------------------- ### Run TestivAI witness on localhost Source: https://testiv.ai/docs/guides/ai-agents Run `npx testivai witness` against a local server to capture every page (crawled or using `--pages`), diff against baselines, and write the report plus `results.json`. Use for apps without a test framework. ```bash npx testivai witness http://localhost:3000 ``` -------------------------------- ### PR comment example with TestivAI visual report Source: https://testiv.ai/docs/github-action Example of the PR comment posted by the Action. It includes a summary line, changed snapshots with diff percentages, DOM noise hints, and approval CLI commands. The comment is upserted per PR and identified by the marker . ```markdown ### 🔍 TestivAI Visual Report ✅ 12 passed · ⚠️ 3 changed · 🆕 2 new #### Changed Snapshots
checkout-page — 0.5% different > 💡 DOM unchanged — pixel diff is likely render noise (anti-aliasing, font hinting). /testivai approve checkout-page
nav-redesign — 8.5% different > 🧱 DOM changed — 2 added, 1 attribute change. /testivai approve nav-redesign
``` -------------------------------- ### Approve and commit baselines Source: https://testiv.ai/docs/frameworks/python On the first run, all snapshots are marked 'new'. Approve them with the CLI and commit the baselines directory to version control. ```bash npx testivai approve --all git add .testivai/baselines/ ``` -------------------------------- ### Configure local mode in .testivai/config.json Source: https://testiv.ai/docs/frameworks/webdriverio Creates the local-mode marker file at the project root. Without this file, the adapter logs a warning and skips report generation, though pixels are still captured. ```json { "mode": "local", "threshold": 0.1, "reportDir": "visual-report", "autoOpen": false, "maxDiffPercent": 0, "noiseAutoPass": false, "stabilize": true, "ignoreSelectors": [] } ``` -------------------------------- ### merge-captures missing shard error output Source: https://testiv.ai/docs/guides/ci-cd Example output from 'merge-captures' when a shard is missing its manifest. It shows the failure message and the explanation that comparing would check only part of the suite. ```text ✗ expected 8 shard(s), received 6 (missing: 4, 7) A shard that crashed or was cancelled reports no manifest. Comparing now would check only part of the suite. ``` -------------------------------- ### Assert delayed content before capture Source: https://testiv.ai/docs/guides/stable-baselines Use this snippet when content arrives on a delay and the settle wait cannot detect it. It asserts that the element is visible before capturing the snapshot, addressing the limitation that the observer only waits for work the page has already started. ```javascript await expect(page.getByTestId('chart')).toBeVisible(); await witness(page, testInfo, 'dashboard'); ```