### Local Development Setup

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Provides the basic commands to clone the repository, navigate into the project directory, and install the necessary npm dependencies to set up the local development environment.

```bash
# 1. Clone and install
git clone https://github.com/YOUR_ORG/playwright-ai-qa-agent
cd playwright-ai-qa-agent
npm ci

```

--------------------------------

### Install and Run Demo App

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Steps to install dependencies and run the demo application locally. Ensure Node.js 20+ is installed.

```bash
cd demo-app
npm install
npm run dev
```

--------------------------------

### Vercel Deploy Steps

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-01-demo-web-app.md

Command-line instructions for deploying the application using Vercel. Includes installation, login, and production deployment.

```bash
npm i -g vercel
vercel login
vercel --prod

```

--------------------------------

### Install Playwright Dependencies

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Install project dependencies using npm ci and then install Playwright Chromium with necessary dependencies. This is a standard setup step for Playwright tests.

```bash
npm ci
install Playwright Chromium (`npx playwright install chromium --with-deps`)
```

--------------------------------

### Playwright Test Setup for Seed Data

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-01-demo-web-app.md

Example code demonstrating how Playwright tests can fetch seed data from the API and inject it into the browser's local storage, along with setting the break mode.

```typescript
const breakMode = process.env.BREAK_MODE ?? "none";
await page.goto(`${BASE_URL}/?qaMode=${breakMode}`);

const seed = await fetch(`${BASE_URL}/api/seed`, { method: "POST" });
const data = await seed.json();
await page.addInitScript((d) => {
  for (const [key, value] of Object.entries(d)) {
    localStorage.setItem(key, value as string);
  }
  // If query param is not used in a specific test, seed fallback mode
  sessionStorage.setItem("demo_break_mode", breakMode);
}, data);

```

--------------------------------

### Playwright Configuration Example

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Illustrates how to configure Playwright for test runs, including different command-line options for various testing scenarios like headless execution against production, headed execution with slow motion against a local server, and break modes for agent testing. Key outputs like JSON reports for the agent, HTML reports, and artifacts are also noted.

```typescript
// tests/playwright.config.ts (illustrative usage, not import)
// Run headless against production:
// BASE_URL=https://playwright-ai-qa-agent.vercel.app npx playwright test -c tests/playwright.config.ts

// Run headed with slow motion against local dev server:
// BASE_URL=http://localhost:3000 PLAYWRIGHT_SLOW_MO=400 npx playwright test --headed -c tests/playwright.config.ts

// Run with a break mode to simulate failures for agent testing:
// QA_MODE=selector-change BASE_URL=https://playwright-ai-qa-agent.vercel.app npx playwright test -c tests/playwright.config.ts

// Key outputs produced:
// test-results/results.json          ← agent reads this
// test-results/html-report/          ← published to GitHub Pages on main/develop pushes
// test-results/artifacts/            ← screenshots, traces, error-context, dom-snapshot per test
```

--------------------------------

### GitHub Actions Workflow for Playwright and AI Agent

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-02-playwright-pipeline.md

This YAML defines a GitHub Actions workflow that checks out code, sets up Node.js, installs dependencies, installs Playwright browsers, runs the Playwright showcase suite, uploads test artifacts, and triggers an AI failure agent if tests fail. It includes options for QA mode injection and preserves failed PR checks.

```yaml
name: Playwright + AI QA Agent

on:
  push:
    branches: [main, develop]
  pull_request:
  workflow_dispatch:
    inputs:
      qa_mode:
        description: 'QA mode to inject for this run'
        required: false
        default: 'none'
        type: choice
        options:
          - none
          - selector-change
          - logic-bug
          - auth-break
          - slow-network

jobs:
  test-and-analyze:
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install chromium --with-deps

      - name: Run Playwright showcase suite
        id: playwright
        env:
          BASE_URL: ${{ secrets.DEMO_APP_URL }}
          QA_MODE: ${{ github.event.inputs.qa_mode || 'none' }}
          CI: true
        run: npx playwright test -c tests/playwright.config.ts
        continue-on-error: true

      - name: Upload test artifacts
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: playwright-results
          path: test-results/
          retention-days: 7

      - name: Run AI Failure Agent
        if: steps.playwright.outcome == 'failure'
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITHUB_SERVER_URL: ${{ github.server_url }}
          GITHUB_REPOSITORY: ${{ github.repository }}
          GITHUB_RUN_ID: ${{ github.run_id }}
          GITHUB_REF_NAME: ${{ github.ref_name }}
          GITHUB_SHA: ${{ github.sha }}
        run: npm run agent

      - name: Mark job failed if tests failed
        if: steps.playwright.outcome == 'failure'
        run: exit 1

```

--------------------------------

### GitHub Actions Workflow for Playwright and AI QA Agent

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

This YAML defines a GitHub Actions workflow that orchestrates the testing pipeline. It includes installing dependencies, running Playwright tests with `continue-on-error`, uploading artifacts, publishing reports, and running the AI failure agent specifically when Playwright tests fail. It supports manual dispatch with configurable inputs for QA modes and agent behavior, and requires specific GitHub Secrets and Repository Variables.

```yaml
# Manual dispatch: Actions → "Playwright + AI QA Agent" → Run workflow
# Inputs:
#   qa_mode: selector-change     # inject a locator-breaking mode
#   enable_agent_reporter: true  # create GitHub Issues for classified failures
#   enable_agent_healer: false   # skip healer PR for this run
#   llm_provider: anthropic      # override repo-default AI provider

# Required GitHub Secrets:
#   ANTHROPIC_API_KEY / OPENAI_API_KEY / GOOGLE_API_KEY  (whichever provider you use)
#   GITHUB_TOKEN is provided automatically by Actions (needs issues:write + pull-requests:write + contents:write)

# Required GitHub Repository Variables (for non-dispatch runs):
#   AI_PROVIDER          e.g. anthropic
#   AI_MODEL             e.g. claude-3-5-sonnet-latest
#   AGENT_ENABLE_BUG_ISSUE  true
#   AGENT_ENABLE_HEAL_PR    false

# The agent step runs only when Playwright fails:
#   if: steps.playwright.outcome == 'failure' && !cancelled()
# The final step always fails the job when Playwright failed:
#   run: exit 1
# This keeps the PR check red while still preserving all artifacts and agent logs.

# To test locally without CI:
# AGENT_ENABLE_IN_CI is not checked for local runs (CI=true is only set in Actions)
# npm run agent   # reads ../test-results/results.json relative to agent/ cwd
```

--------------------------------

### Get LLM Client for Various Providers

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

The `getLlmClient` factory function instantiates the correct concrete LLM client based on `AgentConfig`. It validates that required API keys are set for cloud providers and implements the `LlmClient` interface with `classifyFailure` and `generateFix` methods.

```typescript
import { getLlmClient } from './agent/llm/factory.js'
import { getAgentConfig } from './agent/config.js'
import type { LlmClient } from './agent/llm/types.js'

// Mock client — no API key required, returns deterministic fixture responses
process.env.AI_PROVIDER = 'mock'
const mockClient: LlmClient = getLlmClient(getAgentConfig())
const raw = await mockClient.classifyFailure({
  prompt: 'Classify this failure...', 
  maxTokens: 600,
  temperature: 0,
})
// → '{"category":"BROKEN_LOCATOR","confidence":0.92,...}' (deterministic mock)

// Anthropic client
process.env.AI_PROVIDER = 'anthropic'
process.env.ANTHROPIC_API_KEY = 'sk-ant-...'
process.env.AI_MODEL = 'claude-3-5-sonnet-latest'
const anthropicClient = getLlmClient(getAgentConfig())
const classification = await anthropicClient.classifyFailure({ prompt, maxTokens: 600, temperature: 0 })

// Ollama client (no API key required by default)
process.env.AI_PROVIDER = 'ollama'
process.env.AI_MODEL = 'qwen2.5:7b'
process.env.OLLAMA_BASE_URL = 'http://127.0.0.1:11434'
const ollamaClient = getLlmClient(getAgentConfig())

// OpenAI client
process.env.AI_PROVIDER = 'openai'
process.env.OPENAI_API_KEY = 'sk-...'
process.env.AI_MODEL = 'gpt-4o-mini'
const openaiClient = getLlmClient(getAgentConfig())

// Google Gemini client
process.env.AI_PROVIDER = 'google'
process.env.GOOGLE_API_KEY = '...'
process.env.AI_MODEL = 'gemini-2.5-flash'
const googleClient = getLlmClient(getAgentConfig())
```

--------------------------------

### Get Agent Configuration

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Retrieves and validates all environment variables into a typed `AgentConfig` object. It provides defaults for numeric and boolean variables and throws errors for invalid configurations. Use this to ensure the agent has all necessary settings before execution.

```typescript
import { getAgentConfig } from './agent/config.js'

// Minimal: mock provider, all defaults
// (no environment variables required)
const config = getAgentConfig()
// config.llm.provider         → 'mock'
// config.llm.model            → 'mock-classifier-v1'
// config.thresholds.confidence→ 0.75
// config.limits.maxFailuresPerRun → 3
// config.actions.enableBugIssue  → false
// config.actions.enableHealPr    → false
// config.paths.resultsJson    → '../test-results/results.json'

// Full Anthropic example via environment:
process.env.AI_PROVIDER = 'anthropic'
process.env.AI_MODEL = 'claude-3-5-sonnet-latest'
process.env.ANTHROPIC_API_KEY = 'sk-ant-...'
process.env.AGENT_CONFIDENCE_THRESHOLD = '0.80'
process.env.AGENT_MAX_FAILURES_PER_RUN = '5'
process.env.AGENT_ENABLE_BUG_ISSUE = 'true'
process.env.AGENT_ENABLE_HEAL_PR = 'false'
process.env.AGENT_ENABLE_IN_CI = 'true'
process.env.AGENT_ISSUE_LABELS = 'bug,qa-automated,triage'
process.env.AGENT_RESULTS_JSON_PATH = '../test-results/results.json'

const prodConfig = getAgentConfig()
// Throws if AGENT_CONFIDENCE_THRESHOLD not in [0,1]
// Throws if AGENT_OLLAMA_NUM_CTX_MIN > AGENT_OLLAMA_NUM_CTX_MAX
// Throws if AI_PROVIDER is not one of: mock | anthropic | openai | google | ollama
```

--------------------------------

### Build and Run Ollama in Docker

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Build a Docker image for Ollama and run it using Docker or Docker Compose. Point the OLLAMA_BASE_URL to the running container. Refer to the ollama/ directory for NVIDIA/CPU-only configurations.

```bash
docker build -t qa-agent-ollama ./ollama
docker run …
or
docker compose -f ollama/docker-compose.yml up --build -d
```

--------------------------------

### Vercel Deployment Configuration (`vercel.json`)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-01-demo-web-app.md

Configuration file for Vercel deployment, specifying the framework as Next.js, the build command, and the output directory.

```json
{
  "framework": "nextjs",
  "buildCommand": "npm run build",
  "outputDirectory": ".next"
}

```

--------------------------------

### Run Agent with Real Provider and Issue Creation

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Execute the agent with a real AI provider (e.g., Anthropic) and enable GitHub issue creation for failures above a specified confidence threshold. Requires API keys and repository details.

```bash
AI_PROVIDER=anthropic \
ANTHROPIC_API_KEY=sk-ant-... \
AI_MODEL=claude-3-5-sonnet-latest \
AGENT_ENABLE_BUG_ISSUE=true \
GITHUB_TOKEN=ghp_... \
GITHUB_REPOSITORY=owner/playwright-ai-qa-agent \
AGENT_CONFIDENCE_THRESHOLD=0.75 \
AGENT_LOG_LEVEL=debug \
  npm run agent
```

--------------------------------

### Agent Script in package.json

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-03-ai-failure-agent.md

Defines the script to run the AI Failure Agent using `tsx`. Ensure `tsx` is installed as a dev dependency.

```json
{
  "scripts": {
    "agent": "tsx agent/orchestrator.ts"
  },
  "devDependencies": {
    "@playwright/test": "^1.44.0",
    "tsx": "^4.0.0",
    "typescript": "^5.0.0"
  }
}

```

--------------------------------

### Run Playwright Tests with Different QA Modes

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-02.1-agent-showcase-tests.md

Demonstrates how to execute the Playwright showcase tests under various QA_MODE environments to simulate different failure scenarios for agent analysis.

```bash
# Baseline run: expected pass
QA_MODE=none npx playwright test -c tests/playwright.config.ts

# Agent showcase runs: same tests, expected fail
QA_MODE=selector-change npx playwright test -c tests/playwright.config.ts
QA_MODE=logic-bug npx playwright test -c tests/playwright.config.ts
QA_MODE=auth-break npx playwright test -c tests/playwright.config.ts
QA_MODE=slow-network npx playwright test -c tests/playwright.config.ts

```

--------------------------------

### Run LLM Failure Agent

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Execute the LLM failure agent after test runs to process results. Ensure dependencies are installed and environment variables are set.

```bash
npm run agent
```

--------------------------------

### Break Mode Management (TypeScript)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-01-demo-web-app.md

Defines possible break modes and provides functions to get, set, and initialize the break mode from URL parameters. The break mode affects application behavior for testing purposes.

```typescript
export type BreakMode =
  | "none"
  | "selector-change"
  | "logic-bug"
  | "slow-network"
  | "auth-break";

const BREAK_KEY = "demo_break_mode";

export function getBreakMode(): BreakMode {
  return (sessionStorage.getItem(BREAK_KEY) as BreakMode) ?? "none";
}

export function setBreakMode(mode: BreakMode): void {
  sessionStorage.setItem(BREAK_KEY, mode);
}

export function bootstrapBreakModeFromUrl(): BreakMode {
  const url = new URL(window.location.href);
  const requested = url.searchParams.get("qaMode");
  const allowed = ["none", "selector-change", "logic-bug", "slow-network", "auth-break"];

  if (!requested || !allowed.includes(requested)) {
    return getBreakMode();
  }

  const mode = requested as BreakMode;
  setBreakMode(mode);
  url.searchParams.delete("qaMode");
  window.history.replaceState({}, "", url.toString());
  return mode;
}
```

--------------------------------

### Project Structure Overview

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

This text-based tree displays the directory structure of the project, highlighting key components like the agent, CI workflows, demo application, and tests.

```text
.
├── .cursor/rules/          # Editor / agent conventions
├── .github/workflows/    # CI (Playwright + agent)
├── agent/                  # LLM failure agent (TypeScript)
├── demo-app/               # Next.js TaskFlow app
├── tests/                  # Playwright config + showcase tests
├── ollama/                 # Optional Docker image for local Ollama
└── docs/                   # Design and phase plans
```

--------------------------------

### Run Agent with Mock Provider

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Run the AI agent using a mock provider for local testing without needing API keys. This classifies failed tests based on deterministic mock results and logs detailed information.

```bash
AI_PROVIDER=mock \
AGENT_LOG_LEVEL=info \
AGENT_RESULTS_JSON_PATH=../test-results/results.json \
  npm run agent
```

--------------------------------

### Run Headed Playwright Test with Parameters (Bash)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Execute Playwright tests in headed mode against a production URL using Bash or similar shells. Sets environment variables for base URL, QA mode, and slow motion.

```bash
BASE_URL=https://playwright-ai-qa-agent.vercel.app QA_MODE=none PLAYWRIGHT_SLOW_MO=400 npm run test:e2e:headed -w demo-app
```

--------------------------------

### Run LLM Agent with Mock Provider (PowerShell)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Run the LLM failure agent using a mock AI provider and specifying the results JSON path. Useful for debugging the agent's behavior.

```powershell
$env:AI_PROVIDER='mock'
$env:AGENT_RESULTS_JSON_PATH='../test-results/results.json'
npm run agent
```

--------------------------------

### Run Playwright Tests with QA Mode

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Execute Playwright tests using the specified configuration file. The BASE_URL is set to the public demo, and QA_MODE is determined by the dispatch input or defaults to 'none'. This step uses 'continue-on-error: true' to ensure artifacts are always uploaded.

```bash
npx playwright test -c tests/playwright.config.ts with `BASE_URL` set to the public demo and `QA_MODE` from the dispatch input (or `none`). Step uses `continue-on-error: true` so artifacts always upload when tests fail.
```

--------------------------------

### `loadEnvForAgent` — `agent/env.ts`

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Loads `.env` files from the repository root and the `agent/` directory into `process.env`. It handles UTF-8 BOM, strips stray quotes from `GITHUB_TOKEN`, and applies a special merge for GitHub credentials to avoid overwriting shell-set values.

```APIDOC
## `loadEnvForAgent` — `agent/env.ts`

### Description
Loads `.env` files from the repo root and `agent/` directory into `process.env` without overwriting values already set in the shell. Handles UTF-8 BOM, strips stray quotes from `GITHUB_TOKEN`, and applies a special local-only merge pass for GitHub credentials so developers can store tokens in `.env` without interfering with CI injection.

### Usage
```typescript
import { loadEnvForAgent, stripEnvQuotes } from './agent/env.js'

// Call once at agent startup
loadEnvForAgent()

// Example of stripping quotes from a token value
const clean = stripEnvQuotes("'ghp_abc123'")  // → 'ghp_abc123'
const clean2 = stripEnvQuotes('"ghp_abc123"') // → 'ghp_abc123'
```

### Environment Variables
- `AGENT_DOTENV_OVERRIDES_SHELL_GITHUB=true`: Forces `.env` to override a working shell token (useful for rotating keys locally).
- `AGENT_SKIP_REPO_ROOT_GITHUB_DOTENV=true`: Skips the GitHub dotenv merge entirely (pure CI mode).
```

--------------------------------

### Simulate Locator Failure and Run Agent + Healer

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Simulate a locator failure, run Playwright tests, and then execute the agent with issue creation and auto-healing enabled. This process classifies failures, creates GitHub issues, generates a fix branch, commits corrected code, and opens a pull request.

```bash
QA_MODE=selector-change \
BASE_URL=https://playwright-ai-qa-agent.vercel.app \
  npx playwright test -c tests/playwright.config.ts || true
```

```bash
AI_PROVIDER=anthropic \
ANTHROPIC_API_KEY=sk-ant-... \
AGENT_ENABLE_BUG_ISSUE=true \
AGENT_ENABLE_HEAL_PR=true \
GITHUB_TOKEN=ghp_... \
GITHUB_REPOSITORY=owner/playwright-ai-qa-agent \
  npm run agent
```

--------------------------------

### Run Headed Playwright Test with Parameters (PowerShell)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Execute Playwright tests in headed mode against a production URL using PowerShell. Sets environment variables for base URL, QA mode, and slow motion.

```powershell
$env:BASE_URL='https://playwright-ai-qa-agent.vercel.app'
$env:QA_MODE='none'
$env:PLAYWRIGHT_SLOW_MO='400'
npm run test:e2e:headed -w demo-app
```

--------------------------------

### User Interface and Authentication Types

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-01-demo-web-app.md

Defines the structure for user data and outlines key functions for user registration, login, logout, and session management. Includes a mock authentication break mode.

```typescript
export interface User {
  id: string;
  email: string;
  password: string; // plaintext for demo — fine, no real data
  displayName: string;
}

const USERS_KEY = "demo_users";
const SESSION_KEY = "demo_session";

export function register(email: string, password: string, displayName: string): User {
  const users = getUsers();
  if (users.find((u) => u.email === email)) {
    throw new Error("Email already registered");
  }
  const user: User = { id: crypto.randomUUID(), email, password, displayName };
  localStorage.setItem(USERS_KEY, JSON.stringify([...users, user]));
  return user;
}

export function login(email: string, password: string): User {
  // Break mode: always fail
  if (getBreakMode() === "auth-break") {
    throw new Error("Invalid credentials");
  }

  const user = getUsers().find((u) => u.email === email && u.password === password);
  if (!user) throw new Error("Invalid credentials");

  localStorage.setItem(SESSION_KEY, JSON.stringify(user));
  return user;
}

export function logout() {
  localStorage.removeItem(SESSION_KEY);
}

export function getSession(): User | null {
  const raw = localStorage.getItem(SESSION_KEY);
  return raw ? JSON.parse(raw) : null;
}

function getUsers(): User[] {
  const raw = localStorage.getItem(USERS_KEY);
  return raw ? JSON.parse(raw) : [];
}
```

--------------------------------

### Architecture Diagram

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Illustrates the flow of the AI-powered QA pipeline, from running Playwright tests to classifying failures with an LLM and creating GitHub issues.

```text
GitHub Actions
  |
  v
Run Playwright tests (JSON + HTML reports, screenshots/traces)
  |
  +-- success ------------------------------> ✅ Job passes
  |
  +-- failure
       |
       v
   Agent reads Playwright results.json + failure context
       |
       v
   Claude API classifies failure
       |
  +--> BROKEN_LOCATOR  ---> create `AUTOMATION_BUG` Issue (+ optional linked healer PR)
       |
       +--> REAL_BUG        ---> create GitHub Issue with failure context
       |
       +--> FLAKY           ---> log only (no automated write action)
       |
       +--> ENV_ISSUE       ---> log only (no automated write action)
       |
       v
   CI ends failed (PR checks are red, artifacts preserved)
```

--------------------------------

### Define AgentConfig Structure

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Illustrates the shape of the full runtime configuration for the agent, including LLM provider details, model settings, token limits, retry parameters, and action thresholds. This configuration is built by the getAgentConfig() function.

```typescript
// AgentConfig — full runtime configuration (built by getAgentConfig())
const configShape: Pick<AgentConfig, 'llm' | 'thresholds' | 'actions'> = {
  llm: {
    provider: 'anthropic',
    model: 'claude-3-5-sonnet-latest',
    apiKeyEnvVar: 'ANTHROPIC_API_KEY',
    maxTokens: { classify: 600, heal: 14000 },
    temperature: { classify: 0, heal: 0 },
    retry: { maxAttempts: 3, initialDelayMs: 1000, maxDelayMs: 8000 },
  },
  thresholds: { confidence: 0.75 },
  actions: { enableBugIssue: true, enableHealPr: false },
}
```

--------------------------------

### Structured Logging with Levels and Pretty-Print

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Emits structured key=value logs with four levels: debug, info, warn, and error. Log level is controlled by `AGENT_LOG_LEVEL`, defaulting to 'info'. `warn` and `error` output to stderr, while `info` and `debug` output to stdout. Enable `AGENT_LOG_PRETTY=true` for indented, multi-line output.

```typescript
import { logger } from './agent/logger.js'

// Standard usage
logger.info('Agent starting', {
  provider: 'anthropic',
  model: 'claude-3-5-sonnet-latest',
  threshold: 0.75,
})
// stdout → [2024-01-15T10:30:00.000Z] INFO Agent starting provider="anthropic" model="claude-3-5-sonnet-latest" threshold=0.75

logger.warn('Classification confidence below threshold', {
  testName: 'adds a task',
  confidence: 0.60,
  threshold: 0.75,
})
// stderr → [2024-01-15T10:30:01.000Z] WARN Classification confidence below threshold ...

logger.error('Issue creation failed', { error: '401 Bad credentials' })
// stderr → [2024-01-15T10:30:02.000Z] ERROR Issue creation failed error="401 Bad credentials"

// Pretty-print mode (AGENT_LOG_PRETTY=true):
// [2024-01-15T10:30:00.000Z] INFO Agent starting
//   provider="anthropic" model="claude-3-5-sonnet-latest"

// Debug mode (AGENT_LOG_LEVEL=debug): also emits prompt text, raw LLM responses, file paths
process.env.AGENT_LOG_LEVEL = 'debug'
logger.debug('Submitting classification request', { promptChars: 4200, provider: 'ollama' })
```

--------------------------------

### Create GitHub Issues for Test Failures with `createBugIssue`

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Utilize `createBugIssue` to generate structured GitHub Issues for specific failure classifications like `BROKEN_LOCATOR`. It prevents duplicates by checking for existing issues with the same fingerprint and includes detailed error information, screenshots, and locator suggestions.

```typescript
import { createBugIssue } from './agent/reporter.js'
import { getAgentConfig } from './agent/config.js'
import type { FailureContext, ClassificationResult } from './agent/types.js'

process.env.GITHUB_TOKEN = 'ghp_'
process.env.GITHUB_REPOSITORY = 'owner/playwright-ai-qa-agent'
process.env.AGENT_ENABLE_BUG_ISSUE = 'true'
process.env.AGENT_ISSUE_LABELS = 'bug,automated-qa'

const failure: FailureContext = {
  testName: 'adds a task to the list',
  testFile: 'tasks.spec.ts',
  testSource: '',
  error: "TimeoutError: locator.fill: Timeout exceeded.\nLocator: getByTestId('task-input-field')",
  errorStack: '  at DashboardPage.addTask (dashboard-page.ts:31)',
  runUrl: 'https://github.com/owner/repo/actions/runs/12345',
  branch: 'main',
  commit: 'deadbeef',
}

const classification: ClassificationResult = {
  category: 'BROKEN_LOCATOR',
  confidence: 0.93,
  reason: 'Locator task-input-field resolved to 0 elements.',
  issueTitle: "adds a task: broken locator 'task-input-field'",
  suggestedFix: "Update only data-testid 'task-input-field' to 'task-input-field-v2' (element: input).",
}

await createBugIssue(failure, classification, getAgentConfig())
// → Creates GitHub Issue titled: "[AUTOMATION_BUG] adds a task: broken locator 'task-input-field'"
// → Labels: ['bug', 'automated-qa']
// → Body includes: error block, stack trace, ## Locator Update Needed section, classification table
// → Fingerprint comment: <!-- qa-agent-fp:<sha256> --> (deduplication marker)
// → If open issue with same fingerprint exists → skips creation, logs duplicate URL
```

--------------------------------

### Agent Configuration Interface

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-03-ai-failure-agent.md

Defines the structure for agent configuration, controlling LLM providers, model settings, thresholds, and enabled actions. This ensures a configuration-first design.

```typescript
export interface AgentConfig {
  llm: {
    provider: 'anthropic' | 'openai' | 'google'
    model: string
    apiKeyEnvVar: string
    baseUrl?: string
    maxTokens: {
      classify: number
      heal: number
    }
    temperature: {
      classify: number
      heal: number
    }
  }
  thresholds: {
    confidence: number
  }
  limits: {
    maxFailuresPerRun: number
  }
  actions: {
    enableHealPr: boolean
    enableBugIssue: boolean
  }
  github: {
    baseBranch: string
  }
  paths: {
    resultsJson: string
  }
}
```

--------------------------------

### Run LLM Agent with Ollama (PowerShell)

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Configure and run the LLM failure agent to use Ollama as the AI provider. Sets Ollama model, base URL, log level, results path, and enables issue/PR creation.

```powershell
$env:AI_PROVIDER='ollama'
$env:AI_MODEL='qwen2.5:7b'
$env:OLLAMA_BASE_URL='http://127.0.0.1:11434'
$env:AGENT_LOG_LEVEL='debug'
$env:AGENT_RESULTS_JSON_PATH='../test-results/results.json'
$env:AGENT_ENABLE_BUG_ISSUE='true'
$env:AGENT_ENABLE_HEAL_PR='true'
npm run agent
```

--------------------------------

### Run Playwright Tests Headless

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Execute Playwright tests in headless mode against the live demo application. This command generates a results.json file in the test-results directory.

```bash
BASE_URL=https://playwright-ai-qa-agent.vercel.app \
  npx playwright test -c tests/playwright.config.ts
```

--------------------------------

### Load Environment Variables for Agent

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Loads .env files from the repository root and agent directory. It avoids overwriting existing shell variables and applies special handling for GitHub credentials locally. Use `stripEnvQuotes` to clean token values that may have been stored with quotes.

```typescript
import { loadEnvForAgent, stripEnvQuotes } from './agent/env.js'

// Call once at agent startup (orchestrator.ts calls this first)
loadEnvForAgent()
// Effect: reads <repo-root>/.env, then agent/.env
// Shell variables already set (non-empty) are NOT overwritten
// GITHUB_TOKEN / GITHUB_REPOSITORY are filled from .env when missing (local only)

// stripEnvQuotes — strip leading/trailing quotes from a token value
// Useful when values were stored as: GITHUB_TOKEN='ghp_xxx' (with quotes)
const clean = stripEnvQuotes("'ghp_abc123'")  // → 'ghp_abc123'
const clean2 = stripEnvQuotes('"ghp_abc123"') // → 'ghp_abc123'

// Force .env to override a working shell token (useful when rotating keys locally):
// AGENT_DOTENV_OVERRIDES_SHELL_GITHUB=true npm run agent

// Skip the GitHub dotenv merge entirely (pure CI mode):
// AGENT_SKIP_REPO_ROOT_GITHUB_DOTENV=true npm run agent
```

--------------------------------

### Implement LLM API Retries with `withProviderRetry`

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Use `withProviderRetry` to wrap asynchronous LLM calls. It automatically handles retries for specific HTTP status codes (429, 503, 504) based on configuration. Non-retriable errors are thrown immediately.

```typescript
import { withProviderRetry } from './agent/llm/retry.js'

// Used internally by all cloud LLM clients — example of direct use:
process.env.AI_PROVIDER = 'anthropic'
process.env.ANTHROPIC_API_KEY = 'sk-ant-'
process.env.AGENT_LLM_MAX_ATTEMPTS = '3'
process.env.AGENT_LLM_RETRY_INITIAL_DELAY_MS = '1000'
process.env.AGENT_LLM_RETRY_MAX_DELAY_MS = '8000'

const result = await withProviderRetry('anthropic', async () => {
  const res = await fetch('https://api.anthropic.com/v1/messages', { /* ... */ })
  if (!res.ok) {
    throw new Error(`Anthropic classify request failed: ${res.status} ${await res.text()}`)
    //                                                   ^^^ 429 → retried; 401 → thrown immediately
  }
  return res.json()
})

// Backoff schedule for 3 attempts, initial 1000ms, max 8000ms:
// Attempt 1 fails → wait 1000ms
// Attempt 2 fails → wait 2000ms
// Attempt 3 fails → throw (final)
```

--------------------------------

### Build LLM Prompts for Failure Classification

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Construct detailed LLM prompts using `buildClassificationPrompt`. This function enforces a specific reasoning order and defines the expected JSON response schema. It can optionally prepend a hint for rule-locked `BROKEN_LOCATOR` failures.

```typescript
import { buildClassificationPrompt } from './agent/classifier.js'
import type { FailureContext } from './agent/types.js'

const ctx: FailureContext = {
  testName: 'shows dashboard after login',
  testFile: 'login.spec.ts',
  testSource: '// spec source...',
  error: "Error: expect(page).toHaveURL expected /dashboard/, received /login",
  errorStack: '',
  domSnapshot: '<form data-testid="login-form">...</form>',
  runUrl: '', branch: 'main', commit: 'abc',
}

// locatorRuleExplanationOnly=false → full LLM decision prompt
const prompt = buildClassificationPrompt(ctx, false)
// Returns the multi-section prompt string to send to the LLM

// locatorRuleExplanationOnly=true → rule-locked; LLM provides explanation only
const lockedPrompt = buildClassificationPrompt(ctx, true)
// Prepends: "Deterministic rule (non-negotiable): ... category is fixed as BROKEN_LOCATOR..."

// Expected LLM response shape:
// {
//   "category": "REAL_BUG",
//   "confidence": 0.91,
//   "reason": "URL assertion diff: expected /dashboard but received /login after valid login action.",
//   "issueTitle": "login: expected /dashboard reached /login after valid credentials",
//   "suggestedFix": null
// }
```

--------------------------------

### Enable Healer PR for Broken Locators

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/README.md

Set the AGENT_ENABLE_HEAL_PR environment variable to 'true' to enable the healer PR functionality for broken locators. Ensure necessary GitHub token permissions are configured.

```bash
# Optional — healer PR for BROKEN_LOCATOR (same token needs `contents` + `pull-requests`):
# $env:AGENT_ENABLE_HEAL_PR='true'
```

--------------------------------

### LLM Client Interface

Source: https://github.com/nirarad/playwright-ai-qa-agent/blob/main/docs/plan-03-ai-failure-agent.md

Defines a provider-agnostic interface for Large Language Models, enabling abstraction over different LLM providers.

```typescript
export interface LlmClient {
  classifyFailure(input: {
    prompt: string
    maxTokens: number
    temperature: number
  }): Promise<string>
  generateFix(input: {
    prompt: string
    maxTokens: number
    temperature: number
  }): Promise<string>
}
```

--------------------------------

### `getAgentConfig` — `agent/config.ts`

Source: https://context7.com/nirarad/playwright-ai-qa-agent/llms.txt

Reads all environment variables and returns a fully-typed, validated `AgentConfig` object. It throws descriptive errors for invalid values and provides documented defaults for numeric and boolean variables.

```APIDOC
## `getAgentConfig` — `agent/config.ts`

### Description
Reads all environment variables and returns a fully-typed, validated `AgentConfig` object. Throws descriptive errors for invalid values (non-numeric, out-of-range confidence, invalid provider name, inverted Ollama context window bounds). Every numeric and boolean variable has a documented default.

### Usage
```typescript
import { getAgentConfig } from './agent/config.js'

// Minimal configuration with defaults
const config = getAgentConfig()

// Example with environment variables set for Anthropic provider
process.env.AI_PROVIDER = 'anthropic'
process.env.AI_MODEL = 'claude-3-5-sonnet-latest'
process.env.ANTHROPIC_API_KEY = 'sk-ant-...'
process.env.AGENT_CONFIDENCE_THRESHOLD = '0.80'
// ... other environment variables

const prodConfig = getAgentConfig()
```

### Environment Variables & Configuration
- `AI_PROVIDER`: Specifies the LLM provider (e.g., 'mock', 'anthropic', 'openai', 'google', 'ollama').
- `AI_MODEL`: The specific model to use.
- `ANTHROPIC_API_KEY`: API key for Anthropic.
- `AGENT_CONFIDENCE_THRESHOLD`: Confidence threshold for agent decisions (0 to 1).
- `AGENT_MAX_FAILURES_PER_RUN`: Maximum number of failures allowed per run.
- `AGENT_ENABLE_BUG_ISSUE`: Enable bug issue creation.
- `AGENT_ENABLE_HEAL_PR`: Enable PR healing functionality.
- `AGENT_ENABLE_IN_CI`: Enable agent execution in CI environments.
- `AGENT_ISSUE_LABELS`: Labels to apply to created issues.
- `AGENT_RESULTS_JSON_PATH`: Path to the results JSON file.
- `AGENT_OLLAMA_NUM_CTX_MIN`: Minimum context window size for Ollama.
- `AGENT_OLLAMA_NUM_CTX_MAX`: Maximum context window size for Ollama.

### Error Handling
- Throws if `AGENT_CONFIDENCE_THRESHOLD` is not between 0 and 1.
- Throws if `AGENT_OLLAMA_NUM_CTX_MIN` is greater than `AGENT_OLLAMA_NUM_CTX_MAX`.
- Throws if `AI_PROVIDER` is not a recognized value.
```