### GitHub Actions: Basic Setup Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md A basic GitHub Actions workflow to check out code, set up Node.js, install dependencies, and run Playwright tests, passing Azure DevOps secrets as environment variables. ```yaml name: Playwright Tests on: push: branches: [main] pull_request: branches: [main] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: 18 - run: npm ci - run: npx playwright test env: AZURE_TOKEN: ${{ secrets.AZURE_TOKEN }} AZUREPWDEBUG: '1' - name: Print Test Run ID if: always() run: echo "Test Run ID: $AZURE_PW_TEST_RUN_ID" env: AZURE_PW_TEST_RUN_ID: ${{ env.AZURE_PW_TEST_RUN_ID }} ``` -------------------------------- ### Multi-Project Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example of configuring the Playwright Azure Reporter for multi-project setups, including a custom test point mapper. ```typescript export default defineConfig({ projects: [ { name: 'chromium', use: { ...devices['Desktop Chrome'] }, }, { name: 'firefox', use: { ...devices['Desktop Firefox'] }, }, ], reporter: [ [ '@alex_neo/playwright-azure-reporter', { testPointMapper: async (testCase, testPoints) => { const projectName = testCase.parent.project().name; const configMap = { chromium: '3', firefox: '4', }; const configId = configMap[projectName]; return testPoints.filter((tp) => tp.configuration.id === configId); }, testRunConfig: { configurationIds: [3, 4], }, } ] ] }); ``` -------------------------------- ### Environment-Specific Configurations Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example of configuring different test plan IDs based on the Git branch. ```typescript const planMap = { 'main': 42, // Main test plan 'develop': 41, // Dev test plan 'staging': 100, // Staging plan }; const branch = process.env.GITHUB_REF?.split('/').pop() || 'develop'; const planId = planMap[branch] || 41; export default defineConfig({ reporter: [ [ '@alex_neo/playwright-azure-reporter', { planId } ] ] }); ``` -------------------------------- ### Retry Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example of configuring retries for tests in CI environments. ```typescript export default defineConfig({ use: { retries: process.env.CI ? 2 : 0, // Retry in CI, not locally }, reporter: [ [ '@alex_neo/playwright-azure-reporter', { publishRetryResults: 'last', // Only final result in CI } ] ] }); ``` -------------------------------- ### Conditional Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example of conditionally enabling the reporter based on CI environment and production status. ```typescript const isCI = process.env.CI === 'true'; const isProduction = process.env.SYSTEM_STAGENAME === 'Production'; export default defineConfig({ reporter: isCI ? [ [ '@alex_neo/playwright-azure-reporter', { isDisabled: !isProduction, // Only report in production publishTestResultsMode: 'testRun', logging: true, } ] ] : [], }); ``` -------------------------------- ### Azure DevOps Pipeline: Basic Setup Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md A basic Azure DevOps pipeline YAML configuration to set up Node.js, install dependencies, run Playwright tests, and print the Test Run ID. ```yaml trigger: - main pool: vmImage: 'ubuntu-latest' steps: - task: NodeTool@0 inputs: versionSpec: '18.x' displayName: 'Install Node.js' - script: npm ci displayName: 'Install Dependencies' - script: npx playwright test displayName: 'Run Playwright Tests' name: 'playwright' env: AZURE_TOKEN: $(AZURE_TOKEN) AZUREPWDEBUG: '1' - script: echo "Test Run ID: $(playwright.AZURE_PW_TEST_RUN_ID)" displayName: 'Print Test Run ID' condition: always() ``` -------------------------------- ### Timeout Management Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example of adjusting test timeouts and attachment uploads based on CI environment. ```typescript export default defineConfig({ timeout: process.env.CI ? 60000 : 30000, // Longer timeout in CI reporter: [ [ '@alex_neo/playwright-azure-reporter', { uploadAttachments: process.env.CI ? true : false, } ] ] }); ``` -------------------------------- ### Secure Token Storage - Good Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Demonstrates the recommended way to handle tokens using environment variables. ```typescript token: process.env.AZURE_TOKEN // Use environment variables ``` -------------------------------- ### Secure Token Storage - Bad Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Demonstrates an insecure way to handle tokens by committing them directly. ```typescript token: 'your-actual-token' // Don't do this! ``` -------------------------------- ### GitLab CI Basic Setup Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Basic GitLab CI configuration to run Playwright tests and upload artifacts. ```yaml stages: - test playwright_tests: stage: test image: mcr.microsoft.com/playwright:v1.40.0-focal before_script: - npm ci script: - AZUREPWDEBUG=1 npx playwright test after_script: - echo "Test run completed" artifacts: paths: - test-case-summary.md reports: junit: test-results/*.xml only: - main - /^release\/.*$/ ``` -------------------------------- ### Two-Phase Matching Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example demonstrating the two-phase matching process for the annotation zone, including configuration, test case, and extraction steps. ```typescript // Configuration testCaseIdZone: 'annotation' testCaseIdMatcher: [ /(TestCase)/, // Phase 1: Match type /[\[]([\d,\s]+)[\]]/ // Phase 2: Extract from description ] // Test test('Example', { annotation: { type: 'TestCase', description: '[123, 456]' } }, // Extraction: // Phase 1: type 'TestCase' matches /(TestCase)/ ✓ // Phase 2: description '[123, 456]' matches /[\[]([\d,\s]+)[\]]/ → '123, 456' // Result: ['123', '456'] ); ``` -------------------------------- ### Example: Echoing AZURE_PW_TEST_RUN_ID Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Demonstrates how to access the AZURE_PW_TEST_RUN_ID environment variable after the reporter has run. ```bash # After reporter runs: echo $AZURE_PW_TEST_RUN_ID # 12345 ``` -------------------------------- ### Debug Output Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of debug output showing ID extraction process. ```text azure:pw:debug [_extractMatches] Extracting matches from text: [1] Login test azure:pw:debug [_extractMatches] Using matchers: /\<0xC2><0xA0>[\d,\s]+<0xC2><0xA0]/g azure:pw:debug [_extractMatches] Whole matches found: [1] azure:pw:debug [_extractMatches] Matches found: 1 ``` -------------------------------- ### ID Extraction Logic Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md An example demonstrating the ID extraction process for the title zone, including configuration, test case, and step-by-step extraction. ```typescript // Configuration testCaseIdZone: 'title' testCaseIdMatcher: /[\[]([\d,\s]+)[\]]/ // default // Test test('[1,2] Login @[3]', { tag: '@[4]' }, () => {}); // Extraction steps: // 1. Title: [1,2] → '1,2' // 2. Tags: @[3] → '3', @[4] → '4' // 3. Split: '1,2' → ['1','2'], '3' → ['3'], '4' → ['4'] // 4. Deduplicate: ['1','2','3','4'] // 5. Result: ['1', '2', '3', '4'] ``` -------------------------------- ### Jenkins Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example configuration for Playwright Azure Reporter in Jenkins. ```typescript export default defineConfig({ reporter: [ [ '@alex_neo/playwright-azure-reporter', { orgUrl: 'https://dev.azure.com/my-org', projectName: 'MyProject', planId: 42, token: process.env.AZURE_TOKEN, environment: process.env.BRANCH_NAME || 'unknown', testRunTitle: `Jenkins - ${process.env.BUILD_NUMBER}`, publishTestResultsMode: 'testRun', uploadAttachments: true, } ] ] }); ``` -------------------------------- ### Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of reporter configuration in Playwright config file, enabling logging and setting `testCaseIdMatcher`. ```typescript import { AzureReporterOptions } from '@alex_neo/playwright-azure-reporter'; const config: AzureReporterOptions = { // ... other options ... logging: true, // Enable logs testCaseIdMatcher: /\<0xC2><0xA0>[\d,\s]+<0xC2><0xA0]/, testCaseIdZone: 'title', }; ``` -------------------------------- ### Tag Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Examples of how to include test case IDs within the tags of a test. ```typescript test('Login test', { tag: ['@[1]', '@smoke', '@login'], }, () => {}); test('Multi-case test', { tag: ['@[1]', '@[2]', '@[3]'], }, () => {}); ``` -------------------------------- ### Complete Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md A detailed TypeScript example demonstrating the full range of configuration options for the Playwright Azure Reporter. ```typescript import { defineConfig } from '@playwright/test'; import type { AzureReporterOptions } from '@alex_neo/playwright-azure-reporter'; export default defineConfig({ reporter: [ ['list'], [ '@alex_neo/playwright-azure-reporter', { // Required orgUrl: 'https://dev.azure.com/my-organization', projectName: 'QualityAssurance', planId: 42, // Authentication token: process.env.AZURE_PAT_TOKEN, authType: 'pat', // General logging: true, environment: 'QA', testRunTitle: 'Automated Test Suite Run', // Publishing publishTestResultsMode: 'testRun', publishRetryResults: 'last', testRunConfig: { configurationIds: [3, 4, 5], // Chrome, Firefox, Safari }, // Test case ID matching testCaseIdMatcher: /[\[([\d,\s]+)\]]/, testCaseIdZone: 'title', // Attachments uploadAttachments: true, attachmentsType: ['screenshot', 'video'], uploadLogs: true, // Test point mapping testPointMapper: async (testCase, testPoints) => { const browserName = testCase.parent?.project?.use?.browserName; const configMap: Record = { chromium: '3', firefox: '4', webkit: '5' }; const configId = configMap[browserName] || '3'; return testPoints.filter((tp) => tp.configuration?.id === configId); }, // Summary testCaseSummary: { enabled: true, outputPath: './reports/test-case-summary.md', consoleOutput: true, publishToRun: true, }, // Auto-mark as automated autoMarkTestCasesAsAutomated: { enabled: true, updateAutomatedTestName: true, updateAutomatedTestStorage: true, automatedTestNameFormat: 'titleWithParent', automatedTestStoragePath: (testCase) => { const parts = testCase.location.file.split('/'); const idx = parts.indexOf('tests'); return idx >= 0 ? parts.slice(idx).join('/') : testCase.location.file; }, automatedTestType: (testCase) => { const file = testCase.location?.file || ''; if (file.includes('/e2e/')) return 'End-to-End Test'; return 'Functional Test'; }, }, } as AzureReporterOptions, ], ], }); ``` -------------------------------- ### Test Output Example (No IDs Found) Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of test output when no test case IDs are found for a test case. ```text azure:pw:warn No test points found for test case [] ... ``` -------------------------------- ### isExistingTestRun and testRunId Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example showing how to publish results to an existing test run. ```typescript isExistingTestRun: true testRunId: 12345 publishTestResultsMode: 'testRunADO' ``` -------------------------------- ### Test Output Example (IDs Found) Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of test output when test case IDs are successfully extracted and logged. ```text azure:pw:log [1] Login test - passed azure:pw:log Result published: [1] Login test ``` -------------------------------- ### GitLab CI Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example configuration for Playwright Azure Reporter in GitLab CI. ```typescript export default defineConfig({ reporter: [ [ '@alex_neo/playwright-azure-reporter', { orgUrl: 'https://dev.azure.com/my-org', projectName: 'MyProject', planId: 42, token: process.env.AZURE_TOKEN, environment: process.env.CI_COMMIT_REF_NAME || 'unknown', testRunTitle: `GitLab CI - ${process.env.CI_PIPELINE_ID}`, publishTestResultsMode: 'testRun', } ] ] }); ``` -------------------------------- ### GitHub Actions Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Example configuration for Playwright Azure Reporter in GitHub Actions. ```typescript export default defineConfig({ reporter: [ [ '@alex_neo/playwright-azure-reporter', { orgUrl: 'https://dev.azure.com/my-org', projectName: 'MyProject', planId: 42, token: process.env.AZURE_TOKEN, environment: process.env.GITHUB_REF?.split('/').pop() || 'unknown', testRunTitle: `GitHub Actions - ${process.env.GITHUB_RUN_ID}`, publishTestResultsMode: 'testRun', } ] ] }); ``` -------------------------------- ### Test Title Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Examples of how to include test case IDs directly in the test title using various formats. ```typescript test('[1] Login test', () => {}); test('[10,11,12] Multi-case test', () => {}); test('[1, 2, 3] Spaced format', () => {}); test('[1][2][3] Multiple brackets', () => {}); test('Test [1] with [2] mixed [3] content', () => {}); ``` -------------------------------- ### AutomatedTestId Update Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example showing the format of a generated AutomatedTestId GUID. ```typescript // Before: (not set or old GUID) // After: "a7f3b9d2c4e1f6a8b2d7e9f3a5c8b1d4" (generated GUID) ``` -------------------------------- ### Example: Overriding rootSuiteId with AZURE_PW_ROOT_SUITE_ID Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Shows how to set the AZURE_PW_ROOT_SUITE_ID environment variable to override the rootSuiteId configuration option. ```bash AZURE_PW_ROOT_SUITE_ID=5 npx playwright test ``` -------------------------------- ### Example authType and token configurations Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Authentication method to use. ```typescript // Personal Access Token (default) authType: 'pat' token: process.env.AZURE_PAT ``` ```typescript // OAuth Token authType: 'accessToken' token: process.env.OAUTH_TOKEN ``` ```typescript // Managed Identity authType: 'managedIdentity' applicationIdURI: '499b84ac-1321-427f-aa17-267ca6975798/.default' token: 'not-used' ``` -------------------------------- ### Summary Output Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example of the summary statistics logged onEnd(). ```text auto-mark summary: 5 marked, 2 updated, 3 skipped, 0 failed ``` -------------------------------- ### Configuration File Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/RELEASES.md Example of the .beta-release.json configuration file to customize release behavior. ```json { "betaRelease": { "enabled": true, "branches": ["main", "develop"], "versionStrategy": "patch", "publishTag": "beta", "createGitHubRelease": true, "prBetaRelease": { "enabled": true, "versionStrategy": "patch", "publishTag": "pr-beta", "createGitHubRelease": false } }, "stableRelease": { "enabled": true, "versionStrategy": "patch", "publishTag": "latest", "createGitHubRelease": true } } ``` -------------------------------- ### Installation Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/INDEX.md Installing the Playwright Azure Reporter package using npm. ```bash npm install @alex_neo/playwright-azure-reporter ``` -------------------------------- ### Simple Description Annotation Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of extracting a test case ID from a simple numeric description in an annotation. ```typescript test('Login', { annotation: { type: 'TestCase', description: '12345' }, }, () => {}); ``` -------------------------------- ### Example: Using Standard Azure DevOps Variables in testRunConfig Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Illustrates how to use standard Azure DevOps environment variables like BUILD_BUILDID and RELEASE_RELEASEID within the testRunConfig. ```typescript testRunConfig: { buildId: process.env.BUILD_BUILDID, releaseId: process.env.RELEASE_RELEASEID, } ``` -------------------------------- ### testRunId Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Examples of setting the testRunId, both explicitly and via environment variable. ```typescript // Explicit option (takes precedence) testRunId: 12345 // Or via environment variable // AZURE_PW_TEST_RUN_ID=12345 npx playwright test ``` -------------------------------- ### testRunConfig Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of configuring additional properties for a test run creation API call. ```typescript testRunConfig: { configurationIds: [3, 4, 5], buildId: 42, releaseId: 10, comment: 'Automated test run' } ``` -------------------------------- ### publishRetryResults Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Examples for handling test retry attempts. ```typescript // Publish all retries (shows retry history) publishRetryResults: 'all' // Publish only final result (cleaner results) publishRetryResults: 'last' ``` -------------------------------- ### publishTestResultsMode Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Examples demonstrating the different modes for publishing test results to Azure DevOps. ```typescript publishTestResultsMode: 'testResult' // Batch publish at end publishTestResultsMode: 'testRun' // Update existing run publishTestResultsMode: 'testRunADO' isExistingTestRun: true testRunId: 12345 ``` -------------------------------- ### testCaseIdMatcher Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Examples of different patterns for extracting test case IDs. ```typescript // Default: [123] or [1,2,3] testCaseIdMatcher: /[\[]([\d,\s]+)[\]]/ // Custom: @TC=123 testCaseIdMatcher: /@TC=(\d+)/ // Multiple patterns testCaseIdMatcher: [ /[\[]([\d,\s]+)[\]]/, // [123] /@TC=(\d+)/ // @TC=123 ] // String pattern (converted to RegExp) testCaseIdMatcher: '@TC=(\\d+)' ``` -------------------------------- ### Example 4: Custom Storage Path with Callback Function Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This example shows how to use a callback function for `automatedTestStoragePath` to create relative paths or sanitize file paths for storage. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, updateAutomatedTestName: true, updateAutomatedTestStorage: true, automatedTestNameFormat: 'titleWithParent', // Use callback to create relative path from project root automatedTestStoragePath: (testCase) => { const fullPath = testCase.location.file; const parts = fullPath.split('/'); // Find project folder and return path relative to it const projectIdx = parts.indexOf('my-project'); if (projectIdx >= 0) { return parts.slice(projectIdx + 1).join('/'); } // Fallback to just filename return parts[parts.length - 1]; } } ``` -------------------------------- ### Custom automatedTestStoragePath Callback Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md Example of a custom callback function for `automatedTestStoragePath` that includes error handling and a fallback mechanism. ```typescript automatedTestStoragePath: (testCase) => { try { const parts = testCase.location.file.split('/'); const idx = parts.indexOf('tests'); return idx >= 0 ? parts.slice(idx).join('/') : parts[parts.length - 1]; } catch (error) { console.error('Error in automatedTestStoragePath:', error); return testCase.location.file; // Fallback to full path } } ``` -------------------------------- ### Usage Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/types.md Example configuration for the Playwright Azure Reporter in Playwright's defineConfig. ```typescript import { defineConfig } from '@playwright/test'; import type { AzureReporterOptions } from '@alex_neo/playwright-azure-reporter'; export default defineConfig({ reporter: [ [ '@alex_neo/playwright-azure-reporter', { orgUrl: 'https://dev.azure.com/my-org', projectName: 'MyProject', planId: 42, token: process.env.AZURE_TOKEN, authType: 'pat', logging: true, testRunTitle: 'Automated Test Run', environment: 'QA', publishTestResultsMode: 'testRun', uploadAttachments: true, attachmentsType: ['screenshot', 'video'], uploadLogs: true, testPointMapper: async (testCase, testPoints) => { // Filter by browser configuration if (testCase.parent?.project?.use?.browserName === 'chromium') { return testPoints.filter((tp) => tp.configuration.id === '3'); } return testPoints; }, testRunConfig: { configurationIds: [3, 4, 5], }, testCaseSummary: { enabled: true, outputPath: './test-case-summary.md', consoleOutput: true, publishToRun: true, }, autoMarkTestCasesAsAutomated: { enabled: true, automatedTestNameFormat: 'titleWithParent', automatedTestStoragePath: true, automatedTestType: (testCase) => { if (testCase.location?.file?.includes('/e2e/')) { return 'E2E Test'; } return 'Functional Test'; }, }, } as AzureReporterOptions, ], ], }); ``` -------------------------------- ### Example 1: Enable Auto-Marking with Default Settings Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This example shows how to enable the `autoMarkTestCasesAsAutomated` feature with its default settings in the Playwright configuration. ```typescript import { defineConfig } from '@playwright/test'; import AzureDevOpsReporter from '@alex_neo/playwright-azure-reporter'; export default defineConfig({ reporter: [ [ AzureDevOpsReporter, { orgUrl: 'https://dev.azure.com/your-organization', token: process.env.AZURE_TOKEN, planId: 123, projectName: 'YourProject', logging: true, autoMarkTestCasesAsAutomated: { enabled: true, // Enable the feature updateAutomatedTestName: true, // Will set test name (default) updateAutomatedTestStorage: true // Will set test storage file (default) } } ] ], // ... rest of your config }); ``` -------------------------------- ### Example projectName configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Name of the Azure DevOps project (case-sensitive). ```typescript projectName: 'MyAutomationProject' ``` -------------------------------- ### Custom Title Matchers Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Examples of custom regular expressions for extracting test case IDs from the title zone, including different formats and multiple patterns. ```typescript // Extract @TC=123 format testCaseIdMatcher: /@TC=(\d+)/ test('Login @TC=42', () => {}); // Extracted: ['42'] // Extract TEST123 and test456 formats testCaseIdMatcher: [ /TEST(\d+)/, // TEST123 /test(\d+)/ // test456 ] test('Login TEST100 and test200', () => {}); // Extracted: ['100', '200'] // Custom format with name testCaseIdMatcher: /@Case\[(\d+)\]/ test('Login @Case[1] @Case[2]', () => {}); // Extracted: ['1', '2'] ``` -------------------------------- ### Example 6: Combining Custom Path and Test Type Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This example combines custom storage path logic with dynamic test type assignment based on file location and naming patterns. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, updateAutomatedTestName: true, updateAutomatedTestStorage: true, automatedTestNameFormat: 'titleWithParent', // Custom storage path for relative paths automatedTestStoragePath: (testCase) => { const parts = testCase.location.file.split('/'); const testsIdx = parts.indexOf('tests'); return testsIdx >= 0 ? parts.slice(testsIdx).join('/') : parts[parts.length - 1]; }, // Custom test type based on file patterns automatedTestType: (testCase) => { const fileName = testCase.location.file; // Check file naming patterns if (fileName.includes('.e2e.')) return 'End-to-End Test'; if (fileName.includes('.integration.')) return 'Integration Test'; if (fileName.includes('.unit.')) return 'Unit Test'; // Check folder structure if (fileName.includes('/e2e/')) return 'End-to-End Test'; if (fileName.includes('/integration/')) return 'Integration Test'; if (fileName.includes('/unit/')) return 'Unit Test'; return 'Functional Test'; } } ``` -------------------------------- ### testCaseIdMatcher Extraction Logic Examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Illustrative examples of how testCaseIdMatcher extracts IDs from test titles. ```typescript // Test title: "[1,2,3] Login test" // Matcher: /[\[]([\d,\s]+)[\]]/ // Extracted: ['1', '2', '3'] // Test title: "Login @TC=42 @TC=43" // Matcher: /@TC=(\d+)/ // Extracted: ['42', '43'] ``` -------------------------------- ### Access Token Authentication Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/README.md Example configuration for authenticating with an Access Token. ```typescript { token: 'your-access-token', authType: 'accessToken' } ``` -------------------------------- ### AutomatedTestStorage Update Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example showing the potential values for AutomatedTestStorage after an update. ```typescript // Before: (not set or old value) // After: "login.spec.ts" or "tests/login.spec.ts" ``` -------------------------------- ### testPointMapper Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of a custom testPointMapper function to filter test points by browser. ```typescript testPointMapper: async (testCase, testPoints) => { // Filter by browser const browserName = testCase.parent?.project?.use?.browserName; switch (browserName) { case 'chromium': return testPoints.filter((tp) => tp.configuration?.id === '3'); case 'firefox': return testPoints.filter((tp) => tp.configuration?.id === '4'); case 'webkit': return testPoints.filter((tp) => tp.configuration?.id === '5'); default: return testPoints; } } ``` -------------------------------- ### PAT Authentication Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/README.md Example configuration for authenticating with a Personal Access Token (PAT). ```typescript { token: 'your-personal-access-token', authType: 'pat' // Optional, this is the default } ``` -------------------------------- ### Example 3: Minimal Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md A minimal configuration to enable the `autoMarkTestCasesAsAutomated` feature, relying on default values for other options. ```typescript autoMarkTestCasesAsAutomated: { enabled: true // updateAutomatedTestName and updateAutomatedTestStorage default to true } ``` -------------------------------- ### Logging Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example log messages indicating the status of test case automation updates. ```text azure:pw:log Test case 42 is already automated, checking if update needed azure:pw:log Test case 42 automation details updated // or azure:pw:log Test case 42 automation details unchanged ``` -------------------------------- ### CI/CD Pipeline Integration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md Example configuration for `playwright.config.ts` to integrate Azure DevOps Reporter into a CI/CD pipeline, enabling `autoMarkTestCasesAsAutomated` only when running in CI. ```typescript // playwright.config.ts export default defineConfig({ reporter: [ [ AzureDevOpsReporter, { orgUrl: process.env.AZURE_ORG_URL, token: process.env.AZURE_TOKEN, planId: parseInt(process.env.AZURE_PLAN_ID || '0'), projectName: process.env.AZURE_PROJECT, logging: true, autoMarkTestCasesAsAutomated: { enabled: process.env.CI === 'true', // Only in CI updateAutomatedTestName: true, updateAutomatedTestStorage: true } } ] ] }); ``` -------------------------------- ### Install Specific Beta Version Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/RELEASES.md Command to install a specific beta version of the reporter. ```bash npm install @alex_neo/playwright-azure-reporter@1.13.2-beta.0 ``` -------------------------------- ### Debug Logging Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example debug output when auto-marking is enabled and processing test cases. ```text azure:pw:debug Test case 42 automation status: Not Automated azure:pw:log Marking test case 42 as automated ``` -------------------------------- ### uploadLogs Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of enabling the upload of test stdout and stderr logs. ```typescript uploadLogs: true // Uploads stdout.txt and stderr.txt for each test ``` -------------------------------- ### Constructor Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/azure-devops-reporter.md Example of how to configure and use the AzureDevOpsReporter in Playwright's defineConfig. ```typescript import { defineConfig } from '@playwright/test'; import AzureDevOpsReporter from '@alex_neo/playwright-azure-reporter'; export default defineConfig({ reporter: [ [ AzureDevOpsReporter, { orgUrl: 'https://dev.azure.com/my-organization', projectName: 'MyProject', planId: 42, token: process.env.AZURE_PAT_TOKEN, logging: true, } ], ], }); ``` -------------------------------- ### Install Latest PR Beta Release Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/RELEASES.md Command to install the latest PR beta release of the reporter. ```bash npm install @alex_neo/playwright-azure-reporter@pr-beta ``` -------------------------------- ### createGuid Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/utilities.md Example of generating a GUID using the createGuid function. ```typescript import { createGuid } from '@alex_neo/playwright-azure-reporter'; const id = createGuid(); // Example output: 'a7f3b9d2c4e1f6a8b2d7e9f3a5c8b1d4' ``` -------------------------------- ### Install Specific PR Beta Version Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/RELEASES.md Command to install a specific PR beta version of the reporter. ```bash npm install @alex_neo/playwright-azure-reporter@1.13.3-pr42.feature-branch.a1b2c3d ``` -------------------------------- ### Example 5: Setting Test Type Based on File Location Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This example demonstrates using the `automatedTestType` callback to dynamically set the test type based on the file's location within the project structure. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, updateAutomatedTestName: true, updateAutomatedTestStorage: true, // Automatically set test type based on folder structure automatedTestType: (testCase) => { const filePath = testCase.location.file; if (filePath.includes('/e2e/')) return 'End-to-End Test'; if (filePath.includes('/integration/')) return 'Integration Test'; if (filePath.includes('/unit/')) return 'Unit Test'; if (filePath.includes('/api/')) return 'API Test'; return 'Functional Test'; // default } } ``` -------------------------------- ### Install Latest Merge Beta Release Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/RELEASES.md Command to install the latest merge beta release of the reporter. ```bash npm install @alex_neo/playwright-azure-reporter@beta ``` -------------------------------- ### Example environment configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Optional environment name to prefix test run titles. Useful for distinguishing QA, Staging, Production runs. ```typescript environment: 'QA' // Result: test run title becomes "[QA]: Playwright Test Run" ``` ```typescript environment: 'Staging' // Result: test run title becomes "[Staging]: Playwright Test Run" ``` -------------------------------- ### rootSuiteId Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of setting rootSuiteId to scope test point resolution. ```typescript rootSuiteId: 5 // Only matches test points under suite ID 5 ``` -------------------------------- ### Using Access Token Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/authType-examples.md Configuration example for using an Access Token from the Azure DevOps API with the Azure DevOps Playwright Reporter. ```typescript // playwright.config.ts export default { reporter: [ [ '@alexneo2003/playwright-azure-reporter', { orgUrl: 'https://dev.azure.com/your-org', projectName: 'your-project', planId: 123, token: 'your-access-token', authType: 'accessToken', }, ], ], }; ``` -------------------------------- ### getMimeTypeFromFilename Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/utilities.md Example usage of the getMimeTypeFromFilename function to get MIME types for various file extensions. ```typescript import { getMimeTypeFromFilename } from '@alex_neo/playwright-azure-reporter'; getMimeTypeFromFilename('screenshot.png') // Returns: 'image/png' getMimeTypeFromFilename('/path/to/video.mp4') // Returns: 'video/mp4' getMimeTypeFromFilename('unknown.xyz') // Returns: 'application/octet-stream' ``` -------------------------------- ### Example testRunTitle configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Display name for newly created test runs. Prefixed with [environment]: if environment is set. ```typescript testRunTitle: 'Automated End-to-End Tests' // With environment: // environment: 'QA' // Result: "[QA]: Automated End-to-End Tests" ``` -------------------------------- ### uploadAttachments and attachmentsType Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of enabling attachment uploads and specifying attachment types. ```typescript uploadAttachments: true attachmentsType: ['screenshot', 'video'] ``` -------------------------------- ### Bracketed Format Annotation Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of extracting test case IDs from a bracketed, comma-separated string in an annotation description. ```typescript test('Login', { annotation: { type: 'Test Case', description: '[12345, 67890]', }, }, () => {}); ``` -------------------------------- ### getExtensionFromContentType Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/utilities.md Example usage of the getExtensionFromContentType function to get file extensions from MIME types, including handling of aliases. ```typescript import { getExtensionFromContentType } from '@alex_neo/playwright-azure-reporter'; getExtensionFromContentType('image/png') // Returns: 'png' getExtensionFromContentType('video/mp4') // Returns: 'mp4' getExtensionFromContentType('image/jpeg') // Returns: 'jpeg' (normalized from 'jpg') getExtensionFromContentType('application/unknown') // Returns: 'bin' ``` -------------------------------- ### Publish Test Case Summary to Run Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of enabling the upload of the test case summary as a markdown attachment to the test run. ```typescript testCaseSummary: { publishToRun: true // Upload to Azure DevOps } ``` -------------------------------- ### Enable Test Case Summary Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of enabling test case summary generation and setting its output path, console output, and publish to run options. ```typescript testCaseSummary: { enabled: true, outputPath: './test-case-summary.md', consoleOutput: true, publishToRun: true } ``` -------------------------------- ### Azure DevOps URLs Annotation Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example of extracting test case IDs from Azure DevOps work item URLs in an annotation description. ```typescript test('Login', { annotation: { type: 'Test Case', description: 'https://dev.azure.com/org/project/_workitems/edit/12345, https://dev.azure.com/org/project/_workitems/edit/54321', }, }, () => {}); ``` -------------------------------- ### Example orgUrl configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Full URL of your Azure DevOps organization. Do not include project name. ```typescript orgUrl: 'https://dev.azure.com/my-organization-name' ``` -------------------------------- ### Example token configurations for different auth types Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Authentication token. Content depends on authType. ```typescript // Personal Access Token (PAT) token: 'your-personal-access-token' ``` ```typescript // Access Token (OAuth) token: 'oauth-access-token' ``` ```typescript // Managed Identity (ignored, placeholder required) token: 'not-used' ``` -------------------------------- ### Azure CLI Authentication for Advanced Scenarios Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md Demonstrates how to authenticate using Azure CLI with a service principal before running Playwright tests. ```bash # Azure CLI authentication az login --service-principal -u $AZURE_CLIENT_ID -p $AZURE_CLIENT_SECRET --tenant $AZURE_TENANT_ID # Then run tests AZUREPWDEBUG=1 npx playwright test ``` -------------------------------- ### Edge Cases - Invalid patterns Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md An example of an invalid `testCaseIdMatcher` configuration where a number is provided instead of a string or RegExp, resulting in an error message. ```typescript testCaseIdMatcher: 123 // Not string/RegExp // Error: "Invalid testCaseIdMatcher. Must be a string or RegExp." ``` -------------------------------- ### Set Test Case Summary Output Path Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of specifying the file path for the test case summary markdown. ```typescript testCaseSummary: { outputPath: './reports/test-case-summary.md' } ``` -------------------------------- ### Annotation Zone Requirements - Must provide matcher Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/test-case-id-matching.md Example demonstrating the requirement to provide a `testCaseIdMatcher` when `testCaseIdZone` is set to 'annotation'. A default matcher is used if none is provided, but it's explicitly required. ```typescript testCaseIdZone: 'annotation' // WARNING: testCaseIdMatcher not set, using default /\<0xC2><0xA0>[([\d,\s]+)]/ testCaseIdMatcher: /(TestCase)/ // Required! ``` -------------------------------- ### Relative Path Best Practice Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Demonstrates the recommended way to use relative paths for storage, contrasting it with an absolute path. ```typescript // ✓ Good - relative from project root automatedTestStoragePath: (testCase) => { return path.relative(process.cwd(), testCase.location.file); } // ✗ Bad - absolute path automatedTestStoragePath: true // Full absolute path ``` -------------------------------- ### Example planId configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md ID of the test plan to publish results to. Found in test plan URL: .../_testplans/execute?planId=42 ```typescript planId: 42 ``` -------------------------------- ### Example 2: Enable Auto-Marking Without Updating Test Details Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This example demonstrates how to enable auto-marking while disabling the automatic update of the test name and storage file. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, updateAutomatedTestName: false, // Don't update test name updateAutomatedTestStorage: false // Don't update test storage } ``` -------------------------------- ### Example with error handling for auto-mark automated examples Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/tests/examples/auto-mark-automated-examples.md This code snippet demonstrates how to implement callbacks for `autoMarkTestCasesAsAutomated` with error handling, providing fallback values and logging potential issues. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, automatedTestStoragePath: (testCase) => { try { if (!testCase.location?.file) return 'unknown.spec.ts'; const parts = testCase.location.file.split('/'); const testsIdx = parts.indexOf('tests'); if (testsIdx >= 0 && testsIdx < parts.length - 1) { return parts.slice(testsIdx).join('/'); } return parts[parts.length - 1] || 'unknown.spec.ts'; } catch (error) { console.error('Error in automatedTestStoragePath callback:', error); return testCase.location?.file || 'error.spec.ts'; } }, automatedTestType: (testCase) => { try { const filePath = testCase.location?.file || ''; if (filePath.includes('/e2e/')) return 'End-to-End Test'; if (filePath.includes('/integration/')) return 'Integration Test'; if (filePath.includes('/unit/')) return 'Unit Test'; return 'Functional Test'; } catch (error) { console.error('Error in automatedTestType callback:', error); return 'Unknown Test'; } } } ``` -------------------------------- ### Annotation Mode with Multiple Matchers Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example demonstrating two-step extraction for annotation mode using multiple matchers. ```typescript testCaseIdZone: 'annotation' testCaseIdMatcher: [/(TestCase)/, /[([\d,\s]+)]/] // Test: test('Example', { annotation: { type: 'TestCase', description: '[123, 456]' } }, () => {}); // Result: Extracted ['123', '456'] ``` -------------------------------- ### Token Scope Configuration Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Illustrates a token with insufficient scopes and a comment suggesting regeneration with more scopes. ```typescript // Token with insufficient scopes // Solution: Regenerate token with more scopes ``` -------------------------------- ### Enable Console Output for Test Case Summary Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/configuration.md Example of enabling the printing of the test case summary to the console. ```typescript testCaseSummary: { consoleOutput: true // Print to console } ``` -------------------------------- ### AutomatedTestType Update Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example showing the potential value for AutomatedTestType after an update. ```typescript // Before: (not set) // After: "End-to-End Test" ``` -------------------------------- ### Full Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Comprehensive configuration including custom formats and callbacks for test name and storage path. ```typescript autoMarkTestCasesAsAutomated: { enabled: true, // Update test name field updateAutomatedTestName: true, // Update test storage field updateAutomatedTestStorage: true, // Format for automated test name automatedTestNameFormat: 'titleWithParent', // Storage path handling automatedTestStoragePath: (testCase) => { const parts = testCase.location.file.split('/'); const idx = parts.indexOf('tests'); return idx >= 0 ? parts.slice(idx).join('/') : testCase.location.file; }, // Custom test type callback automatedTestType: (testCase) => { const file = testCase.location?.file || ''; if (file.includes('/e2e/')) return 'End-to-End Test'; if (file.includes('/integration/')) return 'Integration Test'; if (file.includes('/unit/')) return 'Unit Test'; return 'Functional Test'; }, } ``` -------------------------------- ### AutomatedTestName Update Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example showing the potential values for AutomatedTestName after an update. ```typescript // Before: (not set or old value) // After: "Login Test" or "Authentication > Login Test" ``` -------------------------------- ### AutomationStatus Update Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/auto-mark-automation.md Example showing the change in AutomationStatus from 'Not Automated' to 'Automated'. ```typescript // Before: "Not Automated" // After: "Automated" ``` -------------------------------- ### shortID Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/utilities.md Example of generating a short ID using the shortID function. ```typescript import { shortID } from '@alex_neo/playwright-azure-reporter'; const alias = shortID(); // Example output: 'f3a5c8b1d4e7a2f5' ``` -------------------------------- ### Authentication in CI/CD: PAT Configuration Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md TypeScript configuration object showing how to specify 'pat' as the authType when using a Personal Access Token. ```typescript { token: process.env.AZURE_TOKEN, authType: 'pat' } ``` -------------------------------- ### onBegin() Hook Example Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/azure-devops-reporter.md Illustrative comment for the onBegin() hook, indicating its automatic call by the Playwright test runner to create the initial test run and fetch test points. ```typescript // Automatically called by Playwright test runner // Creates initial test run and fetches test points ``` -------------------------------- ### Basic Azure DevOps Pipeline Step Source: https://github.com/alexneo2003/playwright-azure-reporter/blob/main/_autodocs/ci-cd-integration.md A basic script step in an Azure DevOps pipeline to run Playwright tests. ```yaml steps: - script: npx playwright test env: AZURE_TENANT_ID: $(SYSTEM_TEAMTENANT) ```