### Setup and Run Action Source: https://github.com/ctrf-io/github-test-reporter/blob/main/CONTRIBUTING.md Clone the repository, install dependencies, and build/test the action. The `all:action` script is used for this purpose. ```bash git clone https://github.com/ctrf-io/github-test-reporter.git cd github-test-reporter npm install # Add your changes npm run all:action ``` -------------------------------- ### Full example workflow for generating CTRF report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This is a complete GitHub Actions workflow example that checks out code and then runs the npx github-actions-ctrf command to generate a report. It uses `if: always()` to ensure execution regardless of previous step failures. ```yaml name: Example workflow file on: [push] jobs: testing: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Generate Report run: npx github-actions-ctrf path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Local Installation and Execution Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Installs github-actions-ctrf locally within your Node.js project and provides commands to run it. ```bash npm install github-actions-ctrf ``` ```bash ./node_modules/.bin/github-actions-ctrf path-to-your-ctrf-report.json ``` ```json { "scripts": { "report": "github-actions-ctrf path-to-your-ctrf-report.json" } } ``` ```bash npm run report ``` -------------------------------- ### Configure Slack Integration Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/integrations.md Configure the Slack integration to send test results to Slack channels. This example shows how to customize the message title, prefix, suffix, and consolidate results. Requires the `SLACK_WEBHOOK_URL` environment variable. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' integrations-config: | { "slack": { "enabled": true, "action": "results", "options": { "title": "Test Results", "prefix": "Custom prefix", "suffix": "Custom suffix", "consolidated": false, "onFailOnly": false } } } env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} if: always() ``` -------------------------------- ### Run github-actions-ctrf with a specific file Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Use this command to generate a CTRF report from a single JSON file. Ensure Node.js is installed on your GitHub Actions runner. ```bash npx github-actions-ctrf path-to-your-ctrf-report.json ``` -------------------------------- ### Run Specific Version with NPX Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Runs a specific version of github-actions-ctrf without global installation. Useful for locking to a known stable version. ```bash npx github-actions-ctrf@1.2.3 path-to-your-ctrf-report.json ``` -------------------------------- ### Example Failed Test Case Details Source: https://github.com/ctrf-io/github-test-reporter/blob/main/community-reports/failed-detailed/README.md This HTML structure represents a single failed test case within the 'failed-detailed' report. It includes the test name, failure message, stack trace, code snippet, standard output, and standard error if available. The details are expandable using the `
` tag. ```html
Failed Tests ❌ Fail Message
should display title
Error: expect(page).toHaveTitle(expected) failed

Trace:

Error: expect(page).toHaveTitle(expected) failed
Expect "toHaveTitle" with timeout 5000ms 9 × unexpected value "Fast and reliable end-to-end testing for modern web apps | Playwright"
    at .\Projects\Playwright\tests\example.spec.ts:7:22

Snippet:

   5 |
   6 |
   7 |   // Expect a title "to contain" a substring.
>  8 |   await expect(page).toHaveTitle(/Playwright/);
     |                      ^
   9 |
  10 | });

Standard Output:

Navigated URL: https://playwright.dev/

Standard Error:

No standard error available
``` -------------------------------- ### Configure Microsoft Teams Integration Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/integrations.md Configure the Microsoft Teams integration to send test results to Teams channels. This example demonstrates customizing the message title and setting `onFailOnly`. Requires the `TEAMS_WEBHOOK_URL` environment variable. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' integrations-config: | { "teams": { "enabled": true, "action": "results", "options": { "title": "Test Results", "onFailOnly": false } } } env: TEAMS_WEBHOOK_URL: ${{ secrets.TEAMS_WEBHOOK_URL }} if: always() ``` -------------------------------- ### Run github-actions-ctrf with a glob pattern Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command allows generating a CTRF report from multiple JSON files matching a glob pattern. Ensure Node.js is installed on your GitHub Actions runner. ```bash npx github-actions-ctrf "ctrf/*.json" ``` -------------------------------- ### Display Failed Test Details Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Example of how detailed failure information, including error messages and traces, is presented within a collapsible section for a specific test case. ```html
Failed Tests
❌ should display title
Timed out 5000ms waiting for expect(locator).toHaveTitle(expected)
Locator: locator(':root')
Expected pattern: /Playwrc cight/
Received string:  "Fast and reliable end-to-end testing for modern web apps | Playwright"
Call log:
  - expect.toHaveTitle with timeout 5000ms
  - waiting for locator(':root')
  -   locator resolved to <html lang="en" dir="ltr" data-theme="light" data-has-…</html>
  -   unexpected value "Fast and reliable end-to-end testing for modern web apps | Playwright"

Trace:

ProfileTest.js:45
❌ should fail to update profile on network failure
Network Timeout

Trace:

ProfileUpdateTest.js:60
❌ should fail to update profile on network failure
No message available

Trace:

No trace available
``` -------------------------------- ### Iterate Over CTRF Tests Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/build-your-own-report.md An example of how to iterate through individual test results using Handlebars' `each` helper to access test names and statuses. ```hbs {{#each ctrf.tests}} Test: {{this.name}} - Status: {{this.status}} {{/each}} ``` -------------------------------- ### Workflow A: Run Tests and Upload Artifacts Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/fork-pull-requests.md This workflow triggers on pull requests and runs tests, uploading the results as an artifact. It uses `pull_request` for security, requiring only read permissions. ```yaml name: Run Tests on: pull_request: branches: ['**'] jobs: test: runs-on: ubuntu-latest steps: - name: Check out PR code uses: actions/checkout@v4 - name: Run Tests run: | ./run-tests.sh # Replace with your actual test command - name: Upload Test Report Artifact uses: actions/upload-artifact@v4 with: name: testReport path: ./results/ctrf-report.json ``` -------------------------------- ### Configure Multiple Integrations Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/integrations.md Configure multiple integrations like Slack, Teams, and AI in a single GitHub Actions workflow. Ensure the necessary environment variables are set for each integration. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' integrations-config: | { "slack": { "enabled": true, "action": "results" }, "teams": { "enabled": true, "action": "results" }, "ai": { "enabled": true, "action": "openai" } } env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} TEAMS_WEBHOOK_URL: ${{ secrets.TEAMS_WEBHOOK_URL }} OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} if: always() ``` -------------------------------- ### Enable File Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `file-report` input to true to enable the file report. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' file-report: true if: always() ``` -------------------------------- ### Enable Summary Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `summary-report` input to true in your workflow configuration to display a concise table of test statuses. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' summary-report: true if: always() ``` -------------------------------- ### Enable AI Report with OpenAI Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `ai-report` input to true and provide AI configuration, including the provider and model, to generate AI-powered summaries for failed tests. Requires `OPENAI_API_KEY` environment variable. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' ai-report: true ai: | { "provider": "openai", "model": "gpt-4", } if: always() env: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} ``` -------------------------------- ### Workflow B: Download Artifacts and Publish Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/fork-pull-requests.md This workflow triggers after Workflow A completes successfully. It downloads the test report artifact and uses the `github-test-reporter` action to post results to the pull request. It requires `pull-requests: write` permissions. ```yaml name: Publish Test Report on: workflow_run: workflows: ['Run Tests'] types: [completed] permissions: pull-requests: write contents: read jobs: report: runs-on: ubuntu-latest if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Download Test Report Artifact uses: dawidd6/action-download-artifact@v8 with: name: testReport run_id: ${{ github.event.workflow_run.id }} path: artifacts - name: Determine PR number securely id: get_pr env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | HEAD_SHA="${{ github.event.workflow_run.head_sha }}" PR_NUM=$(gh pr list \ --state open \ --json number,headRefOid \ --jq ".[] | select(.headRefOid==\"${HEAD_SHA}\") | .number") if [ -z "$PR_NUM" ]; then echo "No open PR found for head SHA ${HEAD_SHA}" exit 1 fi echo "PR_NUMBER=$PR_NUM" >> $GITHUB_ENV - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: 'artifacts/ctrf-report.json' pull-request: ${{ env.PR_NUMBER }} update-comment: true comment-tag: test-report env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Print GitHub Context for Debugging Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/build-your-own-report.md Print the entire GitHub context to workflow logs using `toJson` and `jq` for debugging. This helps in identifying all available properties for a specific workflow run. ```yaml - name: Print GitHub Context env: CONTEXT: ${{ toJson(github) }} run: echo "$CONTEXT" | jq . ``` -------------------------------- ### Use summary-short Template in Workflow Source: https://github.com/ctrf-io/github-test-reporter/blob/main/community-reports/summary-short/README.md Reference the 'summary-short' template by its name in your workflow file to generate a summary report. ```yaml community-report-name: summary-short ``` -------------------------------- ### Enable Summary Delta Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `summary-delta-report` input to true to display a summary of test results with comparisons to a baseline. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' summary-delta-report: true if: always() ``` -------------------------------- ### Enable Suite List Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Activate the suite list report by setting `suite-list-report` to true in your workflow. This provides a detailed, flat list of all executed tests, grouped by suite. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' suite-list-report: true if: always() ``` -------------------------------- ### Enable GitHub Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `github-report` input to true to enable the GitHub-styled report, which uses GitHub's design system for a native look and feel. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' github-report: true if: always() ``` -------------------------------- ### Configure GitHub Test Reporter Action Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use this snippet to configure the GitHub Test Reporter action in your workflow. Only `report-path` is required. Customize report generation, behavior, and advanced options as needed. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: # Core Configuration report-path: './ctrf/*.json' # Path or glob pattern to the CTRF report JSON file. template-path: './templates/custom-summary.hbs' # Path to the Handlebars template for customizing markdown output. # Reports - Choose as many as you like. Default is false. Choosing none will use default reports. summary-report: true summary-delta-report: false tests-changed-report: false github-report: false test-report: false test-list-report: false failed-report: false fail-rate-report: false flaky-report: false flaky-rate-report: false failed-folded-report: false previous-results-report: false insights-report: false slowest-report: false ai-report: false ai-summary-report: false skipped-report: false suite-folded-report: false suite-list-report: false file-report: false pull-request-report: false commit-report: false custom-report: false community-report: false # Behavior Options summary: true # Post report to the job summary. Default is true pull-request: false # Comment on pull request with report. Default is false issue: '' # Issue number to comment on. Works with standard issues and pull-request. Default is no issue status-check: false # Create a status check for the workflow. Default is false status-check-name: 'Test Reporter Results' # Name of the status check. Default is GitHub Test Reporter Results community-report-name: 'summary-short' # Name of the community report to use. Default is summary-short title: '' # Set a custom title to display on the report. annotate: true # Add failed test annotations. Default is true on-fail-only: false # Add a pull request comment only if tests fail. Default is false exit-on-no-files: false # Exit the workflow with a failure status if no test files are found. Default is false exit-on-fail: false # Exit the workflow with a failure status if any tests fail. Default is false exit-on-empty: false # Exit the workflow with a failure status if no tests are found. Default is false use-suite-name: false # Prefix test names with the suite name for better grouping. Default is false collapse-large-reports: false # Collapse large reports (test-table and test-list) for better readability. Default is false update-comment: false # Update existing Pull Request comment. Default is false overwrite-comment: false # Overwrite existing Pull Request comment. Default is false always-latest-comment: false # Create a new comment if the existing comment is not the latest in the thread. Default is false comment-tag: false # Tag to match Pull Request comment write-ctrf-to-file: 'ctrf/ctrf-report.json' # Path to write the internal processed CTRF report for future processing. Default no write upload-artifact: true # Upload to workflow artifact the processed CTRF report for future processing. Default false # Advanced Options artifact-name: 'ctrf-report' # Name of the artifact containing test reports. Default is ctrf-report previous-results-max: 10 # Maximum number of previous test results to display in the report. Default is 10 fetch-previous-results: false # Always fetch previous workflow runs when using custom templates. Default is false max-workflow-runs-to-check: 400 # Maximum number of workflow runs to check for previous results. Default is 400 max-previous-runs-to-fetch: 100 # Maximum number of previous runs to fetch and process for metrics and reports. Default is 100 baseline: 1 # Baseline report to use for metrics comparison. Number = previous n reports, string = reportId.' Default is 1 (last run) baseline-report-path: 'path/to/baseline/report.json' # Path to a specific CTRF report to use as a baseline.' group-by: 'filePath' # Specify grouping for applicable reports (e.g., suite or file path). Default is filePath always-group-by: false # Force grouping by suite or file path for all reports. Default is false report-order: 'summary-report,failed-report,flaky-report,skipped-report,test-report' # Comma-separated list of report types to specify the order in which reports should be displayed integrations-config: '{}' # JSON configuration for integrations with other developer tools if: always() ``` -------------------------------- ### Publish CTRF Flaky Rate Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md To visualize the flakiness of tests over time, use this command. It requires the `GITHUB_TOKEN` environment variable and uses the `flaky-rate` argument with npx github-actions-ctrf. Artifact upload is also required. ```yaml - name: Publish CTRF Flaky Rate Test Summary Results run: npx github-actions-ctrf flaky-rate path-to-your-ctrf-report.json if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Configure AI Integration for Test Reports Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/integrations.md Use this configuration to enable AI analysis of test results. Requires setting the appropriate environment variable for the AI provider. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' integrations-config: | { "ai": { "enabled": true, "action": "openai", "options": { "model": "gpt-4", "systemPrompt": "Custom system prompt", "frequencyPenalty": 0, "maxTokens": 1000, "presencePenalty": 0, "temperature": 0.7, "topP": 1, "log": false, "maxMessages": 10, "consolidate": false, "deploymentId": "your-azure-deployment-id" } } } env: OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }} if: always() ``` -------------------------------- ### Build Custom Report with Handlebars Template Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `custom-report: true` and `template-path` inputs to generate a report using a custom Handlebars template. This allows for dynamic and customizable report content. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' template-path: './templates/custom-report.hbs' custom-report: true if: always() ``` -------------------------------- ### Enable Insights Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `insights-report` input to true to generate an overview of recent build metrics. This report provides key metrics like average tests, flaky tests, and failed tests. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' insights-report: true if: always() ``` -------------------------------- ### Publish Test Report in GitHub Actions Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Basic configuration to publish test reports. Ensure 'report-path' points to your test results files. ```yaml name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' github-report: true if: always() ``` -------------------------------- ### Publish CTRF Suite List Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command generates a list of tests grouped by suite. It uses the `suite-list` argument with npx github-actions-ctrf. By default, it groups by filePath, but can use the suite property with `--useSuite`. ```yaml - name: Publish CTRF Suite Folded Summary run: npx github-actions-ctrf suite-list path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Enable Commit Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `commit-report` input to `true` in your workflow to publish test results as commit statuses. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' commit-report: true if: always() ``` -------------------------------- ### Enable Suite-Folded Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Configure your workflow to enable the suite-folded report by setting `suite-folded-report` to true. This groups tests within their respective suites for a more organized view. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' suite-folded-report: true if: always() ``` -------------------------------- ### Upload Test Results as Artifacts Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Configure the action to upload test results as artifacts. This is useful for storing and accessing test outputs. ```yaml - name: Upload test results uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' upload-artifact: true if: always() ``` -------------------------------- ### Publish Test Report with Insights Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Enables multi-run insights on failures, flakiness, and test duration. Requires GITHUB_TOKEN with appropriate permissions. ```yaml name: Publish Test Report with Insights uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' summary-delta-report: true insights-report: true flaky-rate-report: true fail-rate-report: true slowest-report: true upload-artifact: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} if: always() ``` -------------------------------- ### Basic Test Results Summary Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/build-your-own-report.md A Handlebars template to generate a test summary with statistics, execution details, and conditional display of failed tests. It utilizes CTRF properties, GitHub context, and helper functions. ```hbs # Test Results Summary ## Overview {{!-- Display basic test statistics --}} 📊 **Test Statistics** - Total Tests: {{ctrf.summary.tests}} - ✅ Passed: {{ctrf.summary.passed}} - ❌ Failed: {{ctrf.summary.failed}} - 🔄 Flaky Tests: {{countFlaky ctrf.tests}} ## Execution Details {{!-- Show timing information --}} ⏱️ **Duration**: {{formatDurationFromTimes ctrf.summary.start ctrf.summary.stop}} 🔍 **Branch**: {{github.branchName}} 👤 **Triggered by**: {{github.actor}} {{!-- Conditionally show failures if they exist --}} {{#if ctrf.summary.failed}} ## Failed Tests {{#each ctrf.tests}} {{#if (eq this.status "failed")}} #### ❌ {{this.name}} {{/if}} {{/each}} {{/if}} ``` -------------------------------- ### Publish Test Report with GitHub Token Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use this snippet to publish test reports. It requires a GITHUB_TOKEN for authentication. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' flaky-rate-report: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} if: always() ``` -------------------------------- ### Enable Test Report Generation Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `test-report` input to true in your workflow configuration to enable general test report generation. Ensure the `report-path` points to your test results. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' test-report: true if: always() ``` -------------------------------- ### Enable Fail Rate Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Configure the GitHub Test Reporter to include the fail rate report in your workflow. This report analyzes test failures over historical runs. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' fail-rate-report: true previous-results-max: 100 if: always() ``` -------------------------------- ### Publish CTRF Flaky Test Summary Results (Failed Rate) Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md To view the failed test rate over time, use this command. It requires the `GITHUB_TOKEN` environment variable and uses the `failed-rate` argument with npx github-actions-ctrf. Artifact upload is also required. ```yaml - name: Publish CTRF Flaky Test Summary Results run: npx github-actions-ctrf failed-rate path-to-your-ctrf-report.json if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Enable Flaky Rate Report with History Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Configure the GitHub Test Reporter to generate a flaky rate report, including historical data for up to 100 previous runs. This provides a comprehensive view of test stability over time. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' flaky-rate-report: true previous-results-max: 100 if: always() ``` -------------------------------- ### Publish CTRF Test List Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command generates a simple list of all tests. It uses the `test-list` argument with npx github-actions-ctrf. ```yaml - name: Publish CTRF test list run: npx github-actions-ctrf test-list path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Publish CTRF Custom Summary Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Generate a custom summary using a handlebars template by specifying the template file path after the report path. ```yaml - name: Publish CTRF Custom summary run: npx github-actions-ctrf custom path-to-your-ctrf-report.json path-to-your-handlebars-template.hbs if: always() ``` -------------------------------- ### Generate Custom Summary with Handlebars Template Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Generates a GitHub Actions summary or PR comment using a custom Handlebars template. This allows for dynamic and customizable output based on your CTRF report and GitHub properties. ```yaml name: Publish CTRF Custom summary run: npx github-actions-ctrf custom path-to-your-ctrf-report.json path-to-your-handlebars-template.hbs if: always() ``` -------------------------------- ### Enable Test List Report Generation Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `test-list-report` input to true in your workflow configuration to generate a detailed, line-by-line breakdown of all executed test cases. This is useful for quickly identifying test statuses and progress. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' test-list-report: true if: always() ``` -------------------------------- ### Enable Pull Request Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `pull-request-report` input to `true` in your workflow to generate a summary of test results directly in PR comments. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' pull-request-report: true if: always() ``` -------------------------------- ### Customize Report Order in Job Summary or PR Comments Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `report-order` parameter with a comma-separated list of report types to control the display order of reports. If not provided, a default order is used. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' summary-report: true failed-report: true flaky-report: true insights-report: true test-report: true # Order reports with the most important information first report-order: 'summary-report,failed-report,flaky-report,insights-report,test-report' if: always() ``` -------------------------------- ### Enable Previous Results Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `previous-results-report` input to true to include historical data in your reports. This helps in tracking trends over time. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' previous-results-report: true if: always() ``` -------------------------------- ### Publish CTRF Test Summary Table Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Use this command in your workflow YAML to generate and publish a test summary table. It appends the `summary` argument to the npx github-actions-ctrf command. ```yaml - name: Publish CTRF Test Summary Results run: npx github-actions-ctrf summary path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Create or Update Comment with Tag Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Appends a new report to an existing tagged comment or creates a new one if none exists. Use this to keep a single comment updated with the latest report. ```bash npx github-actions-ctrf pull-request path-to-your-ctrf-report.json --update-comment --comment-tag "${{ github.workflow }} ${{ github.job }}" ``` -------------------------------- ### Use failed-detailed Community Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/community-reports/failed-detailed/README.md Reference the 'failed-detailed' report by its name in your workflow configuration and add it to your workflow file using the 'uses' directive. Ensure the report path is correctly specified. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' community-report: true community-report-name: failed-detailed if: always() ``` -------------------------------- ### Publish CTRF Suite Folded Summary Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command generates a summary of tests grouped by suite with the tests folded. It uses the `suite-folded` argument with npx github-actions-ctrf. By default, it groups by filePath, but can use the suite property with `--useSuite`. ```yaml - name: Publish CTRF Suite Folded Summary run: npx github-actions-ctrf suite-folded path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Enable Skipped Report in GitHub Actions Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md To enable the skipped report, set the `skipped-report` input to `true` in your workflow configuration. Ensure the `report-path` is correctly specified. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' skipped-report: true if: always() ``` -------------------------------- ### Publish Community Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `community-report: true` and `community-report-name` inputs to publish a report from the community reports collection. This allows users to share custom reports for specific use cases. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' community-report: true community-report-name: summary-short if: always() ``` -------------------------------- ### Configure JUnit to CTRF Integration for Test Reports Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/integrations.md This configuration converts JUnit test results to CTRF format. Specify the output path and tool name for the conversion. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './target/surefire-reports/*.xml' integrations-config: | { "junit-to-ctrf": { "enabled": true, "action": "convert", "options": { "output": "./ctrf-reports/ctrf-report.json", "toolname": "junit-to-ctrf", "useSuiteName": false, "env": { "appName": "my-app" } } } } if: always() ``` -------------------------------- ### Publish Test Report to Pull Request Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `pull-request-report: true` input to add a built-in pull request comment report. Requires a GITHUB_TOKEN with pull request write permission. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' pull-request-report: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} if: always() ``` -------------------------------- ### Publish Test Report with Cobra Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/community-reports/cobra-report/README.md Add the cobra-report template to your workflow file to publish test results. Ensure report JSON files are correctly generated and placed in the specified path. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' community-report: true community-report-name: cobra-report if: always() ``` -------------------------------- ### Publish CTRF Failed Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Use this command to generate a table detailing failed tests. It employs the `failed` argument with npx github-actions-ctrf. ```yaml - name: Publish CTRF Failed Test Summary Results run: npx github-actions-ctrf failed path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Publish CTRF Detailed Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Add this to your workflow YAML to generate a detailed table of test results. It utilizes the `tests` command with npx github-actions-ctrf. ```yaml - name: Publish CTRF Detailed Test Summary Results run: npx github-actions-ctrf tests path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Create Status Check for Workflow Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `status-check: true` input to create a status check for the workflow. You can specify a custom name using `status-check-name`. Requires a GITHUB_TOKEN with status check write permission. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' status-check: true status-check-name: 'GitHub Test Reporter Results' if: always() ``` -------------------------------- ### Publish Test Report with Pull Request Comment Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Generates a pull request comment with a summary of test results. Requires GitHub Token with pull request write permission. ```yaml name: Publish Test Report with Pull Request Comment uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' pull-request-report: true if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Publish CTRF Skipped Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Use this command to display tests that were skipped or pending. It utilizes the `skipped` argument with npx github-actions-ctrf. ```yaml - name: Publish CTRF Skipped Test Summary Results run: npx github-actions-ctrf skipped path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Publish CTRF Historical Results Table Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Use the `historical` command to display results from previous tests in a table format. Ensure artifacts are uploaded for this to work. ```yaml - name: Publish CTRF Historical results table run: npx github-actions-ctrf historical path-to-your-ctrf-report.json if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Enable Tests Changed Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `tests-changed-report` input to true to display tests that have been added or removed compared to a baseline or previous run. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' tests-changed-report: true if: always() ``` -------------------------------- ### Publish CTRF Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Add this command to your workflow YAML to publish general CTRF test summary results. It runs the npx github-actions-ctrf command with the default report generation. ```yaml - name: Publish CTRF Test Summary Results run: npx github-actions-ctrf path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Enable Failed Folded Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Configure the GitHub Test Reporter action to use the failed-folded-report input to enable a compact view of test failures with collapsible sections for details and traces. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' failed-folded-report: true if: always() ``` -------------------------------- ### Upload Test Results as Artifact Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Store CTRF reports as artifacts to ensure they are available for future runs, which is required for some views like historical data. ```yaml - name: Upload test results uses: actions/upload-artifact@v4 with: name: ctrf-report path: path-to-your-ctrf-report.json ``` -------------------------------- ### Publish CTRF Pull Request Comment Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Post test results as a comment on the pull request using the `pull-request` command. Requires GITHUB_TOKEN with write permissions. ```yaml - name: Publish CTRF pull request comment run: npx github-actions-ctrf pull-request path-to-your-ctrf-report.json if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Add Any Report to Pull Request Comment Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md Use the `pull-request: true` input to add any report, including custom ones, to a pull request comment. Requires a GITHUB_TOKEN with pull request write permission. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' flaky-rate-report: true pull-request: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} if: always() ``` -------------------------------- ### Publish CTRF AI Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command generates an AI summary table for CTRF test results. It uses the `ai` argument with the npx github-actions-ctrf command. ```yaml - name: Publish CTRF AI Test Summary Results run: npx github-actions-ctrf ai path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Annotate Failed Tests Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Add annotations for failed tests directly in the GitHub Actions run using the `annotate` command. ```yaml - name: Annotate failed tests run: npx github-actions-ctrf annotate path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Helper Function: getEmoji Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md This code snippet demonstrates the usage of the `getEmoji` helper function, which returns the correct emoji based on test status. It's used internally for reporting. ```typescript getEmoji returns the correct emoji for "passed" getEmoji returns the correct emoji for "failed" getEmoji returns the correct emoji for "skipped" getEmoji returns the correct emoji for "pending" getEmoji returns the correct emoji for "other" getEmoji returns the correct emoji for "build" getEmoji returns the correct emoji for "duration" getEmoji returns the correct emoji for "flaky" getEmoji returns the correct emoji for "tests" getEmoji returns the correct emoji for "result" ``` -------------------------------- ### Reference Cobra Report Template Source: https://github.com/ctrf-io/github-test-reporter/blob/main/community-reports/cobra-report/README.md Reference the cobra-report template by its name in your workflow configuration. ```yaml cobra-report ``` -------------------------------- ### Publish CTRF Community Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Utilize community-built report templates by providing the report template name after the CTRF report JSON file path. ```yaml - name: Publish CTRF Community Report run: npx github-actions-ctrf community path-to-your-ctrf-report.json report-template-name if: always() ``` -------------------------------- ### Publish CTRF Flaky Test Summary Results Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md This command identifies and displays tests that were flaky in the current run. It uses the `flaky` argument with npx github-actions-ctrf. ```yaml - name: Publish CTRF Flaky Test Summary Results run: npx github-actions-ctrf flaky path-to-your-ctrf-report.json if: always() ``` -------------------------------- ### Enable Slowest Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `slowest-report` input to true to generate a detailed report of the slowest tests. This report includes test name, runs, failures, and average duration to help identify performance bottlenecks. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' slowest-report: true if: always() ``` -------------------------------- ### Merge CTRF Reports Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Merge multiple CTRF JSON report files into a single file using the `ctrf merge` command. Replace `` with the path to the reports. ```sh npx ctrf merge ``` -------------------------------- ### Enable Flaky Rate Report Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `flaky-rate-report` input to true to generate a report highlighting flaky tests. This report uses historical data to identify tests that require retries. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' flaky-rate-report: true if: always() ``` -------------------------------- ### Write CTRF Report to File Source: https://github.com/ctrf-io/github-test-reporter/blob/main/README.md This configuration writes the processed CTRF report to a specified file. Use this for reports larger than 1MB or for further processing. ```yaml - name: Write CTRF to File uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' write-ctrf-to-file: './ctrf/ctrf-report.json' if: always() ``` -------------------------------- ### Post Pull Request Comment with Custom View Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/run-on-npx.md Post a pull request comment with a specific view, such as `flaky-rate`, by adding the `--pull-request` argument. This argument is compatible with all views. ```yaml - name: Post PR Comment run: npx github-actions-ctrf flaky-rate ctrf-report.json --pull-request if: always() env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` -------------------------------- ### Enable Failed Report Generation Source: https://github.com/ctrf-io/github-test-reporter/blob/main/docs/report-showcase.md Set the `failed-report` input to true in your workflow configuration to generate a report focusing exclusively on failed test cases. This helps developers quickly diagnose and address issues by providing failure messages. ```yaml - name: Publish Test Report uses: ctrf-io/github-test-reporter@v1 with: report-path: './ctrf/*.json' failed-report: true if: always() ```