Schemathesis

https://schemathesis.readthedocs.io/en/stable/
Admin
Schemathesis is a property-based testing tool that automatically generates tests from OpenAPI and...
Tokens:130,622
Snippets:1,402
Trust Score:9.6
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Data initialization with setup() and before_call()

Source: https://schemathesis.readthedocs.io/en/stable/guides/stateful-testing

Create realistic test data at the start of each scenario. The setup() method creates a test user and stores the user_id, while before_call() injects the user_id into operations that need it.

```python
class APIWorkflow(schema.as_state_machine()):
    def setup(self):
        # Create a test user for this scenario
        case = schema["/users"]["POST"].Case(body={"username": "test_user", "email": "test@example.com"})
        response = case.call()
        self.user_id = response.json()["id"]

    def before_call(self, case):
        # Use the created user in operations that need user_id
        if "user_id" in case.path_parameters:
            case.path_parameters["user_id"] = self.user_id
```

--------------------------------

### Verify setup

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/pytest

Commands to verify that the required tools are installed and accessible.

```bash
git --version
docker compose version
python --version
pytest --version
schemathesis --version
```

--------------------------------

### Verify setup: git, docker compose, uv, curl

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/cli

Commands to verify that Git, Docker Compose, uv, and optionally curl are installed. Run these before proceeding with the tutorial.

```bash
git --version
docker compose version
uv --version
curl --version  # optional

```

--------------------------------

### Setup: Install TraceCov, enable hooks, and run Schemathesis

Source: https://schemathesis.readthedocs.io/en/stable/guides/coverage

Install TraceCov, create a hooks.py file that installs the Schemathesis integration, then run Schemathesis with the SCHEMATHESIS_HOOKS environment variable set to the hooks file. The command generates an HTML report at ./schema-coverage.html after tests complete.

```text
uv pip install tracecov
```

```python
# hooks.py
import tracecov

tracecov.schemathesis.install()
```

```text
SCHEMATHESIS_HOOKS=hooks schemathesis run https://api.example.com/openapi.json
```

--------------------------------

### Examples Phase: Schema with examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/data-generation

Uses the `example` and `examples` keywords from the schema, filling missing parts with defaults or generated data. The example shows a query parameter `limit` with examples [10, 50, 100], producing three test cases.

```yaml
# Schema
parameters:
  - name: limit
    in: query
    schema:
      type: integer
      examples: [10, 50, 100]

# Produces: 3 test cases with limit=10, limit=50, limit=100

```

--------------------------------

### Property-level examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Shows property-level examples for multiple properties in an OpenAPI schema.

```yaml
# Property-level example
properties:
  name:
    type: string
    example: "John Doe"
  age:
    type: integer
    example: 30

```

--------------------------------

### Install dependencies

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/pytest

Commands to create a virtual environment and install pytest and Schemathesis using uv.

```bash
uv venv -p 3.13
source .venv/bin/activate.fish 
uv pip install -U pytest schemathesis
```

--------------------------------

### Database setup with cleanup in Schemathesis tests

Source: https://schemathesis.readthedocs.io/en/stable/guides/hypothesis-strategies

Use @schema.given() with a fixed dictionary strategy to create user data for each Hypothesis example. The test creates a user in the database, uses it in API tests, and cleans up in a finally block to avoid test pollution.

```python
@schema.given(
    user_data=st.fixed_dictionaries(
        {
            "name": st.text(min_size=1, max_size=50),
            "email": st.emails(),
            "role": st.sampled_from(["user", "admin"]),
        }
    )
)
@schema.parametrize()
def test_api_with_db_setup(db, case, user_data):
    # Create user in database for each Hypothesis example
    user_id = db.create_user(user_data)
    try:
        # Use the created user in API tests
        if "user_id" in case.path_parameters:
            case.path_parameters["user_id"] = user_id
        case.call_and_validate()
    finally:
        # Important: cleanup after each example
        db.cleanup_user(user_id)
```

--------------------------------

### Configure example database with generation.database

Source: https://schemathesis.readthedocs.io/en/stable/reference/configuration

Configures storage for examples discovered by Schemathesis. Options include "none", ":memory:" for temporary in-memory storage, or a file path for persistent storage. Default is .hypothesis/examples.

```toml
[generation]
database = ":memory:"

```

```toml
[generation]
database = "./.schemathesis/examples/"

```

--------------------------------

### Single example with 'example' keyword

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Defines a single example value for a property using the 'example' keyword in OpenAPI 3.0+.

```yaml
# Single example using the 'example' keyword
schema:
  type: object
  properties:
    name:
      type: string
      example: "John Doe"

```

--------------------------------

### Location header learning example

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Example showing a POST /users response with a Location header and the learned follow-up operations.

```http
POST /users → 201 Created
Location: /users/123

# Learns: GET /users/123, PUT /users/123, DELETE /users/123

```

--------------------------------

### Authentication flow with setup() and before_call()

Source: https://schemathesis.readthedocs.io/en/stable/guides/stateful-testing

Handle login and token management for protected endpoints. The setup() method logs in and stores the auth token, while before_call() adds the auth header to every request.

```python
import requests


class APIWorkflow(schema.as_state_machine()):
    def setup(self):
        # Login and get auth token
        response = requests.post(
            "http://localhost:8000/auth/login", json={"username": "test_user", "password": "test_password"}
        )
        token = response.json()["access_token"]
        self.auth_headers = {"Authorization": f"Bearer {token}"}

    def before_call(self, case):
        # Add auth to every request
        case.headers = {**case.headers, **self.auth_headers}
```

--------------------------------

### Complete Example with pytest Fixture

Source: https://schemathesis.readthedocs.io/en/stable/guides/schema-conformance

A complete test workflow using a pytest fixture to load the schema and validate both create and get responses.

```python
import pytest
import schemathesis


@pytest.fixture(scope="session")
def api_schema():
    return schemathesis.openapi.from_url("http://api.example.com/openapi.json")


def test_user_workflow(api_schema):
    create_response = requests.post("http://api.example.com/users", json={"name": "Test"})
    user_id = create_response.json()["id"]

    api_schema["/users"]["POST"].validate_response(create_response)

    get_response = requests.get(f"http://api.example.com/users/{user_id}")
    api_schema["/users/{id}"]["GET"].validate_response(get_response)
```

--------------------------------

### Async test functions with pytest-asyncio and pytest-trio

Source: https://schemathesis.readthedocs.io/en/stable/explanations/pytest

Shows how to write asynchronous test functions with Schemathesis. Requires installing pytest-asyncio or pytest-trio; no additional configuration is needed. The example includes both an asyncio-marked test and a trio-marked test.

```python
import pytest
import schemathesis

schema = schemathesis.openapi.from_url("http://127.0.0.1:8080/openapi.json")


@pytest.mark.asyncio
@schema.parametrize()
async def test_api_async(case, client):
    response = await client.request(case.method, case.formatted_path, headers=case.headers)
    schema[case.path][case.method].validate_response(response)


# Or with trio
@pytest.mark.trio
@schema.parametrize()
async def test_api_trio(case, client): ...
```

--------------------------------

### Multiple examples strategy with round-robin

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Schema with multiple examples for 'age' and the round-robin generated test cases using each age value.

```yaml
# Schema
properties:
  name:
    type: string
    example: "John"
  age:
    type: integer
    examples: [25, 30, 35]

```

```json
{"name": "John", "age": 25, ...}
{"name": "John", "age": 30, ...}
{"name": "John", "age": 35, ...}

```

--------------------------------

### Extend TestCase with setUp and tearDown for one-time setup

Source: https://schemathesis.readthedocs.io/en/stable/guides/stateful-testing

Use this snippet to define per-test-run setup and cleanup by extending APIWorkflow.TestCase. The setUp method runs once per test execution, and tearDown cleans up resources after the test, regardless of how many scenarios Hypothesis generates.

```python
class TestAPI(APIWorkflow.TestCase):
    def setUp(self):
        """Create database, start services - runs once per test execution."""

    def tearDown(self):
        """Cleanup resources - runs once per test execution."""

```

--------------------------------

### OpenAPI 2.0 multiple examples with x-examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Defines multiple examples in OpenAPI 2.0 using the 'x-examples' extension.

```yaml
# OpenAPI 2.0 with multiple examples
definitions:
  User:
    type: object
    properties:
      name:
        type: string
    x-examples:
      - name: "John Doe"
      - name: "Jane Smith"

```

--------------------------------

### Step 4: Generate test scenarios - random workflows

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Lists example random workflows that Schemathesis runs, such as create, get, update, delete sequences.

```text
Create user -> Get user -> Update user -> Delete user
Create user -> Create order for that user -> Get order
Create user -> Delete user -> Get user
```

--------------------------------

### Manual OpenAPI Links example

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Example OpenAPI schema with a link definition to explicitly connect POST /users to getUser operation.

```yaml
paths:
  /users:
    post:
      responses:
        '201':
          links:
            GetUser:
              operationId: getUser
              parameters:
                userId: '$response.body#/id'

```

--------------------------------

### schemathesis run --phases=examples,coverage

Source: https://schemathesis.readthedocs.io/en/stable/migration

The new --phases option controls which test phases run. This example runs only the examples and coverage phases.

```bash
schemathesis run --phases=examples,coverage <SCHEMA_URL>

```

--------------------------------

### Clone and start booking API

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/cli

Clones the Schemathesis repository and starts the booking API with Docker Compose. The API will be available at http://localhost:8080/docs.

```bash
git clone https://github.com/schemathesis/schemathesis.git
cd schemathesis/examples/booking
docker compose up -d --build

```

--------------------------------

### Enable fill-missing for examples phase

Source: https://schemathesis.readthedocs.io/en/stable/reference/configuration

Enables example generation for operations without explicit schema examples, using schema defaults and captured response data.

```toml
[phases.examples]
fill-missing = true

```

--------------------------------

### Schema with explicit examples for path parameter

Source: https://schemathesis.readthedocs.io/en/stable/guides/coverage

Adds explicit examples to a path parameter schema to cover boundary values. The 'examples' map uses keys as labels and 'value' for the actual value.

```yaml
/users/{id}:
  get:
    parameters:
      - name: id
        in: path
        schema:
          type: integer
          minimum: 1
        examples:
          existing:
            value: 42
          boundary:
            value: 1

```

--------------------------------

### Example output of st replay with fixed and failed cases

Source: https://schemathesis.readthedocs.io/en/stable/guides/crash-reproduction

Example output from `st replay` showing one fixed and one failed case, with failure details and reproduction commands.

```text
Replaying 2 cases from .schemathesis/crashes/

  + FIXED  GET /users
  x FAILED POST /orders

=================================== FAILURES ===================================
_________________________________ POST /orders _________________________________
1. Test Case ID: Xt9Kp2

- Server error

- Undocumented HTTP status code

    Received: 500
    Documented: 200

[500] Internal Server Error:

    `{"error": "unhandled exception"}`

Reproduce with:

    curl -X POST http://127.0.0.1/orders

    st replay Xt9Kp2

========================== 1 fixed, 1 failed in 0.14s ==========================
```

--------------------------------

### Install Schemathesis with Allure support

Source: https://schemathesis.readthedocs.io/en/stable/guides/allure

Adds the optional Allure extra to Schemathesis, installing allure-python-commons. Requires the Allure CLI separately to generate and view HTML reports.

```bash
uv add 'schemathesis[allure]'

```

--------------------------------

### Schema example and generated test case

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Schema with an example for 'name' and the generated test case where 'John' comes from the example and other values are minimal valid values.

```yaml
# Schema
schema:
  type: object
  properties:
    name:
      type: string
      example: "John"
    age:
      type: integer
    address:
      type: string

```

```json
{"name": "John", "age": 42, "address": "abc"}

```

--------------------------------

### Operation-level example

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Defines an example at the operation level for a request body in OpenAPI 3.0+.

```yaml
# Operation-level example
requestBody:
  content:
    application/json:
      schema:
        $ref: '#/components/schemas/User'
      example:
        name: "John Doe"
        age: 30

```

--------------------------------

### GraphQL schema example for automatic schema analysis

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Example GraphQL schema with Book type, addBook mutation, deleteBook mutation, and book query to illustrate how Schemathesis builds chains automatically.

```graphql
type Book { id: ID! title: String! }

type Mutation {
    addBook(title: String!): Book!     # Producer of Book
    deleteBook(id: ID!): Boolean       # Cleanup of Book
}

type Query {
    book(id: ID!): Book                # Consumer of Book
}

```

--------------------------------

### Shrinking example: before and after

Source: https://schemathesis.readthedocs.io/en/stable/explanations/data-generation

Illustrates how Schemathesis shrinks a failing test case to the minimal example that reproduces the failure. Shrinking is enabled by default; disable with --no-shrink for faster test runs.

```json
{"name": "Very long user name", "age": 42, "metadata": {...}}
```

```json
{"name": "a", "age": 42}  # Only data that triggers a failure
```

--------------------------------

### content_type_conformance missing header output

Source: https://schemathesis.readthedocs.io/en/stable/reference/checks

Example output when the Content-Type header is missing.

```text
- Missing Content-Type header

The following media types are documented in the schema:
- application/json
```

--------------------------------

### OpenAPI schema example for automatic schema analysis

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Example OpenAPI schema showing POST /users and GET /users/{userId} to illustrate how Schemathesis detects relationships and builds sequences.

```yaml
paths:
  /users:
    post:
      responses:
        '201':
          content:
            application/json:
              schema:
                properties:
                  id: {type: string}
                  email: {type: string}

  /users/{userId}:
    get:
      parameters:
        - name: userId
          in: path

```

--------------------------------

### Set generation database to file path

Source: https://schemathesis.readthedocs.io/en/stable/reference/cli

Sets the generation database to a file path for persistent storage of examples.

```bash
$ st run openapi.yaml --generation-database ./schemathesis_examples.db

```

--------------------------------

### Use wildcards for any type or field

Source: https://schemathesis.readthedocs.io/en/stable/guides/fuzz-dictionary

Use '*' for any type or any field. The example binds '*.*.title' to cover every 'title' argument in the schema.

```toml
[parameters]
"*.*.title" = { dictionary = "titles" }

```

--------------------------------

### APIStateMachine.setup()

Source: https://schemathesis.readthedocs.io/en/stable/reference/python

Called once at the beginning of each test scenario.

```APIDOC
## setup()

### Description
Called once at the beginning of each test scenario.

### Method
setup()
```

--------------------------------

### Stateful Phase: Schema with links

Source: https://schemathesis.readthedocs.io/en/stable/explanations/data-generation

Runs when OpenAPI schemas define links between operations, creating sequences where response data feeds into subsequent requests. The example shows a link from POST /users to GET /users/{id}, producing a sequence that extracts the ID and uses it in the GET request.

```yaml
# Schema with links: POST /users → GET /users/{id}

# Produces: POST /users, extract ID, then GET /users/{extracted_id}

```

--------------------------------

### Include/exclude by expression: --include-by / --exclude-by

Source: https://schemathesis.readthedocs.io/en/stable/reference/cli

Include or exclude operations using a custom expression. The expression must start with a JSON Pointer. Example includes operations where the first tag equals 'user'.

```bash
$ st run openapi.yaml --include-by "/tags/0 == 'user'"

```

--------------------------------

### Step 2: Match parameters - link parameters to resources

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Shows how parameters are linked to resources, e.g., userId from User resource and orderId from Order resource.

```text
GET /users/{userId} → Needs "userId" from User resource
GET /orders/{orderId} → Needs "orderId" from Order resource
```

--------------------------------

### Run example-based tests with --phases=examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Runs example-based tests only using the --phases=examples option, skipping other testing phases like coverage, fuzzing, and stateful testing.

```bash
$ st run --phases=examples https://example.schemathesis.io/openapi.json

```

--------------------------------

### Set time budget: --max-time

Source: https://schemathesis.readthedocs.io/en/stable/reference/cli

Sets a time budget in seconds for the whole run. Each phase gets a share, then fuzzing and stateful testing repeat until the budget is spent. Example allows 600 seconds.

```bash
$ st run openapi.yaml --max-time 600

```

--------------------------------

### Targeting Specific Operations with Hooks

Source: https://schemathesis.readthedocs.io/en/stable/guides/extending

Use @schemathesis.hook.apply_to to restrict hook execution to specific paths or operations. The first example targets user endpoints while skipping POST requests, and the second targets a specific GET operation.

```python
# Only apply to user endpoints, skip POST requests
@schemathesis.hook.apply_to(path_regex=r"/users/").skip_for(method="POST")
def map_headers(ctx, headers):
    headers = headers or {}
    headers["X-User-Context"] = "test-user"
    return headers


# Only apply to a specific operation
@schemathesis.hook.apply_to(name="GET /orders/{order_id}")
def map_path_parameters(ctx, path_parameters):
    path_parameters = path_parameters or {}
    path_parameters["order_id"] = "order_12345"
    return path_parameters
```

--------------------------------

### Select test phases with --phases

Source: https://schemathesis.readthedocs.io/en/stable/reference/cli

Specifies which test phases to run. Possible values: examples, coverage, fuzzing, stateful. Default is all phases enabled.

```bash
$ st run openapi.yaml --phases examples,fuzzing

```

--------------------------------

### Run Schemathesis with recommended settings

Source: https://schemathesis.readthedocs.io/en/stable/guides/config-optimization

Recommended configuration for maximum bug detection and API coverage. Use when preparing for production releases, conducting security assessments, or when time allows for extensive testing. For fast development feedback, stick with defaults.

```bash
schemathesis run <schema_url> \
  --max-examples 1000 \
  --continue-on-failure

```

--------------------------------

### Multiple examples with externalValue

Source: https://schemathesis.readthedocs.io/en/stable/explanations/examples

Uses the 'examples' keyword with an externalValue reference. Schemathesis loads and caches external examples during testing.

```yaml
content:
  application/json:
    schema:
      $ref: '#/components/schemas/User'
    examples:
      user:
        summary: "A typical user"
        # Schemathesis will load and cache external examples during testing.
        externalValue: 'http://example.com/examples/user.json'

```

--------------------------------

### FastAPI (ASGI) setup with schemathesis.openapi.from_asgi

Source: https://schemathesis.readthedocs.io/en/stable/guides/python-apps

Loads the OpenAPI schema directly from a FastAPI application using schemathesis.openapi.from_asgi and parametrizes a test that calls and validates each case.

```python
from fastapi import FastAPI
import schemathesis

app = FastAPI()


@app.get("/users")
async def get_users():
    return [{"id": 1, "name": "Alice"}]


# Load schema directly from the app
schema = schemathesis.openapi.from_asgi("/openapi.json", app)


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

```

--------------------------------

### Example output from Schemathesis run

Source: https://schemathesis.readthedocs.io/en/stable/quick-start

Shows the reproduction-ready output for a failing case, including the server error and a curl command to reproduce the failure.

```text
_____________________ POST /improper-input-type-handling _____________________

- Server error

[500] Internal Server Error:

    `{"success":false,"error":"invalid literal for int() with base 10: '\n'"}`

Reproduce with:

    curl -X POST -H 'Content-Type: application/json' \
      -d '{"number": "\n\udbcd." }' \
      https://example.schemathesis.io/improper-input-type-handling

```

--------------------------------

### Example output from running the extension

Source: https://schemathesis.readthedocs.io/en/stable/guides/extending-cli

Shows the expected summary output when running the custom extension with the `--counter-verbose` flag. It lists test execution progress, a counter summary, and confirmation that results were written to `results.txt`.

```text
Starting test #1
✓ Test #1 passed
Starting test #2
✗ Test #2 failed

Counter Summary:
  Total events: 125
  Test cases: 2
  Failures: 1
  Errors: 0
  Results written to: results.txt

```

--------------------------------

### GraphQL Schema usage example

Source: https://schemathesis.readthedocs.io/en/stable/reference/python

Example of how custom scalars are defined and used within a GraphQL schema.

```graphql
scalar Email
scalar PositiveInt

type Query {
  getUser(email: Email!, rating: PositiveInt!): User
}
```

--------------------------------

### Integration with Other Frameworks

Source: https://schemathesis.readthedocs.io/en/stable/guides/hypothesis-strategies

Shows how to use Schemathesis strategies with Hypothesis's @given decorator in a unittest TestCase. Demonstrates creating a pet and then retrieving it, modifying path parameters as needed.

```python
from unittest import TestCase
from hypothesis import given

# Create strategies for specific operations
create_pet_strategy = schema["/pet"]["POST"].as_strategy()
get_pet_strategy = schema["/pet/{id}"]["GET"].as_strategy()


class TestAPI(TestCase):
    @given(case=create_pet_strategy)
    def test_create_pet(self, case):
        response = case.call_and_validate()
        self.assertIn("id", response.json())

    @given(create_case=create_pet_strategy, get_case=get_pet_strategy)
    def test_create_then_get(self, create_case, get_case):
        # Create pet
        create_response = create_case.call_and_validate()
        pet_id = create_response.json()["id"]

        # Get the same pet
        get_case.path_parameters = {"id": pet_id}
        get_response = get_case.call_and_validate()

        self.assertEqual(get_response.json()["id"], pet_id)

```

--------------------------------

### Create schemathesis.toml with generation, timeout, and auth settings

Source: https://schemathesis.readthedocs.io/en/stable/configuration

Create a `schemathesis.toml` file in your project directory to configure test volume, request timeout, and basic authentication. Environment variables like `${USERNAME}` are read from your environment when Schemathesis loads the config.

```toml
# Run more tests for better coverage
generation.max-examples = 500

# Increase timeout for slow APIs
request-timeout = 10.0  # seconds

# Basic authentication
[auth]
basic = { username = "${USERNAME}", password = "${PASSWORD}" }

```

--------------------------------

### Example output of a flaky replay

Source: https://schemathesis.readthedocs.io/en/stable/guides/crash-reproduction

Example output showing a flaky case where a check passed on some replays and failed on others.

```text
  ? FLAKY  GET /report

    ? not_a_server_error  passed 2 of 3 replays
```

--------------------------------

### Run with baseline and example output

Source: https://schemathesis.readthedocs.io/en/stable/guides/baseline

Plain run that loads the baseline file, reports the recorded failures, and exits 0; anything else exits 1. The output shows the baseline header and the known failures count.

```bash
schemathesis run http://localhost:8080/openapi.json

```

```text
Baseline:    schemathesis-baseline.json / 47 entries

...

Baseline:
  Known failures: 47

=========================== No issues found in 12.03s ==========================

```

--------------------------------

### Flask (WSGI) setup with schemathesis.openapi.from_wsgi

Source: https://schemathesis.readthedocs.io/en/stable/guides/python-apps

Loads the OpenAPI schema from a Flask application using schemathesis.openapi.from_wsgi and parametrizes a test that calls and validates each case.

```python
from flask import Flask, jsonify
import schemathesis

app = Flask(__name__)


@app.route("/users")
def get_users():
    return jsonify([{"id": 1, "name": "Alice"}])


@app.route("/openapi.json")
def openapi_spec():
    return {...}  # Your OpenAPI schema


schema = schemathesis.openapi.from_wsgi("/openapi.json", app)


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

```

--------------------------------

### OpenAPI-Aware Authentication with Environment Variables

Source: https://schemathesis.readthedocs.io/en/stable/guides/auth

Set environment variables for API_KEY and TOKEN and run schemathesis with the OpenAPI-aware config.

```bash
export API_KEY="your-api-key"
export TOKEN="your-token"
schemathesis run http://localhost:8000/openapi.json
```

--------------------------------

### Example output of a partially fixed failure

Source: https://schemathesis.readthedocs.io/en/stable/guides/crash-reproduction

Example output showing a case that still fails after some checks were fixed; only the fixed checks' files are removed.

```text
  x FAILED POST /orders

    + not_a_server_error
    + status_code_conformance
    x response_schema_conformance
```

--------------------------------

### Separate schema examples and custom strategies with phases

Source: https://schemathesis.readthedocs.io/en/stable/guides/hypothesis-strategies

When schema examples are present, @schema.given() cannot be used on the same test function due to parameter mismatch. Create separate test functions: one for schema examples without @schema.given() and one for property-based testing with custom strategies, using Phase.explicit and Phase.generate respectively.

```python
from hypothesis import Phase, settings


# 1. One for schema examples (without @schema.given()):
@schema.parametrize()
@settings(phases=[Phase.explicit])
def test_user_endpoints_with_examples(case):
    case.call_and_validate()


# 2. One for property-based testing with your custom strategies:
@schema.given(user_id=st.sampled_from(existing_user_ids))
@schema.parametrize()
@settings(phases=[Phase.generate])
def test_user_endpoints_with_custom_data(case, user_id):
    if "user_id" in case.path_parameters:
        case.path_parameters["user_id"] = user_id
    case.call_and_validate()
```

--------------------------------

### Load custom configuration file

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/pytest

Specify a custom configuration file path using schemathesis.Config.from_path when initializing the schema.

```python
import schemathesis

config = schemathesis.Config.from_path("path-to-my/config.toml")
schema = schemathesis.openapi.from_url("http://127.0.0.1:8080/openapi.json", config=config)
```

--------------------------------

### Generate Allure report with --report-allure-path

Source: https://schemathesis.readthedocs.io/en/stable/faq

Generate Allure-compatible results with Schemathesis and create an Allure report. Requires installing schemathesis[allure] and the Allure CLI.

```bash
schemathesis run your_schema.yaml --report-allure-path allure-results
allure generate allure-results -o allure-report
allure open allure-report
```

--------------------------------

### Negative Testing: Invalid string examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/data-generation

Generates data that should be rejected by the API, deliberately invalid according to the schema. The example schema `{"type": "string", "minLength": 3}` produces negative examples like 42, [], "", "ab".

```json
# Schema: {"type": "string", "minLength": 3}
# Negative examples: 42, [], "", "ab"

```

--------------------------------

### Step 1: Find resources - identify what the API produces

Source: https://schemathesis.readthedocs.io/en/stable/explanations/stateful

Shows how Schemathesis identifies resources produced by operations, such as creating a User or Order resource.

```text
POST /users → Creates "User" resource
POST /orders → Creates "Order" resource
```

--------------------------------

### Example pytest output for Schemathesis tests

Source: https://schemathesis.readthedocs.io/en/stable/explanations/pytest

Example pytest output showing one line per API operation, with each line indicating pass or fail status.

```text
test.py::test_api[GET /users] PASSED    # One line per API operation
test.py::test_api[POST /users] FAILED   # Each runs many Hypothesis examples
test.py::test_api[DELETE /users/{id}] PASSED

```

--------------------------------

### Run Schemathesis with uvx

Source: https://schemathesis.readthedocs.io/en/stable

Run Schemathesis against a demo OpenAPI schema to immediately find real bugs and see how to reproduce them.

```bash
uvx schemathesis run https://example.schemathesis.io/openapi.json

```

--------------------------------

### Positive Testing: Valid string examples

Source: https://schemathesis.readthedocs.io/en/stable/explanations/data-generation

Generates data that should be accepted by the API, valid according to the schema. The example schema `{"type": "string", "minLength": 3}` produces positive examples like "abc", "hello", "test123".

```json
# Schema: {"type": "string", "minLength": 3}
# Positive examples: "abc", "hello", "test123"

```

--------------------------------

### Recommended Schemathesis benchmark configuration

Source: https://schemathesis.readthedocs.io/en/stable/guides/benchmarking

Use this command to run Schemathesis for benchmarking. It sets a time budget, continues after failures, disables shrinking, and suppresses health checks to keep all operations in the measurement.

```bash
schemathesis run <schema_url> \
  --max-time 3600 \
  --continue-on-failure \
  --no-shrink \
  --suppress-health-check all

```

--------------------------------

### Baseline JSON report example

Source: https://schemathesis.readthedocs.io/en/stable/guides/baseline

Example of the JSON report generated with `--report json` for CI gating. Contains baseline counts and IDs for known, new, unobserved, and expired failures.

```json
{
  "baseline": {
    "known": 41,
    "new": 0,
    "recorded": null,
    "unobserved": 6,
    "known_ids": ["f3a91c", "..."],
    "unobserved_ids": ["9b2e04", "..."],
    "expired_ids": [],
    "pruned_ids": null
  }
}
```

--------------------------------

### Run Schemathesis with Docker (Basic Usage)

Source: https://schemathesis.readthedocs.io/en/stable/guides/docker

Runs Schemathesis tests from the official Docker image without installing Python. The image uses free-threaded Python (3.14t) and `-w auto` enables parallel execution across all available CPUs.

```bash
docker run ghcr.io/schemathesis/schemathesis:stable \
  run -w auto https://api.example.com/openapi.json

```

--------------------------------

### Create schemathesis.toml configuration file

Source: https://schemathesis.readthedocs.io/en/stable/tutorials/cli

Create this file in your project root to save core settings and reuse them across your team. The headers value uses the API_TOKEN environment variable, and the file enables continue-on-failure, disables output sanitization, sets max-examples to 500, and enables JUnit reports.

```toml
# Core settings from our previous commands
headers = { Authorization = "Bearer ${API_TOKEN}" }
continue-on-failure = true

[output.sanitization]
enabled = false

[generation] 
max-examples = 500

[reports.junit]
enabled = true

```

--------------------------------

### Static Authentication with CLI Options

Source: https://schemathesis.readthedocs.io/en/stable/guides/auth

Use CLI options like --header, --auth, or --header with an X-API-Key header to authenticate requests for simple cases. Replace the token values with your actual credentials.

```bash
# Bearer token
schemathesis run http://localhost:8000/openapi.json \
  --header "Authorization: Bearer your-token"

# Basic authentication
schemathesis run http://localhost:8000/openapi.json \
  --auth username:password

# API key
schemathesis run http://localhost:8000/openapi.json \
  --header "X-API-Key: your-api-key"
```

--------------------------------

### Run Schemathesis with a static schema file

Source: https://schemathesis.readthedocs.io/en/stable/guides/cicd

Run Schemathesis using a schema file from your repository. The --url flag specifies the API host and port to test against.

```bash
schemathesis run ./openapi.json --url http://api-host:port

```

--------------------------------

### Example JSON report output

Source: https://schemathesis.readthedocs.io/en/stable/guides/cicd

Example JSON report output showing exit code, stop reason, operation counts, test case counts, phases, failures, errors, and warnings. Used to distinguish between a run that selected zero operations and one with real findings.

```json
{
  "exit_code": 1,
  "stop_reason": "completed",
  "operations": { "total": 42, "selected": 40, "tested": 38, "errored": 1, "skipped": 1, "skip_reasons": [] },
  "test_cases": { "generated": 3800, "with_failures": 12, "unique_failures": 4, "without_checks": 0 },
  "phases": { "fuzzing": { "status": "success", "skip_reason": null } },
  "failures": [],
  "errors": [],
  "warnings": {}
}

```

--------------------------------

### Example endpoint with performance issue

Source: https://schemathesis.readthedocs.io/en/stable/guides/targeted

Example API endpoint with a hidden performance issue: inputs with many zeros cause progressively slower responses, and the endpoint fails when the count exceeds 10. This is used to illustrate how targeted testing can find problematic inputs faster than standard testing.

```python
async def performance(request):
    decoded = await request.json()
    number = str(decoded).count("0")
    if number > 0:
        # Progressively slower as zeros increase
        await asyncio.sleep(0.01 * number)
    if number > 10:
        # Fails with too many zeros
        raise ServerError()
    return {"result": "success"}

```

--------------------------------

### Extend state machine inside fixture with setup and before_call

Source: https://schemathesis.readthedocs.io/en/stable/guides/stateful-testing

Use this snippet to extend the state machine inside a pytest fixture, adding custom setup logic and modifying request headers via before_call. The fixture depends on api_schema and auth_service, and returns the extended class for use in tests.

```python
@pytest.fixture
def state_machine(api_schema, auth_service):
    class APIWorkflow(api_schema.as_state_machine()):
        def setup(self):
            # Use fixture dependencies
            self.token = auth_service.get_test_token()

        def before_call(self, case):
            case.headers["Authorization"] = f"Bearer {self.token}"

    return APIWorkflow

```
Tokens:
Raw