### Install ReplicantX CLI Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Installs the ReplicantX command-line interface tool and its dependencies. This is the primary method for setting up ReplicantX for local testing. ```bash pip install replicantx[cli] ``` -------------------------------- ### Install ReplicantX with LLM Support Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This snippet provides pip commands for installing ReplicantX with different levels of LLM provider support. It includes options for installing with all providers, specific providers, or a core installation with the test model. ```bash # Install with all LLM providers pip install replicantx[all] # Install with specific providers pip install replicantx[openai] pip install replicantx[anthropic] # Core installation (includes PydanticAI with test model) pip install replicantx ``` -------------------------------- ### Example Pull Request Description Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md A detailed example of a pull request description, outlining the summary, motivation, changes, testing performed, and any breaking changes or related issues. ```markdown ## Summary Adds parallel execution support to the CLI, allowing multiple scenarios to run concurrently. ## Motivation Running scenarios sequentially can be slow for large test suites. Parallel execution significantly reduces total execution time. ## Changes - Added `--parallel` and `--max-concurrent` CLI options - Implemented `run_scenarios_parallel()` function - Added `parallel: bool` field to ScenarioConfig - Updated documentation with usage examples ## Testing - Added unit tests for parallel execution logic - Tested with 10 concurrent scenarios - Verified proper error handling and cleanup - Confirmed no regression in sequential execution ## Breaking Changes None - all changes are backward compatible. Fixes #45 ``` -------------------------------- ### Install ReplicantX Development Dependencies Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Commands to install the necessary Python packages for developing ReplicantX. This includes editable installation of the project and its requirements. ```bash pip install -e . pip install -r requirements.txt ``` -------------------------------- ### Migration Guide: Legacy to OpenAI Payload Format Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This guide illustrates how to migrate from a legacy payload format to the more compatible OpenAI format. It shows the old configuration (legacy or omitted) and the recommended new configuration using 'openai' for payload_format. ```yaml # Old configuration (still works) replicant: payload_format: legacy # or omit entirely # New recommended configuration replicant: payload_format: openai # More compatible with modern APIs ``` -------------------------------- ### PydanticAI Model String Examples Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This snippet provides examples of PydanticAI model strings used to specify different LLM providers and models within ReplicantX configurations. It includes examples for OpenAI, Anthropic, Google, Groq, and the built-in test model. ```yaml # OpenAI models model: "openai:gpt-4o" model: "openai:gpt-4.1-mini" model: "openai:gpt-4.1-nano" # Anthropic models model: "anthropic:claude-3-5-sonnet-latest" model: "anthropic:claude-3-haiku-20240307" # Google models model: "gemini-1.5-pro" model: "gemini-1.5-flash" # Groq models model: "groq:llama-3.1-8b-instant" model: "groq:mixtral-8x7b-32768" # Test model (no API key needed) model: "test" ``` -------------------------------- ### Install and Run ReplicantX CLI Source: https://context7.com/helixtechnologies/replicantx/llms.txt Commands for installing ReplicantX with CLI support and running test scenarios from YAML files. Supports options for reporting, real-time monitoring, debugging, parallel execution, CI mode, and validation. ```bash # Install ReplicantX with CLI support pip install replicantx[cli] # Run tests with detailed Markdown report replicantx run tests/*.yaml --report report.md # Real-time conversation monitoring with live timestamps replicantx run tests/agent_test.yaml --watch # Technical debugging with HTTP requests and validation details replicantx run tests/agent_test.yaml --debug # Combined monitoring and debugging replicantx run tests/agent_test.yaml --debug --watch # Parallel execution with concurrency control replicantx run tests/*.yaml --parallel --max-concurrent 3 # CI mode (exits 1 on failure) with JSON report replicantx run tests/*.yaml --ci --report results.json # Validate YAML files without running tests replicantx validate tests/*.yaml --verbose # Show version information replicantx --version ``` -------------------------------- ### Download and Run Intelligent Evaluation Example Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This snippet demonstrates how to download a sample ReplicantX test file and run it using intelligent evaluation mode. It also provides instructions on how to switch to keyword-only mode for comparison. ```bash # Download the example test curl -O https://raw.githubusercontent.com/helixtechnologies/replicantx/main/tests/intelligent_evaluation_example.yaml # Run with intelligent evaluation replicantx run intelligent_evaluation_example.yaml --watch # Compare with keyword-only mode by changing goal_evaluation_mode to "keywords" ``` -------------------------------- ### System Prompt Example: Helpful User Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This system prompt configures the agent to act as a polite user who possesses necessary information but requires prompting to recall specific details. ```yaml system_prompt: | You are a polite user trying to achieve your goal. You have the necessary information but need prompting to remember details. ``` -------------------------------- ### Local Development Setup with .env for ReplicantX Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Demonstrates how to create a .env file for local development with ReplicantX, including API keys and target URLs. Shows how ReplicantX automatically loads these variables for running tests. ```bash # Create .env file (ReplicantX automatically loads it!) cat > .env << 'EOF' OPENAI_API_API_KEY=sk-dev-key REPLICANTX_TARGET=dev-api.example.com SUPABASE_URL=https://your-project.supabase.co SUPABASE_ANON_KEY=your-supabase-key EOF # Just run tests - no need to source .env! replicantx run tests/*.yaml # Or export manually (old way still works) export OPENAI_API_KEY=sk-dev-key replicantx run tests/*.yaml ``` -------------------------------- ### Flight Booking Example with Intelligent Evaluation (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md An example configuration for a flight booking scenario using intelligent goal evaluation. It includes detailed facts, system prompts, and specifies the LLM for both goal evaluation and general conversation. ```yaml name: "Smart Flight Booking Test" base_url: "https://api.example.com/chat" auth: provider: noop level: agent replicant: goal: "Book a round-trip business class flight to Paris" facts: name: "Sarah Johnson" email: "sarah@example.com" travel_class: "business" destination: "Paris" departure_city: "New York" travel_date: "next Friday" return_date: "following Monday" budget: "$3000" system_prompt: | You are a customer booking a flight. Provide information when asked but don't volunteer everything upfront. Be conversational and natural. initial_message: "Hi, I'd like to book a flight to Paris." max_turns: 15 # Intelligent goal evaluation goal_evaluation_mode: "intelligent" goal_evaluation_model: "openai:gpt-4o-mini" # Fast, cost-effective model # Still needed for fallback/compatibility completion_keywords: ["booked", "confirmed", "reservation number"] llm: model: "openai:gpt-4o" temperature: 0.7 max_tokens: 150 ``` -------------------------------- ### Problematic Keyword Completion Example Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This example highlights a potential issue with using simple keyword matching for conversation completion. It shows keywords that can lead to false positives if they appear in contexts other than confirming completion. ```yaml # Problematic scenario completion_keywords: ["confirmed", "booked"] # False positive examples: # ❌ "I'll let you know when your booking has been confirmed" (contains "confirmed") # ❌ "Have you booked with us before?" (contains "booked") # ❌ "Your booking confirmation is pending" (contains "booking") ``` -------------------------------- ### ReplicantX Monitoring Tips for CI/CD and Performance Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Provides examples for using ReplicantX flags for monitoring purposes. This includes watching progress during report generation, debugging in CI/CD environments with exit codes, and analyzing performance with verbose output. ```bash # Watch progress while generating a report replicantx run tests/*.yaml --watch --report detailed.md ``` ```bash # Debug mode with CI exit codes replicantx run tests/*.yaml --debug --ci ``` ```bash # Combined with verbose output replicantx run tests/*.yaml --debug --verbose --report performance.json ``` -------------------------------- ### Commit Changes with Conventional Commits Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Example of how to stage changes and commit them using the conventional commit format, which includes a type, optional scope, and a description for better commit history. ```bash git add . git commit -m "feat(agent): add intelligent goal evaluation" ``` -------------------------------- ### Conventional Commit Message Example Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md An example of a conventional commit message format, which helps in standardizing commit history and automating changelog generation. ```bash feat(cli): add parallel execution support ``` -------------------------------- ### GitHub Actions CI Configuration for ReplicantX Tests Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Illustrates how to configure GitHub Actions to run ReplicantX tests. It shows setting up environment variables from GitHub Secrets and installing ReplicantX within the workflow. ```yaml # .github/workflows/test-api.yml jobs: test: runs-on: ubuntu-latest env: # GitHub Secrets → Environment Variables OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} REPLICANTX_TARGET: ${{ secrets.API_TARGET_URL }} steps: - run: pip install replicantx[cli] - run: replicantx run tests/*.yaml --ci # ReplicantX automatically finds the variables! ``` -------------------------------- ### Universal API Configuration Example Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This configuration sets up a universal API test with an OpenAI-compatible format. It defines the API endpoint, authentication, agent level, and specific parameters for the Replicant agent including its goal, facts, system prompt, initial message, and LLM settings. ```yaml name: "Universal API Test" base_url: https://api.example.com/chat auth: provider: noop level: agent replicant: goal: "Test API with OpenAI-compatible format" facts: name: "Test User" email: "test@example.com" system_prompt: | You are a helpful user testing an API. initial_message: "Hello, I'm testing the API." max_turns: 10 completion_keywords: ["complete", "finished", "done"] fullconversation: true # Send full conversation history payload_format: openai # Use OpenAI-compatible format llm: model: "test" temperature: 0.7 max_tokens: 150 ``` -------------------------------- ### Simple Payload Format Example (JSON) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Demonstrates the JSON structure for the simple payload format, containing only a single 'message' field. This is ideal for APIs that only process the current user input. ```json { "message": "Hello, how are you?" } ``` -------------------------------- ### System Prompt Example: Demanding User Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This system prompt configures the agent to act as an impatient user focused on receiving quick results and expecting efficient service, providing information only when directly asked. ```yaml system_prompt: | You are an impatient user who wants quick results. You provide information when asked but expect efficient service. ``` -------------------------------- ### Migration Guide: Simple API Payload Format Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This configuration snippet shows how to set the payload format to 'simple' for interactions with simple APIs. It also indicates that 'fullconversation' is not necessary for this format. ```yaml replicant: payload_format: simple fullconversation: false # Not needed for simple format ``` -------------------------------- ### Legacy Payload Structure Example in JSON Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md An example of the legacy payload structure in JSON format for ReplicantX. This structure includes a message, timestamp, and conversation history, serving as a reference for data exchange in backward-compatible scenarios. ```json { "message": "Hello, how are you?", "timestamp": "2025-07-09T10:30:00", "conversation_history": [ {"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi there!"} ] } ``` -------------------------------- ### OpenAI Payload Format Example (JSON) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Illustrates the JSON structure for the OpenAI payload format, which includes a 'messages' array with role-based content. This format is widely supported and provides full conversation context. ```json { "messages": [ {"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi there!"}, {"role": "user", "content": "How are you?"} ] } ``` -------------------------------- ### GitHub Actions CI/CD for ReplicantX E2E Tests Source: https://context7.com/helixtechnologies/replicantx/llms.txt A GitHub Actions workflow to automate End-to-End (E2E) testing for the ReplicantX project on pull request events. It checks out code, sets up Python, installs ReplicantX, waits for preview deployments, runs E2E tests, and posts the report to the PR. ```yaml # .github/workflows/replicantx.yml name: ReplicantX E2E Tests on: pull_request: types: [opened, synchronize, reopened] jobs: replicantx: runs-on: ubuntu-latest env: # Environment variables from GitHub Secrets OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} SUPABASE_URL: ${{ secrets.SUPABASE_URL }} SUPABASE_ANON_KEY: ${{ secrets.SUPABASE_ANON_KEY }} TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }} TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }} # Dynamic preview URL from Render REPLICANTX_TARGET: pr-${{ github.event.pull_request.number }}-api.onrender.com steps: - name: Checkout repository uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" - name: Install ReplicantX run: pip install "replicantx[cli]" - name: Wait for preview deployment run: | until curl -sf "https://$REPLICANTX_TARGET/health"; do echo "Waiting for preview deployment..." sleep 5 done - name: Run E2E tests run: replicantx run tests/*.yaml --report report.md --ci - name: Post report to PR uses: marocchino/sticky-pull-request-comment@v2 if: always() with: path: report.md ``` -------------------------------- ### Configure LLM for Flight Booking Agent Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This YAML configuration illustrates setting up an agent scenario for flight booking with ReplicantX and PydanticAI. It specifies the agent's goal, relevant facts, a system prompt to guide the agent's persona, and LLM settings. ```yaml level: agent replicant: goal: "Book a business class flight to Paris" facts: name: "Sarah Johnson" destination: "Paris" travel_class: "business" # ... other facts system_prompt: | You are a customer trying to book a flight. You have the necessary information but don't provide all details upfront. # ... other config llm: model: "anthropic:claude-3-5-sonnet-latest" # PydanticAI model string temperature: 0.8 # Response creativity (0.0-1.0) max_tokens: 200 # Maximum response length ``` -------------------------------- ### ReplicantX E2E Tests GitHub Actions Workflow (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md A GitHub Actions workflow to automate end-to-end tests for ReplicantX on pull request events. It checks out code, sets up Python, installs dependencies, waits for a service, runs tests, and posts results. ```yaml name: ReplicantX E2E Tests on: pull_request: { types: [opened, synchronize, reopened] } jobs: replicantx: runs-on: ubuntu-latest env: SUPABASE_URL: ${{ secrets.SUPABASE_URL }} SUPABASE_ANON_KEY: ${{ secrets.SUPABASE_ANON_KEY }} REPLICANTX_TARGET: pr-${{ github.event.pull_request.number }}-helix-api.onrender.com steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: { python-version: "3.11" } - run: pip install "replicantx[cli]" - run: | until curl -sf "https://$REPLICANTX_TARGET/health"; do echo "Waiting for preview…"; sleep 5; done - run: replicantx run tests/*.yaml --report report.md --ci - uses: marocchino/sticky-pull-request-comment@v2 if: always() with: { path: report.md } ``` -------------------------------- ### System Prompt Example: Forgetful Customer Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This system prompt defines the agent's persona as a friendly but scattered customer who tends to forget details and needs multiple prompts to provide information. ```yaml system_prompt: | You are a customer who sometimes forgets details and needs multiple prompts. You're friendly but can be a bit scattered. ``` -------------------------------- ### Update ReplicantX Fork Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Commands to synchronize your local ReplicantX fork with the main project's upstream repository. This ensures your local copy is up-to-date before starting new work. ```bash git fetch upstream git checkout main git merge upstream/main git push origin main ``` -------------------------------- ### Anthropic Payload Format Example (JSON) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Shows the JSON structure for the Anthropic payload format, using a 'messages' array similar to OpenAI but tailored for Anthropic's API. It's suitable for Claude-based applications. ```json { "messages": [ {"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi there!"} ] } ``` -------------------------------- ### Use Environment Variables in ReplicantX YAML Configuration Source: https://context7.com/helixtechnologies/replicantx/llms.txt This YAML example demonstrates how to dynamically inject environment variables into ReplicantX configurations using the {{ env.VAR_NAME }} syntax. It shows how to use variables for the base URL, Supabase authentication details, and LLM model selection. ```yaml # Using environment variables in YAML with {{ env.VAR_NAME }} name: "Test with Environment Variables" base_url: "https://{{ env.REPLICANTX_TARGET }}/api/chat" auth: provider: supabase project_url: "{{ env.SUPABASE_URL }}" api_key: "{{ env.SUPABASE_ANON_KEY }}" email: "{{ env.TEST_USER_EMAIL }}" password: "{{ env.TEST_USER_PASSWORD }}" level: agent replicant: llm: model: "openai:gpt-4o" # Uses OPENAI_API_KEY automatically ``` -------------------------------- ### Custom Evaluation Prompt Configuration (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Allows customization of the evaluation logic for domain-specific scenarios using a detailed prompt. The prompt guides the LLM to look for specific criteria and respond with a structured result, confidence score, and reasoning. ```yaml replicant: goal: "Complete a customer support ticket" goal_evaluation_mode: "intelligent" goal_evaluation_prompt: | Evaluate if the customer support goal is achieved. Look for: 1. Issue resolution confirmation from the agent 2. Ticket number or reference provided 3. Customer satisfaction or acknowledgment 4. Clear closure statements Goal: {goal} User Facts: {facts} Recent Conversation: {conversation} Respond exactly: RESULT: [ACHIEVED or NOT_ACHIEVED] CONFIDENCE: [0.0 to 1.0] REASONING: [Brief explanation] completion_keywords: ["resolved", "ticket created"] ``` -------------------------------- ### RESTful Session Management with URL Placement Source: https://context7.com/helixtechnologies/replicantx/llms.txt Configures session management for RESTful APIs, embedding the session UUID directly into the URL path. This example shows how to structure requests like `/conversations/{uuid}/messages`. ```yaml # tests/restful_session.yaml name: "RESTful Session with URL Placement" base_url: "https://api.example.com" auth: provider: noop level: agent replicant: goal: "Complete customer support ticket" facts: issue_type: "account_access" system_prompt: "You are a customer seeking support." initial_message: "I need help accessing my account." max_turns: 10 session_mode: auto session_format: uuid session_placement: url payload_format: restful_session fullconversation: false # Results in requests to: /conversations/{uuid}/messages ``` -------------------------------- ### Environment-Based Session Configuration with Header in ReplicantX Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This example shows how to configure ReplicantX for environment-based session management, using a session ID defined in an environment variable (`REPLICANTX_SESSION_ID`) and transmitted via an HTTP header. It demonstrates setting the environment variable and the corresponding YAML configuration. ```bash # Set environment variable export REPLICANTX_SESSION_ID="prod_session_abc123" # Use in YAML replicant: session_mode: env session_format: uuid session_placement: header session_variable_name: x-chat-id payload_format: openai_session fullconversation: false ``` -------------------------------- ### Enable Debug Mode in ReplicantX Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Activates detailed logging for ReplicantX test runs, providing insights into HTTP client setup, agent initialization, API requests and responses, and AI processing. This is useful for troubleshooting, performance tuning, and integration debugging. ```bash replicantx run tests/agent_test.yaml --debug ``` -------------------------------- ### Migration Guide: Anthropic API Payload Format Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This configuration snippet demonstrates setting the payload format to 'anthropic' for compatibility with Anthropic APIs. It also specifies that 'fullconversation' should be set to true to maintain conversation context. ```yaml replicant: payload_format: anthropic fullconversation: true # Maintain conversation context ``` -------------------------------- ### Implementing Security Best Practices with .gitignore and .env.example Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Demonstrates how to add the `.env` file to `.gitignore` to prevent committing secrets and how to create a `.env.example` file to document required environment variables for other developers. ```bash # Add .env to .gitignore echo ".env" >> .gitignore # Create .env.example for documentation cat > .env.example << 'EOF' # Copy this file to .env and fill in your values OPENAI_API_KEY=sk-your-openai-key-here REPLICANTX_TARGET=https://your-api-domain.com SUPABASE_URL=https://your-project.supabase.co SUPABASE_ANON_KEY=your-supabase-anon-key-here EOF ``` -------------------------------- ### Creating .env File for ReplicantX with Various Secrets Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Details the process of creating a comprehensive `.env` file for ReplicantX, including keys for multiple LLMs, target API endpoints, and authentication credentials for services like Supabase. ```bash # Create .env file in your project root cat > .env << 'EOF' # LLM API Keys OPENAI_API_KEY=sk-your-openai-key ANTHROPIC_API_KEY=sk-ant-your-anthropic-key # Target API REPLICANTX_TARGET=https://api.yourproject.com # Supabase Authentication SUPABASE_URL=https://your-project.supabase.co SUPABASE_ANON_KEY=your-supabase-anon-key TEST_USER_EMAIL=test@example.com TEST_USER_PASSWORD=testpassword123 # JWT Authentication JWT_TOKEN=your-jwt-token EOF ``` -------------------------------- ### Running ReplicantX Tests and Commands Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Shows common ReplicantX commands for running test files, validating configurations, and generating reports. Assumes that environment variables (potentially from a .env file) are automatically loaded. ```bash # Just run - ReplicantX finds .env automatically! replicantx run tests/*.yaml # Validate test files replicantx validate tests/*.yaml # Generate reports replicantx run tests/*.yaml --report report.md ``` -------------------------------- ### Essential Environment Variables for ReplicantX (Bash) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Lists essential environment variables required for ReplicantX, including those for LLM integration (OpenAI, Anthropic), Supabase authentication, target API configuration, and custom authentication methods. ```bash # LLM Integration (PydanticAI auto-detects these) export OPENAI_API_KEY=sk-your-openai-key export ANTHROPIC_API_KEY=sk-ant-your-anthropic-key # Supabase Authentication export SUPABASE_URL=https://your-project.supabase.co export SUPABASE_ANON_KEY=your-supabase-anon-key # Target API Configuration export REPLICANTX_TARGET=your-api-domain.com # Custom Authentication export JWT_TOKEN=your-jwt-token export MY_API_KEY=your-custom-api-key ``` -------------------------------- ### Automatic Environment Variable Loading (Bash) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Demonstrates how ReplicantX automatically detects and loads environment variables for API keys and configuration from the shell or a .env file. Essential for LLM integration and authentication. ```bash # Your project setup pip install replicantx[cli] # Your environment variables (any of these methods work) export OPENAI_API_KEY=sk-your-key # Shell environment echo "OPENAI_API_KEY=sk-key" > .env # .env file # OR set in your CI/CD platform # ReplicantX automatically finds them! replicantx run tests/*.yaml ``` -------------------------------- ### Configure Groq LLM Provider in YAML Source: https://context7.com/helixtechnologies/replicantx/llms.txt This YAML configuration shows how to integrate Groq's LLM for fast inference, specifically for handling quick customer inquiries about store hours. It highlights the model selection and temperature settings suitable for rapid responses. ```yaml # tests/groq_llm.yaml name: "Using Groq for Fast Inference" base_url: "https://api.example.com/chat" auth: provider: noop level: agent replicant: goal: "Quick customer inquiry" facts: question: "store hours" system_prompt: "You are asking about store hours." initial_message: "What are your store hours?" max_turns: 5 llm: model: "groq:llama-3.1-8b-instant" temperature: 0.5 ``` -------------------------------- ### Setting up GitHub Repository Secrets for ReplicantX Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Provides instructions on how to set up secrets in GitHub repository settings, which can then be accessed by GitHub Actions workflows to securely provide sensitive information like API keys to ReplicantX. ```yaml env: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} REPLICANTX_TARGET: ${{ secrets.REPLICANTX_TARGET }} ``` -------------------------------- ### Configure LLM for Technical Support Agent Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This YAML configuration shows how to set up an agent scenario for technical support using ReplicantX and PydanticAI. It includes defining the agent's goal, facts, system prompt, and LLM parameters like model, temperature, and max tokens. ```yaml level: agent replicant: goal: "Get technical support for my account" facts: name: "Jordan Smith" # ... other facts system_prompt: | You are a customer seeking help with a technical issue. Use your available facts to answer questions naturally. # ... other config llm: model: "openai:gpt-4.1-mini" # PydanticAI model string temperature: 0.7 # Response creativity (0.0-1.0) max_tokens: 150 # Maximum response length ``` -------------------------------- ### Run ReplicantX Tests Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Commands to execute the test suite for ReplicantX. This includes running all tests, running tests with coverage, and running specific test files or scenarios. ```bash # Run all tests python -m pytest # Run with coverage python -m pytest --cov=replicantx # Run specific test files python -m pytest tests/test_scenarios.py # Run ReplicantX scenarios python -m replicantx.cli run tests/*.yaml ``` -------------------------------- ### Run ReplicantX Tests (Bash) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Provides various ReplicantX CLI commands for running test suites. These commands cover basic test execution, CI mode, specific file execution, real-time monitoring, debugging, parallel execution, and validation. ```bash # Run all tests in a directory replicantx run tests/*.yaml --report report.md ``` ```bash # Run with CI mode (exits 1 on failure) replicantx run tests/*.yaml --report report.md --ci ``` ```bash # Run specific test file replicantx run tests/specific_test.yaml ``` ```bash # Real-time conversation monitoring replicantx run tests/*.yaml --watch ``` ```bash # Technical debugging with detailed logs replicantx run tests/*.yaml --debug ``` ```bash # Combined monitoring and debugging replicantx run tests/*.yaml --debug --watch ``` ```bash # Run tests in parallel for faster execution replicantx run tests/*.yaml --parallel ``` ```bash # Run with limited concurrency to prevent API overload replicantx run tests/*.yaml --parallel --max-concurrent 3 ``` ```bash # Validate test files without running replicantx validate tests/*.yaml --verbose ``` -------------------------------- ### Clone ReplicantX Repository Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Instructions to fork and clone the ReplicantX repository to your local machine. This involves forking on GitHub, cloning your fork, and adding the upstream remote for keeping your fork updated. ```bash # Navigate to https://github.com/HelixTechnologies/replicantx # Click the "Fork" button in the top-right corner git clone https://github.com/YOUR_USERNAME/replicantx.git cd replicantx git remote add upstream https://github.com/HelixTechnologies/replicantx.git ``` -------------------------------- ### Fixed Session ID Configuration with Custom Variable Name Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md YAML configuration for ReplicantX using a fixed session ID and a custom variable name. This setup is useful for scenarios where a specific, unchanging session identifier is required, with the session ID embedded in the request body. ```yaml replicant: session_mode: fixed session_id: "test_session_12345" session_format: uuid session_placement: body session_variable_name: conversation_id payload_format: simple_session fullconversation: false ``` -------------------------------- ### Keywords Mode Configuration (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Configures ReplicantX to use simple substring matching for goal evaluation. This mode is the default and offers backwards compatibility. It's suitable for simple scenarios and when performance is critical, as it avoids LLM calls. ```yaml replicant: goal: "Book a flight to Paris" goal_evaluation_mode: "keywords" # Default completion_keywords: ["confirmed", "booked", "reservation number"] ``` -------------------------------- ### Advanced Monitoring and Debugging with ReplicantX CLI Source: https://context7.com/helixtechnologies/replicantx/llms.txt Utilizes ReplicantX's command-line options for real-time monitoring and technical debugging. The `--watch` flag provides live conversation insights, while the `--debug` flag shows detailed technical information about HTTP requests, responses, and AI processing. Both can be combined for full visibility. ```bash # Watch mode: Real-time conversation monitoring # Shows: timestamps, user/assistant messages, step results, timing replicantx run tests/agent_test.yaml --watch # Example output: # [22:04:42] 👥 LIVE CONVERSATION - Starting agent scenario # [22:04:42] 🎯 Goal: Book a business class flight to Paris # [22:04:42] 👤 User: Hi, I'd like to book a flight to Paris. # [22:04:52] ✅ Step 1: PASSED (10.2s) # [22:04:52] 🤖 Assistant: What cabin class would you prefer? # [22:04:53] 👤 User: Business class, please. # Debug mode: Technical details # Shows: HTTP setup, request payloads, response validation, AI processing replicantx run tests/agent_test.yaml --debug # Example output: # 🔍 DEBUG HTTP Client initialized # ├─ base_url: https://api.example.com/chat # ├─ timeout: 120s # ├─ auth_provider: supabase # 🔍 DEBUG HTTP request payload # ├─ message: Hi, I'd like to book a flight to Paris. # ├─ payload_size: 229 chars # 🔍 DEBUG Response validation completed # ├─ total_assertions: 2 # ├─ passed_assertions: 2 # Combined mode: Full visibility replicantx run tests/agent_test.yaml --debug --watch ``` -------------------------------- ### YAML Configuration with Environment Variable References Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Shows how to reference environment variables within ReplicantX YAML configuration files using the `{{ env.VARIABLE_NAME }}` syntax for dynamic settings like base URLs and authentication details. ```yaml name: "API Test" base_url: "https://{{ env.REPLICANTX_TARGET }}/api/chat" auth: provider: supabase project_url: "{{ env.SUPABASE_URL }}" api_key: "{{ env.SUPABASE_ANON_KEY }}" level: agent replicant: facts: api_key: "{{ env.MY_API_KEY }}" llm: model: "openai:gpt-4o" # Uses OPENAI_API_KEY automatically ``` -------------------------------- ### Configure Environment Variables for ReplicantX in .env Source: https://context7.com/helixtechnologies/replicantx/llms.txt This .env file provides a template for environment variables used by ReplicantX. It includes placeholders for API keys, Supabase credentials, target API configuration, and JWT authentication, enabling secure and flexible configuration management. ```bash # .env file # LLM API Keys (auto-detected by PydanticAI) OPENAI_API_KEY=sk-your-openai-key ANTHROPIC_API_KEY=sk-ant-your-anthropic-key # Supabase Configuration SUPABASE_URL=https://your-project.supabase.co SUPABASE_ANON_KEY=your-supabase-anon-key # Target API Configuration REPLICANTX_TARGET=api.yourproject.com # JWT Authentication JWT_TOKEN=your-jwt-token # Test User Credentials TEST_USER_EMAIL=test@example.com TEST_USER_PASSWORD=testpassword123 ``` -------------------------------- ### Basic ReplicantX Test Scenario Configuration Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Defines a basic test scenario for ReplicantX, specifying a name, base URL, authentication provider, and test steps. Includes options for asserting expected content using contains, regex, and exact match. ```yaml name: "Basic Test Scenario" base_url: "https://api.example.com/chat" auth: provider: noop level: basic steps: - user: "User message" expect_contains: ["expected", "text"] expect_regex: "regex_pattern" expect_equals: "exact_match" expect_not_contains: ["forbidden", "text"] ``` -------------------------------- ### Set LLM API Keys via Environment Variables Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This bash snippet shows how to set environment variables for API keys required by various LLM providers used by PydanticAI in ReplicantX. It covers OpenAI, Anthropic, Google AI, and Groq. ```bash # OpenAI export OPENAI_API_KEY=sk-your-api-key # Anthropic export ANTHROPIC_API_KEY=sk-ant-your-api-key # Google AI export GOOGLE_API_KEY=your-google-api-key # Groq export GROQ_API_KEY=your-groq-api-key ``` -------------------------------- ### Configure Anthropic LLM Provider in YAML Source: https://context7.com/helixtechnologies/replicantx/llms.txt This YAML configuration demonstrates how to set up the ReplicantX agent to use Anthropic's Claude model for generating product recommendations. It specifies the model, temperature, and maximum tokens, along with agent-specific goals and system prompts. ```yaml # tests/anthropic_llm.yaml name: "Using Anthropic Claude" base_url: "https://api.example.com/chat" auth: provider: noop level: agent replicant: goal: "Get product recommendations" facts: preference: "eco-friendly products" system_prompt: "You are shopping for eco-friendly products." initial_message: "I'm looking for eco-friendly products." max_turns: 10 llm: model: "anthropic:claude-3-5-sonnet-latest" temperature: 0.8 max_tokens: 200 ``` -------------------------------- ### ReplicantX Session Management with ReplicantX Format and Header Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md This YAML configuration demonstrates setting up ReplicantX session management using the ReplicantX format for session IDs, placed in the HTTP header. It's suitable for scenarios requiring specific ReplicantX session identifiers and header-based communication. ```yaml replicant: session_mode: auto session_format: replicantx # ReplicantX format session_placement: header session_variable_name: x-conversation-id payload_format: openai_session fullconversation: false ``` -------------------------------- ### Basic Test Scenario Configuration (Level 1) Source: https://context7.com/helixtechnologies/replicantx/llms.txt YAML configuration for a basic test scenario. It defines API endpoints, authentication, and steps with fixed user messages and deterministic assertions like 'expect_contains', 'expect_not_contains', 'expect_regex', and 'expect_equals'. ```yaml # tests/basic_api_test.yaml name: "Test AI Agent Conversation" base_url: "https://api.example.com/chat" auth: provider: noop # No authentication level: basic timeout_seconds: 120 max_retries: 3 parallel: false steps: - user: "Hello, I need help with booking a flight" expect_contains: ["flight", "booking"] expect_not_contains: ["error", "failed"] timeout_seconds: 30 - user: "I want to go to Paris" expect_regex: "(?i)paris.*available" timeout_seconds: 30 - user: "Show me business class options" expect_contains: ["business", "class"] expect_equals: "Here are the available business class flights to Paris" ``` -------------------------------- ### Supabase Authentication Configuration Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Configures Supabase as the authentication provider, requiring user credentials and Supabase project details. It utilizes environment variables for sensitive information like URL and API key. ```yaml auth: provider: supabase email: user@example.com password: password123 project_url: "{{ env.SUPABASE_URL }}" api_key: "{{ env.SUPABASE_ANON_KEY }}" ``` -------------------------------- ### Parallel Test Execution with ReplicantX CLI Source: https://context7.com/helixtechnologies/replicantx/llms.txt Demonstrates how to run ReplicantX tests in parallel using the command-line interface. It covers unlimited concurrency, setting a maximum concurrency limit to prevent API overload, and overriding timeout and retry settings. ```bash # Run all tests in parallel (unlimited concurrency) replicantx run tests/*.yaml --parallel # Run with maximum 3 concurrent scenarios (prevent API overload) replicantx run tests/*.yaml --parallel --max-concurrent 3 # Run with timeout and retry overrides replicantx run tests/*.yaml --parallel --timeout 180 --max-retries 5 ``` -------------------------------- ### Run ReplicantX Tests (Bash) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Executes ReplicantX tests using the CLI. This command runs specified YAML test files and generates a report in Markdown format. ```bash replicantx run tests/my_test.yaml --report report.md ``` -------------------------------- ### Agent Test Scenario with Supabase Auth (Level 2) Source: https://context7.com/helixtechnologies/replicantx/llms.txt YAML configuration for an agent-level test scenario using Supabase authentication. It defines goals, facts, system prompts, and conversation parameters for an intelligent agent. Supports PydanticAI integration for LLM models. ```yaml # tests/agent_flight_booking.yaml name: "Flight Booking - Business Class to Paris" base_url: "https://api.example.com/chat" auth: provider: supabase email: "{{ env.TEST_USER_EMAIL }}" password: "{{ env.TEST_USER_PASSWORD }}" project_url: "{{ env.SUPABASE_URL }}" api_key: "{{ env.SUPABASE_ANON_KEY }}" level: agent timeout_seconds: 300 max_retries: 3 validate_politeness: false replicant: goal: "Book a round-trip business class flight to Paris for next weekend" facts: name: "Sarah Johnson" email: "sarah.johnson@example.com" travel_class: "business" destination: "Paris" departure_city: "New York" travel_date: "next Friday" return_date: "following Monday" passengers: 1 budget: "$3000" preferences: "aisle seat, vegetarian meal" system_prompt: | You are a customer trying to book a flight. You have all the necessary information but you're a typical user who doesn't provide all details upfront. Answer questions naturally based on your available facts. Be conversational and helpful. initial_message: "Hi, I'd like to book a flight to Paris for next weekend." max_turns: 15 completion_keywords: ["booked", "confirmed", "reservation number"] fullconversation: true payload_format: openai llm: model: "openai:gpt-4o" temperature: 0.7 max_tokens: 150 goal_evaluation_mode: intelligent goal_evaluation_model: "openai:gpt-4o-mini" ``` -------------------------------- ### Report Generation with ReplicantX CLI Source: https://context7.com/helixtechnologies/replicantx/llms.txt Generates test execution reports in either Markdown or JSON format using the ReplicantX command-line interface. These reports provide detailed summaries of test suites, individual scenario results, conversation history, and error details, aiding in analysis and programmatic processing. ```bash # Generate Markdown report replicantx run tests/*.yaml --report report.md # Generate JSON report for programmatic processing replicantx run tests/*.yaml --report results.json # Report includes: # - Test suite summary (passed/failed scenarios, success rate, duration) # - Individual scenario results (steps, assertions, timing) # - Goal evaluation details (method, confidence, reasoning) # - Complete conversation history for agent tests # - Justification for pass/fail decisions # - Error messages and stack traces ``` -------------------------------- ### Run Custom Test with Asyncio Source: https://context7.com/helixtechnologies/replicantx/llms.txt Executes a custom asynchronous test function using Python's asyncio library. This is typically the entry point for running tests. ```python if __name__ == "__main__": asyncio.run(run_custom_test()) ``` -------------------------------- ### PydanticAI Customer Support Configuration (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Configuration for a ReplicantX customer support agent using PydanticAI. It defines the agent's goal, facts, system prompt, and LLM parameters for handling billing issues. ```yaml name: "Customer Support - Billing Issue" base_url: https://api.example.com/support auth: provider: noop level: agent replicant: goal: "Get customer support for billing issue" facts: name: "Alex Chen" account_number: "ACC-12345" issue_type: "billing" last_payment: "$99.99 on Jan 15th" system_prompt: | You are a customer who is polite but slightly frustrated about a billing issue. You have the necessary account information but may need prompting to remember specific details. initial_message: "Hi, I have a question about my recent bill." max_turns: 12 completion_keywords: ["resolved", "ticket created", "issue closed"] fullconversation: true # Send full conversation history with each request payload_format: openai # Use OpenAI-compatible format llm: model: "openai:gpt-4o" # PydanticAI model string temperature: 0.8 max_tokens: 120 ``` -------------------------------- ### Hybrid Mode Configuration (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Combines LLM evaluation with keyword fallback. It attempts LLM analysis first and resorts to keyword matching if the LLM is uncertain. This provides a balance between smart evaluation and reliable fallback, making it cost-effective and production-ready. ```yaml replicant: goal: "Get help with billing issue" goal_evaluation_mode: "hybrid" goal_evaluation_model: "openai:gpt-4o-mini" completion_keywords: ["resolved", "ticket created", "issue closed"] ``` -------------------------------- ### Create Feature Branch in Git Source: https://github.com/helixtechnologies/replicantx/blob/master/CONTRIBUTING.md Demonstrates how to create new branches in Git for different types of work, such as features, bug fixes, or documentation updates, following the project's branching conventions. ```bash git checkout -b feature/your-feature-name # or git checkout -b fix/your-bug-fix # or git checkout -b docs/your-documentation-update ``` -------------------------------- ### Migration Strategy to Intelligent/Hybrid Mode (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Outlines a phased approach for migrating to more advanced goal evaluation modes. Phase 1 involves testing Intelligent mode, Phase 2 adopting Hybrid mode for safety, and Phase 3 making Intelligent/Hybrid the default for new tests. ```yaml # Update specific tests to use intelligent evaluation goal_evaluation_mode: "intelligent" ``` ```yaml # Use hybrid for safety while gaining intelligence goal_evaluation_mode: "hybrid" ``` ```yaml # Eventually make intelligent/hybrid the default for new tests goal_evaluation_mode: "intelligent" ``` -------------------------------- ### Intelligent Mode Configuration (YAML) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Enables LLM-powered analysis for goal evaluation, understanding context and intent. This mode is recommended for its ability to reduce false positives and provide reasoning. It requires specifying the LLM model to be used. ```yaml replicant: goal: "Book a business class flight to Paris" goal_evaluation_mode: "intelligent" goal_evaluation_model: "openai:gpt-4o-mini" # Optional: separate model for evaluation completion_keywords: ["confirmed", "booked"] # Still required for compatibility ``` -------------------------------- ### Watch Mode for ReplicantX Tests (Bash) Source: https://github.com/helixtechnologies/replicantx/blob/master/README.md Demonstrates the usage of the `--watch` flag in the ReplicantX CLI to enable real-time conversation monitoring during test execution. This mode provides live updates on the conversation flow, agent messages, and step results. ```bash replicantx run tests/agent_test.yaml --watch ```