### SPARC-Bench Getting Started and Core Concepts Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/README.md Provides guidance on getting started with SPARC-Bench, including links to quick start and installation guides. It also outlines the core concepts of the framework, such as benchmark types, test cases, metrics, the Benchmark Manager, and E2B integration. ```markdown ## Getting Started To quickly get started with SPARC-Bench, refer to our [Quick Start Guide](./getting-started/quick-start.md). For a more detailed setup process, check the [Installation Guide](./getting-started/installation.md). ## Core Concepts SPARC-Bench operates on several key concepts: - **Benchmark Types**: Different categories of tests (HumanEval, SWE-bench, RedCode) - **Test Cases**: Individual tests within each benchmark - **Metrics**: Quantitative measurements of performance (accuracy, efficiency, safety, adaptability) - **Benchmark Manager**: Coordinates the execution of benchmarks and collects results - **E2B Integration**: Secure sandbox environment for code execution For a deeper understanding of these concepts, see the [Core Concepts](./concepts/core-concepts.md) documentation. ``` -------------------------------- ### Getting Started with Agentic Edge Functions Source: https://github.com/agenticsorg/edge-agents/blob/main/README.md Instructions on how to get started with Agentic Edge Functions, including repository cloning, Supabase setup, environment variable configuration, and deployment. ```markdown ## Getting Started To get started with these edge functions: 1. Clone this repository 2. Set up a Supabase project 3. Configure environment variables 4. Deploy functions to your Supabase project For beginners, we recommend following our [step-by-step tutorials](./docs/tutorials/README.md) to learn how to create, deploy, and test edge functions. For detailed instructions, see the [Getting Started Guide](./docs/getting_started.md). ``` -------------------------------- ### Manual SPARC2 Installation and Alias Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Provides steps for manually installing SPARC2 by cloning the repository, caching dependencies with Deno, and setting up an alias for command execution. ```bash git clone https://github.com/agentics-org/sparc2.git cd sparc2 deno cache --reload src/cli/cli.ts alias sparc2="deno run --allow-read --allow-write --allow-env --allow-net --allow-run /path/to/sparc2/src/cli/cli.ts" ``` -------------------------------- ### Guides Overview Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/README.md Guides for getting started, deploying edge functions, contributing to the repository, and beginner tutorials for Agentic Edge Functions. ```en - [Getting Started](./getting_started.md) - Guide to getting started with the repository - [Deployment Guide](./deployment.md) - Guide to deploying edge functions - [Contributing Guidelines](./contributing.md) - Guidelines for contributing to the repository - [Tutorials](./tutorials/README.md) - Step-by-step beginner tutorials for Agentic Edge Functions ``` -------------------------------- ### Install SPARC2 via NPM Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Installs the SPARC2 command-line tool globally or as a development dependency in a Node.js project using npm. ```bash npm install -g @agentics.org/sparc2 npm install --save-dev @agentics.org/sparc2 ``` -------------------------------- ### Clone Repository and Install Dependencies Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md Steps to clone the DevOps Agent repository and install its Python dependencies. ```bash git clone https://github.com/yourusername/devops-agent.git cd devops-agent pip install -r requirements.txt ``` -------------------------------- ### Environment Variables Setup Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/quick-start.md Creates a .env file in the scripts/sparc2 directory to store the E2B API key. This key is necessary for secure code execution during benchmarking. ```bash E2B_API_KEY=your_e2b_api_key_here ``` -------------------------------- ### SPARC2 TOML Configuration Example Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Demonstrates a basic TOML configuration file for SPARC2, specifying execution modes, logging preferences, rollback settings, and model choices for different tasks. ```toml # SPARC 2.0 Configuration (TOML) [execution] mode = "semi" # Options: automatic, semi, manual, custom diff_mode = "file" # Options: file, function processing = "sequential" # Options: parallel, sequential, concurrent, swarm [logging] enable = true vector_logging = true [rollback] checkpoint_enabled = true temporal_rollback = true [models] reasoning = "gpt-4o" # For architecture, planning, problem solving instruct = "gpt-4o" # For instructing code changes ``` -------------------------------- ### Verify SPARC2 Installation Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Checks if the SPARC2 CLI has been installed correctly by displaying its version. ```bash sparc2 --version ``` -------------------------------- ### SPARC-Bench Configuration (config.toml) Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/installation.md Example TOML configuration file for SPARC-Bench, defining benchmark settings, metrics to include, and agent parameters like sizes and parallelism. ```toml [benchmark] name = "SPARC2 Benchmark" version = "1.0.0" [metrics] include = ["accuracy", "efficiency", "safety", "adaptability"] [agent] sizes = ["small", "medium", "large"] tokenCacheEnabled = true maxParallelAgents = 2 ``` -------------------------------- ### CLI: Work with GitHub Repositories Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md CLI commands for interacting with GitHub, such as listing repositories, getting README content, and listing branches. ```bash # List repositories in an organization python -m src.cli github list-repos --org your-organization # Get README content python -m src.cli github get-readme owner/repo-name # List branches python -m src.cli github list-branches owner/repo-name ``` -------------------------------- ### List Supabase Projects Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Lists the Supabase projects linked to your CLI. This is a troubleshooting step to verify project linkage. ```bash supabase projects list ``` -------------------------------- ### Deno Script with Full Permissions Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/installation.md Example of running a Deno script with comprehensive permissions (read, write, environment, network) to avoid common permission errors. ```bash deno run --allow-read --allow-write --allow-env --allow-net your-script.ts ``` -------------------------------- ### Installation and Setup Source: https://github.com/agenticsorg/edge-agents/blob/main/supabase/functions/openai-agent-sdk/README.md Instructions for cloning the repository, navigating to the function directory, and installing dependencies using Deno. ```bash # Clone the repository git clone your-repo-url # Navigate to the function directory cd supabase/functions/openai-agent-sdk # Install dependencies deno cache --reload index.ts ``` -------------------------------- ### Supabase CLI Project Linking Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Logs into the Supabase CLI and links the local repository to a specific Supabase project using the project reference ID. ```bash supabase login supabase link --project-ref your-project-ref ``` -------------------------------- ### CLI: Get GitHub Repository Information Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md Command to retrieve details about a GitHub repository using the CLI. ```bash python -m src.cli github get-repo owner/repo-name ``` -------------------------------- ### Clone Repository and Navigate Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Clones the Agentic Edge Functions repository from GitHub and changes the current directory to the cloned repository. ```bash git clone https://github.com/agentics-foundation/edge-agents.git cd edge-agents ``` -------------------------------- ### Clone SPARC-Bench Repository Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/installation.md Clones the SPARC-Bench repository from GitHub. This is one of the options for installing the framework. ```bash git clone https://github.com/your-org/edge-agents.git cd edge-agents ``` -------------------------------- ### Standalone SPARC-Bench Installation Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/installation.md Installs SPARC-Bench as a standalone tool by cloning its specific repository. ```bash git clone https://github.com/your-org/sparc-bench.git cd sparc-bench ``` -------------------------------- ### Start SPARC2 MCP Server Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/examples/README.md Demonstrates how to start the MCP server using SPARC2. ```bash ./scripts/sparc2/sparc mcp ``` ```bash ./scripts/sparc2/examples/08-mcp-server.sh ``` -------------------------------- ### Running the SPARC2 SSE Example Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/examples/README-SSE.md This snippet shows how to make the example script executable and then run it. It's the primary way to start the SPARC2 SSE example. ```bash # Make the script executable chmod +x run-sse-example.sh # Run the example ./run-sse-example.sh ``` -------------------------------- ### SPARC2 CLI and MCP Server Usage Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/DENO-INSTALLATION.md Demonstrates basic commands for interacting with SPARC2 after Deno has been successfully installed. ```bash # Using the CLI sparc2 --help # Using the MCP server sparc2-mcp ``` -------------------------------- ### Manual Installation and Setup Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/README.md Installs SPARC2 by cloning the repository and setting up dependencies using Deno. ```bash # Clone the repository git clone https://github.com/agentics-org/sparc2.git cd sparc2 # Install dependencies deno cache --reload src/cli/cli.ts ``` -------------------------------- ### Setup and Configuration Source: https://github.com/agenticsorg/edge-agents/blob/main/supabase/functions/openai-agent-sdk/examples/README.md Demonstrates the initial setup steps for the OpenAI Agent SDK, including copying environment files, configuring API keys, and making scripts executable. ```bash cp .env.example .env # OpenAI Configuration OPENAI_API_KEY=your_openai_api_key # Supabase Configuration SUPABASE_URL=https://your-project-id.supabase.co SUPABASE_ANON_KEY=your_supabase_anon_key chmod +x *.sh ``` -------------------------------- ### Basic Deno Edge Function Example Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md A simple Deno-based edge function that accepts a JSON payload with a 'name' property and returns a personalized greeting. It uses Deno's standard HTTP server. ```typescript import { serve } from "https://deno.land/std@0.168.0/http/server.ts"; serve(async (req) => { const { name } = await req.json(); return new Response( JSON.stringify({ message: `Hello, ${name}!` }), { headers: { "Content-Type": "application/json" } } ); }); ``` -------------------------------- ### Clone Repository Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/quick-start.md Clones the SPARC-Bench repository and navigates into the project directory. This is the first step to setting up the benchmarking tool. ```bash git clone https://github.com/your-org/edge-agents.git cd edge-agents ``` -------------------------------- ### SPARC-Bench CLI: Run All Benchmarks in Parallel with Verbose Output Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/quick-start.md Runs all available benchmarks concurrently with verbose logging enabled via the SPARC-Bench CLI. Requires Deno and appropriate permissions. ```bash deno run --allow-read --allow-write --allow-env --allow-net sparc-bench.ts run --all --parallel --verbose ``` -------------------------------- ### Start Local Supabase Functions Server Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Starts the Supabase local development server for functions, with JWT verification disabled. This is a troubleshooting step for local development issues. ```bash supabase functions serve --no-verify-jwt ``` -------------------------------- ### Supabase Local Development Server Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Starts the Supabase local development server and serves edge functions. The `--no-verify-jwt` flag bypasses JWT verification for local testing. ```bash supabase start supabase functions serve --no-verify-jwt ``` -------------------------------- ### SPARC-Bench CLI: Run Specific Benchmark Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/quick-start.md Uses the SPARC-Bench CLI to run a specific benchmark identified by its name. Requires Deno and appropriate permissions. ```bash deno run --allow-read --allow-write --allow-env --allow-net sparc-bench.ts run --benchmark sparc2-code-analysis ``` -------------------------------- ### Enable Debug Mode Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md How to enable debug logging for the DevOps Agent CLI to get more detailed output. ```bash python -m src.cli --debug ec2 list-instances ``` -------------------------------- ### Configure AWS and GitHub Credentials Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md Instructions for setting up environment variables with AWS and GitHub credentials. ```bash cp env.example .env # Edit .env file: AWS_ACCESS_KEY_ID=your-access-key-id AWS_SECRET_ACCESS_KEY=your-secret-access-key AWS_REGION=us-east-1 GITHUB_TOKEN=your-github-personal-access-token ``` -------------------------------- ### Manual SPARC2 Installation with Deno Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/tutorials/basic-usage.md Provides instructions for manually installing SPARC2 by cloning the repository and using Deno for dependency management and execution. Includes creating a shell alias for convenience. ```bash # Clone the repository git clone https://github.com/agentics-org/sparc2.git cd sparc2 # Install dependencies deno cache --reload src/cli/cli.ts # Create an alias for easier use alias sparc2="deno run --allow-read --allow-write --allow-env --allow-net --allow-run /path/to/sparc2/src/cli/cli.ts" ``` -------------------------------- ### Serve Specific Function Locally Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Starts the Supabase local development server and serves a specific edge function. The `--no-verify-jwt` flag bypasses JWT verification for local testing. ```bash supabase functions serve function-name --no-verify-jwt ``` -------------------------------- ### CLI: Manage EC2 Instances Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md Common CLI commands for managing EC2 instances, including listing, creating, stopping, and starting. ```bash # List running instances python -m src.cli ec2 list-instances --state running # Create a new instance python -m src.cli ec2 create-instance --name web-server --type t2.micro --ami-id ami-0c55b159cbfafe1f0 # Stop an instance python -m src.cli ec2 stop-instance i-1234567890abcdef0 # Start an instance python -m src.cli ec2 start-instance i-1234567890abcdef0 ``` -------------------------------- ### Update Supabase CLI Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Updates the Supabase CLI to the latest version. This is a troubleshooting step for deployment issues. ```bash npm install -g supabase@latest ``` -------------------------------- ### OpenAI Agent MCP Server Usage Example Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agentic-mcp/docs/plans/implementation.md An example demonstrating how to initialize and configure an OpenAI Agent MCP server, register tools, and start the server. This provides a practical guide for setting up the agent. ```typescript const server = new OpenAIAgentMCPServer({ name: 'openai-agent', version: '1.0.0', openai: { apiKey: process.env.OPENAI_API_KEY, defaultModel: 'gpt-4o-mini' }, tracing: { enabled: true, level: 'debug' }, tools: { enabled: ['research', 'database', 'support'], config: { database: { url: process.env.SUPABASE_URL, key: process.env.SUPABASE_KEY } } }, guardrails: { enabled: true, rules: [defaultGuardrail] } }); // Register tools server.registerTool(researchTool); server.registerTool(databaseTool); server.registerTool(supportTool); // Start server await server.serve(); ``` -------------------------------- ### Analyze Code with SPARC2 Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Shows how to analyze a JavaScript file using the SPARC2 CLI, including creating a sample file and running the analysis command. ```bash # Create a test file echo 'function add(a, b) { return a + b; }\nfunction multiply(a, b) { return a + b; }' > test.js # Analyze the file sparc2 analyze --files test.js ``` -------------------------------- ### Serve Specific Function Locally for Testing Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Starts the Supabase local development server and specifically serves the 'new-function'. The `--no-verify-jwt` flag is used to disable JWT verification during local development. ```bash supabase functions serve new-function --no-verify-jwt ``` -------------------------------- ### Execute Code with SPARC2 Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/getting-started.md Illustrates how to execute a code file using the SPARC2 CLI, which runs the code in a secure sandbox and displays the results. ```bash sparc2 execute --file test.js ``` -------------------------------- ### Deploy Edge Functions Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Deploys edge functions to the Supabase project. Can deploy a specific function or all functions in the project. ```bash supabase functions deploy function-name supabase functions deploy ``` -------------------------------- ### Local Deployment: Basic Setup Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/deployment/deployment-options.md Steps to clone the repository and run SPARC-Bench locally using Deno. Requires Deno to be installed. ```bash git clone https://github.com/your-org/edge-agents.git cd edge-agents/scripts/sparc-bench # Make sure Deno is installed curl -fsSL https://deno.land/x/install/install.sh | sh deno run --allow-read --allow-write --allow-env --allow-net sparc-bench.ts run --all ``` -------------------------------- ### Development Setup Commands Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/README.md Commands to clone the repository, set up a virtual environment, install dependencies, run tests, and check code style. ```bash git clone https://github.com/agentics-foundation/agentic-devops.git cd agentic-devops python -m venv venv source venv/bin/activate pip install -r requirements-dev.txt python -m pytest flake8 src tests black src tests ``` -------------------------------- ### Run First Benchmark Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/quick-start.md Executes a basic benchmark using Deno, allowing read, write, environment, and network access. This command runs all three benchmark types (HumanEval, SWE-bench, RedCode) and generates a report. ```bash cd scripts/sparc2 den run --allow-read --allow-write --allow-env --allow-net run-sparc2-benchmark.ts ``` -------------------------------- ### View Supabase Function Logs Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Retrieves logs for deployed Supabase edge functions. Can fetch logs for a specific function or all functions. ```bash supabase functions logs function-name supabase functions logs ``` -------------------------------- ### Python Library: AWS EC2 and GitHub Integration Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/agents/devops/docs/quickstart.md Example of using the DevOps Agent Python library to interact with AWS EC2 and GitHub services. ```python from devops_agent.aws.ec2 import EC2Service from devops_agent.github.github import GitHubService from devops_agent.core.credentials import get_credential_manager # Get credentials cred_manager = get_credential_manager() aws_creds = cred_manager.get_aws_credentials() github_creds = cred_manager.get_github_credentials() # Initialize services ec2 = EC2Service(credentials=aws_creds) github = GitHubService(token=github_creds.token) # List EC2 instances instances = ec2.list_instances(filters=[{'Name': 'instance-state-name', 'Values': ['running']}]) for instance in instances: print(f"Instance ID: {instance['InstanceId']}") # Get GitHub repository details repo = github.get_repository("owner/repo-name") print(f"Repository: {repo['full_name']}") print(f"Description: {repo['description']}") ``` -------------------------------- ### Deploy Supabase Edge Function Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Deploys a Supabase Edge Function using the Supabase CLI. This command uploads your function to the Supabase platform. ```bash supabase functions deploy new-function ``` -------------------------------- ### JavaScript WebSocket Client Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md A simple JavaScript client to connect to a WebSocket-based edge function, send messages, and handle incoming messages and connection events. ```javascript // Simple WebSocket client const ws = new WebSocket('wss://your-project-ref.supabase.co/functions/v1/function-name'); ws.onopen = () => { console.log('Connected'); ws.send(JSON.stringify({ message: 'Hello' })); }; ws.onmessage = (event) => { console.log('Received:', event.data); }; ws.onerror = (error) => { console.error('Error:', error); }; ws.onclose = () => { console.log('Disconnected'); }; ``` -------------------------------- ### Configure API Keys Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc2/docs/tutorials/basic-usage.md Sets up the necessary API keys for OpenAI and E2B by creating a `.env` file in the project's root directory. ```bash OPENAI_API_KEY=your_openai_api_key E2B_API_KEY=your_e2b_api_key ``` -------------------------------- ### Hello World Example Function Source: https://github.com/agenticsorg/edge-agents/blob/main/README.md Documentation for a simple 'hello-world' example function, serving as a basic illustration for Agentic Edge Functions. ```markdown - [hello-world](./docs/utility_functions/hello-world.md) - Simple example function ``` -------------------------------- ### Test Local Function Deployment Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Tests a locally deployed Supabase Edge Function using curl. It sends a POST request to the local endpoint with a JSON payload. ```bash curl -X POST http://localhost:54321/functions/v1/new-function \ -H "Content-Type: application/json" \ -d '{"name": "World"}' ``` -------------------------------- ### Clone and Set Up Repository Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/contributing.md Instructions for cloning the repository, setting up the upstream remote, and starting Supabase local development. ```bash git clone https://github.com/your-username/edge-agents.git cd edge-agents git remote add upstream https://github.com/agentics-foundation/edge-agents.git npm install -g supabase supabase start --no-verify-jwt ``` -------------------------------- ### Create New Edge Function Structure Source: https://github.com/agenticsorg/edge-agents/blob/main/docs/getting_started.md Creates the necessary directory structure and an initial TypeScript file for a new edge function within the Supabase functions directory. ```bash # Create a new function directory mkdir -p supabase/functions/new-function # Create the main function file touch supabase/functions/new-function/index.ts ``` -------------------------------- ### Run SPARC-Bench Verification Test Source: https://github.com/agenticsorg/edge-agents/blob/main/scripts/sparc-bench/docs/getting-started/installation.md Executes a test script using Deno to verify the SPARC-Bench installation. Requires read and environment permissions. ```bash cd scripts/sparc-bench deno run --allow-read --allow-env test-sparc2.ts ```