### Defining Examples in Instructions Source: https://github.com/zhang-henry/coevoskills/blob/main/meta_skills/skill-creator/SKILL.md Pattern for providing input-output examples to guide model behavior. ```markdown ## Commit message format **Example 1:** Input: Added user authentication with JWT tokens Output: feat(auth): implement JWT-based authentication ``` -------------------------------- ### Install CoEvoSkills environment Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Clone the repository, sync dependencies with uv, and initialize the environment configuration. ```bash git clone https://github.com/Zhang-Henry/CoEvoSkills.git cd CoEvoSkills uv sync --locked cp .env.example .env docker info ``` -------------------------------- ### Install Flash Attention 2 Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/simpo-code-reproduction/environment/SimPO/README.md Installs the Flash Attention 2 library without build isolation. ```shell python -m pip install flash-attn --no-build-isolation ``` -------------------------------- ### Install Alignment Handbook Dependencies Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/simpo-code-reproduction/environment/SimPO/README.md Clones the alignment-handbook repository and installs its package dependencies. ```shell git clone https://github.com/huggingface/alignment-handbook.git cd ./alignment-handbook/ python -m pip install . ``` -------------------------------- ### Configure OpenAI Codex Subscription Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Install and authenticate the Codex CLI for ChatGPT subscription access. ```bash npm install -g @openai/codex codex login codex login status export CODEX_CHATGPT_AUTH_FILE="$HOME/.codex/auth.json" export CODEX_CLI_BINARY="$(command -v codex)" export CODEX_REASONING_EFFORT="low" ``` -------------------------------- ### Example problem.json structure Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/jax-computing-basics/instruction.md The configuration file format used to define task IDs, descriptions, input paths, and output paths. ```json [ { "id": a unique problem id, "description": a language description of the task, "input": input data path, for example: "data/x.npy", "output": output result path, for example: "basic_reduce.npy" }, ..., ] ``` -------------------------------- ### Run SimPO Task End-to-End Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/simpo-code-reproduction/evo-simpo-loss/SKILL.md Executes the full task pipeline including package installation, code injection, and validation. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-simpo-loss/scripts') from run_task import run_simpo_task # Run the complete task end-to-end run_simpo_task( project_dir='/root/SimPO', output_path='/root/loss.npz', python_info_path='/root/python_info.txt', ) ``` -------------------------------- ### Dialogue Parser Usage Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/dialogue-parser/evo-dialogue-parser/SKILL.md Example of running the end-to-end parsing pipeline and validating the resulting graph structure. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-dialogue-parser/scripts') from utils import run_end_to_end, parse_script, validate_graph # End-to-end: parse script and write outputs graph = run_end_to_end('/app/script.txt', '/app/dialogue.json', '/app/dialogue.dot') # Validate is_valid, issues = validate_graph(graph) assert is_valid, f"Validation failed: {issues}" print("All validations passed") ``` -------------------------------- ### Detect and Verify Java Environment Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/background_docs/fix-druid-loophole-cve/druid-cve-2021-25646-background.md Set the JAVA_HOME environment variable based on the installed JDK and verify the versions of Java and Maven. ```bash # Detect JAVA_HOME from the installed JDK export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which javac)))) # Verify java -version mvn -v ``` -------------------------------- ### Calculate PPSETMODE ioctl number Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/syzkaller-ppdev-syzlang/syzkaller-ppdev-syzlang/SKILL.md Example calculation for the PPSETMODE ioctl using the _IOW macro formula. ```text 0x40000000 | (4 << 16) | (0x70 << 8) | 0x80 = 0x40047080 = 1074032768 ``` -------------------------------- ### Example TPP Task Entry Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/pddl-tpp-planning/instruction.md JSON structure defining the domain, problem, and output paths for a TPP task. ```json [ { "id": "problem_id", "domain": ".../xxx.pddl", "problem": ".../yyy.pddl", "plan_output": "xxx/problem_id.txt" }, ..., ] ``` -------------------------------- ### Execute End-to-End Fuzzing Workflow Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/setup-fuzzing-py/evo-python-fuzzing/SKILL.md Run the complete fuzzing pipeline including discovery, analysis, driver generation, setup, execution, and validation. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-python-fuzzing/scripts') from utils import run_all, validate_all # End-to-end: discover, analyze, generate, setup, fuzz, validate results = run_all( base_dir='/app', libraries_file='/app/libraries.txt', timeout=10 ) # Validate all artifacts exist and logs are valid all_ok = validate_all('/app', '/app/libraries.txt') print(f"All OK: {all_ok}") ``` -------------------------------- ### Example Wyckoff Analysis Output Format Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/crystallographic-wyckoff-position-analysis/instruction.md The expected structure of the returned dictionary, including multiplicities and rational-formatted coordinates. ```json { "wyckoff_multiplicity_dict": {"a": 4, "c": 8}, "wyckoff_coordinates_dict": { "a": ["0", "1/2", "1/2"], "c": ["3/8", "1/9", "8/9"] } } ``` -------------------------------- ### Index Tutorial Videos Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/video-tutorial-indexer/evo-video-indexer/SKILL.md Demonstrates the full workflow from audio extraction to JSON index generation. Requires the scripts directory to be in the Python path. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-video-indexer/scripts') from transcribe import extract_audio, transcribe_audio, save_transcript from align_chapters import align_chapters_to_transcript, validate_chapters, generate_index # Step 1: Extract audio audio_path = extract_audio('/root/tutorial_video.mp4', '/tmp/audio.wav') # Step 2: Transcribe segments = transcribe_audio(audio_path, model_name='base') save_transcript(segments, '/root/transcript_segments.json') # Step 3: Define chapter titles (from task) chapter_titles = [ "What we'll do", "How we'll get there", # ... all chapter titles ... ] # Step 4: Align chapters to transcript chapters = align_chapters_to_transcript(chapter_titles, segments, first_time=0) # Step 5: Validate errors = validate_chapters(chapters, duration=1382, expected_count=29) if errors: print("Validation errors:", errors) # Step 6: Generate output generate_index(chapters, title='In-Depth Floor Plan Tutorial Part 1', duration=1382, output_path='/root/tutorial_index.json') ``` -------------------------------- ### Initialize Environment and Import Skills Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/travel-planning/solution/solve.ipynb Configures the system path to include necessary skill modules and imports the required classes. ```python import sys sys.path.append("../environment/skills/search_cities/scripts") sys.path.append("../environment/skills/search_accommodations/scripts") sys.path.append("../environment/skills/search_restaurants/scripts") sys.path.append("../environment/skills/search_attractions/scripts") sys.path.append("../environment/skills/search_driving_distance/scripts") from search_accommodations import Accommodations from search_attractions import Attractions from search_cities import Cities from search_driving_distance import GoogleDistanceMatrix from search_restaurants import Restaurants ``` -------------------------------- ### Grading JSON Schema Structure Source: https://github.com/zhang-henry/coevoskills/blob/main/meta_skills/skill-creator/references/schemas.md Example of the grading.json output format generated by the grader agent. ```json { "expectations": [ { "text": "The output includes the name 'John Smith'", "passed": true, "evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'" }, { "text": "The spreadsheet has a SUM formula in cell B10", "passed": false, "evidence": "No spreadsheet was created. The output was a text file." } ], "summary": { "passed": 2, "failed": 1, "total": 3, "pass_rate": 0.67 }, "execution_metrics": { "tool_calls": { "Read": 5, "Write": 2, "Bash": 8 }, "total_tool_calls": 15, "total_steps": 6, "errors_encountered": 0, "output_chars": 12450, "transcript_chars": 3200 }, "timing": { "executor_duration_seconds": 165.0, "grader_duration_seconds": 26.0, "total_duration_seconds": 191.0 }, "claims": [ { "claim": "The form has 12 fillable fields", "type": "factual", "verified": true, "evidence": "Counted 12 fields in field_info.json" } ], "user_notes_summary": { "uncertainties": ["Used 2023 data, may be stale"], "needs_review": [], "workarounds": ["Fell back to text overlay for non-fillable fields"] }, "eval_feedback": { "suggestions": [ { "assertion": "The output includes the name 'John Smith'", "reason": "A hallucinated document that mentions the name would also pass" } ], "overall": "Assertions check presence but not correctness." } } ``` -------------------------------- ### Verify project build and quality Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/spring-boot-jakarta-migration/instruction.md Run these Maven commands to ensure the project compiles correctly and passes all existing unit tests after migration. ```bash mvn clean compile ``` ```bash mvn test ``` -------------------------------- ### Normalize defect logs with Python Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/manufacturing-codebook-normalization/evo-defect-normalizer/SKILL.md Demonstrates how to import the normalizer and validator, process logs using default or custom configurations, and validate the resulting output. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-defect-normalizer/scripts') from normalizer import normalize_logs from validator import validate_output data_dir = '/app/data' output_path = '/app/output/solution.json' # Use defaults records = normalize_logs(data_dir, output_path) # Or override policy records = normalize_logs(data_dir, output_path, config={ 'unknown_threshold': 15, 'conf_min': 0.45, }) print(f"Processed {len(records)} records") errors, warnings, stats = validate_output(output_path, data_dir) print(f"Stats: {stats}") if not errors: print("VALIDATION PASSED") ``` -------------------------------- ### Register and run a custom agent adapter Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Commands to prepare and execute a task using a custom agent registered via an import path. ```bash export RUN_ID=custom-agent-smoke uv run python scripts/prepare_tasks.py \ --condition skill-only \ --skill-set validated \ --tasks 3d-scan-calc MODEL=/ \ AGENT=my-agent \ scripts/run_condition.sh skill-only \ --only-tasks 3d-scan-calc \ --agent-import-path my-agent=my_package.agents:MyAgent ``` -------------------------------- ### RTTM Diarization Output Format Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/speaker-diarization-subtitles/instruction.md The RTTM file records speaker segments with start times and durations. ```text SPEAKER input 1 0.804000 0.860000 spk00 SPEAKER input 1 23.820000 1.170000 spk01 ``` -------------------------------- ### Launch SimPO Training Scripts Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/simpo-code-reproduction/environment/SimPO/README.md Commands to initiate training for different model architectures using accelerate and DeepSpeed configurations. Adjust hardware-specific parameters like num_processes and batch size as needed. ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/mistral-7b-base-simpo.yaml ``` ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/mistral-7b-instruct-simpo.yaml ``` ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-base-simpo.yaml ``` ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo.yaml ``` ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo-v2.yaml ``` -------------------------------- ### Transcribe video with word-level timestamps Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/video-filler-word-remover/evo-filler-word-detector/SKILL.md Uses Whisper to generate a list of words with start and end times. ```python from transcribe import transcribe_video words = transcribe_video("/root/input.mp4", model_name="base", cache_path="/root/cache/transcript.json") # Returns: [{"word": "hello", "start": 0.0, "end": 0.5}, ...] ``` -------------------------------- ### Execute Supply-Side Model Workflow Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/shock-analysis-supply/evo-supply-shock/SKILL.md Initializes the environment, runs the model builder, and validates the resulting Excel workbook. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-supply-shock/scripts') from data_collection import collect_pwt_data, collect_weo_data, collect_ecb_cfc_data from excel_builder import build_supply_model from validator import validate_workbook # End-to-end entry point build_supply_model( template_path='/root/test-supply.xlsx', output_path='/root/test-supply.xlsx' ) # Validate validate_workbook('/root/test-supply.xlsx') ``` -------------------------------- ### Organize Documents Workflow Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/organize-messy-files/evo-doc-organizer/SKILL.md Demonstrates the full pipeline for organizing documents from a source directory into categorized output folders and validating the results. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-doc-organizer/scripts') from utils import organize_documents, validate_organization # Organize all documents from source into category folders source_dir = '/root/papers/all' output_dir = '/root/papers' assignments = organize_documents(source_dir, output_dir) # Validate the organization validate_organization(source_dir, output_dir) ``` -------------------------------- ### Implement Solution Script Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/crystallographic-wyckoff-position-analysis/evo-wyckoff/SKILL.md Shows how to import and utilize the analysis function within a custom solution script. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-wyckoff/scripts') from utils import analyze_wyckoff_position_multiplicities_and_coordinates # The function is directly importable and usable # Example: result = analyze_wyckoff_position_multiplicities_and_coordinates('/root/cif_files/FeS2_mp-226.cif') ``` -------------------------------- ### Identify Malicious JavaScript Payload Source: https://github.com/zhang-henry/coevoskills/blob/main/tasks/fix-druid-loophole-cve/instruction.md Example of an authenticated request using a JavaScript filter to execute arbitrary system commands. ```http POST /druid/indexer/v1/sampler HTTP/1.1 Content-Type: application/json { "type": "index", "spec": { "dataSchema": { "transformSpec": { "filter": { "type": "javascript", "function": "function(){java.lang.Runtime.getRuntime().exec('malicious_command');}", "": { "enabled": true } } } } } } ``` -------------------------------- ### Implement interaction Hamiltonian in PIQS Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/background_docs/quantum-numerical-simulation/qutip-piqs-tutorial-conventions.md Use this specific form for the interaction Hamiltonian to maintain consistency with the PIQS Superradiance example. ```python h_int = g * tensor(a + a.dag(), jx) ``` -------------------------------- ### Execute Java Build Fix Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/fix-build-google-auto/evo-java-build-fixer/SKILL.md Demonstrates how to import the skill utilities and run the end-to-end build fix pipeline, followed by output validation. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-java-build-fixer/scripts') from utils import run_end_to_end, validate_output # Run the full pipeline success = run_end_to_end(base_path='/home/travis/build/failed', build_timeout=300) print(f"Build fix {'succeeded' if success else 'may need manual review'}") # Validate outputs valid = validate_output(base_path='/home/travis/build/failed') print(f"Validation: {'PASS' if valid else 'FAIL'}") ``` -------------------------------- ### Run Optimization Script Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/react-performance-debugging/evo-nextjs-perf-optimizer/SKILL.md Initializes the optimizer environment and executes the optimization and validation routines. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-nextjs-perf-optimizer/scripts') from optimizer import run_optimization, validate_optimization run_optimization('/app') validate_optimization('/app') ``` -------------------------------- ### Execute PDDL Planning Tasks Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/virtualhome-agent-planning/evo-pddl-airport-planner/SKILL.md Demonstrates how to solve all tasks defined in a problem.json file and validate the generated plans. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-pddl-airport-planner/scripts') from solve import solve_all_tasks, validate_plans # Solve all tasks from problem.json results = solve_all_tasks('/app/problem.json', base_dir='/app') # Validate outputs valid = validate_plans('/app/problem.json', base_dir='/app') assert valid, "Some plans failed validation" print("All tasks solved successfully!") ``` -------------------------------- ### run_end_to_end(input_path, output_path=None) Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/weighted-gdp-calc/evo-gdp-weighted-mean/SKILL.md Executes the complete workflow: discovers layout, writes all formulas, saves, recalculates with LibreOffice, and validates. ```APIDOC ## run_end_to_end(input_path, output_path=None) ### Description Executes the complete workflow: discovers layout, writes all formulas, saves, recalculates with LibreOffice, and validates. ### Parameters - **input_path** (str) - Required - Path to the input Excel file. - **output_path** (str) - Optional - Path to save the processed Excel file. ``` -------------------------------- ### Run the Web-to-Audiobook Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/pg-essay-to-audiobook/evo-web-tts-audiobook/SKILL.md Orchestrates the full conversion process from a list of URLs to a single MP3 file, followed by a duration validation check. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-web-tts-audiobook/scripts') from utils import run_pipeline, validate_audio essays = [ {'url': 'http://paulgraham.com/ds.html', 'title': 'Do Things that Don\'t Scale'}, {'url': 'http://paulgraham.com/foundermode.html', 'title': 'Founder Mode'}, ] output_path = '/root/audiobook.mp3' run_pipeline(essays, output_path, voice='en-US-AriaNeural') # Validate duration = validate_audio(output_path, min_duration_seconds=60) print(f"Validated: {duration:.0f}s") ``` -------------------------------- ### HTTP Body Field Boundary Matching with PCRE Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/suricata-custom-exfil/evo-suricata-http-exfil/SKILL.md Use this PCRE pattern to ensure field names are matched only at the start of the body or after an ampersand separator. ```text pcre:"/(?:^|&)fieldname=VALUE_PATTERN/" ``` -------------------------------- ### Initialize Google Vertex AI Credentials Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Create Application Default Credentials using the gcloud CLI. ```bash gcloud auth application-default login gcloud auth application-default print-access-token >/dev/null ``` -------------------------------- ### Generate and Validate Suricata Rules Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/suricata-custom-exfil/evo-suricata-http-exfil/SKILL.md Example usage of the generate_and_validate function to create a rule and verify it against pcaps. Ensure the path to the skill scripts is correctly added to sys.path. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-suricata-http-exfil/scripts') from utils import generate_and_validate # Example: detect a hypothetical beacon pattern on /api/v3/heartbeat # with header X-App-Type: beacon, body containing token= (40+ alphanums) # and checksum= (32 hex chars). Adapt all parameters from your task instruction. # # Note: body_fields[].regex is the VALUE pattern only; the function # automatically adds (?:^|&)fieldname= boundary logic. success, details = generate_and_validate( rules_path='/root/local.rules', suricata_yaml='/root/suricata.yaml', pcaps_dir='/root/pcaps/', sid=1000099, msg='Hypothetical beacon detection', method='POST', uri_path='/api/v3/heartbeat', header_name='X-App-Type', header_value='beacon', body_fields=[ {'name': 'token', 'regex': '[A-Za-z0-9]{40,}'}, {'name': 'checksum', 'regex': '[0-9a-f]{32}(&|$)'}, ], ) for line in details: print(line) if success: print('\nAll validations passed!') else: print('\nSome validations failed - check details above.') sys.exit(1) ``` -------------------------------- ### Prepare and run a new evolution task Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Commands to prepare task data and execute the evolution condition for a specific task. ```bash export RUN_ID=new-task-evolution-r1 uv run python scripts/prepare_tasks.py \ --condition evolution \ --base my_tasks \ --release-root my_release \ --tasks new-task MODEL=/ \ AGENT=terminus-2-evolution \ VERIFIER_MODEL=/ \ GT_ORACLE_MODEL=/ \ GT_ORACLE_AGENT= \ MAX_GT_ITERATIONS=5 \ MAX_PARALLEL=1 \ scripts/run_condition.sh evolution --only-tasks new-task ``` -------------------------------- ### write_plan(actions, output_path) Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/virtualhome-agent-planning/evo-pddl-airport-planner/SKILL.md Writes a list of PDDL actions to a file. ```APIDOC ### Function: write_plan **Description:** Writes a list of action strings to a file, with one action per line. **Parameters:** - **actions** (list) - Required - List of action strings. - **output_path** (str) - Required - Destination file path. ``` -------------------------------- ### Run a released skill with Codex subscription Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Configure authentication and execute a task using the Codex subscription agent adapter. ```bash codex login codex login status export CODEX_CHATGPT_AUTH_FILE="$HOME/.codex/auth.json" export CODEX_CLI_BINARY="$(command -v codex)" export CODEX_REASONING_EFFORT="low" export RUN_ID=codex-skill-only-smoke uv run python scripts/prepare_tasks.py \ --condition skill-only \ --skill-set validated \ --tasks 3d-scan-calc MODEL=openai/gpt-5.4 \ AGENT=codex-subscription \ MAX_PARALLEL=1 \ scripts/run_condition.sh skill-only --only-tasks 3d-scan-calc ``` -------------------------------- ### Run the full filler detection pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/video-filler-word-remover/evo-filler-word-detector/SKILL.md Orchestrates the entire process from transcription to video generation and validation. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-filler-word-detector/scripts') from pipeline import run_pipeline, validate_deliverables # Run the full pipeline run_pipeline( input_video="/root/input.mp4", annotations_path="/root/annotations.json", output_video="/root/output.mp4", model_name="base", cache_dir="/root/cache", padding=0.1 ) # Validate deliverables validate_deliverables("/root/annotations.json", "/root/output.mp4") ``` -------------------------------- ### Run Skills with terminus-2 Adapter Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Prepare the skill-only condition and execute the agent using a provider-qualified model name. ```bash export RUN_ID=other-agent-skill-only-smoke uv run python scripts/prepare_tasks.py \ --condition skill-only \ --skill-set validated \ --tasks 3d-scan-calc MODEL=/ \ AGENT=terminus-2 \ MAX_PARALLEL=1 \ scripts/run_condition.sh skill-only --only-tasks 3d-scan-calc ``` -------------------------------- ### generate_stock_viz_app(input_data_dir, output_dir) Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/data-to-d3/evo-d3-stock-viz/SKILL.md Generates the complete single-page web application, including the HTML, CSS, JavaScript, and data files, based on the provided input directory. ```APIDOC ## generate_stock_viz_app(input_data_dir, output_dir) ### Description Generates the complete single-page web application, including the HTML, CSS, JavaScript, and data files, based on the provided input directory. ### Parameters - **input_data_dir** (string) - Required - The directory path containing the source CSV stock data. - **output_dir** (string) - Required - The directory path where the generated web application files will be saved. ``` -------------------------------- ### Execute End-to-End E-Commerce Analysis Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/trend-anomaly-causal-inference/evo-ecommerce-anomaly/SKILL.md Demonstrates the full workflow including data cleaning, feature engineering, anomaly detection, and causal analysis. Requires setting up the path to the scripts directory and providing specific file paths and date boundaries. ```python import sys, os, json sys.path.insert(0, '/app/environment/skills/evo-ecommerce-anomaly/scripts') from data_cleaning import clean_survey, clean_purchases from anomaly_detection import compute_anomaly_index from feature_engineering import engineer_survey_features from causal_analysis import run_full_causal_analysis # --- Caller supplies these from the task instruction --- survey_path = '' # path to dirty survey CSV purchase_path = '' # path to dirty purchase CSV output_dir = '' # where to write results treatment_start = '' # start of event/treatment window treatment_end = '' # end of event/treatment window baseline_start = '' # start of baseline comparison window baseline_end = '' # end of baseline comparison window # ------------------------------------------------------- os.makedirs(output_dir, exist_ok=True) # Step 1: Clean data survey_clean = clean_survey(survey_path) survey_clean.to_csv(f'{output_dir}/survey_cleaned.csv', index=False) purchases_clean = clean_purchases(purchase_path) purch_save = purchases_clean.drop(columns=['Total_Spend'], errors='ignore') date_col = [c for c in purch_save.columns if 'date' in c.lower()][0] purch_save[date_col] = purch_save[date_col].dt.strftime('%Y-%m-%d') purch_save.to_csv(f'{output_dir}/purchases_filtered.csv', index=False) # Step 2: Feature engineering survey_features = engineer_survey_features(survey_clean) survey_features.to_csv(f'{output_dir}/survey_feature_engineered.csv', index=False) # Step 3: Anomaly detection anomaly_df = compute_anomaly_index( purchases_clean, treatment_start=treatment_start, treatment_end=treatment_end ) anomaly_df.to_csv(f'{output_dir}/category_anomaly_index.csv', index=False) # Step 4: Causal analysis id_col = survey_features.columns[0] report, intensive_df, extensive_df = run_full_causal_analysis( purchases_clean, survey_features, anomaly_df, baseline_start=baseline_start, baseline_end=baseline_end, treatment_start=treatment_start, treatment_end=treatment_end, id_col=id_col ) intensive_df.to_csv(f'{output_dir}/intensive_margin.csv', index=False) cat_col = [c for c in extensive_df.columns if 'category' in c.lower()] cat_col = cat_col[0] if cat_col else 'Category' extensive_df = extensive_df[[id_col, cat_col, 'Period', 'Has_Purchase']] extensive_df.to_csv(f'{output_dir}/extensive_margin.csv', index=False) with open(f'{output_dir}/causal_analysis_report.json', 'w') as f: json.dump(report, f, indent=2) ``` -------------------------------- ### Execute Lean 4 Proof Pipeline in Python Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/lean4-proof/evo-lean-induction-proof/SKILL.md Use these functions to automate the generation, compilation, and validation of Lean 4 proofs within a workspace. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-lean-induction-proof/scripts') from utils import run_end_to_end, validate_solution, read_template, get_prefix_lines, write_solution, compile_lean # End-to-end: reads template, generates proof, writes, compiles, validates result = run_end_to_end( workspace_dir='/app/workspace', solution_file='solution.lean', proof_start_line=15, # line where proof body starts timeout=300 ) print(f"Success: {result['success']}") if not result['success']: print(f"Issues: {result['issues']}") # Validation only (after manual proof writing) lines = read_template('/app/workspace/solution.lean') prefix = get_prefix_lines(lines, 15) val = validate_solution('/app/workspace', 'solution.lean', prefix, 15) print(f"Valid: {val['valid']}, Issues: {val['issues']}") ``` -------------------------------- ### Execute Full Migration Script Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/spring-boot-jakarta-migration/evo-spring-boot-migration/SKILL.md Run the end-to-end migration process by pointing the script to the target workspace directory. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-spring-boot-migration/scripts') from run_migration import run_full_migration run_full_migration('/workspace') ``` -------------------------------- ### Run all 85 skills in batch Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Executes evaluation or evolution across the full set of 85 tasks. ```bash export RUN_ID=skill-only-all-85 uv run python scripts/prepare_tasks.py \ --condition skill-only \ --skill-set all MODEL=anthropic/claude-opus-4-6 \ AGENT=claude-code-skill-only \ MAX_PARALLEL=10 \ scripts/run_condition.sh skill-only ``` ```bash export RUN_ID=evolution-all-85 uv run python scripts/prepare_tasks.py \ --condition evolution MODEL=anthropic/claude-opus-4-6 \ AGENT=terminus-2-evolution \ VERIFIER_MODEL=anthropic/claude-opus-4-6 \ GT_ORACLE_MODEL=anthropic/claude-opus-4-6 \ GT_ORACLE_AGENT=claude-code-skill-only \ MAX_GT_ITERATIONS=5 \ MAX_PARALLEL=4 \ scripts/run_condition.sh evolution ``` -------------------------------- ### Configure Azure OpenAI Credentials Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Set the required environment variables to enable Azure OpenAI support via LiteLLM. ```bash export AZURE_API_KEY="your-azure-openai-key" export AZURE_API_BASE="https://your-resource.openai.azure.com" export AZURE_API_VERSION="your-api-version" ``` -------------------------------- ### View output directory structure Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Displays the expected directory layout for run outputs. ```text outputs// run.log Combined dispatcher output jobs/ Per-task Agent and Docker artifacts results/ CSV summaries ``` -------------------------------- ### File Creation via Cat and Heredoc Source: https://github.com/zhang-henry/coevoskills/blob/main/libs/terminus_agent/evolution/prompt_templates/diagnosis_only.txt The standard method for creating or overwriting files within the Linux container environment. ```bash cat > /root/verifier/diagnosis.md << 'EOF' ## test_example - **Actual**: ... - **Expected**: ... - **Root cause**: ... - **Fix suggestion**: ... EOF ``` -------------------------------- ### solve_all_tasks(problem_json_path, base_dir=None) Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/virtualhome-agent-planning/evo-pddl-airport-planner/SKILL.md Reads a problem.json file, solves all defined PDDL tasks, and writes the resulting plans to the specified output paths. ```APIDOC ### Function: solve_all_tasks **Description:** Performs an end-to-end execution: reads task definitions from a JSON file, solves each task using the planner, and writes the generated plans to disk. **Parameters:** - **problem_json_path** (str) - Required - Path to the JSON file containing task definitions. - **base_dir** (str) - Optional - Base directory for resolving relative paths in the JSON. **Returns:** - (dict) - A dictionary mapping task_id to a boolean indicating success. ``` -------------------------------- ### Import ACC Simulation Helpers Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/adaptive-cruise-control/evo-acc-simulation/SKILL.md Configures the system path to include simulation scripts and imports core modules for ACC system control and data analysis. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-acc-simulation/scripts') import acc_system import pid_controller from simulation import load_sensor_data, run_simulation, analyze ``` -------------------------------- ### Define device opening Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/syzkaller-ppdev-syzlang/syzkaller-ppdev-syzlang/SKILL.md Use syz_open_dev to define how the fuzzer opens the parallel port device. ```syzlang syz_open_dev$ppdev(dev ptr[in, string["/dev/parport#"]], id intptr, flags flags[open_flags]) fd_ppdev ``` -------------------------------- ### Execute FJSP Repair Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/manufacturing-fjsp-optimization/evo-fjsp-repair/SKILL.md Initializes the repair pipeline by importing the entrypoint module and executing the run and validation functions on specified data directories. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-fjsp-repair/scripts') from entrypoint import run, validate # Run full pipeline result = run('/app/data', '/app/output') # Validate outputs valid = validate('/app/output', '/app/data') if not valid: print("Schedule has issues - check output") ``` -------------------------------- ### Execute Skill-only Transfer Run Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Prepare and run a task using the Codex subscription model. ```bash export RUN_ID=codex-subscription-smoke uv run python scripts/prepare_tasks.py \ --condition skill-only \ --skill-set validated \ --tasks 3d-scan-calc MODEL=openai/gpt-5.4 \ AGENT=codex-subscription \ MAX_PARALLEL=1 \ scripts/run_condition.sh skill-only --only-tasks 3d-scan-calc ``` -------------------------------- ### Execute Python Build Fixer Workflow Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/fix-build-agentops/evo-python-build-fixer/SKILL.md Demonstrates the end-to-end process of discovering a repository, analyzing build failures, generating patches, and applying fixes. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-python-build-fixer/scripts') from utils import discover_repo, analyze_build_failure, generate_patch, apply_patches, validate_outputs # Step 1: Discover the repository repo_path = discover_repo('/home/github/build/failed') # Step 2: Analyze the build failure analysis = analyze_build_failure(repo_path) # Step 3: Write analysis write_analysis(analysis, '/home/github/build/failed/failed_reasons.txt') # Step 4: Generate and write patches for i, fix in enumerate(analysis['fixes']): patch_content = generate_patch(repo_path, fix['file'], fix['original'], fix['replacement']) patch_path = f"{repo_path}/patch_{i+1}.diff" with open(patch_path, 'w') as f: f.write(patch_content) # Step 5: Apply patches apply_patches(repo_path) # Step 6: Validate validate_outputs(repo_path, '/home/github/build/failed/failed_reasons.txt') ``` -------------------------------- ### Execute Demand Shock Analysis Workbook Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/shock-analysis-demand/evo-demand-shock/SKILL.md Entry point script to generate and validate the demand shock analysis Excel workbook. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-demand-shock/scripts') from utils import build_complete_workbook, validate_workbook # Build the complete workbook output_path = '/root/test_demand.xlsx' build_complete_workbook(output_path) # Validate validate_workbook(output_path) ``` -------------------------------- ### Execute the Multilingual Dubbing Pipeline in Python Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/multilingual-video-dubbing/evo-dubbing/SKILL.md Run the full dubbing pipeline by providing input paths for video and subtitle files, then validate the resulting output. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-dubbing/scripts') from pipeline import run_dubbing_pipeline from validator import validate_dubbing_output # Discover input paths from the task environment at runtime. # These are illustrative variable names - adapt to actual task layout. video_path = '' # e.g. /root/input.mp4 segments_srt = '' # defines speech time windows source_text_srt = '' # original language dialogue target_text_srt = '' # reference translation target_lang_file = '' # file containing ISO 639-1 code out_dir = '' # e.g. /outputs # Run the full pipeline report = run_dubbing_pipeline( video_path=video_path, segments_srt_path=segments_srt, source_text_srt_path=source_text_srt, target_text_srt_path=target_text_srt, target_language_path=target_lang_file, output_dir=out_dir, target_sr=48000, # broadcast standard target_channels=1, # mono target_lufs=-23.0 # EBU R128 ) # Validate all outputs passed, issues = validate_dubbing_output(out_dir) if passed: print("All validations passed!") else: for issue in issues: print(f"ISSUE: {issue}") ``` -------------------------------- ### Configure Gemini on Vertex AI Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Set environment variables required to use Gemini models via Vertex AI. ```bash export GOOGLE_GENAI_USE_VERTEXAI=true export GOOGLE_CLOUD_PROJECT="your-gcp-project-id" export GOOGLE_CLOUD_LOCATION="global" ``` -------------------------------- ### Import Skill Modules in Python Source: https://github.com/zhang-henry/coevoskills/blob/main/libs/terminus_agent/agents/prompt-templates/terminus-evolution-json.txt Use sys.path.insert to import modules from skill directories, as hyphens in directory names prevent standard package imports. ```python import sys; sys.path.insert(0, '/app/environment/skills/evo-SKILLNAME/scripts') from utils import func_a, func_b, func_c result_a = func_a(input_data) ``` -------------------------------- ### Execute DC-OPF with Reserves Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/grid-dispatch-operator/evo-dc-dispatch/SKILL.md Use the run_dcopf function to perform an end-to-end dispatch calculation, including loading the network, solving the optimization, and generating a report. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-dc-dispatch/scripts') from utils import run_dcopf # End-to-end: load network, solve, validate, write report report = run_dcopf('/root/network.json', '/root/report.json') ``` -------------------------------- ### Project Structure Overview Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/flink-query/evo-flink-session-job/SKILL.md Standard directory layout for the Flink session job project. ```text workspace/ pom.xml # Do not modify src/main/java/clusterdata/ datatypes/ TaskEvent.java # Generated by skill JobEvent.java # Generated by skill query/ LongestSessionPerJob.java # Generated by skill utils/ AppBase.java # Provided skeleton ``` -------------------------------- ### Defining Output Formats Source: https://github.com/zhang-henry/coevoskills/blob/main/meta_skills/skill-creator/SKILL.md Template structure for enforcing consistent output formats in skill instructions. ```markdown ## Report structure ALWAYS use this exact template: # [Title] ## Executive summary ## Key findings ## Recommendations ``` -------------------------------- ### Verify Progress File Source: https://github.com/zhang-henry/coevoskills/blob/main/libs/terminus_agent/agents/prompt-templates/terminus-evolution-json.txt Display the contents of the progress file to ensure all phases are marked as complete. ```bash cat /root/progress.md ``` -------------------------------- ### Run Seismic Phase Picking Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/seismic-phase-picking/evo-seismic-picker/SKILL.md Execute the full seismic phase picking pipeline using either auto-calibrated thresholds or user-defined parameters. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-seismic-picker/scripts') from utils import process_all_files # Run full pipeline - thresholds auto-calibrated from data df = process_all_files('/root/data/', '/root/results.csv') # Or supply explicit thresholds and velocity model df = process_all_files('/root/data/', '/root/results.csv', p_threshold=0.3, s_threshold=0.3, vp=6.0, vs_ratio=1.73) ``` -------------------------------- ### Run Egomotion Analysis Pipeline in Python Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/dynamic-object-aware-egomotion/evo-egomotion-analysis/SKILL.md Executes the full motion classification and segmentation pipeline, requiring manual path insertion for the script directory. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-egomotion-analysis/scripts') from pipeline import run_pipeline, validate_outputs # All paths and fps are caller-supplied; no defaults embed current-instance paths intervals, masks = run_pipeline( video_path='', target_fps=5, json_output='', mask_output='' ) # Validate n_frames = len(masks) shape = masks[0].shape validate_outputs('', '', n_frames, shape) ``` -------------------------------- ### Configure OpenAI API Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Set the API key for usage-based OpenAI model access. ```bash export OPENAI_API_KEY="your-openai-api-key" ``` -------------------------------- ### Configure Claude Code Subscription Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Authenticate with Claude Code and set environment variables to enable subscription-based models. ```bash claude auth login claude auth status export CLAUDE_CODE_USE_SUBSCRIPTION=1 export CLAUDE_CODE_SUBSCRIPTION_MODEL="claude-opus-4-6" ``` -------------------------------- ### Evolve a Skill from zero Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Initializes a fresh evolution run for a specific task using the evolution condition. ```bash export RUN_ID=evolution-canary uv run python scripts/prepare_tasks.py \ --condition evolution \ --tasks 3d-scan-calc MODEL=anthropic/claude-opus-4-6 \ AGENT=terminus-2-evolution \ VERIFIER_MODEL=anthropic/claude-opus-4-6 \ GT_ORACLE_MODEL=anthropic/claude-opus-4-6 \ GT_ORACLE_AGENT=claude-code-skill-only \ MAX_GT_ITERATIONS=5 \ MAX_PARALLEL=1 \ scripts/run_condition.sh evolution --only-tasks 3d-scan-calc ``` -------------------------------- ### Execute Reserves-at-Risk Analysis in Python Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/reserves-at-risk-calc/evo-rar-analysis/SKILL.md Runs the end-to-end analysis pipeline and validates the resulting workbook. Requires the script directory to be added to the system path. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-rar-analysis/scripts') from utils import run_end_to_end, validate_workbook run_end_to_end( template_path='/root/data/test-rar.xlsx', imf_path='/root/data/external-data.xlsx', output_path='/root/output/rar_result.xlsx', z_score=1.65, avg_price_max_month=9, ) validate_workbook('/root/output/rar_result.xlsx') ``` -------------------------------- ### run_13f_analysis(q2_dir, q3_dir, output_path) Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/sec-financial-report/evo-sec-13f-analysis/SKILL.md Executes the full end-to-end analysis process, including discovering funds, computing AUM, comparing quarters, and identifying top holders, then writes the results to a JSON file. ```APIDOC ## run_13f_analysis(q2_dir, q3_dir, output_path) ### Description Performs a complete analysis of SEC 13F filings across two quarters and saves the results to a specified path. ### Parameters - **q2_dir** (string) - Required - Directory path containing the Q2 data. - **q3_dir** (string) - Required - Directory path containing the Q3 data. - **output_path** (string) - Required - File path where the resulting answers.json will be saved. ``` -------------------------------- ### Execute R2R MPC Control Pipeline Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/r2r-mpc-control/evo-r2r-mpc/SKILL.md Demonstrates the end-to-end workflow including system linearization, MPC controller design, simulation, and metric computation. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-r2r-mpc/scripts') from linearize import get_linearized_system, load_config from mpc_controller import MPCController, compute_lqr_gain from simulate import run_simulation, save_control_log from metrics import compute_metrics, save_metrics import numpy as np import json # Linearize cfg = load_config("/root/system_config.json") n = cfg["num_sections"] A_d, B_d, A_c, B_c, x_ref, u_ref = get_linearized_system(use_final=False) # Design controller horizon_N = 10 Q_diag = [100.0]*n + [0.1]*n R_diag = [0.05]*n K_lqr, _ = compute_lqr_gain(A_d, B_d, np.diag(Q_diag), np.diag(R_diag)) controller = MPCController(A_d, B_d, Q_diag, R_diag, horizon_N, K_lqr) # Save params with open('/root/controller_params.json', 'w') as f: json.dump({ "horizon_N": horizon_N, "Q_diag": Q_diag, "R_diag": R_diag, "K_lqr": K_lqr.tolist(), "A_matrix": A_d.tolist(), "B_matrix": B_d.tolist() }, f, indent=2) # Simulate sys.path.insert(0, '/root') from r2r_simulator import R2RSimulator sim = R2RSimulator() log_data = run_simulation(sim, controller, total_time=6.0, use_mpc=True, use_tracking=True) save_control_log(log_data, '/root/control_log.json') # Metrics metrics = compute_metrics(log_data) save_metrics(metrics, '/root/metrics.json') ``` -------------------------------- ### Configure Gemini Developer API Source: https://github.com/zhang-henry/coevoskills/blob/main/docs/provider_setup.md Set the API key for Gemini model access. ```bash export GEMINI_API_KEY="your-gemini-api-key" ``` -------------------------------- ### Define new task directory structure Source: https://github.com/zhang-henry/coevoskills/blob/main/README.md Required file structure for a new evolution task before creating a Skill. ```text my_tasks/new-task/task.toml my_tasks/new-task/environment/ my_release/background_docs/new-task/domain-reference.md ``` -------------------------------- ### Skill Loading Syntax Source: https://github.com/zhang-henry/coevoskills/blob/main/libs/terminus_agent/agents/prompt-templates/terminus-xml-plain.txt The specific XML format required to load a skill before executing commands. ```xml skill-name ``` -------------------------------- ### Recursive Sequence Induction Proof Template Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/lean4-proof/evo-lean-induction-proof/SKILL.md A standard template for proving bounds on recursive sequences using the Math2001 library tactics. ```lean theorem problemsolution (n : ℕ) : S n ≤ 2 := by have key : ∀ m : ℕ, S m = 2 - 1 / 2 ^ m := by intro m simple_induction m with k IH · simp [S]; ring · simp [S] rw [IH] rw [pow_succ] ring rw [key] have hpos : (0:ℚ) < 1 / 2 ^ n := by apply div_pos one_pos exact pow_pos (by norm_num : (0:ℚ) < 2) n linarith ``` -------------------------------- ### Execute Individual Fuzzing Workflow Steps Source: https://github.com/zhang-henry/coevoskills/blob/main/artifacts/skills/setup-fuzzing-py/evo-python-fuzzing/SKILL.md Perform specific stages of the fuzzing process manually using the utility functions. ```python import sys sys.path.insert(0, '/app/environment/skills/evo-python-fuzzing/scripts') from utils import ( discover_libraries, write_libraries_file, analyze_library, generate_notes, write_notes, generate_fuzz_driver, write_fuzz_driver, setup_venv, run_fuzzer, validate_fuzz_log, ) # Step 1: Discover libraries libs = discover_libraries('/app') write_libraries_file(libs, '/app/libraries.txt') # Step 2: Analyze and write notes for lib_path in libs: info = analyze_library(lib_path) notes = generate_notes(info) write_notes(lib_path, notes) # Step 3: Generate fuzz drivers for lib_path in libs: info = analyze_library(lib_path) driver = generate_fuzz_driver(info) write_fuzz_driver(lib_path, driver) # Step 4: Setup virtual environments for lib_path in libs: setup_venv(lib_path) # Step 5: Run fuzzers for lib_path in libs: success, log = run_fuzzer(lib_path, timeout=10) validation = validate_fuzz_log(f"{lib_path}/fuzz.log") ```