### Installation Instructions Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/README.md Provides commands to clone the repository, navigate into the directory, and install dependencies. ```bash git clone https://github.com/muratcankoylan/llm-as-judge-skills.git cd llm-as-judge-skills npm install ``` -------------------------------- ### AI SDK 6: Input Examples for Model Guidance Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/skills/tool-design/tool-design.md Provides `inputExamples` to guide the model in generating correct inputs for a tool. This is particularly useful for complex tools where understanding the expected structure and values is crucial. ```typescript const complexTool = tool({ description: "Create a calendar event", parameters: eventSchema, inputExamples: [ { title: "Team Standup", date: "2024-01-15", time: "09:00", duration: 30, attendees: ["alice@example.com", "bob@example.com"] } ], execute: async (input) => { ... } }); ``` -------------------------------- ### Install Plugin via Command Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/README.md Directly install the 'context-engineering' plugin using this command. This installs all 15 skills. ```bash /plugin install context-engineering@context-engineering-marketplace ``` -------------------------------- ### Clone and Install Project Dependencies Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/CONTRIBUTING.md Steps to clone the repository, navigate into the project directory, and install necessary npm packages. ```bash git clone https://github.com/muratcankoylan/llm-as-judge-skills.git cd llm-as-judge-skills npm install ``` -------------------------------- ### Install and Run Router Benchmark Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/researcher/benchmarks/router/results-published/2026-05-19.md Installs dependencies and runs the router benchmark using specified models and configurations. Ensure CURSOR_API_KEY is set. ```bash cd researcher/benchmarks/sdk-runner npm install export CURSOR_API_KEY= node --experimental-strip-types src/runRouter.ts --models claude-opus-4-7,composer-2,gemini-3.1-pro,gpt-5.5 --reps 3 --seed 1 --max-budget-usd 15 ``` -------------------------------- ### Install Reasoning Trace Optimizer Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/README.md Install the Reasoning Trace Optimizer package. Navigate to the project directory first. ```bash cd examples/interleaved-thinking pip install -e . ``` -------------------------------- ### Example Agent Prompt After Optimization Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/SKILL.md This example demonstrates an optimized system prompt and guidelines, showing improved performance with issues resolved and a higher score. ```text System: You are a research assistant focused on finding accurate information. IMPORTANT GUIDELINES: - Always verify search results before summarizing - If a tool returns an error, try an alternative approach - Keep track of your original goal throughout the task - Validate findings against multiple sources when possible Issue: None Score: 85/100 Patterns: None detected ``` -------------------------------- ### Example Agent Prompt Before Optimization Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/SKILL.md This example shows a system prompt and issue description before optimization, highlighting potential problems like tool confusion and goal abandonment. ```text System: You are a helpful assistant. Issue: Agent called wrong tools, lost track of goal after 3 turns Score: 45/100 Patterns: tool_confusion, goal_abandonment ``` -------------------------------- ### Install Anthropic SDK Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/docs/m2-1.md Install the Anthropic SDK for Python or Node.js. ```Python pip install anthropic ``` ```Node.js npm install @anthropic-ai/sdk ``` -------------------------------- ### Configuration Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/README.md Shows how to set up the environment variables for API keys and model selection. ```bash OPENAI_API_KEY=your_openai_api_key_here OPENAI_MODEL=gpt-5.2 ``` -------------------------------- ### Install and Run Router Benchmark Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/researcher/benchmarks/router/README.md Navigate to the SDK runner directory, install dependencies, and then run the router benchmark. Use `router:dry-run` to preview the plan and cost, and `router:run` to execute the benchmark, specifying a maximum budget. ```bash cd researcher/benchmarks/sdk-runner npm install npm run router:dry-run # see the plan and cost forecast npm run router:run -- --max-budget-usd 5 # execute (after exporting CURSOR_API_KEY) ``` -------------------------------- ### Install Dependencies and Set API Key Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/researcher/benchmarks/sdk-runner/README.md Navigate to the SDK runner directory, install npm packages, and set the CURSOR_API_KEY environment variable. ```bash cd researcher/benchmarks/sdk-runner npm install export CURSOR_API_KEY="cursor_..." ``` -------------------------------- ### Build Examples Function Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/SKILL.md Constructs multiple training examples from a given chunk of text, instruction, and author, varying system prompts and templates. ```python def build_examples(chunk, instruction, author, variants=2): examples = [] for i in range(variants): system = SYSTEM_PROMPTS[i % len(SYSTEM_PROMPTS)] template = PROMPT_TEMPLATES[(chunk.id + i) % len(PROMPT_TEMPLATES)] user = template.format(author=author, desc=instruction) examples.append({"messages": [ {"role": "system", "content": system}, {"role": "user", "content": user}, {"role": "assistant", "content": chunk.text} ]}) return examples ``` -------------------------------- ### Install Book SFT Pipeline Plugin Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/README.md Use these commands to add the marketplace and install the book-sft-pipeline plugin for Claude Code environments. ```bash # Add the marketplace first /plugin marketplace add muratcankoylan/Agent-Skills-for-Context-Engineering # Install the book-sft-pipeline plugin /plugin install book-sft-pipeline@context-engineering-marketplace ``` -------------------------------- ### Install macOS Daemon Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/CLAUDE.md Install the macOS daemon for continuous operation using launchd. ```bash researcher/orchestration/launchd/install.sh ``` -------------------------------- ### Install Tinker Cookbook Locally Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt Clone the tinker-cookbook repository and install it locally in editable mode. This is recommended for development and code browsing. ```bash git clone https://github.com/thinking-machines-lab/tinker-cookbook.git cd tinker-cookbook # Switch to your virtual environment pip install -e . ``` -------------------------------- ### Complete Example: Sampling from an Image Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt This example demonstrates how to create a training client, save weights for sampling, and perform sampling from an image input. It requires the requests and transformers libraries. The ModelInput is constructed with image data and encoded text chunks. ```python import requests import tinker from transformers import AutoTokenizer model_name = "Qwen/Qwen3-VL-30B-A3B-Instruct" tokenizer = AutoTokenizer.from_pretrained(model_name) service_client = tinker.ServiceClient() training_client = await service_client.create_lora_training_client_async(base_model=model_name, rank=32) sampling_client = await training_client.save_weights_and_get_sampling_client_async(name="sampler") # Grab an image and ask a question image_data = requests.get("https://thinkingmachines.ai/blog/on-policy-distillation/images/chess.png").content model_input = tinker.ModelInput(chunks=[ tinker.types.EncodedTextChunk(tokens=tokenizer.encode("<|im_start|>user\n<|vision_start|>")), tinker.types.ImageChunk(data=image_data, format="png"), tinker.types.EncodedTextChunk(tokens=tokenizer.encode("<|vision_end|>What is this?<|im_end|>\n<|im_start|>assistant\n")) ]) result = await sampling_client.sample_async(prompt=model_input, num_samples=1, sampling_params=tinker.types.SamplingParams(max_tokens=100)) print(tokenizer.decode(result.sequences[0].tokens)) ``` -------------------------------- ### Install Tinker SDK Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt Install the Tinker SDK using pip. This command makes both the Python SDK and the tinker CLI available for use. ```bash pip install tinker ``` -------------------------------- ### Direct Score Tool Usage Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/tools/evaluation/direct-score.md Demonstrates how to use the directScore tool by providing a sample response, prompt, criteria, and rubric configuration. The example shows the expected input format for executing the evaluation. ```typescript const result = await directScore.execute({ response: "Machine learning is a subset of AI that enables systems to learn from data...", prompt: "Explain machine learning to a beginner", criteria: [ { name: "Accuracy", description: "Technical correctness of explanations", weight: 0.4 }, { name: "Clarity", description: "Understandable for a beginner", weight: 0.3 }, { name: "Completeness", description: "Covers key concepts adequately", weight: 0.3 } ], rubric: { scale: "1-5", levelDescriptions: { "1": "Completely fails criterion", "2": "Major deficiencies", "3": "Acceptable but improvable", "4": "Good with minor issues", "5": "Excellent, no issues" } } }); ``` -------------------------------- ### Prompt Design: Guard Rails Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/prompts/index.md Example of including guard rails and constraints in a prompt. This guides the LLM to avoid common pitfalls and biases. ```markdown Important Guidelines: - Do NOT prefer responses simply because they are longer - Do NOT prefer responses based on their position (A vs B) - Focus on the specified criteria ``` -------------------------------- ### BDI Temporal Interval Examples Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/skills/bdi-mental-states/references/rdf-examples.md Illustrates the representation of temporal intervals for planning and execution. Use these to specify start and end times for events or processes. ```turtle ex:TimeInterval_morning_commute a bdi:TimeInterval ; bdi:hasStartTime "2026-01-04T08:30:00"^^xsd:dateTime ; bdi:hasEndTime "2026-01-04T09:00:00"^^xsd:dateTime . ``` ```turtle ex:TimeInterval_planning_phase a bdi:TimeInterval ; bdi:hasStartTime "2026-01-04T08:31:00"^^xsd:dateTime ; bdi:hasEndTime "2026-01-04T08:34:00"^^xsd:dateTime . ``` ```turtle ex:TimeInterval_execution a bdi:TimeInterval ; bdi:hasStartTime "2026-01-04T08:35:00"^^xsd:dateTime ; bdi:hasEndTime "2026-01-04T08:52:00"^^xsd:dateTime . ``` -------------------------------- ### Chain-of-Thought Prompting Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/optimization_artifacts/iteration_2/trace.txt Use this structure to encourage step-by-step reasoning for complex tasks. It guides the model through identifying the problem, breaking it down, solving components, and combining results. ```plaintext Solve this problem step by step: 1. First, identify what is being asked 2. Break down into components 3. Solve each component 4. Combine results ``` -------------------------------- ### Thread Call to Action (CTA) Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/content/templates/thread.md The final tweet in a thread, guiding the audience on what to do next. This could include actions like following, subscribing, replying, or sharing. ```text [PLACEHOLDER: Follow, subscribe, reply, share, etc.] ``` -------------------------------- ### Add Individual Skill to Claude Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/README.md Copy a single skill's SKILL.md file into your project's .claude/skills/ directory to use it without installing the full plugin. This example adds the 'context-fundamentals' skill. ```bash # Example: add just the context-fundamentals skill mkdir -p .claude/skills curl -o .claude/skills/context-fundamentals.md \ https://raw.githubusercontent.com/muratcankoylan/Agent-Skills-for-Context-Engineering/main/skills/context-fundamentals/SKILL.md ``` -------------------------------- ### Initialize BDIMentalStateStore with Triple Store Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/skills/bdi-mental-states/references/framework-integration.md Initializes the BDIMentalStateStore by configuring the SPARQLUpdateStore with endpoint URLs for queries and updates, and setting up an RDF graph. ```python from rdflib import Graph from rdflib.plugins.stores.sparqlstore import SPARQLUpdateStore class BDIMentalStateStore: def __init__(self, endpoint: str): self.store = SPARQLUpdateStore() self.store.open((endpoint + "/query", endpoint + "/update")) self.graph = Graph(store=self.store, identifier="http://example.org/bdi") ``` -------------------------------- ### Dynamic Skill Loading Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/skills/filesystem-context/SKILL.md Lists available skills with brief descriptions. Full skill files should be loaded dynamically using `read_file` when a task requires them, optimizing token usage. ```text Available skills (load with read_file when relevant): - database-optimization: Query tuning and indexing strategies - api-design: REST/GraphQL best practices - testing-strategies: Unit, integration, and e2e testing patterns ``` -------------------------------- ### Writer Agent Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/tools/orchestration/delegate-to-agent.md Example of delegating a task to the 'writer' agent to create an executive summary. ```APIDOC ## Writer Agent ### Description Write an executive summary of research findings. ### Method `delegateToAgent.execute` ### Parameters - **agentName** (enum: "writer") - Required - Name of the agent to delegate to. - **task** (string) - Required - Clear description of what the agent should do. - **context** (object) - Optional - Context and dependencies. - **previousOutputs** (array[string]) - Optional - Outputs from prior steps this agent needs. - **constraints** (array[string]) - Optional - Requirements or limitations to observe. - **expectedOutput** (object) - Optional - Output requirements. - **format** (enum: "text") - Required - Expected output format. - **maxLength** (number) - Optional - Maximum length constraint. ### Request Example ```typescript await delegateToAgent.execute({ agentName: "writer", task: "Write an executive summary of these research findings", context: { previousOutputs: [researchFindings], constraints: ["Maximum 500 words", "Non-technical audience"] }, expectedOutput: { format: "text", maxLength: 2500 } }); ``` ``` -------------------------------- ### Run Basic Supervised Learning Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt Execute a basic supervised learning fine-tuning script from the command line. This script fine-tunes a Llama-3.1-8B model on the NoRobots dataset. ```bash python -m tinker_cookbook.recipes.sl_basic ``` -------------------------------- ### Evaluator Agent Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/tools/orchestration/delegate-to-agent.md Example of delegating a task to the 'evaluator' agent to assess response quality. ```APIDOC ## Evaluator Agent ### Description Evaluate the quality of a response against accuracy and clarity criteria. ### Method `delegateToAgent.execute` ### Parameters - **agentName** (enum: "evaluator") - Required - Name of the agent to delegate to. - **task** (string) - Required - Clear description of what the agent should do. - **context** (object) - Optional - Context and dependencies. - **documents** (array[string]) - Optional - Relevant documents or data. - **constraints** (array[string]) - Optional - Requirements or limitations to observe. - **expectedOutput** (object) - Optional - Output requirements. - **format** (enum: "structured") - Required - Expected output format. ### Request Example ```typescript await delegateToAgent.execute({ agentName: "evaluator", task: "Evaluate the quality of this response against accuracy and clarity criteria", context: { documents: [responseToEvaluate], constraints: ["Use 1-5 scale", "Include justification"] }, expectedOutput: { format: "structured" } }); ``` ``` -------------------------------- ### Voice in Action: Example 2 Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/identity/voice.md Provide a second example of content that captures your voice well. ```text [PLACEHOLDER: Another example] ``` -------------------------------- ### Example File Organization for Agent Discoverability Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/skills/filesystem-context/SKILL.md This structure demonstrates how to organize files and directories to facilitate agent navigation and discoverability. Use consistent naming conventions and include timestamps or IDs in scratch files for disambiguation. ```text project/ scratch/ # Temporary working files tool_outputs/ # Large tool results plans/ # Active plans and checklists memory/ # Persistent learned information preferences.yaml # User preferences patterns.md # Learned patterns skills/ # Loadable skill definitions agents/ # Sub-agent workspaces ``` -------------------------------- ### Analyst Agent Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/tools/orchestration/delegate-to-agent.md Example of delegating a task to the 'analyst' agent to analyze trade-offs between scoring methods. ```APIDOC ## Analyst Agent ### Description Analyze the trade-offs between direct scoring and pairwise comparison. ### Method `delegateToAgent.execute` ### Parameters - **agentName** (enum: "analyst") - Required - Name of the agent to delegate to. - **task** (string) - Required - Clear description of what the agent should do. - **context** (object) - Optional - Context and dependencies. - **documents** (array[string]) - Optional - Relevant documents or data. - **constraints** (array[string]) - Optional - Requirements or limitations to observe. - **expectedOutput** (object) - Optional - Output requirements. - **format** (enum: "structured") - Required - Expected output format. ### Request Example ```typescript await delegateToAgent.execute({ agentName: "analyst", task: "Analyze the trade-offs between direct scoring and pairwise comparison", context: { documents: [evaluationData], constraints: ["Quantitative where possible"] }, expectedOutput: { format: "structured" } }); ``` ``` -------------------------------- ### User Query and Model's Thinking Process for Tool Selection Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/interleaved-thinking/docs/interleavedthinking.md This example illustrates a user's query about the weather and the model's internal thought process for selecting and preparing to use the 'get_weather' tool. It details the reasoning behind choosing the tool and formatting the location argument. ```nushell 👤 User> How's the weather in San Francisco? 💬 Model> Alright, the user is asking about the weather in San Francisco. This is a straightforward request that I can handle using the tools provided to me. I see that I have access to a tool called "get_weather" which can provide weather information for a location. Looking at the parameters, it requires a "location" parameter which should be a string in the format of "city and state, e.g. San Francisco, US". In this case, the user has already specified the location as "San Francisco", which is a major city in California, US. I need to format this properly for the tool call. Following the example format in the tool description, I should format it as "San Francisco, US". The user didn't specify any other parameters or requirements, so a simple weather query should be sufficient. I don't need to ask for clarification since they've provided a clear location. Let me prepare the tool call to get the weather information for San Francisco. I'll use the "get_weather" tool with the location parameter set to "San Francisco, US". This should return the current weather conditions for San Francisco, which is what the user is asking about. Once I get the weather information back from the tool, I'll be able to provide the user with details about the current weather in San Francisco, such as temperature, conditions (sunny, cloudy, rainy, etc.), and possibly other relevant information like humidity or wind speed if that data is available. So I'll proceed with making the tool call to get_weather with the location parameter. 🔧 Tool> get_weather(San Francisco, US) ``` -------------------------------- ### Voice in Action: Example 1 Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/identity/voice.md Paste an example post or content that effectively represents your defined voice. ```text [PLACEHOLDER: Paste an example post/content that represents your voice] ``` -------------------------------- ### Researcher Agent Example Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/tools/orchestration/delegate-to-agent.md Example of delegating a task to the 'researcher' agent to gather information on LLM evaluation best practices. ```APIDOC ## Researcher Agent ### Description Research current best practices for LLM evaluation. ### Method `delegateToAgent.execute` ### Parameters - **agentName** (enum: "researcher") - Required - Name of the agent to delegate to. - **task** (string) - Required - Clear description of what the agent should do. - **context** (object) - Optional - Context and dependencies. - **constraints** (array[string]) - Optional - Requirements or limitations to observe. - **expectedOutput** (object) - Optional - Output requirements. - **format** (enum: "markdown") - Required - Expected output format. ### Request Example ```typescript await delegateToAgent.execute({ agentName: "researcher", task: "Research current best practices for LLM evaluation", context: { constraints: ["Focus on 2024 publications", "Include citations"] }, expectedOutput: { format: "markdown" } }); ``` ``` -------------------------------- ### Organizing System Prompts Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/skills/context-fundamentals/SKILL.md This example demonstrates how to structure system prompts by placing critical information at the beginning and end, and using explicit section boundaries for better model parsing. It follows a common pattern for defining background, instructions, and output expectations. ```markdown You are a Python expert helping a development team. Current project: Data processing pipeline in Python 3.9+ - Write clean, idiomatic Python code - Include type hints for function signatures - Add docstrings for public functions - Follow PEP 8 style guidelines Provide code blocks with syntax highlighting. Explain non-obvious decisions in comments. ``` -------------------------------- ### ID Generation Convention Examples Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/references/file-formats.md Examples of ID generation conventions using type, date, and time or a unique slug. ```text - `idea_20241229_143022` - `contact_johndoe` - `post_20241229_160000` - `bm_20241229_120000` ``` -------------------------------- ### Configure Environment Variables Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/CONTRIBUTING.md Copy the example environment file and add your OpenAI API key to the .env file for development. ```bash cp env.example .env # Add your OPENAI_API_KEY to .env ``` -------------------------------- ### Initialize File System Agent with Sandbox Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/docs/vercel_tool.md This code demonstrates the setup of a file system agent using Vercel's Sandbox. It writes local files into the sandbox and defines a tool to execute bash commands within that environment. ```typescript import { Sandbox } from "@vercel/sandbox"; import { files } from './semantic-catalog' import { tool, ToolLoopAgent } from "ai"; import { ExecuteSQL } from "@/lib/tools"; const sandbox = await Sandbox.create(); await sandbox.writeFiles(files); const executeCommandTool(sandbox: Sandbox) { return tool({ /* ... */ execute: async ({ command }) => { const result = await sandbox.exec(command); } }); } ``` -------------------------------- ### Content Creation Workflow Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/SKILLS-MAPPING.md Illustrates a skills chain and file loading process for content creation, demonstrating context optimization by loading only necessary modules. ```text User: "Write a post about building in public" Skills Chain: 1. context-fundamentals → Load only identity module 2. memory-systems → Retrieve voice patterns from voice.md 3. context-optimization → Don't load network/operations 4. tool-design → Use content templates as structured scaffolds Files Loaded: - SKILL.md (50 tokens) - Routing - identity/IDENTITY.md (80 tokens) - Module instructions - identity/voice.md (200 tokens) - Voice patterns - identity/brand.md (scan for pillars) - Topic validation Total: ~400 tokens vs loading entire brain (~5000 tokens) ``` -------------------------------- ### LLM-as-Judge Skills Project Setup Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/CLAUDE.md Commands for setting up and managing the llm-as-judge-skills project. Requires Node.js >= 18. ```bash cd examples/llm-as-judge-skills ``` ```bash npm install ``` ```bash npm run build # tsc ``` ```bash npm test # vitest (19 tests) ``` ```bash npm run lint # eslint ``` ```bash npm run format # prettier ``` ```bash npm run typecheck # tsc --noEmit ``` -------------------------------- ### Install Digital Brain as a Claude Code Skill Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/digital-brain-skill/README.md Clone the Digital Brain repository into your Claude configuration directory for user-wide or project-specific installation. ```bash # User-wide installation git clone https://github.com/muratcankoylan/digital-brain-skill.git \ ~/.claude/skills/digital-brain # Or project-specific git clone https://github.com/muratcankoylan/digital-brain-skill.git \ .claude/skills/digital-brain ``` -------------------------------- ### Build and Test Project Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/CONTRIBUTING.md Commands to build the project and run the test suite. ```bash npm run build npm test ``` -------------------------------- ### Build Supervised Example with Tinker Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt Use `build_supervised_example` to create a `ModelInput` and token loss weights for supervised learning. The output is then formatted for visualization. ```python model_input, weights = renderer.build_supervised_example(messages) from tinker_cookbook.utils.format_colorized import format_colorized print(format_colorized(model_input.to_ints(), weights, tokenizer)) ``` -------------------------------- ### Example Workflow Input Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/llm-as-judge-skills/prompts/agent-system/orchestrator-prompt.md An example of a complex task input provided to the Orchestrator Agent, including the main task and specific requirements for the workflow. ```plaintext Task: Create a comprehensive report on LLM evaluation best practices. Requirements: 1. Research current methods and tools 2. Analyze trade-offs between approaches 3. Write an executive summary 4. Evaluate the quality of the final report ``` -------------------------------- ### Shortcut: Save Weights and Get Sampling Client Source: https://github.com/muratcankoylan/agent-skills-for-context-engineering/blob/main/examples/book-sft-pipeline/references/tinker.txt A convenience method that combines saving weights for sampling and creating a sampling client in one step. Useful for quick testing. ```python sampling_client = training_client.save_weights_and_get_sampling_client(name="0000") ```