### Setup Development Environment Source: https://promptpack.org/docs/processes/contributing Commands to clone the repository, install documentation dependencies, and launch the local development server. ```bash # Clone the repository git clone https://github.com/altairalabs/promptpack-spec.git cd promptpack-spec # Install documentation dependencies cd promptpack-docs npm install # Start development server npm start ``` -------------------------------- ### Install PromptKit CLIs Source: https://promptpack.org/docs/ecosystem/promptkit-runtime Global installation command for the promptarena and packc command-line tools. ```bash npm install -g @altairalabs/promptarena @altairalabs/packc ``` -------------------------------- ### Preview documentation locally Source: https://promptpack.org/docs/processes/contributing Run this command to start the local development server and preview changes. ```bash npm start ``` -------------------------------- ### Test Documentation Locally Source: https://promptpack.org/docs/spec/VERSIONING Command to start the local development server for verifying versioning changes. ```bash npm start # Verify: # - Version switcher works # - Archived docs are accessible # - Links work correctly ``` -------------------------------- ### Install LangChain.js Integration Source: https://promptpack.org/docs/ecosystem/integrations Install the official PromptPack integration package for LangChain.js via npm. ```bash npm install @promptpack/langchain ``` -------------------------------- ### Define MultimodalExample Source: https://promptpack.org/docs/rfcs/multimodal-support Example structure for including media within a prompt message. ```json { "examples": [ { "name": "image-analysis", "description": "Basic image analysis", "role": "user", "parts": [ { "type": "text", "text": "What's in this image?" }, { "type": "image", "media": { "file_path": "examples/photo.jpg", "mime_type": "image/jpeg", "detail": "high", "caption": "Sample photo" } } ] } ] } ``` -------------------------------- ### Configure PromptPacks via Kubernetes Source: https://promptpack.org/docs/spec/file-format Example of embedding a prompt pack within a Kubernetes ConfigMap. ```yaml # Kubernetes ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: ai-packs data: customer-support.json: | { "id": "customer-support", "version": "1.0.0", ... } ``` -------------------------------- ### PromptPack YAML configuration example Source: https://promptpack.org/docs/processes/contributing A template for defining a PromptPack resource, including metadata and prompt specifications. ```yaml # Good: Clear, complete, realistic apiVersion: v1 kind: PromptPack metadata: name: customer-support version: 1.0.0 description: Customer support automation system spec: prompts: - name: greeting template: "Hello {{customer_name}}, how can I help you?" variables: - name: customer_name type: string required: true ``` -------------------------------- ### Version Control Diff Example Source: https://promptpack.org/docs/spec/file-format Demonstrates how JSON structure facilitates clear tracking of changes in version control systems. ```json { "prompts": { "support": { - "system_template": "You are a support agent.", + "system_template": "You are a friendly support agent.", "parameters": { - "temperature": 0.7 + "temperature": 0.8 } } } } ``` -------------------------------- ### Define state parameters Source: https://promptpack.org/docs/rfcs/workflow-extension Example structure for passing parameters to a state to modify prompt behavior. ```json { "planning": { "prompt_task": "planning", "parameters": { "detail_level": "high" } } } ``` -------------------------------- ### PromptPack Compilation Metadata Source: https://promptpack.org/docs/spec/file-format Example of the compilation metadata block included in a pack file. ```json { "compilation": { "compiled_with": "packc-v1.0.0", "created_at": "2025-10-31T12:00:00Z", "schema": "v1" } } ``` -------------------------------- ### CI/CD GitHub Actions Source: https://promptpack.org/docs/ecosystem/promptkit-runtime Configuration examples for integrating promptarena and packc into GitHub Actions workflows. ```yaml - uses: AltairaLabs/PromptKit/.github/actions/promptarena-action@v1 with: scenario: scenarios/regression.scenario.yaml ``` ```yaml - uses: AltairaLabs/PromptKit/.github/actions/packc-action@v1 with: config: config.arena.yaml output: dist/app.pack.json ``` -------------------------------- ### Add media configuration to existing prompt Source: https://promptpack.org/docs/rfcs/multimodal-support Example of updating an existing prompt schema to enable media support by adding the media configuration object. ```json { "prompts": { "existing-prompt": { "system_template": "...", // Add media configuration "media": { "enabled": true, "supported_types": ["image"] } } } } ``` -------------------------------- ### Minimal PromptPack Definition Source: https://promptpack.org/docs/rfcs/core-schema A basic example of a PromptPack configuration file using the required schema version and structure. ```json { "$schema": "https://promptpack.org/schema/v1/promptpack.schema.json", "id": "hello-world", "name": "Hello World Pack", "version": "1.0.0", "template_engine": { "version": "v1", "syntax": "{{variable}}" }, "prompts": { "greeting": { "id": "greeting", "name": "Greeter", "version": "1.0.0", "system_template": "Say hello to {{name}}." } } } ``` -------------------------------- ### Complete PromptPack Configuration Source: https://promptpack.org/docs/getting-started A comprehensive example of a feedback-system.promptpack.json file integrating schema, template engines, evaluation metrics, prompt logic, and workflow states. ```json { "$schema": "https://promptpack.org/schema/latest/promptpack.schema.json", "id": "feedback-system", "name": "Customer Feedback System", "version": "1.4.0", "description": "Complete customer feedback collection, analysis, and response system", "template_engine": { "version": "v1", "syntax": "{{variable}}", "features": ["basic_substitution", "fragments"] }, "evals": [ { "id": "empathy-score", "description": "Monitors empathy level across all customer-facing responses", "type": "llm_judge", "trigger": "sample_turns", "sample_percentage": 20, "metric": { "name": "promptpack_empathy_score", "type": "gauge", "range": { "min": 1, "max": 5 } }, "params": { "judge_prompt": "Rate the empathy level of this customer interaction from 1 (cold/robotic) to 5 (warm/genuinely caring).", "model": "gpt-4o", "passing_score": 4 } } ], "prompts": { "collector": { "id": "collector", "name": "Feedback Collector", "version": "1.0.0", "system_template": "You are a feedback collector for {{company}}.\n\n{{fragments.company_context}}\n\n{{fragments.response_guidelines}}\n\nAsk about their experience and categorize feedback as positive, neutral, or negative.", "variables": [ { "name": "company", "type": "string", "required": true, "example": "Acme Corp" } ], "tools": ["lookup_customer", "log_feedback"], "parameters": { "temperature": 0.7, "max_tokens": 300 }, "validators": [ { "type": "banned_words", "enabled": true, "fail_on_violation": true, "params": { "words": ["don't care", "not my problem", "deal with it"] } }, { "type": "pii_detection", "enabled": true, "fail_on_violation": true } ] }, "analyzer": { "id": "analyzer", "name": "Sentiment Analyzer", "version": "1.0.0", "system_template": "You are a sentiment analysis specialist for {{company}}.\n\nAnalyze the customer feedback and return a JSON object with: sentiment (positive/neutral/negative), confidence (0-1), key_themes (array), and suggested_action.", "variables": [ { "name": "company", "type": "string", "required": true } ], "parameters": { "temperature": 0.2, "max_tokens": 500 }, "evals": [ { "id": "classification-accuracy", "description": "Checks whether sentiment classification matches the feedback content", "type": "llm_judge", "trigger": "every_turn", "metric": { "name": "promptpack_classification_accuracy", "type": "boolean" }, "params": { "judge_prompt": "Does the sentiment classification accurately reflect the customer's feedback? Answer true or false.", "model": "gpt-4o" } } ] }, "responder": { "id": "responder", "name": "Response Drafter", "version": "1.0.0", "system_template": "You are a customer response specialist for {{company}}.\n\n{{fragments.response_guidelines}}\n\nDraft a personalized follow-up response based on the feedback analysis.", "variables": [ { "name": "company", "type": "string", "required": true } ], "parameters": { "temperature": 0.6, "max_tokens": 500 } } }, "workflow": { "version": 1, "entry": "collect", "states": { ``` -------------------------------- ### Define a session-bound variable Source: https://promptpack.org/docs/spec/schema-guide Example of a variable that is automatically populated from the session context. ```json { "name": "user_id", "type": "string", "required": true, "binding": { "kind": "session", "field": "user_id", "auto_populate": true } } ``` -------------------------------- ### Complete Document Analyzer Example Source: https://promptpack.org/docs/guides/add-composition A full configuration demonstrating classification and conditional routing to specific extractors. ```json { "$schema": "https://promptpack.org/schema/v1.5.0/promptpack.schema.json", "id": "document-analyzer", "name": "Document Analyzer", "version": "1.0.0", "template_engine": { "version": "v1", "syntax": "{{variable}}" }, "prompts": { "doc_classifier": { "id": "doc_classifier", "name": "Classifier", "version": "1.0.0", "system_template": "Classify the document. Return JSON: { \"type\": \"research_paper\" | \"general\" }.\n\n{{input}}", "parameters": { "temperature": 0.0 } }, "research_paper_extractor": { "id": "research_paper_extractor", "name": "Paper Extractor", "version": "1.0.0", "system_template": "Extract title, authors, abstract, findings.\n\n{{input}}", "parameters": { "temperature": 0.2 } }, "general_doc_extractor": { "id": "general_doc_extractor", "name": "General Extractor", "version": "1.0.0", "system_template": "Extract a summary and main entities.\n\n{{input}}", "parameters": { "temperature": 0.2 } } }, "workflow": { "version": 1, "entry": "main", "states": { "main": { "orchestration": "composition", "composition": "analyze_document", "terminal": true } } }, "compositions": { "analyze_document": { "version": 1, "description": "Classify a document and route to a type-specific analyzer.", "steps": [ { "id": "classify", "kind": "prompt", "prompt_task": "doc_classifier", "input": "${input.text}" }, { "id": "route", "kind": "branch", "predicate": { "path": "${classify.output.type}", "op": "equals", "value": "research_paper" }, "then": "extract_paper", "else": "extract_general" }, { "id": "extract_paper", "kind": "prompt", "prompt_task": "research_paper_extractor", "input": "${input.text}" }, { "id": "extract_general", "kind": "prompt", "prompt_task": "general_doc_extractor", "input": "${input.text}" } ] } } } ``` -------------------------------- ### Conceptual CLI Utility Commands Source: https://promptpack.org/docs/ecosystem/community-tools Example commands for potential future CLI tools to validate, lint, and initialize PromptPack files. ```bash # What could exist (doesn't exist yet) promptpack validate my-file.promptpack.yml promptpack lint --fix my-file.promptpack.yml promptpack init --template basic ``` -------------------------------- ### Define Skill Directory Structure Source: https://promptpack.org/docs/guides/add-skills Example directory layout for organizing skill-based knowledge sources. ```text skills/ ├── billing/ → billing policies, refund rules, account procedures ├── technical/ → troubleshooting guides, diagnostic procedures └── compliance/ → regulatory requirements (preload for all states) ``` -------------------------------- ### Progressive Skill Loading Example Source: https://promptpack.org/docs/guides/add-skills Shows the refactored structure using external skills instead of embedded templates. ```json { "id": "customer-support", "version": "1.3.1", "skills": [ "./skills/billing", { "path": "./skills/compliance", "preload": true }, { "name": "escalation-protocol", "description": "Steps for escalating unresolved issues", "instructions": "When an issue cannot be resolved within 3 exchanges:\n1. Acknowledge the complexity\n2. Collect case details\n3. Create escalation ticket\n4. Provide ticket reference" } ], "prompts": { "billing": { "id": "billing", "name": "Billing Support", "version": "1.1.0", "system_template": "You are a billing specialist.\n\nUse your available skills for refund policies, compliance rules, and escalation procedures." } } } ``` -------------------------------- ### Extended Tool Policy Example Source: https://promptpack.org/docs/rfcs/agent-loops A proposed JSON structure for adding loop semantics to the existing tool_policy. ```json { "tool_policy": { "max_rounds": 50, "on_error": "retry", "result_bindings": { "run_tests.output": "artifacts.test_results" } } } ``` -------------------------------- ### Configure Sampling Trigger Source: https://promptpack.org/docs/guides/add-evals Example configuration for an evaluation using a sampling trigger with a specified percentage. ```json { "id": "safety-audit", "type": "llm_judge", "trigger": "sample_turns", "sample_percentage": 10, "metric": { "name": "promptpack_safety_violations", "type": "counter" }, "params": { "judge_prompt": "Review the response for safety concerns. Return the count of violations.", "model": "gpt-4o" } } ``` -------------------------------- ### Define a simple two-state workflow in JSON Source: https://promptpack.org/docs/rfcs/workflow-extension This example demonstrates a basic workflow configuration with two states, 'analyze' and 'execute', linked by an event transition. ```json { "$schema": "https://promptpack.org/schema/v1/promptpack.schema.json", "id": "simple-agent", "name": "Simple Agent Workflow", "version": "1.0.0", "template_engine": { "version": "v1", "syntax": "{{variable}}" }, "prompts": { "analyze": { "id": "analyze", "name": "Analysis Phase", "version": "1.0.0", "system_template": "Analyze the input data and determine the required approach. When analysis is complete, emit the AnalysisComplete event." }, "execute": { "id": "execute", "name": "Execution Phase", "version": "1.0.0", "system_template": "Execute the determined approach based on the analysis." } }, "workflow": { "version": 1, "entry": "analyze", "states": { "analyze": { "prompt_task": "analyze", "description": "Initial analysis phase to understand requirements", "on_event": { "AnalysisComplete": "execute" }, "persistence": "transient", "orchestration": "internal" }, "execute": { "prompt_task": "execute", "description": "Execution phase to perform the task", "persistence": "persistent", "orchestration": "internal" } } } } ``` -------------------------------- ### Define a Tool with JSON Schema Parameters Source: https://promptpack.org/docs/spec/schema-guide Example of a tool definition including a name, description, and structured input parameters with validation rules. ```json "tools": { "create_ticket": { "name": "create_ticket", "description": "Create a support ticket with title, description, and priority", "parameters": { "type": "object", "properties": { "title": { "type": "string", "description": "Ticket title" }, "priority": { "type": "string", "enum": ["low", "medium", "high", "urgent"] } }, "required": ["title"] } } } ``` -------------------------------- ### Define a Workflow Configuration Source: https://promptpack.org/docs/spec/schema-guide Example of a workflow configuration using states, prompt tasks, and event-based transitions. ```json "workflow": { "version": 1, "entry": "triage", "states": { "triage": { "prompt_task": "triage", "description": "Route incoming requests to the appropriate specialist", "on_event": { "billing": "billing_support", "technical": "tech_support", "resolved": "closing" } }, "billing_support": { "prompt_task": "billing", "on_event": { "resolved": "closing", "escalate": "human_handoff" }, "persistence": "persistent" }, "tech_support": { "prompt_task": "technical", "on_event": { "resolved": "closing", "escalate": "human_handoff" }, "persistence": "persistent" }, "closing": { "prompt_task": "closing", "on_event": {}, "orchestration": "internal" }, "human_handoff": { "prompt_task": "handoff", "on_event": {}, "orchestration": "external" } } } ``` -------------------------------- ### PromptPack JSON Structure Example Source: https://promptpack.org/docs/spec/file-format A standard PromptPack definition file containing metadata and prompt templates with variable definitions. ```json { "id": "customer-support", "name": "Customer Support Pack", "version": "1.0.0", "prompts": { "support": { "system_template": "You are a helpful {{role}} for {{company}}.", "variables": [ { "name": "role", "type": "string", "required": true, "description": "The assistant's role" } ] } } } ``` -------------------------------- ### Define a logical OR predicate Source: https://promptpack.org/docs/spec/schema-guide Example of a predicate using the any_of combinator to evaluate multiple conditions. ```yaml predicate: any_of: - path: "${assess.output.confidence}" op: less_than value: 0.8 - path: "${assess.output.complexity}" op: greater_than value: 7 ``` -------------------------------- ### Define Model Overrides Source: https://promptpack.org/docs/spec/schema-guide Example of configuring model-specific template modifications and parameters for a specific model identifier. ```json "model_overrides": { "claude-3-opus": { "system_template_prefix": "\n", "parameters": { "temperature": 0.5 } } } ``` -------------------------------- ### Define Evals in PromptPack Source: https://promptpack.org/docs/spec/schema-guide Example configuration for pack-level evaluations, including both structural validation and LLM-based quality assessment. ```json "evals": [ { "id": "json_format", "type": "json_valid", "trigger": "every_turn", "description": "Verify the assistant always returns valid JSON", "metric": { "name": "promptpack_json_valid", "type": "boolean" } }, { "id": "tone-check", "type": "llm_judge", "trigger": "sample_turns", "sample_percentage": 10, "params": { "judge_prompt": "Rate the response tone on a 1-5 scale for professionalism.", "model": "gpt-4o", "passing_score": 4 }, "metric": { "name": "promptpack_tone_score", "type": "gauge", "range": { "min": 1, "max": 5 } } } ] ``` -------------------------------- ### Configure response validators Source: https://promptpack.org/docs/spec/schema-guide Example array of validator objects used to enforce guardrails on LLM responses. ```json "validators": [ { "type": "banned_words", "enabled": true, "fail_on_violation": true, "params": { "words": ["impossible", "can't help"] } }, { "type": "pii_detection", "enabled": true, "fail_on_violation": true } ] ``` -------------------------------- ### Define a validated variable Source: https://promptpack.org/docs/spec/schema-guide Example of a variable with an enum validation rule to restrict input to specific values. ```json { "name": "priority", "type": "string", "required": true, "description": "Support ticket priority level", "validation": { "enum": ["low", "medium", "high", "urgent"] } } ``` -------------------------------- ### Define LLM generation parameters Source: https://promptpack.org/docs/spec/schema-guide Example configuration for controlling model output characteristics like temperature and token limits. ```json "parameters": { "temperature": 0.7, "max_tokens": 1500, "top_p": 0.9 } ``` -------------------------------- ### Define Artifact Slots Source: https://promptpack.org/docs/spec/schema-guide Example of defining named artifact slots with specific MIME types and update modes. ```json "artifacts": { "commit_sha": { "type": "text/plain", "description": "Latest generated commit" }, "test_report": { "type": "application/json", "description": "Test runner summary" }, "iteration_log": { "type": "text/plain", "mode": "append", "description": "Per-visit log accumulated across the loop" } } ``` -------------------------------- ### Configure Optional Media Support Source: https://promptpack.org/docs/rfcs/multimodal-support Demonstrates how to define prompts with or without media support in the configuration. ```json { "prompts": { "text-only": { "system_template": "Text-only prompt", // No media field }, "vision": { "system_template": "Vision prompt", "media": { "enabled": true, "supported_types": ["image"] } } } } ``` -------------------------------- ### Define Object Schema Source: https://promptpack.org/docs/rfcs/template-variables Example schema definition for an object type variable. ```json { "name": "user", "type": "object", "description": "User object with nested properties" } ``` -------------------------------- ### Using promptarena CLI Source: https://promptpack.org/docs/ecosystem/promptkit-runtime Common commands for scaffolding projects, inspecting configurations, and executing scenarios. ```bash # Scaffold a project from a template promptarena init my-project --template iot-maintenance-demo # Inspect resolved config promptarena config-inspect # Run a scenario across providers promptarena run --scenario scenarios/hardware-faults.scenario.yaml # Red-team / self-play scenarios promptarena run --scenario scenarios/redteam-selfplay.scenario.yaml # Launch the local results viewer promptarena view ``` -------------------------------- ### Configure PromptConfig in YAML Source: https://promptpack.org/docs/rfcs/multimodal-support Demonstrates the equivalent configuration structure using YAML syntax for Kubernetes-style prompt definitions. ```yaml apiVersion: promptkit.altairalabs.ai/v1alpha1 kind: PromptConfig metadata: name: vision-assistant version: 1.0.0 spec: system_template: "You are an expert image analyst." media: enabled: true supported_types: - image image: max_size_mb: 10 allowed_formats: - jpeg - png - webp default_detail: high examples: - name: simple-analysis role: user parts: - type: text text: "What's in this image?" - type: image media: file_path: examples/photo.jpg mime_type: image/jpeg detail: high ``` -------------------------------- ### Configure Skills via String Paths Source: https://promptpack.org/docs/guides/add-skills The simplest method to register skill directories in the pack root. ```json { "skills": [ "./skills/billing", "./skills/technical" ] } ``` -------------------------------- ### Preserve comments in templates Source: https://promptpack.org/docs/rfcs/yaml-format Example showing how comments within the YAML source are handled. ```yaml # This is a customer support prompt # Updated: 2024-10-31 system_template: | You are a support agent. # This comment is preserved in the template ``` -------------------------------- ### Configure Sidebar Navigation Source: https://promptpack.org/docs/spec/VERSIONING Example configuration for adding archived versions to the Docusaurus sidebar. ```javascript { type: 'category', label: 'Archive', collapsed: true, items: [ { type: 'category', label: 'v1.1', collapsed: true, items: [ 'spec/v1.1/overview', 'spec/v1.1/examples', // ... other v1.1 docs ], }, // ... older versions ], } ``` -------------------------------- ### Declare Prometheus Metric Source: https://promptpack.org/docs/guides/add-evals Example of defining a gauge metric within the evaluation configuration. ```json { "metric": { "name": "promptpack_tone_score", "type": "gauge", "range": { "min": 1, "max": 5 } } } ``` -------------------------------- ### Top-Level Artifacts Declaration Example Source: https://promptpack.org/docs/rfcs/agent-loops A proposed JSON structure for declaring artifacts at the pack level. ```json { "artifacts": { "code": { "type": "text/plain" }, "test_results": { "type": "application/json" } } } ``` -------------------------------- ### Define workflow-level metadata Source: https://promptpack.org/docs/rfcs/workflow-extension Example structure for adding versioning and analytics metadata to a workflow definition. ```json { "workflow": { "version": 1, "entry": "sparring", "metadata": { "analytics_id": "workflow-v1", "workflow_version": "2.1.0" }, "states": { ... } } } ``` -------------------------------- ### Create a new documentation branch Source: https://promptpack.org/docs/processes/contributing Use this command to initialize a new branch for documentation improvements. ```bash git checkout -b docs/improve-examples ``` -------------------------------- ### Single-File Deployment Structure Source: https://promptpack.org/docs/spec/file-format A complete PromptPack file containing prompts, tools, fragments, and engine configuration. ```json { "prompts": { "support": {...}, "sales": {...} }, "tools": { "lookup_order": {...}, "create_ticket": {...} }, "fragments": { "greeting": "...", "policies": "..." }, "template_engine": { "version": "v1", "syntax": "{{var}}" } } ``` -------------------------------- ### Prompt Contextual Structure Source: https://promptpack.org/docs/spec/file-format Example of adding descriptive metadata to prompt definitions for better readability. ```json { "prompts": { "support": { "name": "General Support Assistant", "description": "Handles general customer inquiries and directs to specialists", "system_template": "..." } } } ``` -------------------------------- ### Manage Testing Costs Source: https://promptpack.org/docs/ecosystem/arena-testing Use mock providers or lower-cost models to minimize expenses during the development phase. ```bash # Use mock provider during development promptarena run --mock-provider # Test with cheaper models first promptarena run --provider gpt-3.5-turbo ``` -------------------------------- ### Configure Multi-Prompt Architecture Source: https://promptpack.org/docs/rfcs/core-schema Demonstrates defining multiple specialized prompts within the prompts map. ```json { "prompts": { "support": { "system_template": "..." }, "sales": { "system_template": "..." }, "technical": { "system_template": "..." } } } ```