### Run Command Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of using the `run` command with specific model parameters and a prompt.

```bash
gh models run --max-tokens 2048 --temperature 0.5 openai/gpt-4o-mini "your prompt"
```

--------------------------------

### View Specific Model Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/view-command.md

Demonstrates how to view details for a specific model, such as 'openai/gpt-4o'.

```bash
gh models view openai/gpt-4o
```

--------------------------------

### View Different Provider Model Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/view-command.md

Shows how to view a model from a different provider, like 'cohere/cohere-command-r'.

```bash
gh models view cohere/cohere-command-r
```

--------------------------------

### Generate Command Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of using the `generate` command with high effort, a specific ground truth model, a session file, and template variables.

```bash
gh models generate \
  --effort high \
  --groundtruth-model "openai/gpt-4.1" \
  --session-file session.json \
  --var domain=biology \
  prompt.yml
```

--------------------------------

### Get Model Details Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Retrieves detailed information about a specific model using its registry, name, and version. Useful for understanding model capabilities like max tokens and supported languages.

```go
details, err := client.GetModelDetails(ctx, "registry", "gpt-4o", "2024-05")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Max tokens: %d\n", details.MaxInputTokens)
fmt.Println("Supported languages:", details.SupportedLanguages)
```

--------------------------------

### Example: SetParameterByName

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Demonstrates how to use SetParameterByName to configure temperature and max-tokens.

```go
mp := &ModelParameters{}
mp.SetParameterByName("temperature", "0.5")
mp.SetParameterByName("max-tokens", "1000")
```

--------------------------------

### CLI Usage Examples

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Demonstrates how to use the 'gh models eval' command from the command line. Examples show basic usage, JSON output, and specifying an organization.

```bash
gh models eval prompt.yml
gh models eval --json prompt.yml
gh models eval --org my-org prompt.yml
```

--------------------------------

### Interactive Mode Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/view-command.md

Illustrates running the command in interactive mode, which displays a selection list of available models.

```bash
gh models view
# Displays selection list
```

--------------------------------

### Create Default Azure Client Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Creates an Azure client with default configuration using a GitHub authentication token. This is the simplest way to initialize the client.

```go
token, _ := auth.TokenForHost("github.com")
client, err := azuremodels.NewDefaultAzureClient(token)
if err != nil {
    log.Fatal(err)
}
```

--------------------------------

### Check Go Version

Source: https://github.com/github/gh-models/blob/main/DEV.md

Verify that the installed Go version meets the minimum requirement of 1.22.

```shell
go version
go version go1.22.x <arch>
```

--------------------------------

### Eval Command Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of using the `eval` command to output results in JSON format and attribute usage to an organization.

```bash
gh models eval --json --org my-org evaluation.yml > results.json
```

--------------------------------

### Get Chat Completion Stream Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Performs inference with a streaming response. Automatically enables streaming for most models, but disables it for o1 models. Handles automatic retries on rate limits.

```go
req := azuremodels.ChatCompletionOptions{
    Messages: []azuremodels.ChatMessage{...},
    Model: "openai/gpt-4o",
    MaxTokens: util.Ptr(2048),
}
resp, err := client.GetChatCompletionStream(ctx, req, "")
if err != nil {
    // Handle error
}
for {
    completion, err := resp.Reader.Read()
    if errors.Is(err, io.EOF) {
        break
    }
    // Process completion
}
```

--------------------------------

### Prompt Description Field Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of the 'description' field in a .prompt.yml file, providing an explanation for users and documentation.

```yaml
description: "Description of what this prompt does"
```

--------------------------------

### Create Azure Client with Custom Configuration Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Creates an Azure client with explicit configuration, allowing customization of the HTTP client and other settings. Use this when default configurations are insufficient.

```go
httpClient := &http.Client{Timeout: 30 * time.Second}
cfg := azuremodels.NewCustomAzureClientConfig(...)
client := azuremodels.NewAzureClient(httpClient, token, cfg)
```

--------------------------------

### Prompt Model Field Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of the 'model' field in a .prompt.yml file, specifying the model identifier for inference.

```yaml
model: "openai/gpt-4o-mini"
```

--------------------------------

### Install GitHub Models CLI Extension

Source: https://github.com/github/gh-models/blob/main/README.md

Install the GitHub Models CLI extension using the `gh extension install` command. Ensure the `gh` CLI is installed and authenticated.

```shell
gh extension install https://github.com/github/gh-models
```

--------------------------------

### Azure Client API Reference

Source: https://github.com/github/gh-models/blob/main/_autodocs/README.md

This Go comment indicates where to find examples for the Azure Client. Refer to the linked API reference for detailed usage.

```go
// See [Azure Client](api-reference/azure-client.md) for examples
```

--------------------------------

### List Models Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Retrieves a list of all available models from the catalog, including their metadata. This can be used to display available models to the user or for filtering.

```go
models, err := client.ListModels(ctx)
if err != nil {
    log.Fatal(err)
}
for _, model := range models {
    fmt.Printf("%s - %s\n", model.ID, model.FriendlyName)
}
```

--------------------------------

### Run Inference in REPL Mode

Source: https://github.com/github/gh-models/blob/main/README.md

Start the extension in REPL mode for interactive prompt-response sessions. Use `/help` for commands within REPL mode.

```shell
gh models run
```

--------------------------------

### Prompt Name Field Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of the 'name' field in a .prompt.yml file, used as a human-readable identifier for the prompt.

```yaml
name: "Prompt Identifier"
```

--------------------------------

### Get Model Details

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Fetches detailed information about a specific model using its registry, name, and version. Displays the model's description, license, and context limits.

```go
details, err := client.GetModelDetails(
    ctx,
    summary.Registry,
    summary.Name,
    summary.Version,
)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Description: %s\n", details.Description)
fmt.Printf("License: %s\n", details.License)
fmt.Printf("Context: %s\n", details.ContextLimits())
```

--------------------------------

### Template Variables Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Illustrates how to use template variables from test data within message and evaluator prompts in the .prompt.yml file.

```yaml
testData:
  - name: "Alice"
    topic: "AI"
messages:
  - role: user
    content: "{{name}} asks about {{topic}}"
evaluators:
  - name: "relevance"
    llm:
      prompt: "Is the response about {{topic}}?"
```

--------------------------------

### Example Test Output Without Authentication

Source: https://github.com/github/gh-models/blob/main/integration/README.md

Illustrates the output when running integration tests without GitHub authentication, showing skipped tests.

```text
=== RUN   TestIntegrationHelp
--- PASS: TestIntegrationHelp (0.05s)
=== RUN   TestIntegrationList
    integration_test.go:90: Skipping integration test - no GitHub authentication available
--- SKIP: TestIntegrationList (0.04s)
```

--------------------------------

### Generate with Template Variables Bash Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Example of passing template variables to the generate command for use in prompts.

```bash
gh models generate \
  --var domain="biology" \
  --var difficulty="advanced" \
  prompt.yml
```

--------------------------------

### Generate Tests with Custom Intent

Source: https://github.com/github/gh-models/blob/main/README.md

Use the `--instruction-intent` flag to provide custom system instructions for the intent generation phase. This example focuses on edge cases for prompt generation.

```bash
gh models generate --instruction-intent "Focus on edge cases" my_prompt.prompt.yml
```

--------------------------------

### Model Parameter Override - topP

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of how to configure nucleus sampling for response generation using the --top-p flag.

```bash
gh models run --top-p 0.9 model "prompt"
```

--------------------------------

### Get Human-Readable Context Limits

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Returns a human-readable summary of a model's context limits, such as 'up to 128000 input tokens and 4096 output tokens'.

```go
details := model.ModelDetails()
fmt.Println(details.ContextLimits())
```

--------------------------------

### Check Code Style and Tests

Source: https://github.com/github/gh-models/blob/main/CONTRIBUTING.md

Run the make check command to ensure code passes tests and adheres to the linter's style guide.

```bash
make check
```

--------------------------------

### Model Parameter Override - temperature

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of how to control the randomness of the model's output using the --temperature flag.

```bash
gh models run --temperature 0.2 model "prompt"
```

--------------------------------

### Run Inference with Piped Input

Source: https://github.com/github/gh-models/blob/main/README.md

Process text piped from another command as input for inference. This example summarizes the README.md file.

```shell
cat README.md | gh models run openai/gpt-4o-mini "summarize this text"
```

--------------------------------

### Plugin Evaluator Configuration

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Example of using a plugin evaluator, specified by 'uses' with a 'github/' namespace. This allows leveraging built-in or community evaluators.

```yaml
evaluators:
  - name: "similarity"
    uses: "github/similarity"
```

--------------------------------

### Bash Example: Model Not Found

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Shows the error when a specified model does not exist. Use 'gh models list' to find available models or run interactively.

```bash
gh models run nonexistent/model "prompt"
# Error: Model not found
```

--------------------------------

### Create New Run Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Creates a new Cobra command for the run functionality. This is typically used for CLI setup.

```go
func NewRunCommand(cfg *command.Config) *cobra.Command
```

--------------------------------

### Model Parameter Override - maxTokens

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Example of how to set the maximum number of tokens for a model's response using a command-line flag.

```bash
gh models run --max-tokens 2048 model "prompt"
```

--------------------------------

### Bash Example: Invalid Model Format

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Illustrates the error when a model identifier does not follow the 'provider/name' format. Use the correct format, e.g., 'openai/gpt-4o-mini'.

```bash
gh models run "invalid model" "prompt"
# Error: invalid model format
```

--------------------------------

### Bash Example: Empty Variable Key

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Demonstrates the error when a variable key is an empty string. Variable keys must be non-empty; use the '--var key=value' format.

```bash
gh models run --var =value
# Error: variable key cannot be empty
```

--------------------------------

### String Evaluator Configurations

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Examples of string evaluator configurations within a .prompt.yml file. These evaluators check for specific string patterns in the model's response.

```yaml
evaluators:
  - name: "contains-check"
    string:
      contains: "expected text"
  - name: "exact-match"
    string:
      equals: "exact response"
  - name: "starts-check"
    string:
      startsWith: "Response:"
  - name: "ends-check"
    string:
      endsWith: "?"
```

--------------------------------

### Create List Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/list-command.md

Creates a new Cobra command for the list functionality within the CLI. This function is part of the command's setup.

```go
func NewListCommand(cfg *command.Config) *cobra.Command
```

--------------------------------

### JSON Output Example

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Shows the JSON output format when the '--json' flag is used with the 'gh models eval' command. This format is suitable for programmatic processing.

```json
{
  "name": "My Evaluation",
  "description": "...",
  "model": "openai/gpt-4o",
  "testResults": [...],
  "summary": {
    "totalTests": 2,
    "passedTests": 2,
    "failedTests": 0,
    "passRate": 100.0
  }
}
```

--------------------------------

### Bash Example: Duplicate Variable Keys

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Shows the error when the same variable key is provided more than once using the --var flag. Ensure each variable key is unique or use the last provided value.

```bash
gh models run --var name=Alice --var name=Bob
# Error: duplicate variable key 'name'
```

--------------------------------

### Basic CLI Usage

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Demonstrates basic command-line invocation of the 'run' command with a model and prompt.

```bash
gh models run [model] [prompt]
```

--------------------------------

### Build the Project

Source: https://github.com/github/gh-models/blob/main/DEV.md

Compile the project to generate the executable binary. This binary can then be run locally.

```shell
make build
```

--------------------------------

### Build and Run Integration Tests

Source: https://github.com/github/gh-models/blob/main/DEV.md

Compile the project and then execute integration tests against live endpoints. Ensure the binary is built first.

```shell
# Build the binary first
make build

# Run integration tests
cd integration
go test -v
```

--------------------------------

### CLI Usage with File and Variables

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Shows how to use the 'run' command with a prompt file and template variables.

```bash
gh models run --file prompt.yml --var name=value
```

--------------------------------

### Model Not Found Error

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Example of an error message when a specified model does not exist.

```text
Error: The specified model 'invalid/model' is not found. Run 'gh models list' to see available models
```

--------------------------------

### Run Command with Parameters

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Execute a model with a text prompt and adjust parameters like temperature.

```bash
gh models run openai/gpt-4o-mini --temperature 0.2 "code review this function"
```

--------------------------------

### Basic Generate Command Usage

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Demonstrates the basic usage of the generate command with a prompt file.

```bash
gh models generate prompt.yml
```

--------------------------------

### Invalid Parameter Format Error

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Example of an error message indicating an incorrect format for a variable parameter.

```text
Error: invalid variable format 'key', expected 'key=value'
```

--------------------------------

### Run All Tests with Make

Source: https://github.com/github/gh-models/blob/main/CONTRIBUTING.md

Use the make command to run all tests, providing a convenient shortcut for the testing process.

```bash
make test
```

--------------------------------

### Run Basic Single-Shot Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Execute a model with a simple text prompt.

```bash
gh models run openai/gpt-4o-mini "what is machine learning?"
```

--------------------------------

### NewConfig Constructor

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/command-config.md

Creates a new command configuration with explicitly provided output writers, API client, and terminal settings. Use this when you need precise control over all configuration parameters.

```go
func NewConfig(out, errOut io.Writer, client azuremodels.Client, isTerminalOutput bool, width int) *Config
```

```go
cfg := command.NewConfig(os.Stdout, os.Stderr, client, true, 80)
```

--------------------------------

### Run Command with Prompt File and Variables

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Execute a model using a prompt defined in a file and pass custom variables.

```bash
gh models run \
  --file customer_response.yml \
  --var name="John Smith" \
  --var issue="billing error"
```

--------------------------------

### NewTablePrinter Method

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/command-config.md

Initializes and returns a table printer that is configured according to the terminal settings of the Config object. Use this to display data in a structured, terminal-friendly table format.

```go
func (c *Config) NewTablePrinter() tableprinter.TablePrinter
```

```go
printer := cfg.NewTablePrinter()
printer.AddHeader([]string{"ID", "NAME"})
printer.AddField("openai/gpt-4o")
printer.AddField("GPT-4o")
printer.EndRow()
printer.Render()
```

--------------------------------

### Basic Chat Completion with Streaming

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Demonstrates how to perform a basic chat completion request and stream the response content. Ensure to handle io.EOF for stream termination.

```go
ctx := context.Background()
client, _ := azuremodels.NewDefaultAzureClient(token)

req := azuremodels.ChatCompletionOptions{
    Messages: []azuremodels.ChatMessage{
        {
            Role:    azuremodels.ChatMessageRoleUser,
            Content: util.Ptr("What is AI?"),
        },
    },
    Model:       "openai/gpt-4o-mini",
    MaxTokens:   util.Ptr(1024),
    Temperature: util.Ptr(0.7),
    Stream:      true,
}

resp, err := client.GetChatCompletionStream(ctx, req, "")
if err != nil {
    log.Fatal(err)
}
def resp.Reader.Close()

var response strings.Builder
for {
    completion, err := resp.Reader.Read()
    if errors.Is(err, io.EOF) {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    for _, choice := range completion.Choices {
        if choice.Delta != nil && choice.Delta.Content != nil {
            response.WriteString(*choice.Delta.Content)
        }
    }
}

fmt.Println(response.String())
```

--------------------------------

### PromptPexTest Go Struct

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Represents a single generated test case, including input, expected output, and reasoning.

```go
type PromptPexTest struct {
    Input      string // Test input/variables
    Expected   string // Expected model output
    Predicted  string // Actual model output (after groundtruth)
    Reasoning  string // Generation reasoning
    Scenario   string // Test scenario description
}
```

--------------------------------

### LLM Evaluator Configuration

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

An example of an LLM evaluator configuration in .prompt.yml. This uses another model to evaluate the response based on a provided prompt and choices.

```yaml
evaluators:
  - name: "coherence"
    llm:
      modelId: "openai/gpt-4.1"
      prompt: "Is this coherent? {{completion}}"
      systemPrompt: "You are an evaluator"
      choices:
        - choice: "yes"
          score: 1.0
        - choice: "no"
          score: 0.0
```

--------------------------------

### Run Integration Tests Locally

Source: https://github.com/github/gh-models/blob/main/integration/README.md

Execute integration tests from the integration directory using the Go test runner.

```bash
go test -v
```

--------------------------------

### Load Prompt File from Disk

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Loads and parses a prompt file from disk. Use this function to read prompt configurations from YAML files.

```go
pf, err := prompt.LoadFromFile("my_prompt.prompt.yml")
if err != nil {
    log.Fatal(err)
}
```

--------------------------------

### View Model Interactively

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/view-command.md

Opens an interactive selection menu to choose a model. Run without arguments to activate.

```bash
gh models view  # Interactive selection
```

--------------------------------

### PromptPexContext Struct

Source: https://github.com/github/gh-models/blob/main/_autodocs/types.md

Represents the complete context for a generation session, including run ID, prompt details, options, intent, and generated rules.

```go
type PromptPexContext struct {
    RunID        string
    Prompt       *prompt.File
    PromptHash   string
    Options      *PromptPexOptions
    Intent       *string
    Rules        []string
    InverseRules []string
    InputSpec    *string
    Tests        []PromptPexTest
}
```

--------------------------------

### PromptPexOptions Struct Definition

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Defines the configuration structure for PromptPex test generation, including instructions, models, and generation parameters.

```go
type PromptPexOptions struct {
    Instructions       *PromptPexPrompts
    Models             *PromptPexModelAliases
    TestsPerRule       int
    RulesPerGen        int
    MaxRules           int
    IntentMaxTokens    int
    InputSpecMaxTokens int
    Effort             string
    Prompt             string
    Verbose            bool
}
```

--------------------------------

### Upgrade GitHub Models CLI Extension

Source: https://github.com/github/gh-models/blob/main/README.md

Upgrade an existing installation of the `gh models` extension to the latest version using the `gh extension upgrade` command.

```shell
gh extension upgrade github/gh-models
```

--------------------------------

### Pipe Input to Run Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Demonstrates streaming input from stdin to the 'run' command for processing.

```bash
cat data.txt | gh models run openai/gpt-4o-mini "summarize this"
```

--------------------------------

### Generate with Custom Instructions Bash Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Provides custom instructions for intent and test generation.

```bash
gh models generate \
  --instruction-intent "Focus on edge cases" \
  --instruction-tests "Generate adversarial tests" \
  prompt.yml
```

--------------------------------

### CLI Usage with Organization

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Illustrates specifying an organization for usage attribution when running a model.

```bash
gh models run --org my-org openai/gpt-4o-mini "your prompt"
```

--------------------------------

### NewConfigWithTerminal Constructor

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/command-config.md

Creates a new command configuration by inferring settings from a terminal instance. This is convenient for applications that primarily interact with a user's terminal.

```go
func NewConfigWithTerminal(terminal term.Term, client azuremodels.Client) *Config
```

```go
terminal := term.FromEnv()
cfg := command.NewConfigWithTerminal(terminal, client)
```

--------------------------------

### PromptPexContext Go Struct

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Defines the complete context for a generation session, including prompt details, options, and generated artifacts.

```go
type PromptPexContext struct {
    RunID        string            // Unique session identifier
    Prompt       *prompt.File      // Parsed prompt file
    PromptHash   string            // Hash of original prompt
    Options      *PromptPexOptions // Configuration used
    Intent       *string           // Generated intent
    Rules        []string          // Positive output rules
    InverseRules []string          // Negative output rules
    InputSpec    *string           // Input specification
    Tests        []PromptPexTest   // Generated test cases
}
```

--------------------------------

### Invalid Variable Formats

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Examples of incorrect command-line flag formats for defining variables, highlighting errors like missing equals signs, empty keys, or duplicate keys.

```bash
# ❌ Error: missing =
gh models run --var invalid_format

# ❌ Error: empty key
gh models run --var =value

# ❌ Error: duplicate key
gh models run --var x=1 --var x=2

# ✓ Valid
gh models run --var name=Alice --var value="hello world"
```

--------------------------------

### Example of Evaluator Not Found Error

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

This snippet demonstrates a configuration where a plugin evaluator reference is invalid. The system will report an error if the specified 'uses' path does not point to a valid evaluator.

```yaml
evaluators:
  - name: "unknown"
    uses: "plugin/nonexistent"
```

--------------------------------

### Prompt File Mode with Variables

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Uses a prompt file and provides template variables for model inference.

```bash
gh models run --file my_prompt.yml --var name=Alice --var topic=ML
```

--------------------------------

### PopulateFromFlags Method

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Loads model parameters from command-line flags. Handles parsing and validation of flag values.

```go
func (mp *ModelParameters) PopulateFromFlags(flags *pflag.FlagSet) error
```

--------------------------------

### Bash Example: Invalid Variable Format

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Illustrates the error message when a --var flag is used with an incorrect format (missing '='). Use the 'key=value' format for variables.

```bash
gh models run --var invalid_format  # Missing '='
# Error: invalid variable format 'invalid_format', expected 'key=value'
```

--------------------------------

### Ptr

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/util-package.md

Creates a pointer to any value, which is useful for optional configuration fields.

```APIDOC
## Ptr

### Description
Creates a pointer to any given value. This is particularly useful for optional configuration fields where a nil pointer signifies an unset value.

### Signature
```go
func Ptr[T any](value T) *T
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **value** (`T`) - Required - The value for which to create a pointer.

### Returns
- **`*T`** - A pointer to the provided value.

### Request Example
```go
maxTokens := util.Ptr(2048)
// maxTokens: *int = &2048

temperature := util.Ptr(0.7)
// temperature: *float64 = &0.7
```

### Response
None

### Errors
None
```

--------------------------------

### LoadFromFile

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Loads and parses a prompt file from disk. It handles file reading, YAML parsing, and validation of prompt configurations.

```APIDOC
## LoadFromFile

### Description
Loads and parses a prompt file from disk.

### Method
func

### Parameters
#### Path Parameters
- `filePath` (string) - Required - Path to `.prompt.yml` file

### Returns
`(*File, error)` — Parsed File or error

### Errors:
- File not found or unreadable
- Invalid YAML syntax
- Invalid `responseFormat` value
- Missing required fields

### Example:
```go
pf, err := prompt.LoadFromFile("my_prompt.prompt.yml")
if err != nil {
    log.Fatal(err)
}
```
```

--------------------------------

### Bash Example: Reserved Variable Name

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Highlights the error when 'input' is used as a variable name with the generate command. 'input' is reserved; use a different name like '--var prompt_input=value'.

```bash
gh models run --var input=value
# Error: 'input' is a reserved variable name and cannot be used with --var
```

--------------------------------

### Attach HTTP Logging to Context

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Attaches an HTTP logging filename to the context for debugging purposes. This function creates or appends to the specified log file, logging request details. It removes the file before logging starts for a fresh log.

```go
ctx := azuremodels.WithHTTPLogFile(context.Background(), "api-calls.log")
resp, err := client.GetChatCompletionStream(ctx, req, "")
// Writes HTTP request details to api-calls.log
```

--------------------------------

### View Model Details Using List Output

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/list-command.md

Shows how to find a model ID using 'gh models list' and then use that ID to view specific model details with the 'gh models view' command.

```bash
gh models list
# Get model ID
gh models view openai/gpt-4o-mini
```

--------------------------------

### Generate Test Cases for Prompts

Source: https://github.com/github/gh-models/blob/main/README.md

Generate comprehensive test cases for prompts using the PromptPex methodology. This helps ensure prompt robustness.

```shell
gh models generate my_prompt.prompt.yml
```

--------------------------------

### Build gh-models Binary

Source: https://github.com/github/gh-models/blob/main/integration/README.md

Build the gh-models CLI binary. This is a prerequisite for running integration tests.

```bash
cd ..
script/build
```

--------------------------------

### Generate Command Usage with Session File Flag

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Demonstrates loading or saving session state for resumable generation using the '--session-file' flag.

```bash
gh models generate --session-file session.json prompt.yml
```

--------------------------------

### Run All Tests

Source: https://github.com/github/gh-models/blob/main/CONTRIBUTING.md

Execute all tests in the repository to ensure code changes pass.

```bash
go test -v ./...
```

--------------------------------

### Run a prompt with variables

Source: https://github.com/github/gh-models/blob/main/_autodocs/README.md

Execute a prompt defined in a YAML file, providing dynamic values for template variables using the --var flag. This is useful for testing prompts with different inputs.

```bash
gh models run --file prompt.yml \
  --var name="Alice" \
  --var topic="AI"
```

--------------------------------

### Generate Tests with GitHub Models CLI

Source: https://github.com/github/gh-models/blob/main/_autodocs/README.md

Use this command to generate tests with specified effort and ground truth model. Ensure 'prompt.yml' is in the current directory.

```bash
gh models generate --effort high \
  --groundtruth-model "openai/gpt-4.1" \
  prompt.yml
```

--------------------------------

### Basic Prompt Evaluation

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Execute a basic evaluation of a prompt defined in a YAML file. This is the simplest way to test your prompts.

```bash
gh models eval my_prompt.yml
```

--------------------------------

### Build Chat Completion Options

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Creates a ChatCompletionOptions request object from a prompt file configuration and a list of messages. This is useful for setting up API requests.

```go
opts := pf.BuildChatCompletionOptions(messages)
// opts.Model = pf.Model
// opts.MaxTokens = pf.ModelParameters.MaxTokens
// opts.Temperature = pf.ModelParameters.Temperature
// opts.ResponseFormat = pf.ResponseFormat
```

--------------------------------

### Authenticate with GitHub CLI

Source: https://github.com/github/gh-models/blob/main/integration/README.md

Optional step to authenticate with GitHub CLI. This allows integration tests to access live GitHub endpoints.

```bash
gh auth login
```

--------------------------------

### Generate Tests Using Session File

Source: https://github.com/github/gh-models/blob/main/README.md

Load or create a session file to manage and continue the test generation process.

```shell
gh models generate --session-file my_prompt.session.json my_prompt.prompt.yml
```

--------------------------------

### Handling Context Cancellation in Go

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Demonstrates how to check if an error is due to context cancellation using `errors.Is`. This is common in operations that can be aborted.

```go
if errors.Is(err, context.Canceled) {
    fmt.Println("Operation cancelled")
}
```

--------------------------------

### PromptPexTest Struct

Source: https://github.com/github/gh-models/blob/main/_autodocs/types.md

Defines a single generated test case, including input, expected output, predicted output, reasoning, and scenario.

```go
type PromptPexTest struct {
    Input      string
    Expected   string
    Predicted  string
    Reasoning  string
    Scenario   string
}
```

--------------------------------

### List All Available Models

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/list-command.md

Use this command to display all chat models available through the gh-models extension. No arguments or flags are required.

```bash
gh models list
```

--------------------------------

### File Struct Definition

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Defines the structure of a complete .prompt.yml file, including metadata, model configuration, messages, and evaluation settings.

```go
type File struct {
    Name            string
    Description     string
    Model           string
    ModelParameters ModelParameters
    ResponseFormat  *string
    JsonSchema      *JsonSchema
    Messages        []Message
    TestData        []TestDataItem
    Evaluators      []Evaluator
}
```

--------------------------------

### Evaluate Prompts with YAML File

Source: https://github.com/github/gh-models/blob/main/README.md

Evaluate prompts defined in a `.prompt.yml` file against a model. This command runs test cases and displays results.

```shell
gh models eval my_prompt.prompt.yml
```

--------------------------------

### Run Command in Interactive Mode

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Enter an interactive mode to select a model and engage in a REPL session.

```bash
gh models run
# Prompts for model selection, then enters REPL
```

--------------------------------

### Save Prompt File to Disk

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Saves a prompt file to disk. Use this method to write modified prompt configurations back to a file.

```go
err := pf.SaveToFile("updated_prompt.prompt.yml")
```

--------------------------------

### Session File JSON Structure

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Illustrates the JSON structure for saving and loading generation session state.

```json
{
  "runId": "uuid-string",
  "prompt": { /* prompt file content */ },
  "promptHash": "hash",
  "options": { /* generation options */ },
  "intent": "What the prompt does",
  "rules": ["rule 1", "rule 2"],
  "inverseRules": ["inverse rule 1"],
  "inputSpec": "Input specification",
  "tests": [
    {
      "input": "test input",
      "expected": "expected output",
      "predicted": "actual output",
      "reasoning": "why this test",
      "scenario": "test scenario"
    }
  ]
}
```

--------------------------------

### BuildChatCompletionOptions

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/prompt-package.md

Creates a ChatCompletionOptions request object from a prompt file configuration and a list of messages.

```APIDOC
## BuildChatCompletionOptions

### Description
Creates a ChatCompletionOptions request from the prompt file configuration.

### Method
func (f *File)

### Parameters
#### Path Parameters
- `messages` ([]azuremodels.ChatMessage) - Required - Messages to include in request

### Returns
`azuremodels.ChatCompletionOptions` — Configured request object

### Example:
```go
opts := pf.BuildChatCompletionOptions(messages)
// opts.Model = pf.Model
// opts.MaxTokens = pf.ModelParameters.MaxTokens
// opts.Temperature = pf.ModelParameters.Temperature
// opts.ResponseFormat = pf.ResponseFormat
```
```

--------------------------------

### Config Struct Definition

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/command-config.md

Defines the structure for command configuration, including output writers, API client, and terminal-specific settings.

```go
type Config struct {
    Out              io.Writer
    ErrOut           io.Writer
    Client           azuremodels.Client
    IsTerminalOutput bool
    TerminalWidth    int
}
```

--------------------------------

### Run Command with Organization Context

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Specify an organization context for the 'run' command to attribute usage for billing and auditing. Replace 'my-org' with your organization name.

```bash
gh models run --org my-org model "prompt"
```

--------------------------------

### PromptPexPrompts Struct Definition

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Defines the structure for custom system instructions used in each phase of the PromptPex pipeline.

```go
type PromptPexPrompts struct {
    InputSpec          string
    OutputRules        string
    InverseOutputRules string
    Intent             string
    Tests              string
}
```

--------------------------------

### Generate Command Usage with Effort Flag

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Shows how to specify the effort level for test generation using the '--effort' flag.

```bash
gh models generate --effort high prompt.yml
```

--------------------------------

### Create a Git Tag for Release

Source: https://github.com/github/gh-models/blob/main/DEV.md

Tag the current commit on the main branch to trigger a new release. This is necessary for the production build.

```shell
git tag v0.0.x main
git push origin tag v0.0.x
```

--------------------------------

### ContextLimits

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Returns a human-readable summary of the model's context limits, such as input and output token capacities. This provides an easy way to understand a model's operational constraints.

```APIDOC
## ContextLimits

### Description
Returns human-readable context limit summary.

### Signature:
```go
func (m *ModelDetails) ContextLimits() string
```

### Returns:
`string` — Formatted string like "up to 128000 input tokens and 4096 output tokens"

### Example:
```go
details := model.ModelDetails()
fmt.Println(details.ContextLimits())
// Output: up to 128000 input tokens and 4096 output tokens
```
```

--------------------------------

### Generate with Session File Bash Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Generates tests and saves the session state to a specified file.

```bash
gh models generate --session-file session.json prompt.yml
# Generates tests, saves to session.json
```

--------------------------------

### Model Parameters Configuration

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Configure model behavior such as maximum tokens, temperature for randomness, and topP for nucleus sampling. These fields are optional.

```yaml
modelParameters:
  maxTokens: 2048
  temperature: 0.7
  topP: 0.95
```

--------------------------------

### Run Command with Organization Context

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Execute a model within a specific organization's context.

```bash
gh models run --org acme-corp openai/gpt-4o-mini "team prompt"
```

--------------------------------

### ModelParameters.PopulateFromFlags

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Loads model parameters from the provided command-line flags.

```APIDOC
## PopulateFromFlags

### Description
Load parameters from command flags.

### Signature
```go
func (mp *ModelParameters) PopulateFromFlags(flags *pflag.FlagSet) error
```

### Parameters
#### Path Parameters
- **flags** (*pflag.FlagSet) - Required - Flag set to parse

### Returns
error - Parse error if invalid value

### Flags
- `"max-tokens"`
- `"temperature"`
- `"top-p"`
```

--------------------------------

### Template Variable Syntax

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Illustrates the basic syntax for template variables used in prompts. Variables are case-sensitive and are not substituted if missing.

```bash
gh models run --var name=Alice --var topic=AI prompt.yml
# Available: {{name}}, {{topic}}
```

--------------------------------

### View Specific Model

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/view-command.md

Displays detailed information for a specified model ID. Ensure the model ID is in the format 'publisher/name'.

```bash
gh models view openai/gpt-4o-mini
```

--------------------------------

### Create Unauthenticated Azure Client

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Creates a mock client for unauthenticated access when a token is not available. This client returns no models.

```go
client = azuremodels.NewUnauthenticatedClient()
// Displays: "No GitHub token found. Please run 'gh auth login'"
```

--------------------------------

### Create or Resume Session Bash Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Command to create a new generation session or resume an existing one using a session file.

```bash
# Create new or resume existing session
gh models generate --session-file my.session.json prompt.yml
```

--------------------------------

### Default Model Aliases for PromptPex

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Shows the default model assignments for various purposes within the PromptPex pipeline.

```go
Rules:       "openai/gpt-4o"
Tests:       "openai/gpt-4o"
Groundtruth: "openai/gpt-4o"
Eval:        "openai/gpt-4o"
```

--------------------------------

### Handling RateLimitError in Go

Source: https://github.com/github/gh-models/blob/main/_autodocs/errors.md

Demonstrates how to catch and handle RateLimitError, including retrying the operation after a specified duration. This is useful when API rate limits are exceeded.

```go
package main

import (
	"errors"
	"fmt"
	"time"
	"your_module/azuremodels"
)

func main() {
	var err error // Assume err is an existing error variable

	var rateLimitErr *azuremodels.RateLimitError
	if errors.As(err, &rateLimitErr) {
		fmt.Printf("Rate limited, retry after: %v\n", rateLimitErr.RetryAfter)
		time.Sleep(rateLimitErr.RetryAfter)
		// Retry operation
	}
}
```

--------------------------------

### NewRunCommand

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/run-command.md

Creates a new Cobra command instance for the 'run' functionality, allowing CLI integration.

```APIDOC
## NewRunCommand

### Description
Creates a new run command for the CLI.

### Signature
```go
func NewRunCommand(cfg *command.Config) *cobra.Command
```

### Returns
* cobra.Command - Cobra command with run functionality
```

--------------------------------

### Create Pointer to Value

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/util-package.md

Generates a pointer to any given value. This is useful for optional configuration fields where a nil pointer signifies absence.

```Go
func Ptr[T any](value T) *T

```

```Go
maxTokens := util.Ptr(2048)
// maxTokens: *int = &2048

temperature := util.Ptr(0.7)
// temperature: *float64 = &0.7

```

--------------------------------

### List and Filter Models

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Retrieves a list of available models, sorts them, and filters for chat-compatible models. The output displays the model ID and its friendly name.

```go
models, _ := client.ListModels(ctx)
azuremodels.SortModels(models)

// Filter to chat models
chatModels := make([]*azuremodels.ModelSummary, 0)
for _, model := range models {
    if model.IsChatModel() {
        chatModels = append(chatModels, model)
    }
}

for _, model := range chatModels {
    fmt.Printf("%s (%s)\n", model.ID, model.FriendlyName)
}
```

--------------------------------

### EvaluationSummary Struct

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/eval-command.md

Defines the Go struct for the overall evaluation summary, including name, description, model, test results, and statistics.

```go
type EvaluationSummary struct {
    Name        string       // Evaluation name
    Description string       // Evaluation description
    Model       string       // Model being evaluated
    TestResults []TestResult // Results per test case
    Summary     Summary      // Overall statistics
}
```

--------------------------------

### Detecting Rate Limit Errors

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Shows how to check if an error is a rate limit error using errors.As and access its properties.

```go
var rateLimitErr *azuremodels.RateLimitError
if errors.As(err, &rateLimitErr) {
    fmt.Printf("Retry after: %v\n", rateLimitErr.RetryAfter)
}
```

--------------------------------

### Set Generation Effort to Low

Source: https://github.com/github/gh-models/blob/main/_autodocs/configuration.md

Use the 'low' effort setting for fast experimentation with few rules and tests. This is suitable for quick validation and initial testing.

```bash
gh models generate --effort low prompt.yml
```

--------------------------------

### Retrieve HTTP Log Filename from Context

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Retrieves the HTTP log filename from the context. Returns the log file path if set, otherwise an empty string.

```go
logFile := azuremodels.HTTPLogFileFromContext(ctx)
if logFile != "" {
    fmt.Printf("Logging to: %s\n", logFile)
}
```

--------------------------------

### Generate Command Usage with Groundtruth Model Flag

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Illustrates using the '--groundtruth-model' flag to specify a model for generating expected outputs.

```bash
gh models generate --groundtruth-model "openai/gpt-4.1" prompt.yml
```

--------------------------------

### NewUnauthenticatedClient

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/azure-client.md

Creates a mock client for unauthenticated access. This is useful when an authentication token is not available, and it provides a client that returns no models.

```APIDOC
## NewUnauthenticatedClient

### Description
Creates a mock client for unauthenticated access (used when token not available).

### Signature:
```go
func NewUnauthenticatedClient() Client
```

### Returns:
`Client` — Mock client that returns no models

### Usage:
```go
client = azuremodels.NewUnauthenticatedClient()
// Displays: "No GitHub token found. Please run 'gh auth login'"
```
```

--------------------------------

### Define Model Details Structure

Source: https://github.com/github/gh-models/blob/main/_autodocs/types.md

Provides detailed information about a model, including its description, license, and supported modalities.

```Go
type ModelDetails struct {
    Description               string
    Evaluation                string
    License                   string
    LicenseDescription        string
    Notes                     string
    Tags                      []string
    SupportedInputModalities  []string
    SupportedOutputModalities []string
    SupportedLanguages        []string
    MaxOutputTokens           int
    MaxInputTokens            int
    RateLimitTier             string
}
```

--------------------------------

### GitHub Models CLI Project Structure

Source: https://github.com/github/gh-models/blob/main/_autodocs/README.md

This illustrates the directory structure of the gh-models project, showing the organization of commands, packages, and internal modules.

```tree
gh-models/
├── cmd/
│   ├── run/           # Run command implementation
│   ├── eval/          # Eval command implementation
│   ├── generate/      # Generate command (PromptPex)
│   ├── list/          # List command implementation
│   ├── view/          # View command implementation
│   └── root.go        # Root command with all subcommands
├── pkg/
│   ├── command/       # Shared Config type
│   ├── prompt/        # Prompt file types and utilities
│   └── util/          # Helper functions
├── internal/
│   ├── azuremodels/   # Azure API client
│   ├── modelkey/      # Model identifier parsing
│   └── sse/           # Server-sent events (streaming)
└── main.go            # Entry point
```

--------------------------------

### Generate with Template Variables Bash Command

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/generate-command.md

Uses template variables to customize prompt generation.

```bash
gh models generate \
  --var category="math" \
  --var level="high-school" \
  prompt.yml
```

--------------------------------

### WriteToOut Method

Source: https://github.com/github/gh-models/blob/main/_autodocs/api-reference/command-config.md

Writes a given string message to the standard output writer configured in the Config. This method is useful for logging or displaying information during command execution.

```go
func (c *Config) WriteToOut(message string)
```

```go
cfg.WriteToOut("Starting model inference...\n")
```

--------------------------------

### Run Inference in Single-Shot Mode

Source: https://github.com/github/gh-models/blob/main/README.md

Execute a single prompt against a specified model and exit. This mode is useful for quick queries.

```shell
gh models run openai/gpt-4o-mini "why is the sky blue?"
```