### Run Basic Examples

Source: https://github.com/justi/ruby_llm-contract/blob/main/examples/README.md

Execute the basic test adapter examples without needing API keys. These cover incremental layers from plain prompt to trace inspection.

```bash
ruby examples/00_basics.rb
ruby examples/01_fallback_showcase.rb
ruby examples/03_summarize_with_keywords.rb
ruby examples/04_summarize_and_translate.rb
ruby examples/05_eval_dataset.rb
ruby examples/06_retry_variants.rb
```

--------------------------------

### Run Real LLM Example

Source: https://github.com/justi/ruby_llm-contract/blob/main/examples/README.md

Execute an example that requires a real LLM provider API key or a local Ollama server.

```bash
ruby examples/02_real_llm_minimal.rb
```

--------------------------------

### Install ruby_llm-contract Gem

Source: https://github.com/justi/ruby_llm-contract/blob/main/README.md

Add this line to your Gemfile to install the ruby_llm-contract gem.

```ruby
gem "ruby_llm-contract"
```

--------------------------------

### Configure RubyLLM and Contract Gem

Source: https://github.com/justi/ruby_llm-contract/blob/main/README.md

Configure the OpenAI API key and default model for RubyLLM. The Contract gem also needs to be configured, though an empty block is sufficient for basic setup.

```ruby
RubyLLM.configure do |c|
  c.openai_api_key = ENV["OPENAI_API_KEY"]
  c.default_model  = "gpt-4.1-mini"   # used when a Step has no explicit model
end

# Required: boots the gem so `Step.run` knows how to talk to your LLM.
# Empty block is fine. Pass options here if you need them (e.g. `c.logger`).
RubyLLM::Contract.configure { }
```

--------------------------------

### Global Stub for All Steps

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/testing.md

Use `stub_all_steps` to provide a single canned response for all steps in an example. This is ideal for high-level integration tests where step output details are not critical.

```ruby
stub_all_steps(response: { default: true })
```

--------------------------------

### Minimal Multimodal Input Example

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/multimodal_input.md

This snippet demonstrates how to define a contract step for extracting data from an attached PDF. It includes setting an attachment token estimate, max cost, and a retry policy. Use this when your contract needs to process file attachments.

```ruby
class ExtractInvoiceData < RubyLLM::Contract::Step::Base
  prompt "Extract invoice fields from the attached PDF. Return JSON."

  output_schema do
    string :vendor
    string :invoice_number
    number :total_amount
    string :currency, enum: %w[USD EUR PLN GBP]
  end

  validate("currency present") { |o, _| !o[:currency].nil? }

  # REQUIRED when max_cost is set and the contract receives an attachment.
  # Conservative estimate of attachment input tokens (provider/model-specific).
  attachment_token_estimate 15_000   # ~12 PDF pages at ~1250 tokens/page

  max_cost 0.10

  retry_policy do
    escalate "gpt-4.1-mini",
             { model: "gpt-5", reasoning_effort: "high" }
  end
end

result = ExtractInvoiceData.run(
  "Look for vendor, amount, currency.",
  context: { attachment: "tmp/invoice.pdf" }
)

result.status         # => :ok
result.parsed_output  # => { vendor: "...", invoice_number: "...", ... }
result.trace[:cost]   # => 0.0042  (total across attempts)
```

--------------------------------

### RSpec Setup for Ruby LLM Contract

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/testing.md

Include the RSpec contract library in your spec_helper.rb to gain access to testing matchers and helpers.

```ruby
require "ruby_llm/contract/rspec"
```

--------------------------------

### Setup Contract Testing in Rails

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/rails_integration.md

Add the contract testing require statement to your RSpec or Minitest helper file.

```ruby
require "ruby_llm/contract/rspec"    # or ruby_llm/contract/minitest
```

--------------------------------

### Gate CI with pass_eval

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/eval_first.md

Use `pass_eval` to gate your Continuous Integration process. Ensure a minimum score is met for the specified evaluation. Refer to Getting Started for the full matcher chain.

```ruby
pass_eval("regression").with_context(model: "...").with_minimum_score(0.8)
```

--------------------------------

### Retry Policy DSL Example

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Use this DSL to define your retry policy within your step. The models are tried in order, falling back to the next model if the current one fails validation on the hardest eval.

```ruby
retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini]
```

--------------------------------

### Define Prompt Structure with AST Nodes

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/prompt_ast.md

Use the `prompt do ... end` block to define structured prompts with various node types like `system`, `rule`, `section`, `example`, and `user`. This approach ensures clean composition, diffing, and snapshot testing.

```ruby
prompt do
  system  "You summarize articles for a UI card."     # system message
  rule    "Return valid JSON only."                   # appended as separate system message
  section "AUDIENCE", "Rails developers"              # labeled system message: [AUDIENCE]\n...
  example input:  "Ruby 3.4 ships frozen strings...", # user/assistant few-shot pair
          output: '{"tldr":"...","takeaways":[...],"tone":"analytical"}'
  user    "{input}"                                   # user message with interpolation
end
```

--------------------------------

### Implement Model Fallback with Retry Policy

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/best_practices.md

Define a contract step that uses `output_schema` and `validate` to ensure output quality. The `retry_policy` specifies a list of models to try in order, starting with cheaper options and falling back to more expensive ones if validation fails. This approach optimizes costs by using the cheapest viable model for each task.

```ruby
class SummarizeArticle < RubyLLM::Contract::Step::Base
  output_schema do
    string :tldr
    array  :takeaways, of: :string, min_items: 3, max_items: 5
    string :tone, enum: %w[neutral positive negative analytical]
  end

  validate("TL;DR fits the card")  { |o, _| o[:tldr].length <= 200 }
  validate("takeaways are unique") { |o, _| o[:takeaways] == o[:takeaways].uniq }

  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
end
```

--------------------------------

### RSpec Contract Test Example

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/rails_integration.md

Write RSpec tests to verify that LLM contract steps pass and produce expected outputs. Note the handling of JSONB/JSON column round-tripping.

```ruby
RSpec.describe ArticlesController do
  it "saves the summary when the step passes" do
    stub_step(SummarizeArticle, response: {
      tldr: "...", takeaways: %w[a b c], tone: "analytical"
    })

    post :summarize, params: { id: article.id }

    # NOTE: jsonb/json column round-trips as string keys on reload.
    expect(article.reload.summary["tldr"]).to eq("...")
  end
end
```

--------------------------------

### Runtime Call Path for Step::Base

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/relation_to_agent.md

Illustrates the execution flow when using Step::Base, showing its interaction with Adapters::RubyLLM and RubyLLM.chat.

```text
Step.run(input)
  → Runner
  → Adapters::RubyLLM
  → RubyLLM.chat(model: ...)
  → ... .ask(prompt)
```

--------------------------------

### Initialize Test Adapter with Sequential Responses

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/testing.md

Use this to simulate multiple LLM calls in sequence. Provide an array of responses, where each element is returned for a subsequent call.

```ruby
# Multiple sequential responses (one per call)
adapter = RubyLLM::Contract::Adapters::Test.new(
  responses: [
    { tldr: "...", takeaways: %w[a b c], tone: "neutral" },
    { tldr: "...", takeaways: %w[x y z], tone: "analytical" }
  ]
)

result = SummarizeArticle.run("article text", context: { adapter: adapter })
result.ok?  # => true
```

--------------------------------

### Batch Generation with Parallel Threads

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

Demonstrates how to perform parallel batch generation using Ruby threads with the SummarizeBatch step. Ensure Rails executor wrap for proper context.

```ruby
class SummarizeBatch < RubyLLM::Contract::Step::Base
  output_schema do
    array :summaries do
      object do
        string :article_id
        string :tldr
      end
    end
  end
  retry_policy models: %w[gpt-4.1-mini gpt-4.1]
end

# Your orchestrator
threads = 10.times.map do |i|
  Thread.new { Rails.application.executor.wrap { SummarizeBatch.run(input(i)) } }
end
results = threads.map(&:value)
```

--------------------------------

### Define BuildArticleCard Step

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

The final step shapes the output for a UI card, including a headline, summary, hashtags, and a sentiment icon. It enforces a headline character limit and ensures the summary is the verbatim TL;DR from the previous step.

```ruby
class BuildArticleCard < RubyLLM::Contract::Step::Base
  input_type Hash

  output_schema do
    string :headline
    string :summary
    array  :hashtags, of: :string
    string :sentiment_icon, enum: %w[😐 🙂 ⚠️ 🧠]
  end

  prompt do
    rule "Headline <= 70 chars. Summary is the incoming tldr reprinted verbatim."
    rule "Pick sentiment_icon from: 😐 neutral, 🙂 positive, ⚠️ negative, 🧠 analytical."
    user "TL;DR: {tldr}\nTone: {tone}\nHashtags: {hashtags}"
  end

  validate("summary is the tldr verbatim") { |o, input| o[:summary] == input[:tldr] }
```

--------------------------------

### Validate No Markdown Headings in TL;DR (Ruby)

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/best_practices.md

Checks that the TL;DR section does not contain markdown headings (lines starting with 1 to 6 '#' characters). This prevents markdown leakage into plain-text outputs.

```ruby
validate("no markdown headings in the TL;DR") do |o, |
  !o[:tldr].match?(/^\#{1,6}\s/)
end
```

--------------------------------

### Compare AI Models for Cost Efficiency

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

Compare different AI models for a specific evaluation case to find the most cost-effective option that meets performance criteria. Use `best_for` to find the cheapest model above a certain score threshold.

```ruby
comparison = SummarizeArticle.compare_models("regression",
  candidates: [{ model: "gpt-4.1-nano" }, { model: "gpt-4.1-mini" }])

comparison.print_summary
comparison.best_for(min_score: 0.95)  # => cheapest model at >= 95%
```

--------------------------------

### Estimate LLM Evaluation Costs

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/eval_first.md

Use `estimate_eval_cost` to get a cost projection for running evaluations on specified models without making actual LLM calls. This is helpful for budgeting and deciding which models to use for regression testing in CI.

```ruby
SummarizeArticle.estimate_eval_cost("regression",
  models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1])
# => { "gpt-4.1-nano" => 0.00041, "gpt-4.1-mini" => 0.0018, "gpt-4.1" => 0.0092 }
```

--------------------------------

### Before: Raw HTTP Call

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

This snippet shows the original implementation using a raw HTTP client call to an LLM service before adopting the ruby_llm-contract.

```ruby
class ArticleSummaryService
  def call(article_text)
    response = LlmClient.new(model: "gpt-4o-mini").call(prompt(article_text))
    JSON.parse(response[:content], symbolize_names: true)
  end
end
```

--------------------------------

### Initialize Test Adapter with String JSON Response

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/testing.md

Use this when the expected LLM response is a JSON string. The adapter will return this exact string for the first call.

```ruby
# String JSON
adapter = RubyLLM::Contract::Adapters::Test.new(
  response: '{"tldr":"...","takeaways":["a","b","c"],"tone":"neutral"}'
)

result = SummarizeArticle.run("article text", context: { adapter: adapter })
result.ok?  # => true
```

--------------------------------

### Stubbing RubyLLM::Chat for Attachment Testing

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/multimodal_input.md

Use this RSpec example to stub RubyLLM::Chat to verify that attachments are correctly forwarded to the chat.ask method. This is useful for testing adapter call shapes without needing a full adapter implementation.

```ruby
RSpec.describe ExtractInvoiceData do
  it "forwards attachment to chat.ask" do
    expect_any_instance_of(RubyLLM::Chat).to receive(:ask)
      .with(anything, with: "fixtures/invoice.pdf")
      .and_return(double(content: '{"vendor":"X",...}', input_tokens: 200, output_tokens: 50))

    result = described_class.run("extract", context: { attachment: "fixtures/invoice.pdf" })
    expect(result.status).to eq(:ok)
  end
end
```

--------------------------------

### Run and Inspect SummarizeArticle Contract Results

Source: https://github.com/justi/ruby_llm-contract/blob/main/README.md

Demonstrates how to execute the SummarizeArticle contract and access its status, parsed output, and detailed trace information.

```ruby
result = SummarizeArticle.run(article_text)
result.status           # => :ok  (or :validation_failed if all steps fail)
result.parsed_output    # => { tldr: "...", takeaways: [...], tone: "..." }
result.trace[:model]    # => "gpt-4.1-mini"  (winning step)
result.trace[:cost]     # => 0.000520        (total across all attempts)
```

```ruby
result.trace[:attempts]
# => [
#      {
#        attempt: 1,
#        model: "gpt-4.1-nano",
#        status: :validation_failed,
#        usage: { input_tokens: 256, output_tokens: 84 },
#        latency_ms: 45,
#        cost: 0.000100
#      },
#      {
#        attempt: 2,
#        model: "gpt-4.1-mini",
#        status: :ok,
#        usage: { input_tokens: 256, output_tokens: 92 },
#        latency_ms: 92,
#        cost: 0.000420
#      }
#    ]
```

--------------------------------

### Preflight Checks for Cost and Resource Limits

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/why.md

Implement preflight checks using `max_input`, `max_output`, and `max_cost` to prevent runaway costs on pathological inputs. These parameters set hard limits before execution.

```ruby
max_input, max_output, max_cost
```

--------------------------------

### Optimize Against Real Models

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Optimize your retry policy by making live API calls. Set `LIVE=1` to enable real calls and `RUNS=3` to average results over multiple runs, which is crucial for models with high variance.

```bash
LIVE=1 RUNS=3 rake ruby_llm_contract:optimize \
  STEP=SummarizeArticle \
  CANDIDATES=gpt-4.1-nano,gpt-4.1-mini@low,gpt-4.1-mini,gpt-4.1
```

--------------------------------

### Configure Unknown Pricing Behavior

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Set `on_unknown_pricing` to `:warn` to receive a warning instead of a refusal when pricing information is missing for `max_cost`. The default is `:refuse`.

```ruby
max_cost 0.01, on_unknown_pricing: :warn
```

--------------------------------

### Running a Contract and Inspecting Results

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Execute a contract and examine the status, parsed output, and detailed trace of model attempts. The trace includes information about each model tried, its status, cost, and latency.

```ruby
result = SummarizeArticle.run(article_text)

result.status           # => :ok
result.parsed_output    # => { tldr: "...", takeaways: [...], tone: "analytical" }
result.trace[:model]    # => "gpt-4.1-mini"  (first model that passed)
result.trace[:cost]     # => 0.00052  (sum of all attempts)
result.trace[:attempts]
# => [
#   { attempt: 1, model: "gpt-4.1-nano", status: :validation_failed,
#     cost: 0.00010, latency_ms: 45, ... },
#   { attempt: 2, model: "gpt-4.1-mini", status: :ok,
#     cost: 0.00042, latency_ms: 92, ... }
# ]
```

--------------------------------

### Initialize Test Adapter with Hash Response

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/testing.md

Use this when the expected LLM response is a Ruby Hash. The adapter automatically converts the hash to a JSON string before returning it.

```ruby
# Hash — auto-converted to JSON
adapter = RubyLLM::Contract::Adapters::Test.new(
  response: { tldr: "...", takeaways: %w[a b c], tone: "neutral" }
)

result = SummarizeArticle.run("article text", context: { adapter: adapter })
result.ok?  # => true
```

--------------------------------

### Directory Structure for Step Classes

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/rails_integration.md

Recommended directory for placing contract step classes in a Rails application.

```ruby
app/contracts/summarize_article.rb   # class SummarizeArticle
```

--------------------------------

### Run and Inspect Pipeline Results

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

Execute a pipeline and inspect its success status, outputs by step, and trace information like total cost and latency.

```ruby
result = ArticleCardPipeline.run(article_text, context: { adapter: adapter })
result.ok?                          # => true
result.outputs_by_step[:summarize]  # => { tldr: "...", takeaways: [...], tone: "analytical" }
result.outputs_by_step[:card]       # => { headline: "...", summary: "...", ... }
result.trace.total_cost             # => 0.000128 (all steps combined)
result.trace.total_latency_ms       # => 2340
```

--------------------------------

### Directly Run a Step with Overrides

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Execute a specific step directly with custom configurations, such as overriding the retry policy or specifying a model. This is helpful for debugging when a step fails unexpectedly or when you need to inspect the raw output.

```ruby
context: { retry_policy_override: nil, model: "gpt-4.1" }
```

--------------------------------

### Compare Models for Selection

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/eval_first.md

Compare different models against a 'regression' evaluation to identify the best performing candidate. This is done after prompt stability is achieved.

```ruby
comparison = SummarizeArticle.compare_models(
  "regression",
  candidates: [{ model: "gpt-4.1-nano" }, { model: "gpt-4.1-mini" }, { model: "gpt-4.1" }]
)

comparison.best_for(min_score: 0.95)
```

--------------------------------

### Basic Prompt as a Plain String

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/prompt_ast.md

A simple prompt can be defined as a plain string, which will be automatically wrapped as a single user message within the AST.

```ruby
prompt "Summarize this article for a UI card. {input}"
```

--------------------------------

### Compare Prompts with compared_with

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/eval_first.md

Change prompts only through comparison in CI. This ensures any regression blocks the merge by comparing against a previous version.

```ruby
pass_eval(...).compared_with(SummarizeArticleV1)
```

--------------------------------

### Guard Against Empty/Placeholder Output

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/best_practices.md

Use validation rules to ensure that LLM outputs, like 'tldr' and 'takeaways', are substantive and not boilerplate. This prevents UI rendering issues caused by empty or unhelpful content.

```ruby
output_schema do
  string :tldr
  array  :takeaways, of: :string, min_items: 3, max_items: 5
  string :tone, enum: %w[neutral positive negative analytical]
end

validate("tldr is substantive") do |o, _|
  o[:tldr].to_s.split.length >= 5   # at least five words
end

validate("no boilerplate takeaways") do |o, _|
  o[:takeaways].none? { |t|
    t.downcase.start_with?("this article")
  }
end
```

--------------------------------

### Compare Models with Production Mode

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Use `compare_models` with `production_mode` to analyze the effective cost of model chains, including fallback strategies. This helps in understanding the true cost per successful output when validators reject a portion of the results.

```ruby
SummarizeArticle.compare_models(
  "dense_article",
  candidates: [{ model: "gpt-5-nano" }, { model: "gpt-5-mini", reasoning_effort: "low" }],
  production_mode: { fallback: "gpt-5-mini" }
).print_summary
```

--------------------------------

### Rails Initializer Configuration for Ruby LLM Contract

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/rails_integration.md

Configure API keys and default model/adapter for Ruby LLM Contract within a Rails application.

```ruby
# config/initializers/ruby_llm_contract.rb
RubyLLM.configure do |c|
  c.openai_api_key    = ENV.fetch("OPENAI_API_KEY", nil)
  c.anthropic_api_key = ENV.fetch("ANTHROPIC_API_KEY", nil)
end

RubyLLM::Contract.configure do |c|
  c.default_model   = Rails.env.production? ? "gpt-5-mini" : "gpt-5-nano"
  c.default_adapter = RubyLLM::Contract::Adapters::RubyLLM.new
end
```

--------------------------------

### Estimating Cost with Attachments

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/multimodal_input.md

For pre-flight estimate tests, use the .estimate_cost method with input and attachment parameters. No adapter stubbing is required for this type of test.

```ruby
# For pre-flight estimate tests, just call .estimate_cost(input: ..., attachment: ...) -- no adapter stub needed.
```

--------------------------------

### LLM Contract Configuration in Ruby

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/why.md

Configure maximum input tokens, output tokens, and cost limits for LLM API calls. Set up a retry policy to escalate to more expensive models only on failure.

```ruby
max_input  2_000   # refuses before calling the API if tokens exceed budget
max_output 4_000
max_cost   0.01    # refuses if estimated cost exceeds cap
retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]  # cheap first, escalate only on failure
```

--------------------------------

### Optimizing Traffic Costs with Retry Policies

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/why.md

Manage costs by applying `retry_policy` and `optimize_retry_policy` to ensure that traffic, especially the 80/20 segment, utilizes premium model rates efficiently. This helps balance cost and performance.

```ruby
retry_policy
optimize_retry_policy
```

--------------------------------

### Database Migration for LLM Call Logs

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/rails_integration.md

A sample Rails migration to create the `ai_call_logs` table for storing detailed information about each LLM interaction. This includes fields for step, model, status, latency, token usage, cost, and validation errors.

```ruby
# rails g model AiCallLog step:string model:string status:string ...
create_table :ai_call_logs do |t|
  t.string  :step, null: false
  t.string  :model
  t.string  :status, null: false
  t.integer :latency_ms
  t.integer :input_tokens
  t.integer :output_tokens
  t.decimal :cost, precision: 10, scale: 6
  t.jsonb   :validation_errors, default: []
  t.timestamps
end
add_index :ai_call_logs, :step
add_index :ai_call_logs, :status
```

--------------------------------

### Handle Exceeded Budget Limits

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Use `max_input`, `max_output`, or `max_cost` as preflight checks. The LLM call is skipped if an estimate exceeds these limits, resulting in zero tokens spent and zero cost.

```ruby
result = SummarizeArticle.run(giant_10mb_document)
result.status            # => :limit_exceeded
result.validation_errors
# => ["Input token limit exceeded: estimated 32000 tokens (heuristic ±30%), max 2000"]
```

--------------------------------

### Define ArticleCardPipeline

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

This defines the complete pipeline by chaining the SummarizeArticle, GenerateHashtags, and BuildArticleCard steps in sequence, aliasing them for clarity within the pipeline.

```ruby
class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
  step SummarizeArticle, as: :summarize
  step GenerateHashtags, as: :tag
  step BuildArticleCard, as: :card
end
```

--------------------------------

### Schema vs. Manual Validation

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/output_schema.md

Compares manual validation methods with the concise schema declaration for output validation. Use schema for a single, declarative approach.

```ruby
# WITHOUT schema — many validates:
validate("tldr must be a string")          { |o| o[:tldr].is_a?(String) }
validate("takeaways must be an array")     { |o| o[:takeaways].is_a?(Array) }
validate("takeaways 3 to 5")               { |o| (3..5).cover?(o[:takeaways].size) }
ALLOWED_TONES = %w[neutral positive negative analytical].freeze
validate("tone must be an allowed label") { |o| ALLOWED_TONES.include?(o[:tone]) }

# WITH schema — one declaration:
output_schema do
  string :tldr
  array  :takeaways, of: :string, min_items: 3, max_items: 5
  string :tone, enum: %w[neutral positive negative analytical]
end
```

--------------------------------

### Estimate Cost with Attachment

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/multimodal_input.md

Use `estimate_cost` with the `attachment:` kwarg for budget planning. This includes an estimate for attachment tokens in the total input token count.

```ruby
ExtractInvoiceData.estimate_cost(
  input: "Look for vendor, amount...",
  attachment: "tmp/invoice.pdf"
)
# => { model: "gpt-4.1-mini",
#      input_tokens: 15_320,
#      output_tokens_estimate: 256,
#      estimated_cost: 0.0123 }
```

--------------------------------

### Compare Models Directly

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Use this method to test a specific hypothesis by directly comparing models on a given task. This is useful for isolating performance differences without rerunning the entire optimization pass.

```ruby
SummarizeArticle.compare_models("critical_tone", candidates: [{ model: "gpt-5-mini", reasoning_effort: "medium" }], runs: 3)
```

--------------------------------

### Register Custom Model Pricing

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Register pricing for custom or fine-tuned models using `RubyLLM::Contract::CostCalculator.register_model` to avoid `max_cost` failures when pricing is unknown.

```ruby
RubyLLM::Contract::CostCalculator.register_model("ft:gpt-4o-custom",
  input_per_1m: 3.0, output_per_1m: 6.0)
```

--------------------------------

### Propagating Step Overrides in Threads

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

Shows how to manually propagate step adapter overrides when spawning new threads for parallel execution in tests. This ensures thread-local overrides are correctly applied.

```ruby
overrides = RubyLLM::Contract.step_adapter_overrides.dup
Thread.new { RubyLLM::Contract.step_adapter_overrides = overrides; SummarizeBatch.run(input) }
```

--------------------------------

### Tribunal and Contract Together: CI and Production Flow

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/relation_to_tribunal.md

Combines Tribunal for CI-based quality gating and Contract for runtime validation in production. Tribunal prevents regressions before merge, while Contract ensures valid outputs for every user request.

```text
CI (before merge):
  define_eval(frozen dataset) ──► run Step ──► [Tribunal grades each case]
                                                       │
                                                       ▼
                                                regression gate
                                           (prevents quality drift over time)
                                                       │
                                                       ▼
                                              ✅ merge allowed
                                                       │
                                                       ▼
PROD (every request):
  your code ──► Step.run ──► LLM ──► [contract] ──► output ──► your code
                                         ▲
                                         │ keeps bad outputs out of prod
```

--------------------------------

### Run Offline Check

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/optimizing_retry_policy.md

Perform an initial offline check to verify wiring and ensure evals load and candidates parse. This uses `sample_response` and does not make API calls.

```bash
rake ruby_llm_contract:optimize \
  STEP=SummarizeArticle \
  CANDIDATES=gpt-4.1-nano,gpt-4.1-mini@low,gpt-4.1-mini,gpt-4.1
```

--------------------------------

### Define SummarizeArticle Step

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

This step summarizes an article for a UI card, extracting a TL;DR, key takeaways, and a tone. It includes validation to ensure the TL;DR fits within a 200-character limit.

```ruby
class SummarizeArticle < RubyLLM::Contract::Step::Base
  prompt <<~PROMPT
    Summarize this article for a UI card. Return a short TL;DR,
    3 to 5 key takeaways, and a tone label.

    {input}
  PROMPT

  output_schema do
    string :tldr
    array  :takeaways, of: :string, min_items: 3, max_items: 5
    string :tone, enum: %w[neutral positive negative analytical]
  end

  validate("TL;DR fits the card") { |o, _| o[:tldr].length <= 200 }
end
```

--------------------------------

### After: Define Contract

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

This snippet defines a new service contract using ruby_llm-contract, specifying the model, prompt structure, output schema, validation rules, and retry policy.

```ruby
class SummarizeArticle < RubyLLM::Contract::Step::Base
  model "gpt-4.1-mini"

  prompt do
    system "You summarize articles for a UI card."
    rule "Return valid JSON only."
    user "{input}"
  end

  output_schema do
    string :tldr
    array  :takeaways, of: :string, min_items: 3, max_items: 5
    string :tone, enum: %w[neutral positive negative analytical]
  end

  validate("TL;DR fits the card") { |o, _| o[:tldr].length <= 200 }
  retry_policy models: %w[gpt-4.1-mini gpt-4.1]
end
```

--------------------------------

### Handle Pipeline Fail-Fast Behavior

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

Observe how pipelines stop execution when a step fails validation, preventing downstream steps from running and saving resources.

```ruby
# Summarize returns a TL;DR over 200 chars → the "TL;DR fits the card" validate fails
adapter = RubyLLM::Contract::Adapters::Test.new(response: {
  tldr: "x" * 500,
  takeaways: %w[one two three],
  tone: "neutral"
})

result = ArticleCardPipeline.run("article text", context: { adapter: adapter })
result.failed?        # => true
result.failed_step    # => :summarize (validate rejected; retries exhausted)
# tag and card never run — no downstream tokens spent on garbage
```

--------------------------------

### Configure Rake Task for CI Gate

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

Set up a Rake task for Continuous Integration to enforce quality gates on your LLM contracts. Configure minimum score, maximum cost, and whether to fail on regressions.

```ruby
# Rakefile
require "ruby_llm/contract/rake_task"
RubyLLM::Contract::RakeTask.new do |t|
  t.minimum_score = 0.8
  t.maximum_cost = 0.05
  t.fail_on_regression = true
  t.save_baseline = false # read-only in CI; refresh baselines in a separate workflow
end
```

--------------------------------

### Replace Caller: Before and After

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/migration.md

This snippet demonstrates how to replace the old caller logic with the new contract-based approach, including handling successful and failed calls.

```ruby
# Before
parsed = ArticleSummaryService.new.call(article_text)
Article.update!(summary: parsed["tldr"])

# After
result = SummarizeArticle.run(article_text)
if result.ok?
  Article.update!(summary: result.parsed_output[:tldr])
else
  Rails.logger.warn "Summary failed: #{result.status}"
end
```

--------------------------------

### Define GenerateHashtags Step

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

This step takes the output from SummarizeArticle, preserves the summary fields, and adds hashtags suitable for social posts. It requires the input to be a Hash and validates that the tone is preserved.

```ruby
class GenerateHashtags < RubyLLM::Contract::Step::Base
  input_type Hash

  output_schema do
    # Carry through the summary fields downstream consumers (and the next step) need.
    string :tldr
    array  :takeaways, of: :string
    string :tone, enum: %w[neutral positive negative analytical]
    # Add new field.
    array  :hashtags, of: :string, min_items: 2, max_items: 5
  end

  prompt do
    rule "Preserve tldr / takeaways / tone exactly as given."
    user "Article summary: {tldr}\nTone: {tone}\nGenerate 2 to 5 concise hashtags."
  end

  validate("tone preserved") { |o, input| o[:tone] == input[:tone] }
```

--------------------------------

### Opt-Out of Fail-Closed Per Step

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/multimodal_input.md

To avoid strict fail-closed behavior for unknown attachment sizes, set `on_unknown_attachment_size` to `:warn`. This logs a warning instead of refusing the call.

```ruby
class FlexibleContract < RubyLLM::Contract::Step::Base
  on_unknown_attachment_size :warn   # log a warning instead of refusing
  max_cost 0.05
end
```

--------------------------------

### Configuring Model Fallbacks with Reasoning Effort

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Define a chain of models for retries, allowing subsequent attempts to use more reasoning effort. This is useful for complex tasks where initial, faster models might fail.

```ruby
retry_policy models: [
  "gpt-5-nano",                                      # attempt 1: cheap + fast
  { model: "gpt-5-mini", reasoning_effort: "high" }  # attempt 2: stronger + more reasoning
]
```

--------------------------------

### Compare Prompt Versions with AB Eval in Ruby

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/eval_first.md

Use 'ab' eval to compare two prompt versions on the same set of regression cases. This allows for clean prompt iteration by comparing outputs on identical inputs.

```ruby
diff = SummarizeArticleV2.compare_with(
  SummarizeArticleV1,
  eval: "regression",
  model: "gpt-4.1-mini"
)

diff.safe_to_switch?  # => true if no cases regressed
```

--------------------------------

### Override Per-Step Model Configuration

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/pipeline.md

Define a pipeline and specify different models for individual steps using the `model:` option.

```ruby
class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
  step SummarizeArticle, as: :summarize, model: "gpt-4.1-mini"
  step GenerateHashtags, as: :tag,       model: "gpt-4.1-nano"
  step BuildArticleCard, as: :card,      model: "gpt-4.1-nano"
end
```

--------------------------------

### Gate CI on Evaluation Score and Cost

Source: https://github.com/justi/ruby_llm-contract/blob/main/docs/guide/getting_started.md

Use RSpec to gate CI builds based on minimum score and maximum cost thresholds for a given evaluation.

```ruby
# RSpec — blocks merge if accuracy drops or cost spikes
expect(SummarizeArticle).to pass_eval("regression")
  .with_minimum_score(0.8)
  .with_maximum_cost(0.01)
```