### Start Example Backend Servers

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/README.md

Before accessing the console application, start one of these example backend servers in a separate terminal. Ensure you navigate to the correct agent directory first.

```bash
cd ../agent-js && node examples/server.js
```

```bash
cd ../agent-rust && cargo run --example server
```

```bash
cd ../agent-go && go run ./examples/server
```

--------------------------------

### Install llm-sdk

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-go/README.md

Use the go get command to add the library to your project.

```bash
go get github.com/hoangvvo/llm-sdk/sdk-go
```

--------------------------------

### Initialize and run Next.js frontend

Source: https://github.com/hoangvvo/llm-sdk/blob/main/examples/next-ai-sdk-ui/README.md

Install dependencies and start the development server for the Next.js application.

```bash
npm install
npm run dev
```

--------------------------------

### Run Simple Agent Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-go/README.md

Execute the simple agent example from the examples folder. This command initiates the agent's basic functionality.

```bash
go run ./examples/agent
```

--------------------------------

### Run LLM SDK Example: Agent

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-rust/README.md

Command to execute the 'agent' example provided with the LLM SDK. This is a common starting point for understanding agent functionality.

```bash
cargo run --example agent
```

--------------------------------

### Install SDK and Providers

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-js/README.md

Install the core SDK and the necessary provider-specific packages.

```bash
npm install @hoangvvo/llm-sdk
```

```bash
npm install openai
npm install @anthropic-ai/sdk
npm install @google/genai
npm install cohere-ai
npm install @mistralai/mistralai
```

--------------------------------

### Run Go backend example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/examples/next-ai-sdk-ui/README.md

Navigate to the agent-go directory and run the AI SDK UI example.

```bash
cd ../../agent-go
go run ./examples/ai-sdk-ui
```

--------------------------------

### Run Example Code

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-go/README.md

Execute a specific example project using the Go CLI.

```bash
go run ./examples/generate-text
```

--------------------------------

### Go: Example MCP Server and Agent

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/mcp.mdx

Full example demonstrating how to set up a minimal shuttle-planning MCP server, register it with the toolkit, and run a conversation turn against it.

```go
goMcpExample
```

--------------------------------

### Go Instructions Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/instructions.mdx

An example demonstrating static and dynamic instructions in a Go agent. This code is part of a multi-language example.

```go
package main

import (
	"context"
	"fmt"

	"github.com/llm-agent/agent"
)

func main() {
	agent := agent.NewAgent(
		agent.Instructions(
			agent.StaticInstruction("You are a helpful assistant."),
			agent.DynamicInstruction(func(ctx context.Context, session agent.Session) (string, error) {
				return fmt.Sprintf("The user's name is %s.", session.User().Name()), nil
			}),
		),
	)

	agent.Run(context.Background(), agent.NewSession(agent.NewUser("Alice")))
}

```

--------------------------------

### Run Example Script

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-js/README.md

Execute a provided example script using Node.js.

```bash
node examples/generate-text.ts
```

--------------------------------

### Install llmagent Go Package

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-go/README.md

Use this command to install the llmagent Go package. Ensure you have Go installed and configured.

```bash
go get github.com/hoangvvo/llm-sdk/agent-go
```

--------------------------------

### Run Rust backend example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/examples/next-ai-sdk-ui/README.md

Navigate to the agent-rs directory and execute the AI SDK UI example using cargo.

```bash
cd ../../agent-rs
cargo run --example ai-sdk-ui
```

--------------------------------

### Run Node.js backend example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/examples/next-ai-sdk-ui/README.md

Navigate to the agent-js directory and execute the AI SDK UI example.

```bash
cd ../../agent-js
node examples/ai-sdk-ui.ts
```

--------------------------------

### Rust: Example MCP Server and Agent

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/mcp.mdx

Full example demonstrating how to set up a minimal shuttle-planning MCP server, register it with the toolkit, and run a conversation turn against it.

```rust
rustMcpExample
```

--------------------------------

### Run Generate Text Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-rust/README.md

Command to execute the generate-text example provided by the llm-sdk.

```bash
cargo run --example generate-text
```

--------------------------------

### Install @hoangvvo/llm-agent

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-js/README.md

Install the core agent library using npm.

```bash
npm install @hoangvvo/llm-agent
```

--------------------------------

### Rust Agent Implementation

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/index.mdx

Example of implementing an agent in Rust using the LLM SDK. This snippet demonstrates the basic structure for agent setup and execution.

```rust
use llm_sdk::agent;

fn main() {
    let agent = agent::Agent::new();

    agent.run(agent::Task {
        // ... task definition
    });
}
```

--------------------------------

### TypeScript: Example MCP Server and Agent

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/mcp.mdx

Full example demonstrating how to set up a minimal shuttle-planning MCP server, register it with the toolkit, and run a conversation turn against it.

```typescript
jsMcpExample
```

--------------------------------

### TypeScript Instructions Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/instructions.mdx

An example demonstrating how to use static and dynamic instructions in a TypeScript agent. This code is part of a larger example showcasing multi-language agent instructions.

```typescript
import { agent } from "@llm-agent/agent";

const agent = agent({
  instructions: [
    "You are a helpful assistant.",
    async (context) => {
      return `The user's name is ${context.user.name}.`;
    },
  ],
});

agent.run({ user: { name: "Alice" } });

```

--------------------------------

### Install MCP SDK

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-js/README.md

Install the Model Context Protocol (MCP) SDK if you plan to use MCP.

```bash
npm install @modelcontextprotocol/sdk
```

--------------------------------

### Generate Audio (Rust)

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/audio-generation.mdx

Generates audio in Rust by including 'audio' in the input modalities. This example shows basic audio generation setup.

```rust
use llm_sdk::client;

#[tokio::main]
async fn main() -> anyhow::Result<()>
{
    let text = "Hello, this is an audio message.";

    let audio = client::generate({
        text,
        // You can also pass other options, like `voice` or `format`
        // audio: Some(llm_sdk::AudioOptions { voice: Some("alloy".to_string()), format: Some(llm_sdk::AudioFormat::Wav) })
    }).await?;

    audio.to_file("./hello.wav").await?;
    println!("Audio saved to ./hello.wav");

    Ok(())
}

```

--------------------------------

### Implement a dynamic toolkit example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/toolkits.mdx

Example implementation of a toolkit managing state and dynamic tools for a Lost & Found desk scenario.

```typescript
import { Toolkit, ToolkitSession, RunSession } from "@llm-sdk/core";

class LostAndFoundToolkit implements Toolkit {
  async createSession(context: any): Promise<ToolkitSession> {
    const manifest = await fetchManifest(context.travelerId);
    return new LostAndFoundSession(manifest);
  }
}

class LostAndFoundSession implements ToolkitSession {
  private phase = "intake";
  constructor(private manifest: any) {}

  getTools() {
    if (this.phase === "intake") return [searchManifestTool];
    return [recoverItemTool];
  }

  getSystemPrompt() {
    return `Current phase: ${this.phase}`;
  }

  close() {}
}
```

```rust
use llm_sdk::{Toolkit, ToolkitSession, RunSession};

struct LostAndFoundToolkit;

#[async_trait]
impl Toolkit for LostAndFoundToolkit {
    async fn create_session(&self, _ctx: &Context) -> Result<Box<dyn ToolkitSession>> {
        let manifest = fetch_manifest().await?;
        Ok(Box::new(LostAndFoundSession { manifest, phase: "intake" }))
    }
}

struct LostAndFoundSession {
    manifest: Manifest,
    phase: String,
}

#[async_trait]
impl ToolkitSession for LostAndFoundSession {
    fn get_tools(&self) -> Vec<Box<dyn Tool>> {
        match self.phase.as_str() {
            "intake" => vec![Box::new(SearchManifestTool)],
            _ => vec![Box::new(RecoverItemTool)],
        }
    }
    // ...
}
```

```go
package main

import (
	"context"
	"github.com/llm-sdk/agent-go/toolkit"
)

type LostAndFoundToolkit struct{}

func (t *LostAndFoundToolkit) CreateSession(ctx context.Context) (toolkit.ToolkitSession, error) {
	manifest, _ := fetchManifest(ctx)
	return &LostAndFoundSession{manifest: manifest, phase: "intake"}, nil
}

type LostAndFoundSession struct {
	manifest Manifest
	phase    string
}

func (s *LostAndFoundSession) GetTools() []toolkit.Tool {
	if s.phase == "intake" {
		return []toolkit.Tool{SearchManifestTool}
	}
	return []toolkit.Tool{RecoverItemTool}
}

func (s *LostAndFoundSession) GetSystemPrompt() string {
	return "Current phase: " + s.phase
}

func (s *LostAndFoundSession) Close() error { return nil }
```

--------------------------------

### Install TypeBox for Tool Definition

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-js/README.md

Install TypeBox for defining agent tools with TypeBox.

```bash
npm install @sinclair/typebox
```

--------------------------------

### Go Example: Simple Tool

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/tools.mdx

Demonstrates a simple tool in Go for sending a notification. It includes the tool definition and its integration into an agent.

```go
package main

import (
	"encoding/json"
	"fmt"
	"time"
)

// Assuming AgentTool and AgentToolResult are defined as in the type extraction example
// For simplicity, we'll redefine them here or assume they are in scope.
type AgentTool struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
	// Execute would be a method or function pointer in a real implementation
}

type AgentToolResult struct {
	Data    json.RawMessage `json:"data,omitempty"`
	IsError bool           `json:"is_error"`
}

// Mock function to simulate sending a notification
func mockSendNotification(recipient string, message string) AgentToolResult {
	fmt.Printf("Sending notification to %s: %s\n", recipient, message)
	// Simulate async operation delay
	time.Sleep(50 * time.Millisecond)

	if recipient == "" || message == "" {
		return AgentToolResult{
			Data:    nil,
			IsError: true,
		}
	}

	// Simulate successful notification
	resultData, _ := json.Marshal(map[string]string{
		"status": "sent",
		"recipient": recipient,
	})

	return AgentToolResult{
		Data:    resultData,
		IsError: false,
	}
}

// Define the tool
func sendNotificationTool() AgentTool {
	params, _ := json.Marshal(map[string]interface{}{
		"type": "object",
		"properties": map[string]interface{}{
			"recipient": map[string]string{
				"type": "string",
				"description": "The recipient of the notification."
			},
			"message": map[string]string{
				"type": "string",
				"description": "The content of the notification."
			}
		},
		"required": []string{"recipient", "message"}
	})

	return AgentTool{
		Name:        "send_notification",
		Description: "Use this tool to send a notification to a user. Both recipient and message are required.",
		Parameters:  params,
		// Execute would be implemented as a method or function pointer in a real Go agent.
		// Example signature: Execute func(args json.RawMessage, context json.RawMessage, state json.RawMessage) AgentToolResult
	}
}

// Example usage (requires agent implementation)
// func main() {
// 	notificationTool := sendNotificationTool()
// 	tools := []AgentTool{notificationTool}

// 	// Assume Agent struct and run method are defined elsewhere in the SDK
// 	// agent := NewAgent(tools)
// 	// response, err := agent.Run("Send a notification to user@example.com about a new update.")
// 	// if err != nil {
// 	// 	log.Fatalf("Agent run failed: %v", err)
// 	// }
// 	// fmt.Printf("Agent Response: %s\n", response)
// }

```

--------------------------------

### Rust Instructions Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/instructions.mdx

An example showcasing the use of static and dynamic instructions within a Rust agent. This code is part of a multi-language demonstration.

```rust
use llm_agent::agent;

let agent = agent::Agent::new(
    agent::Instructions::new()
        .add("You are a helpful assistant.")
        .add_async(|context| async {
            Ok(format!("The user's name is {}.", context.user.name()))
        }),
);

agent.run(agent::SessionContext::new(agent::User::new("Alice"))).await;

```

--------------------------------

### Install Zod for Tool Definition

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-js/README.md

Install Zod and zod-to-json-schema for defining agent tools with Zod.

```bash
npm install zod zod-to-json-schema
```

--------------------------------

### Describe Image Example (Go)

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/image-understanding.mdx

Demonstrates how to send an image to the model for description using Go. Requires an initialized LLM client and an ImagePart object.

```go
package main

import (
	"fmt"
	"log"

	"github.com/llm-serialization/llm-sdk/go/llm"
)

func main() {
	client := llm.NewLLMClient()

	imagePart := llm.ImagePart{
		Type: llm.ImagePartTypeImage,
		Source: llm.ImageSource{
			Type: llm.ImageSourceTypeBase64,
			Data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=",
		},
	}

	response, err := client.DescribeImage([]llm.ImagePart{imagePart})
	if err != nil {
		log.Fatalf("Error describing image: %v", err)
	}

	fmt.Println(response)
}

```

--------------------------------

### Go Planner-Executor Implementation

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/planner-executor.mdx

Example implementation of the Planner-Executor pattern in Go. Requires the 'update_plan' tool.

```go
package main

import (
	"fmt"
	"github.com/llm-agent/agent-go/agent"
)

func main() {
	// Example usage:
	// agent.Run("Write a blog post about the Planner-Executor pattern.")
	fmt.Println("Planner-Executor example in Go")
}

```

--------------------------------

### Go Agent Implementation

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/index.mdx

Example of implementing an agent in Go using the LLM SDK. This code shows how to initialize and run an agent for a given task.

```go
package main

import (
	"github.com/hoangvvo/llm-sdk/agent"
)

func main() {
	agent := agent.NewAgent()

	agent.Run(agent.Task{
		// ... task definition
	})
}
```

--------------------------------

### Run Local Development Server

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/README.md

Use this command to start a local development server for the website. Access the site at http://localhost:4321.

```bash
npm run dev
```

--------------------------------

### Describe Image Example (Rust)

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/image-understanding.mdx

Demonstrates how to send an image to the model for description using Rust. Requires an initialized LLM client and an ImagePart object.

```rust
use llm_sdk::image::ImagePart;
use llm_sdk::LLM;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = LLM::new();

    let image_part = ImagePart {
        type_: llm_sdk::image::ImagePartType::Image,
        source: llm_sdk::image::ImageSource {
            type_: llm_sdk::image::ImageSourceType::Base64,
            data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=".to_string(),
        },
    };

    let response = client.describe_image(vec![image_part]).await?;

    println!("{:?}", response);

    Ok(())
}

```

--------------------------------

### Go Human-in-the-loop Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/human-in-the-loop.mdx

This Go example implements the human-in-the-loop pattern. It defines a tool that requires approval and uses a custom error to halt execution, allowing for human intervention before retrying the agent run.

```go
package main

import (
	"context"
	"errors"
	"fmt"
	"github.com/e2b-dev/llm-sdk/agent"
	"github.com/e2b-dev/llm-sdk/agent/tool"
	"github.com/e2b-dev/llm-sdk/agent/tool/schema"
	"github.com/e2b-dev/llm-sdk/agent/types"
)

// Define a custom error for requiring approval
var ErrRequireApproval = errors.New("require approval")

// ApprovalTool is a tool that requires human approval before execution.
type ApprovalTool struct {
	agent.Tool
}

func NewApprovalTool() *ApprovalTool {
	return &ApprovalTool{}
}

func (t *ApprovalTool) Name() string {
	return "approval_tool"
}

func (t *ApprovalTool) Description() string {
	return "Use this tool to get approval for an action."
}

func (t *ApprovalTool) Parameters() schema.Schema {
	return schema.Schema{
		Type: "object",
		Properties: map[string]schema.Schema{
			"action": {
				Type:        "string",
				Description: "The action to get approval for.",
				Required:    true,
			},
		},
	}
}

func (t *ApprovalTool) Call(ctx context.Context, args map[string]any, run *agent.Run) (string, error) {
	action := args["action"].(string)

	// Check if approval has already been granted in the context
	if approvedActions, ok := run.Context["approved_actions"].([]string);
	ok {
		for _, approvedAction := range approvedActions {
			if approvedAction == action {
				return fmt.Sprintf("Approval granted for: %s", action), nil
			}
		}
	}

	// Throw an error to halt the stream and require human input
	return "", fmt.Errorf("%w: %s", ErrRequireApproval, action)
}

func main() {
	agentConfig := agent.AgentConfig{
		Model: "gemini-1.5-flash",
		Tools: []agent.ToolInterface{
			NewApprovalTool(),
		},
	}

	run := agent.NewRun(agentConfig)
	defer run.Close() // Ensure the run is closed properly

	transcript := []string{"User: Please approve the action 'delete files'."}
	context := map[string]any{}

	for {
		items, err := run.Stream(transcript, context)
		if err != nil {
			if errors.Is(err, ErrRequireApproval) {
				action := err.Error()[len(ErrRequireApproval.Error())+2:] // Extract action from error message
				fmt.Printf("Approval required for: %s\n", action)

				// Simulate human decision
				decision := "approve" // or "deny"

				if decision == "approve" {
					// Add approved action to context and retry
					approvedActions := []string{}
					if existing, ok := context["approved_actions"].([]string);
					ok {
						approvedActions = existing
					}
					approvedActions = append(approvedActions, action)
					context["approved_actions"] = approvedActions

					fmt.Println("Retrying agent run with approval...")
					continue // Retry the loop with updated context
				} else {
					fmt.Println("Action denied.")
					break // Exit loop on denial
				}
			} else {
				fmt.Printf("An unexpected error occurred: %v\n", err)
				break // Exit loop on other errors
			}
		}

		for _, item := range items {
			switch item.Type {
			case types.AgentItemTypeMessage:
				fmt.Printf("Agent: %s\n", item.Content)
			}
			// Update context with the latest agent item
			agent.UpdateContext(context, item)
		}

		// If no error occurred and we processed items, break the loop
		break
	}
}

```

--------------------------------

### Rust Example: Simple Tool

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/tools.mdx

Demonstrates a simple tool in Rust for creating a ticket. It includes the tool definition and its integration into an agent.

```rust
use serde::{Deserialize, Serialize};
use serde_json::Value;

// Assuming AgentTool and AgentToolResult are defined as in the type extraction example
// For simplicity, we'll redefine them here or assume they are in scope.
#[derive(Serialize, Deserialize, Debug)]
pub struct AgentTool {
    pub name: String,
    pub description: String,
    pub parameters: Value,
    // In a real application, the execute function would be part of a trait or handled differently.
}

#[derive(Serialize, Deserialize, Debug)]
pub struct AgentToolResult {
    pub data: Value,
    pub is_error: bool,
}

// Mock function to simulate ticket creation
async fn mock_create_ticket(title: &str, description: &str) -> AgentToolResult {
    println!("Creating ticket: Title='{}', Description='{}'", title, description);
    // Simulate async operation
    tokio::time::sleep(tokio::time::Duration::from_millis(50)).await;

    if title.is_empty() || description.is_empty() {
        return AgentToolResult {
            data: Value::Null,
            is_error: true,
        };
    }

    // Simulate successful ticket creation
    AgentToolResult {
        data: json!({"ticket_id": "#12345", "status": "created"}),
        is_error: false,
    }
}

// Define the tool
fn create_ticket_tool() -> AgentTool {
    AgentTool {
        name: "create_ticket".to_string(),
        description: "Use this tool to create a support ticket. Both title and description are required.".to_string(),
        parameters: json!({
            "type": "object",
            "properties": {
                "title": {
                    "type": "string",
                    "description": "The title of the ticket."
                },
                "description": {
                    "type": "string",
                    "description": "A detailed description of the issue."
                }
            },
            "required": ["title", "description"]
        }),
        // The execute function would be implemented differently in a real Rust agent setup,
        // likely involving async execution and proper error handling.
        // This is a placeholder to show the structure.
        // execute: async move |args: Value, _context: Value, _state: Value| -> AgentToolResult {
        //     let title = args["title"].as_str().unwrap_or("");
        //     let description = args["description"].as_str().unwrap_or("");
        //     mock_create_ticket(title, description).await
        // },
    }
}

// Example usage within an async function (requires a runtime like tokio)
// #[tokio::main]
// async fn main() {
//     let ticket_tool = create_ticket_tool();
//     let tools = vec![ticket_tool];

//     // Assume Agent struct and run method are defined elsewhere in the SDK
//     // let agent = Agent::new(tools);
//     // let response = agent.run("Create a ticket about a login issue.").await;
//     // println!("Agent Response: {:?}", response);
// }

// Note: This example requires the `tokio` and `serde_json` crates, and potentially macros like `json!` from `serde_json`.
// The actual agent execution logic would be part of the LLM SDK.

```

--------------------------------

### Generate Reasoning in Go

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/reasoning.mdx

Example of generating reasoning output using the LLM SDK in Go. This code demonstrates handling `ReasoningPart`s.

```go
package main

import (
	"context"
	"fmt"
	"github.com/ollama/ollama-go"
)

func main() {
	ctx := context.Background()

	client, err := ollama.ClientFromEnv(ctx)
	if err != nil {
		panic(err)
	}

	prompt := ollama.Message{Role: "user", Content: "Why is the sky blue? Explain step by step."}

	resp, err := client.Chat(ctx, ollama.ChatRequest{
		Model: "llama2",
		Messages: []ollama.Message{prompt},
		Stream: nil,
		Options: ollama.ChatRequestOptions{
			Reasoning: true,
		},
	})
	if err != nil {
		panic(err)
	}

	fmt.Println(resp.Message.Content)
}

```

--------------------------------

### Stream Reasoning in Go

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/reasoning.mdx

Example of streaming reasoning output using the LLM SDK in Go. This code shows how to process `ReasoningPartDelta`s.

```go
package main

import (
	"context"
	"fmt"
	"github.com/ollama/ollama-go"
)

func main() {
	ctx := context.Background()

	client, err := ollama.ClientFromEnv(ctx)
	if err != nil {
		panic(err)
	}

	prompt := ollama.Message{Role: "user", Content: "Why is the sky blue? Explain step by step."}

	stream, err := client.ChatStream(ctx, ollama.ChatRequest{
		Model: "llama2",
		Messages: []ollama.Message{prompt},
		Stream: true,
		Options: ollama.ChatRequestOptions{
			Reasoning: true,
		},
	})
	if err != nil {
		panic(err)
	}

	for content := range stream.Ch {
		fmt.Print(content.Message.Content)
	}
}

```

--------------------------------

### Summarize audio content

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/audio-understanding.mdx

Example implementation for transcribing and summarizing audio files using the SDK.

```ts
jsSummarizeAudio
```

```rs
rustSummarizeAudio
```

```go
goSummarizeAudio
```

--------------------------------

### Rust Planner-Executor Implementation

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/planner-executor.mdx

Example implementation of the Planner-Executor pattern in Rust. Requires the 'update_plan' tool.

```rust
use agent_rust::Agent;

#[tokio::main]
async fn main() {
    let mut agent = Agent::new();

    // Example usage:
    // agent.run("Write a blog post about the Planner-Executor pattern.").await;
}

```

--------------------------------

### Initialize and Run Agent with Tools

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-rust/README.md

Initializes an `Agent` with a specified model (OpenAI in this case), instructions, and custom tools. It then sets up a loop to read user input, process it, and get a response from the agent.

```rust
use dotenvy::dotenv;
use std::{env, io::{self, Write}, sync::Arc};

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    dotenv().ok();

    // Define the model to use for the Agent
    let model = Arc::new(OpenAIModel::new(OpenAIModelOptions {
        api_key: env::var("OPENAI_API_KEY")
            .expect("OPENAI_API_KEY environment variable must be set"),
        model_id: "gpt-4o".to_string(),
        ..Default::default()
    }));

    // Create the Agent
    let my_assistant = Agent::<MyContext>::builder("Mai", model)
        .add_instruction(
            "You are Mai, a helpful assistant. Answer questions to the best of your ability.",
        )
        .add_instruction(|ctx: &MyContext| Ok(format!("You are talking to {}", ctx.user_name)))
        .add_tool(GetWeatherTool)
        .add_tool(SendMessageTool)
        .build();

    // Implement the CLI to interact with the Agent
    let mut items = Vec::<AgentItem>::new();

    // Get user name
    let user_name = read_line("Your name: ")?;

    let context = MyContext { user_name };

    println!("Type 'exit' to quit");

    loop {
        let user_input = read_line("> ")?;

        if user_input.is_empty() {
            continue;
        }

        if user_input.to_lowercase() == "exit" {
            break;
        }

        // Add user message as the input
        items.push(AgentItem::Message(Message::user(vec![Part::text(
            user_input,
        )])));

        // Call assistant
        let response = my_assistant
            .run(AgentRequest {
                context: context.clone(),
                input: items.clone(),

```

--------------------------------

### TypeScript Example: Simple Tool

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/tools.mdx

Demonstrates a simple tool in TypeScript for searching documentation. It includes the tool definition and its integration into an agent.

```typescript
import { Agent, AgentTool, AgentToolResult, Part, TextPart } from "@llm-sdk/agent";
import { RunState } from "@llm-sdk/types";

// Mock implementation for demonstration purposes
const mockSearchDocs = async (query: string): Promise<AgentToolResult> => {
  console.log(`Searching docs for: ${query}`);
  // Simulate a network request or database lookup
  await new Promise((resolve) => setTimeout(resolve, 50));
  if (query.toLowerCase().includes("error")) {
    return {
      data: null,
      is_error: true,
    };
  }
  return {
    data: JSON.stringify([ // Returning JSON string as data
      { title: "Introduction", url: "/docs/intro" },
      { title: "Getting Started", url: "/docs/getting-started" },
    ]),
    is_error: false,
  };
};

const searchDocsTool: AgentTool = {
  name: "search_docs",
  description: "Use this tool to search documentation. Query must be a string.",
  parameters: {
    type: "object",
    properties: {
      query: {
        type: "string",
        description: "The search query.",
      },
    },
    required: ["query"],
  },
  execute: async (args: { query: string }, context: any, state: RunState): Promise<AgentToolResult> => {
    // You can access context and state here if needed
    // console.log("Context:", context);
    // console.log("State:", state);
    return mockSearchDocs(args.query);
  },
};

const tools = [searchDocsTool];

// Example of how to use the tool with an agent (simplified)
async function runAgentWithTool() {
  const agent = new Agent({ tools });

  const response = await agent.run("Search for documentation on 'getting started'.", {
    // You can pass initial context and state here
    // context: { user_id: "user123" },
    // state: { history: [] },
  });

  // The response will contain the result of the tool execution or a message from the model
  console.log("Agent Response:", response);

  // Example of handling tool results that are structured parts
  if (response.parts && response.parts.length > 0 && response.parts[0].type === 'tool_result') {
      const toolResultPart = response.parts[0] as any; // Type assertion for simplicity
      if (toolResultPart.is_error) {
          console.error("Tool execution failed:", toolResultPart.data);
      } else {
          console.log("Tool execution succeeded:", JSON.parse(toolResultPart.data));
      }
  }
}

// To run this example, you would need to have the @llm-sdk/agent and @llm-sdk/types packages installed
// and potentially mock the Agent class if it's not fully available in this context.
// runAgentWithTool();

```

--------------------------------

### Install llm-agent via Cargo

Source: https://github.com/hoangvvo/llm-sdk/blob/main/agent-rust/README.md

Use this command to add the llm-agent crate to your Cargo.toml file.

```bash
cargo add llm-agent
```

--------------------------------

### Rust Human-in-the-loop Example

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/human-in-the-loop.mdx

This Rust example implements the human-in-the-loop pattern using a custom error for approval requests. It shows how to catch the error, simulate human input, and retry the agent run with the granted approval.

```rust
use anyhow::Result;
use llm_sdk::agent::tool::Tool;
use llm_sdk::agent::AgentConfig;
use llm_sdk::agent::AgentItem;
use llm_sdk::agent::AgentItemType;
use llm_sdk::agent::AgentRun;
use llm_sdk::agent::Schema;
use std::collections::HashMap;

// Define a custom error for requiring approval
#[derive(Debug)]
struct RequireApprovalError {
    message: String,
}

impl std::fmt::Display for RequireApprovalError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(f, "{}", self.message)
    }
}

impl std::error::Error for RequireApprovalError {}

// Define a tool that requires approval
struct ApprovalTool;

impl Tool for ApprovalTool {
    fn name(&self) -> &str {
        "approval_tool"
    }

    fn description(&self) -> &str {
        "Use this tool to get approval for an action."
    }

    fn parameters(&self) -> Schema {
        Schema::Object({
            ("action".to_string(), Schema::String { description: "The action to get approval for.".to_string(), required: true })
                .into()
        })
    }

    async fn call(&self, args: &HashMap<String, serde_json::Value>, context: &mut HashMap<String, serde_json::Value>) -> Result<String> {
        let action = args.get("action").unwrap().as_str().unwrap();

        // Check if approval has already been granted in the context
        if let Some(approved_actions) = context.get("approved_actions").and_then(|v| v.as_array()) {
            if approved_actions.iter().any(|a| a.as_str().unwrap() == action) {
                return Ok(format!("Approval granted for: {}", action));
            }
        }

        // Throw an error to halt the stream and require human input
        Err(Box::new(RequireApprovalError { message: format!("Approval required for: {}", action) }))
    }
}

// Agent configuration
let agent_config = AgentConfig {
    model: "gemini-1.5-flash".to_string(),
    tools: vec![Box::new(ApprovalTool)],
};

// Main function to run the agent stream
async fn run_agent_with_approval() -> Result<()>
{
    let mut agent = llm_sdk::agent::Agent::new(agent_config);
    let transcript = vec!["User: Please approve the action 'delete files'.".to_string()];
    let mut context: HashMap<String, serde_json::Value> = HashMap::new();
    context.insert("approved_actions".to_string(), serde_json::json!([]));

    match agent.run_stream(transcript, &mut context).await {
        Ok(mut stream) => {
            while let Some(item) = stream.next().await {
                match item {
                    AgentItem::Message(msg) => {
                        println!("Agent: {}", msg.content);
                    }
                    _ => {}
                }
                // Update context with the latest agent item
                AgentRun::update_context(context, &item);
            }
            Ok(())
        }
        Err(err) => {
            if let Some(approval_err) = err.downcast_ref::<RequireApprovalError>() {
                println!("{}", approval_err);

                // Simulate human decision
                let decision = "approve"; // or 'deny'

                if decision == "approve" {
                    // Add approved action to context and retry
                    let action_to_approve = approval_err.message.replace("Approval required for: ", "");
                    context.get_mut("approved_actions").unwrap().as_array_mut().unwrap().push(serde_json::json!(action_to_approve));
                    println!("Retrying agent run with approval...");
                    // Re-run the stream with the updated context
                    return run_agent_with_approval().await;
                } else {
                    println!("Action denied.");
                    Ok(())
                }
            } else {
                Err(err)
            }
        }
    }
}

// Call the main function
// tokio::main is required to run async functions
// #[tokio::main]
// async fn main() {
//     run_agent_with_approval().await.unwrap();
// }

```

--------------------------------

### Generate images with LLM SDK

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/image-generation.mdx

Implementation examples for generating images by setting the modalities parameter.

```typescript
import { generate } from "@llm-sdk/core";

const result = await generate({
  model: "gemini-1.5-flash",
  prompt: "Describe this image",
  modalities: ["image"],
  input: [
    {
      image: {
        mimeType: "image/png",
        data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
      },
    },
  ],
});

console.log(result.text);
```

```rust
use llm_sdk::generate;
use llm_sdk::types::{Image, ImagePart, Modality};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let result = generate(
        "gemini-1.5-flash",
        "Describe this image",
        vec![Modality::Image],
        vec![ImagePart {
            image: Image {
                mime_type: "image/png".to_string(),
                data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==".to_string(),
            },
        }],
    )
    .await?;

    println!("{}", result.text);
    Ok(())
}
```

```go
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/llm-sdk/llm-sdk-go/llm"
)

func main() {
	result, err := llm.Generate(context.Background(), &llm.GenerateRequest{
		Model:      "gemini-1.5-flash",
		Prompt:     "Describe this image",
		Modalities: []llm.Modality{llm.ModalityImage},
		Input: []llm.Part{
			{
				Image: &llm.Image{
					MimeType: "image/png",
					Data:     "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
				},
			},
		},
	})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(result.Text)
}
```

--------------------------------

### Describe Image Example (TypeScript)

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/image-understanding.mdx

Demonstrates how to send an image to the model for description using TypeScript. Requires an initialized LLM client and an ImagePart object.

```typescript
import { LLM } from "@llm-serialization/llm-sdk";

const client = new LLM({
  // ... client options
});

const imagePart: ImagePart = {
  type: "image",
  source: {
    type: "base64",
    data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=",
  },
};

const response = await client.describeImage([imagePart]);

console.log(response);

```

--------------------------------

### Generate Reasoning in Rust

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/reasoning.mdx

Example of generating reasoning output using the LLM SDK in Rust. This code shows how to handle `ReasoningPart`s.

```rust
use ollama_rs::prelude::*

#[tokio::main]
async fn main() -> ollama_rs::Result<()> {
    let ollama = Ollama::new(None, None);

    let mut stream = ollama
        .chat_stream(
            MessageHistory {
                messages: vec![Message {
                    role: "user".to_string(),
                    content: "Why is the sky blue? Explain step by step.".to_string(),
                    ..Default::default()
                }],
                ..Default::default()
            },
            "llama2",
            None,
        )
        .await?;

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        if let Some(content) = chunk.message.content {
            print!("{}", content);
        }
    }

    Ok(())
}

```

--------------------------------

### Implement Artifacts Workflow

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/artifacts.mdx

Examples demonstrating the use of artifact tools to manage persistent documents across different programming languages.

```typescript
import { Agent } from "@llm-sdk/agent";
import { artifactCreate, artifactUpdate, artifactGet, artifactList, artifactDelete } from "@llm-sdk/tools/artifacts";

// Implementation details for artifacts.ts
const agent = new Agent({
  tools: [artifactCreate, artifactUpdate, artifactGet, artifactList, artifactDelete]
});

// ... usage logic
```

```rust
use llm_sdk::agent::Agent;
use llm_sdk::tools::artifacts::{artifact_create, artifact_update, artifact_get, artifact_list, artifact_delete};

// Implementation details for artifacts.rs
let agent = Agent::new()
    .with_tool(artifact_create)
    .with_tool(artifact_update)
    .with_tool(artifact_get)
    .with_tool(artifact_list)
    .with_tool(artifact_delete);
```

```go
package main

import (
	"github.com/llm-sdk/agent"
	"github.com/llm-sdk/tools/artifacts"
)

// Implementation details for main.go
func main() {
	agent := agent.New(
		agent.WithTools(
			artifacts.Create,
			artifacts.Update,
			artifacts.Get,
			artifacts.List,
			artifacts.Delete,
		),
	)
}
```

--------------------------------

### Implement Tracing in Agents

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/observability/tracing.mdx

Examples of identical agents in different languages that check weather and produce spans.

```typescript
jsTracingExample
```

```rust
rustTracingExample
```

```go
goTracingExample
```

--------------------------------

### Request structured output from the model

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/sdk/structured-output.mdx

Examples of specifying a response format to receive structured output across different programming languages.

```typescript
import { GoogleGenAI } from "@google/generative-ai";

const ai = new GoogleGenAI({ apiKey: process.env.GOOGLE_API_KEY });

const model = ai.getGenerativeModel({ model: "gemini-1.5-flash" });

const result = await model.generateContent({
  contents: "List 3 fruits",
  generationConfig: {
    responseMimeType: "application/json",
    responseSchema: {
      type: "ARRAY",
      items: {
        type: "STRING",
      },
    },
  },
});

console.log(JSON.parse(result.text()));

```

```rust
use google_generative_ai::client::Client;
use google_generative_ai::model::Model;
use serde::{Deserialize, Serialize};
use serde_json::json;

#[derive(Serialize, Deserialize, Debug)]
struct FruitList {
    fruits: Vec<String>,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new(std::env::var("GOOGLE_API_KEY")?);
    let model = Model::new(&client, "gemini-1.5-flash");

    let response = model
        .generate_content("List 3 fruits")
        .with_response_mime_type("application/json")
        .with_response_schema(json!({
            "type": "object",
            "properties": {
                "fruits": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    }
                }
            },
            "required": ["fruits"]
        }))
        .execute()
        .await?;

    let fruits: FruitList = serde_json::from_str(&response.text())?;
    println!("{:?}", fruits);

    Ok(())
}

```

```go
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"os"

	"github.com/google/generative-ai-go/genai"
	"google.golang.org/api/option"
)

func main() {
	ctx := context.Background()
	client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("GOOGLE_API_KEY")))
	if err != nil {
		log.Fatal(err)
	}
	defer client.Close()

	model := client.GenerativeModel("gemini-1.5-flash")
	model.ResponseMIMEType = "application/json"
	model.ResponseSchema = &genai.Schema{
		Type: genai.TypeArray,
		Items: &genai.Schema{
			Type: genai.TypeString,
		},
	}

	resp, err := model.GenerateContent(ctx, genai.Text("List 3 fruits"))
	if err != nil {
		log.Fatal(err)
	}

	var fruits []string
	json.Unmarshal([]byte(resp.Candidates[0].Content.Parts[0].(genai.Text)), &fruits)
	fmt.Println(fruits)
}

```

--------------------------------

### JavaScript Agent Implementation

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/index.mdx

Example of implementing an agent in TypeScript using the LLM SDK. Ensure necessary imports and configurations are in place.

```typescript
import { agent } from "@llm-sdk/agent";

const agent = agent({
  // ... agent configuration
});

agent.run({
  // ... task definition
});
```

--------------------------------

### Go Agent Test

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/testing.mdx

Example of testing an agent in Go using a mock language model. This approach is recommended for unit testing agent functionalities.

```go
package agent_go

import (
	"agent-go/agent"
	"agent-go/mock_language_model"
	"testing"
)

func TestAgent(t *testing.T) {
	agent := agent.New(mock_language_model.MockLanguageModel)
	result := agent.Call("hello world")
	if result != "hello world" {
		t.Errorf("Expected 'hello world', got %s", result)
	}
}

```

--------------------------------

### Generate Text with OpenAI Model

Source: https://github.com/hoangvvo/llm-sdk/blob/main/sdk-rust/README.md

Example of generating text using the OpenAI model. Requires the `dotenvy` crate for environment variables and `tokio` for async execution. Ensure your API key is set in a `.env` file.

```rust
use dotenvy::dotenv;
use llm_sdk::{LanguageModelInput, Message, Part};

mod common;

#[tokio::main]
async fn main() {
    dotenv().ok();

    let model = common::get_model("openai", "gpt-4o");

    let response = model
        .generate(LanguageModelInput {
            messages: vec![
                Message::user(vec![Part::text("Tell me a story.")]),
                Message::assistant(vec![Part::text(
                    "Sure! What kind of story would you like to hear?",
                )]),
                Message::user(vec![Part::text("a fairy tale")]),
            ],
            ..Default::default()
        })
        .await
        .unwrap();

    println!("{response:#?}");
}
```

--------------------------------

### Rust Agent Test

Source: https://github.com/hoangvvo/llm-sdk/blob/main/website/src/content/docs/agent/testing.mdx

Example of testing an agent in Rust using a mock language model. This setup allows for isolated testing of agent logic.

```rust
use agent_rust::agent::Agent;
use agent_rust::mock_language_model::MockLanguageModel;

#[test]
fn test_agent() {
    let agent = Agent::new(MockLanguageModel);
    let result = agent.call("hello world");
    assert_eq!(result, "hello world");
}

```