### Install Schema-Stream with Bun, NPM, or PNPM

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/schema-stream/getting-started.mdx

Install the schema-stream and zod libraries using your preferred package manager. This is the first step to using schema-stream in your project.

```bash
bun add schema-stream zod
```

```bash
npm install schema-stream zod
```

```bash
pnpm add schema-stream zod
```

--------------------------------

### Install Hack Dance Island AI Packages

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Installs the 'evalz', 'openai', 'zod', and '@instructor-ai/instructor' packages using different package managers.

```bash
bun add evalz openai zod @instructor-ai/instructor
```

```bash
npm install evalz openai zod @instructor-ai/instructor
```

```bash
pnpm add evalz openai zod @instructor-ai/instructor
```

--------------------------------

### Install Stream Hooks and Zod Stream (Bun, npm, pnpm)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/stream-hooks/getting-started.mdx

Instructions for installing the stream-hooks, zod, and zod-stream libraries using different package managers. Ensure these dependencies are installed before proceeding.

```bash
bun add stream-hooks zod zod-stream
```

```bash
npm install stream-hooks zod zod-stream
```

```bash
pnpm add stream-hooks zod zod-stream
```

--------------------------------

### Install ZodStream and Dependencies (Bun, npm, pnpm)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Installs the necessary packages 'zod-stream', 'zod', and 'openai' using different package managers. This is the first step to integrate ZodStream into your project.

```bash
bun add zod-stream zod openai
```

```bash
npm install zod-stream zod openai
```

```bash
pnpm add zod-stream zod openai
```

--------------------------------

### Quick Start with useJsonStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/stream-hooks/getting-started.mdx

A basic example demonstrating how to use the `useJsonStream` hook from `stream-hooks` to initiate a stream, handle incoming data with a Zod schema, and manage loading states. This requires a backend API endpoint at '/api/ai/chat'.

```typescript
import { useJsonStream } from "stream-hooks"
import { z } from "zod"

export function ChatComponent() {
  const { loading, startStream, stopStream, data } = useJsonStream({
    schema: z.object({
      content: z.string()
    }),
    onReceive: data => {
      console.log("incremental update to final response model", data)
    }
  })

  const submit = async () => {
    try {
      await startStream({
        url: "/api/ai/chat",
        method: "POST",
        body: {
          messages: [
            {
              content: "yo",
              role: "user"
            }
          ]
        }
      })
    } catch (e) {
      console.error(e)
    }
  }

  return (
    <div>
      {data?.content}

      <button onClick={submit} disabled={loading}>
        Start
      </button>

      <button onClick={stopStream}>
        Stop
      </button>
    </div>
  )
}
```

--------------------------------

### Process Streaming Chunks with Error Handling in TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Shows how to process chunks from a streaming response, including path tracking for progressive updates and final validation. This example also implicitly covers error handling provided by `zod-stream` during stream processing. Requires `zod-stream` and potentially an OpenAI client setup.

```typescript
const stream = await client.create({
  completionPromise: async () => response.body,
  response_model: { schema }
});

let finalResult

// Path tracking for progressive updates
for await (const chunk of stream) {
  finalResult = chunk
  // Check which paths are complete
  console.log("Completed paths:", chunk._meta._completedPaths);
  console.log("Current path:", chunk._meta._activePath);
}

// Final validation happens after stream completes
const isValid = finalResult._meta._isValid
```

--------------------------------

### TOOLS Mode Configuration for OpenAI (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Illustrates the resulting OpenAI configuration object when using the `TOOLS` mode. This includes `tool_choice` and `tools` properties, with the function's parameters generated from a provided schema.

```typescript
// Results in OpenAI tool configuration
{
  tool_choice: {
    type: "function",
    function: { name: "Analysis" }
  },
  tools: [{
    type: "function",
    function: {
      name: "Analysis",
      description: "Extract sentiment and keywords",
      parameters: {/* Generated from schema */}
    }
  }],
  // ... other existing params are preserved
}
```

--------------------------------

### FUNCTIONS Mode Configuration for OpenAI (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Shows the structure of an OpenAI configuration object for the legacy `FUNCTIONS` mode. It specifies `function_call` and `functions`, similar to `TOOLS` mode but using older OpenAI API conventions.

```typescript
// Results in OpenAI function configuration
{
  function_call: { name: "Analysis" },
  functions: [{
    name: "Analysis",
    description: "Extract sentiment and keywords",
    parameters: {/* Generated from schema */}
  }],
  // ... other existing params are preserved
}
```

--------------------------------

### Progressive UI Updates with Schema-Stream

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/schema-stream/getting-started.mdx

Shows how to use SchemaStream for progressive UI updates by defining default data and using the `onKeyComplete` callback to update loading states as data streams in. This provides a better user experience during data loading.

```typescript
const schema = z.object({
  analysis: z.object({
    sentiment: z.string(),
    keywords: z.array(z.string()),
    summary: z.string()
  }),
  metadata: z.object({
    processedAt: z.string(),
    wordCount: z.number()
  })
});

const parser = new SchemaStream(schema, {
  // Show loading states initially
  defaultData: {
    analysis: {
      sentiment: "analyzing...",
      keywords: ["loading..."],
      summary: "generating summary..."
    }
  },
  onKeyComplete({ activePath, completedPaths }) {
    // Update UI loading states based on completion
    updateLoadingStates(activePath, completedPaths);
  }
});
```

--------------------------------

### Configure OpenAI Params with Response Model (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Uses `withResponseModel` to configure OpenAI parameters for the TOOLS mode. It defines a Zod schema for response data and integrates it into the OpenAI chat completion request, enabling streaming.

```typescript
import { withResponseModel } from "zod-stream";
import { z } from "zod";

const schema = z.object({
  sentiment: z.string(),
  keywords: z.array(z.string()),
  confidence: z.number()
});

// Configure for OpenAI tools mode
const params = withResponseModel({
  response_model: {
    schema,
    name: "Analysis",
    description: "Extract sentiment and keywords"
  },
  mode: "TOOLS",
  params: {
    messages: [{ role: "user", content: "Analyze this text..." }],
    model: "gpt-4"
  }
});

const completion = await oai.chat.completions.create({
  ...params,
  stream: true
});
```

--------------------------------

### Schema-Stream with Zod-Stream for Validation

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/schema-stream/getting-started.mdx

Illustrates how SchemaStream can be integrated with zod-stream for enhanced validation and error handling. This example shows setting up a parser with type defaults and piping the stream through a validation transform stream.

```typescript
const streamParser = new SchemaStream(response_model.schema, {
  typeDefaults: {
    string: null,
    number: null,
    boolean: null
  },
  onKeyComplete: ({ activePath, completedPaths }) => {
    _activePath = activePath;
    _completedPaths = completedPaths;
  }
});

// Create parser with validation stream
const parser = streamParser.parse({ 
  handleUnescapedNewLines: true
});

// Add validation in transform stream
const validationStream = new TransformStream({
  transform: async (chunk, controller) => {
    try {
      const parsedChunk = JSON.parse(decoder.decode(chunk));
      const validation = await schema.safeParseAsync(parsedChunk);
      
      controller.enqueue(encoder.encode(JSON.stringify({
        ...parsedChunk,
        _meta: {
          _isValid: validation.success,
          _activePath,
          _completedPaths
        }
      })));
    } catch (e) {
      controller.error(e);
    }
  }
});

// Chain streams
stream
  .pipeThrough(parser)
  .pipeThrough(validationStream);
```

--------------------------------

### JSON_SCHEMA Mode Configuration for OpenAI (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Explains the configuration for `JSON_SCHEMA` mode, which uses OpenAI's `response_format` with a `schema` property. The system message is tailored to guide the model in producing JSON that conforms to the provided schema.

```typescript
// Results in JSON schema-based configuration
{
  response_format: {
    type: "json_object",
    schema:{/* Schema without name and description */}
  },
  messages: [
    {
      role: "system",
      content: "
        Given a user prompt, you will return fully valid JSON based on the following description.
        You will return no other prose. You will take into account any descriptions or required parameters within the schema
        and return a valid and fully escaped JSON object that matches the schema and those instructions.

        description: ${definition.description}
      "
    },
    // ... user messages are preserved
  ]
}
```

--------------------------------

### MESSAGE_BASED Mode Configuration (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Describes the configuration for `MESSAGE_BASED` mode. This mode is similar to `JSON` mode but omits the `response_format` property, relying solely on the system message to enforce JSON output structure.

```typescript
// Similar to JSON mode but without response_format
{
  messages: [
    {
      role: "system",
      content: "
        Given a user prompt, you will return fully valid JSON based on the following description and schema.
        You will return no other prose. You will take into account any descriptions or required parameters within the schema
        and return a valid and fully escaped JSON object that matches the schema and those instructions.

        description: ${definition.description}
        json schema: ${JSON.stringify(definition)}
      "
    },
    // ... user messages are preserved
  ]
}
```

--------------------------------

### ZodStream Client Initialization and Streaming Extraction (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Demonstrates how to initialize the ZodStream client, define a Zod schema for data extraction, and create a streaming response from an API. It processes chunks of data as they arrive, logging validation status and parsing paths.

```typescript
import ZodStream from "zod-stream";
import { z } from "zod";

const client = new ZodStream({
  debug: true  // Enable debug logging
});

// Define your extraction schema
const schema = z.object({
  content: z.string(),
  metadata: z.object({
    confidence: z.number(),
    category: z.string()
  })
});

// Create streaming extraction
const stream = await client.create({
  completionPromise: async () => {
    const response = await fetch("/api/extract", {
      method: "POST",
      body: JSON.stringify({ prompt: "..." })
    });
    return response.body;
  },
  response_model: {
    schema,
    name: "ContentExtraction"
  }
});

// Process with validation metadata
for await (const chunk of stream) {
  console.log({
    data: chunk,              // Partial extraction result
    isValid: chunk._meta._isValid,    // Current validation state
    activePath: chunk._meta._activePath,    // Currently processing path
    completedPaths: chunk._meta._completedPaths  // Completed paths
  });
}
```

--------------------------------

### zod-stream Response Parsers (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Demonstrates the usage of built-in parsers provided by `zod-stream` for handling different LLM response formats. It shows automatic detection and specific parsers for tool arguments, function arguments, and raw JSON content.

```typescript
import { 
  OAIResponseParser,
  OAIResponseToolArgsParser,
  OAIResponseFnArgsParser,
  OAIResponseJSONParser
} from "zod-stream";

// Automatic format detection
const result = OAIResponseParser(response);

// Format-specific parsing
const toolArgs = OAIResponseToolArgsParser(response);
const fnArgs = OAIResponseFnArgsParser(response);
const jsonContent = OAIResponseJSONParser(response);
```

--------------------------------

### Progressive Data Processing with ZodStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Illustrates progressive processing where dependent data is handled as soon as relevant parts of the stream are complete. This allows for early UI updates, parallel processing, and optimistic loading without waiting for the entire response.

```typescript
// Define schema for a complex analysis
const schema = z.object({
  user: z.object({
    id: z.string(),
    preferences: z.object({
      theme: z.string(),
      language: z.string()
    })
  }),
  content: z.object({
    title: z.string(),
    body: z.string(),
    metadata: z.object({
      keywords: z.array(z.string()),
      category: z.string()
    })
  }),
  recommendations: z.array(z.object({
    id: z.string(),
    score: z.number(),
    reason: z.string()
  }))
});

// Process data as it becomes available
for await (const chunk of stream) {
  // Start personalizing UI as soon as user preferences are ready
  if (isPathComplete(['user', 'preferences'], chunk)) {
    applyUserTheme(chunk.user.preferences.theme);
    setLanguage(chunk.user.preferences.language);
  }

  // Begin content indexing once we have title and keywords
  if (isPathComplete(['content', 'metadata', 'keywords'], chunk) && 
      isPathComplete(['content', 'title'], chunk)) {
    indexContent({
      title: chunk.content.title,
      keywords: chunk.content.metadata.keywords
    });
  }

  // Start fetching recommended content in parallel
  chunk._meta._completedPaths.forEach(path => {
    if (path[0] === 'recommendations' && path.length === 2) {
      const index = path[1] as number;
      const recommendation = chunk.recommendations[index];
      
      if (recommendation?.id) {
        prefetchContent(recommendation.id);
      }
    }
  });
}
```

--------------------------------

### Basic Schema-Stream Usage for Parsing

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/schema-stream/getting-started.mdx

Demonstrates the basic usage of SchemaStream for parsing streaming JSON data. It includes defining a Zod schema, creating a parser, setting up event listeners for key completion, and reading results from the stream with type inference.

```typescript
import { SchemaStream } from 'schema-stream';
import { z } from 'zod';

// Define your schema
const schema = z.object({
  users: z.array(z.object({
    name: z.string(),
    age: z.number()
  })),
  metadata: z.object({
    total: z.number(),
    page: z.number()
  })
});

// Create parser with optional defaults
const parser = new SchemaStream(schema, {
  metadata: { total: 0, page: 1 }
});

// Track completion paths
parser.onKeyComplete(({ completedPaths }) => {
  console.log('Completed:', completedPaths);
});

// Parse streaming data
const stream = parser.parse();
response.body.pipeThrough(stream);

// Read results with full type inference
const reader = stream.readable.getReader();
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  
  const result = JSON.parse(decoder.decode(value));
  // result is fully typed as z.infer<typeof schema>
  console.log(result);
}
```

--------------------------------

### Composite Evaluation Example

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Demonstrates creating a composite evaluator by combining multiple evaluator types ('entities-recall', accuracy, and a model-graded quality check) with specified weights. The data provided must satisfy the requirements of all included evaluators.

```typescript
// Can combine different evaluator types
const compositeEval = createWeightedEvaluator({
  evaluators: {
    entities: createContextEvaluator({ type: "entities-recall" }),
    accuracy: createAccuracyEvaluator({
      weights: { 
        factual: 0.9,   // High weight on exact matches
        semantic: 0.1    // Low weight on similar terms
      }
    }),
    quality: createEvaluator({
      client: oai,
      model: "gpt-4-turbo",
      evaluationDescription: "Rate quality"
    })
  },
  weights: {
    entities: 0.3,
    accuracy: 0.4,
    quality: 0.3
  }
});

// Must provide all required fields for each evaluator type
await compositeEval({
  data: [{
    prompt: "Summarize the earnings call",
    completion: "CEO Jane Smith announced 15% growth",
    expectedCompletion: "The CEO reported strong growth",
    groundTruth: "CEO discussed Q3 performance",
    contexts: [
      "CEO Jane Smith presented Q3 results",
      "Company saw 15% growth in Q3 2023"
    ]
  }]
});
```

--------------------------------

### Model-Graded Evaluator Example

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Demonstrates how to create and use a model-graded evaluator. This evaluator uses an LLM (e.g., 'gpt-4-turbo') to rate a completion based on a prompt. It requires a client, model, and evaluation description.

```typescript
type ModelGradedData = {
  prompt: string;
  completion: string;
  expectedCompletion?: string;  // Ignored for this evaluator type
}

const modelEval = createEvaluator({
  client: oai,
  model: "gpt-4-turbo",
  evaluationDescription: "Rate the response"
});


await modelEval({
  data: [{
    prompt: "What is TypeScript?",
    completion: "TypeScript is a typed superset of JavaScript"
  }]
});
```

--------------------------------

### Enabling Debug Logging for ZodStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Demonstrates how to enable detailed debug logging for the ZodStream client by passing the 'debug: true' option during initialization. This logging provides insights into stream initialization, validation results, path completion, and errors.

```typescript
const client = new ZodStream({ debug: true });

// Logs will include:
// - Stream initialization
// - Validation results
// - Path completion
// - Errors with full context
```

--------------------------------

### Handle Streaming Responses with TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Demonstrates how to create and handle streaming API responses using `zod-stream` in TypeScript. It shows the creation of a streaming endpoint and the conversion of a readable stream to an async generator for processing chunks. Requires the `zod-stream` library.

```typescript
import { OAIStream, readableStreamToAsyncGenerator } from "zod-stream";

// Create streaming response
app.post("/api/stream", async (req, res) => {
  const completion = await oai.chat.completions.create({
    ...params,
    stream: true
  });

  return new Response(
    OAIStream({ res: completion })
  );
});

// Convert stream to async generator
const generator = readableStreamToAsyncGenerator(stream);
for await (const chunk of generator) {
  console.log(chunk);
}
```

--------------------------------

### Nested Data Processing with Schema-Stream

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/schema-stream/getting-started.mdx

Demonstrates how to process nested data structures effectively using SchemaStream. The `onKeyComplete` callback is used to track specific paths within the nested JSON, allowing for targeted business logic execution as data segments complete.

```typescript
const schema = z.object({
  users: z.array(z.object({
    id: z.string(),
    profile: z.object({
      name: z.string(),
      email: z.string(),
      preferences: z.object({
        theme: z.string(),
        notifications: z.boolean()
      })
    }),
    activity: z.array(z.object({
      timestamp: z.string(),
      action: z.string()
    }))
  }))
});

const parser = new SchemaStream(schema);

// Track specific paths for business logic
parser.onKeyComplete(({ activePath, completedPaths }) => {
  const path = activePath.join('.');
  
  // Process user profiles as they complete
  if (path.match(/users\.\d+\.profile$/)) {
    processUserProfile(/* ... */);
  }
  
  // Process activity logs in batches
  if (path.match(/users\.\d+\.activity\.\d+$/)) {
    batchActivityLog(/* ... */);
  }
});
```

--------------------------------

### Stream Metadata Structure (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Defines the structure of the metadata object included with each streamed chunk. This metadata provides information about the validation status, the current parsing path, and all completed paths within the stream.

```typescript
type CompletionMeta = {
  _isValid: boolean;           // Schema validation status
  _activePath: (string | number)[];     // Current parsing path
  _completedPaths: (string | number)[][]; // All completed paths
}

// Example chunk
{
  content: "partial content...",
  metadata: {
    confidence: 0.95
  },
  _meta: {
    _isValid: false,  // Not valid yet
    _activePath: ["metadata", "category"],
    _completedPaths: [
      ["content"],
      ["metadata", "confidence"]
    ]
  }
}
```

--------------------------------

### Generating Schema Stubs with ZodStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Shows how to generate typed stub objects for initialization using a Zod schema. This is useful for pre-filling data structures or providing default values before the actual data is streamed.

```typescript
const schema = z.object({
  users: z.array(z.object({
    name: z.string(),
    age: z.number()
  }))
});

const client = new ZodStream();
const stub = client.getSchemaStub({
  schema,
  defaultData: {
    users: [{ name: "loading...", age: 0 }]
  }
});
```

--------------------------------

### Install zod-stream and Dependencies

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/zod-stream/README.md

Instructions for installing zod-stream and its common dependencies (zod, openai) using npm, pnpm, and bun package managers.

```bash
# npm
npm install zod-stream zod openai

# pnpm
pnpm add zod-stream zod openai

# bun
bun add zod-stream zod openai
```

--------------------------------

### Accuracy Evaluator Example

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Shows how to create and use an accuracy evaluator. This evaluator compares a completion against an expected completion, requiring a 'completion' and 'expectedCompletion' field in the data. Weights can be provided for factual and semantic accuracy.

```typescript
type AccuracyData = {
  completion: string;
  expectedCompletion: string;  // Required for accuracy comparison
}

const accuracyEval = createAccuracyEvaluator({
  weights: { factual: 0.5, semantic: 0.5 }
});

await accuracyEval({
  data: [{
    completion: "TypeScript adds types to JavaScript",
    expectedCompletion: "TypeScript is JavaScript with type support"
  }]
});
```

--------------------------------

### JSON Mode Configuration for OpenAI (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Details the configuration for `JSON` mode, which instructs OpenAI to return a direct JSON object. It sets the `response_format` to `json_object` and modifies the system message to enforce JSON output based on a schema.

```typescript
// Results in direct JSON response configuration
{
  response_format: { type: "json_object" },
  messages: [
    {
      role: "system",
      content: "
        Given a user prompt, you will return fully valid JSON based on the following description and schema.
        You will return no other prose. You will take into account any descriptions or required parameters within the schema
        and return a valid and fully escaped JSON object that matches the schema and those instructions.

        description: ${definition.description}
        json schema: ${JSON.stringify(definition)}
      "
    },
    // ... user messages are preserved
  ]
}
```

--------------------------------

### Google Generative AI SDK Installation via Bun

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/llm-client/README.md

Installs the Google Generative AI SDK using the Bun package manager. This SDK is necessary for integrating Google's Gemini models via the llm-polyglot library.

```bash
bun add @google/generative-ai
```

--------------------------------

### Install llm-polyglot and OpenAI SDK

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/llm-client/README.md

Installs the base llm-polyglot library along with the OpenAI SDK for core LLM functionalities. Additional provider-specific SDKs can be installed as needed.

```bash
# Base installation
npm install llm-polyglot openai

# Provider-specific SDKs (as needed)
npm install @anthropic-ai/sdk    # For Anthropic
npm install @google/generative-ai # For Google/Gemini
```

--------------------------------

### Progressive Updates with useJsonStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/stream-hooks/getting-started.mdx

Illustrates how to leverage the `useJsonStream` hook for progressive UI updates. The `onReceive` callback processes incoming data chunks, allowing for partial updates to the UI as specific data paths become available, defined by a complex Zod schema.

```typescript
const AnalysisComponent = () => {
  const { data } = useJsonStream({
    schema: z.object({
      user: z.object({
        preferences: z.object({
          theme: z.string(),
          language: z.string()
        })
      }),
      content: z.object({
        title: z.string(),
        body: z.string()
      })
    }),
    onReceive: (chunk) => {
      // Start personalizing as soon as preferences are available
      if (isPathComplete(['user', 'preferences'], chunk)) {
        applyTheme(chunk.user.preferences.theme);
      }

      // Begin content rendering when title is ready
      if (isPathComplete(['content', 'title'], chunk)) {
        updateTitle(chunk.content.title);
      }
    }
  });

  return <div>{/* Your UI */}</div>;
};

```

--------------------------------

### Install llm-polyglot for Google Gemini (Bun)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/llm-polyglot/google.mdx

Installs the llm-polyglot library along with necessary dependencies (openai and @google/generative-ai) for Google Gemini integration using Bun. Ensure you have Bun installed to run this command.

```bash
bun add llm-polyglot openai @google/generative-ai
```

--------------------------------

### Anthropic SDK Installation via Bun

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/llm-client/README.md

Installs the official Anthropic SDK using the Bun package manager. This SDK is required for utilizing the 'anthropic' provider with the llm-polyglot library.

```bash
bun add @anthropic-ai/sdk
```

--------------------------------

### zod-stream Response Modes Enum (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Defines an enumeration of available response modes supported by `zod-stream`. These modes dictate how the LLM structures its output, such as using function calling, tools, or direct JSON.

```typescript
import { MODE } from "zod-stream";

const modes = {
  FUNCTIONS: "FUNCTIONS",   // OpenAI function calling
  TOOLS: "TOOLS",          // OpenAI tools API
  JSON: "JSON",            // Direct JSON response
  MD_JSON: "MD_JSON",      // JSON in markdown blocks
  JSON_SCHEMA: "JSON_SCHEMA", // JSON with schema validation
  THINKING_MD_JSON: "THINKING_MD_JSON" // JSON with thinking in markdown blocks (deepseek r1)
} as const;
```

--------------------------------

### Track Path Completion Status with TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/getting-started.mdx

Illustrates how to monitor the completion status of specific paths using the `isPathComplete` utility from `zod-stream`. This function checks if a given active path has been completed based on the provided metadata. Requires the `zod-stream` library.

```typescript
import { isPathComplete } from "zod-stream";

const activePath = ["analysis", "sentiment"];
const isComplete = isPathComplete(activePath, {
  _meta: {
    _completedPaths: [["analysis", "sentiment"]],
    _activePath: ["analysis", "keywords"],
    _isValid: false
  }
});
```

--------------------------------

### Install llm-polyglot for Google Gemini (npm)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/llm-polyglot/google.mdx

Installs the llm-polyglot library along with necessary dependencies (openai and @google/generative-ai) for Google Gemini integration using npm. Ensure you have npm installed to run this command.

```bash
npm install llm-polyglot openai @google/generative-ai
```

--------------------------------

### Instructor Integration Example (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/schemaStream/README.md

Illustrates the integration of SchemaStream with the Instructor library for high-level AI response modeling and extraction.

```typescript
const client = Instructor({
  client: oai,
  mode: "TOOLS"
});

const result = await client.chat.completions.create({
  response_model: { schema: yourSchema } // yourSchema would be compatible with SchemaStream
  // ...
});
```

--------------------------------

### Install llm-polyglot for Google Gemini (pnpm)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/llm-polyglot/google.mdx

Installs the llm-polyglot library along with necessary dependencies (openai and @google/generative-ai) for Google Gemini integration using pnpm. Ensure you have pnpm installed to run this command.

```bash
pnpm add llm-polyglot openai @google/generative-ai
```

--------------------------------

### Context Evaluator Types and Usage

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Illustrates the creation and usage of different context evaluators: 'entities-recall', 'precision', 'recall', and 'relevance'. Each evaluator requires specific data fields like 'prompt', 'completion', 'groundTruth', and 'contexts'.

```typescript
type ContextData = {
  prompt: string;
  completion: string;
  groundTruth: string;   // Required for context evaluation
  contexts: string[];    // Required for context evaluation
}

// Entities Recall - Checks named entities
const entitiesEval = createContextEvaluator({
  type: "entities-recall"
});

// Precision - Checks accuracy against context
const precisionEval = createContextEvaluator({
  type: "precision"
});

// Recall - Checks information coverage
const recallEval = createContextEvaluator({
  type: "recall"
});

// Relevance - Checks contextual relevance
const relevanceEval = createContextEvaluator({
  type: "relevance"
});

// Example usage
const data = {
  prompt: "What did the CEO say about Q3?",
  completion: "CEO Jane Smith reported 15% growth in Q3 2023",
  groundTruth: "The CEO announced strong Q3 performance",
  contexts: [
    "CEO Jane Smith presented Q3 results",
    "Company saw 15% revenue growth in Q3 2023"
  ]
};

await entitiesEval({ data: [data] });   // Focuses on "Jane Smith", "Q3", "2023"
await precisionEval({ data: [data] });  // Checks factual accuracy
await recallEval({ data: [data] });     // Checks information completeness
await relevanceEval({ data: [data] });  // Checks contextual relevance
```

--------------------------------

### Process Product Enrichment Stream - TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/examples.mdx

Processes streaming product data for e-commerce enrichment. It initializes product cards, updates pricing and inventory displays, optimizes product SEO, and prefetches related products.

```typescript
for await (const chunk of stream) {
  // 1. Initialize product display with basic info
  if (isPathComplete(['basic'], chunk)) {
    initializeProductCard(chunk.basic);
  }

  // 2. Handle pricing and inventory updates
  if (isPathComplete(['pricing', 'final'], chunk)) {
    updatePriceDisplay(chunk.pricing.final);
    
    if (isPathComplete(['inventory', 'status'], chunk)) {
      updateBuyButton({
        price: chunk.pricing.final,
        status: chunk.inventory.status
      });
    }
  }

  // 3. Handle SEO optimization
  if (isPathComplete(['enrichment', 'seoDescription'], chunk) &&
      isPathComplete(['enrichment', 'searchKeywords'], chunk)) {
    optimizeProductSEO({
      description: chunk.enrichment.seoDescription,
      keywords: chunk.enrichment.searchKeywords
    });
  }

  // 4. Handle related products
  if (isPathComplete(['enrichment', 'relatedProducts'], chunk)) {
    prefetchRelatedProducts(chunk.enrichment.relatedProducts);
  }
}
```

--------------------------------

### Install Anthropic SDK with llm-polyglot (Bun, npm, pnpm)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/llm-polyglot/anthropic.mdx

Installs the necessary packages for Anthropic API integration, including llm-polyglot, openai, and the official Anthropic SDK, across different package managers.

```bash
bun add llm-polyglot openai @anthropic-ai/sdk
```

```bash
npm install llm-polyglot openai @anthropic-ai/sdk
```

```bash
pnpm add llm-polyglot openai @anthropic-ai/sdk
```

--------------------------------

### Error Handling with useJsonStream (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/stream-hooks/getting-started.mdx

Demonstrates robust error handling for data streams using the `useJsonStream` hook. The `onError` callback captures stream errors, and the hook provides an `error` state and a `reset` function to manage and recover from errors in the UI.

```typescript
function StreamComponent() {
  const { error, reset } = useJsonStream({
    schema,
    onError: (err) => {
      console.error("Stream error:", err);
    }
  });

  if (error) {
    return (
      <div>
        <p>Error: {error.message}</p>
        <button onClick={reset}>Retry</button>
      </div>
    );
  }

  return <div>{/* Your UI */}</div>;
}

```

--------------------------------

### ZodStream Integration Example (TypeScript)

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/schemaStream/README.md

Shows an example of how ZodStream utilizes SchemaStream to add validation and OpenAI integration for data extraction from streaming responses.

```typescript
// Example of zod-stream using schema-stream
const zodStream = new ZodStream();
const extraction = await zodStream.create({
  completionPromise: stream,
  response_model: { 
    schema: yourSchema,
    name: "Extract"
  }
});
```

--------------------------------

### Process Document Stream - TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/examples.mdx

Processes streaming document data, initiating indexing when metadata is available, processing section annotations, and generating a preview when summary information is complete.

```typescript
for await (const chunk of stream) {
  // 1. Start document indexing when metadata arrives
  if (isPathComplete(['metadata'], chunk)) {
    indexDocument({
      title: chunk.metadata.title,
      topics: chunk.metadata.topics
    });
  }

  // 2. Process section annotations as they complete
  chunk._meta._completedPaths.forEach(path => {
    if (path[0] === 'sections' && isPathComplete([...path, 'annotations'], chunk)) {
      const sectionIndex = path[1] as number;
      const section = chunk.sections[sectionIndex];
      processAnnotations({
        heading: section.heading,
        annotations: section.annotations
      });
    }
  });

  // 3. Generate preview when required fields are available
  if (isPathComplete(['summary', 'abstract'], chunk) && 
      isPathComplete(['summary', 'readingTime'], chunk)) {
    generatePreview({
      abstract: chunk.summary.abstract,
      readingTime: chunk.summary.readingTime
    });
  }
}
```

--------------------------------

### Define E-commerce Product Schema - TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/examples.mdx

Defines the TypeScript schema for e-commerce product data using Zod. It includes structures for basic product information, pricing, inventory, and enrichment details like SEO descriptions and related products.

```typescript
const productSchema = z.object({
  basic: z.object({
    id: z.string(),
    name: z.string(),
    category: z.string()
  }),
  pricing: z.object({
    base: z.number(),
    discounts: z.array(z.object({
      type: z.string(),
      amount: z.number()
    })),
    final: z.number()
  }),
  inventory: z.object({
    status: z.string(),
    locations: z.array(z.object({
      id: z.string(),
      quantity: z.number()
    }))
  }),
  enrichment: z.object({
    seoDescription: z.string(),
    searchKeywords: z.array(z.string()),
    relatedProducts: z.array(z.string())
  })
});
```

--------------------------------

### Process Market Data Stream - TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/zod-stream/examples.mdx

Handles streaming market data by iterating through chunks and performing actions based on completed data paths. It visualizes trends, processes competitor data, and plans budget allocation.

```typescript
for await (const chunk of stream) {
  // 1. Visualize market trends as soon as they're available
  if (isPathComplete(['marketData', 'trends'], chunk)) {
    initializeCharts(chunk.marketData.trends);
  }

  // 2. Process competitor data as each competitor entry completes
  chunk._meta._completedPaths.forEach(path => {
    if (path[0] === 'competitors' && path.length === 2) {
      const competitor = chunk.competitors[path[1] as number];
      fetchCompetitorData(competitor.name);
    }
  });

  // 3. Handle budget planning when both requirements are met
  if (isPathComplete(['recommendations', 'immediate'], chunk) && 
      isPathComplete(['recommendations', 'budget'], chunk)) {
    planBudgetAllocation({
      actions: chunk.recommendations.immediate,
      budget: chunk.recommendations.budget
    });
  }
}
```

--------------------------------

### Perform Chat Completions with Anthropic, Google, and OpenAI-compatible Providers

Source: https://context7.com/hack-dance/island-ai/llms.txt

Illustrates how to use the unified client to create chat completions across different LLM providers. This example shows a basic completion request and how to process the response, including accessing the generated content. It requires the respective provider's API key and model name.

```typescript
// Use consistent OpenAI-style API across all providers
const anthropicCompletion = await anthropicClient.chat.completions.create({
  model: "claude-3-opus-20240229",
  max_tokens: 1000,
  messages: [
    { role: "system", content: "You are a helpful assistant" },
    { role: "user", content: "Explain quantum computing in simple terms" }
  ]
});

console.log('Anthropic response:', anthropicCompletion.choices[0].message.content);
```

--------------------------------

### TypeScript: Direct HTTP Streaming with zod-stream and OpenAI

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/instructor-js.mdx

This example demonstrates how to build custom streaming endpoints using zod-stream and OpenAI. It configures OpenAI parameters with a defined Zod schema and returns a streaming response. The client-side consumption shows how to pipe the stream through a parser for partial data updates.

```typescript
import { OAIStream, SchemaStream } from "zod-stream";
import { withResponseModel } from "zod-stream";
import OpenAI from "openai";
import { z } from "zod";

const oai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  organization: process.env.OPENAI_ORG_ID
});

// Define your schema
const schema = z.object({
  content: z.string(),
  users: z.array(z.object({
    name: z.string(),
  })),
});

// API Route Example (Next.js)
export async function POST(request: Request) {
  const { messages } = await request.json();

  // Configure OpenAI parameters with schema
  const params = withResponseModel({
    response_model: { 
      schema: schema, 
      name: "Users extraction and message" 
    },
    params: {
      messages,
      model: "gpt-4",
    },
    mode: "TOOLS",
  });

  // Create streaming completion
  const extractionStream = await oai.chat.completions.create({
    ...params,
    stream: true,
  });

  // Return streaming response
  return new Response(
    OAIStream({ res: extractionStream })
  );
}

// Client-side consumption
async function consumeStream() {
  const response = await fetch('/api/extract', {
    method: 'POST',
    body: JSON.stringify({
      messages: [{ role: 'user', content: 'Your prompt here' }]
    })
  });

  const parser = new SchemaStream(schema);
  const stream = parser.parse();

  response.body
    ?.pipeThrough(stream)
    .pipeTo(new WritableStream({
      write(chunk) {
        const data = JSON.parse(new TextDecoder().decode(chunk));
        // Use partial data as it arrives
        console.log('Partial data:', data);
      }
    }));
}
```

--------------------------------

### Context Evaluation Setup in TypeScript

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/index.mdx

Creates a context evaluator to measure how well an output utilizes and stays faithful to provided context. Supports different evaluation types like 'precision' or 'recall'.

```typescript
const contextEval = createContextEvaluator({ 
  type: "precision"  // or "recall", "relevance", "entities-recall" 
});
```

--------------------------------

### OpenAI JSON Mode Configuration (Object)

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/zod-stream/README.md

Details the configuration for OpenAI's JSON mode, specifying `response_format` as `{"type": "json_object"}` and including system and user messages to guide the model towards JSON output.

```json
{
  "response_format": { "type": "json_object" },
  "messages": [
    {
      "role": "system",
      "content": "Return JSON matching schema..."
    },
    // ... user messages
  ]
}
```

--------------------------------

### Handling Progressive Updates with useJsonStream

Source: https://github.com/hack-dance/island-ai/blob/main/public-packages/hooks/README.md

Illustrates how to utilize the `onReceive` callback within `useJsonStream` to process data chunks as they arrive. This example shows partial personalization and UI updates based on specific data paths becoming available, using a nested Zod schema.

```typescript
const AnalysisComponent = () => {
  const { data } = useJsonStream({
    schema: z.object({
      user: z.object({
        preferences: z.object({
          theme: z.string(),
          language: z.string()
        })
      }),
      content: z.object({
        title: z.string(),
        body: z.string()
      })
    }),
    onReceive: (chunk) => {
      // Start personalizing as soon as preferences are available
      if (isPathComplete(['user', 'preferences'], chunk)) {
        applyTheme(chunk.user.preferences.theme);
      }

      // Begin content rendering when title is ready
      if (isPathComplete(['content', 'title'], chunk)) {
        updateTitle(chunk.content.title);
      }
    }
  });

  return <div>{/* Your UI */}</div>;
};
```

--------------------------------

### Configure OpenAI Client for Together.ai Provider

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/llm-polyglot/index.mdx

This TypeScript snippet demonstrates how to configure the llm-polyglot client to use the Together.ai provider. It specifies the provider as 'openai' and sets the 'baseURL' to Together.ai's API endpoint. The example then shows how to create a chat completion using a model hosted by Together.ai.

```typescript
const client = createLLMClient({
  provider: "openai",
  baseURL: "https://api.together.xyz/v1"
});

// Use any Together-hosted model
const completion = await client.chat.completions.create({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  messages: [{ role: "user", content: "Hello!" }]
});
```

--------------------------------

### Stream OpenAI Completions with zod-stream

Source: https://github.com/hack-dance/island-ai/blob/main/README.md

This example shows how to use zod-stream to integrate with OpenAI for structured data extraction. It defines an ExtractionSchema using Zod, configures OpenAI parameters with this schema and a specific response mode ('TOOLS'), and then streams the completions. Finally, it processes the results using ZodStream, iterating over progressively updated data.

```typescript
import { OAIStream } from "zod-stream";
import { withResponseModel } from "zod-stream";
import { z } from "zod";

// Define extraction schema
const ExtractionSchema = z.object({
  users: z.array(z.object({
    name: z.string(),
    handle: z.string(),
    twitter: z.string()
  })).min(3),
  location: z.string(),
  budget: z.number()
});

// Configure OpenAI params with schema
const params = withResponseModel({
  response_model: { 
    schema: ExtractionSchema, 
    name: "Extract" 
  },
  params: {
    messages: [{ role: "user", content: textBlock }],
    model: "gpt-4"
  },
  mode: "TOOLS"
});

// Stream completions
const stream = OAIStream({
  res: await oai.chat.completions.create({
    ...params,
    stream: true
  })
});

// Process results
const client = new ZodStream();
const extractionStream = await client.create({
  completionPromise: () => stream,
  response_model: { 
    schema: ExtractionSchema, 
    name: "Extract" 
  }
});

for await (const data of extractionStream) {
  console.log('Progressive update:', data);
}

```

--------------------------------

### Create LLM-based evaluators with evalz

Source: https://context7.com/hack-dance/island-ai/llms.txt

This snippet demonstrates how to create LLM-based evaluators using the 'evalz' library. It shows how to set up both score-based (0-1 scale) and binary (true/false) evaluators, configuring them with a client, model, evaluation description, and optional system messages. The examples include evaluating multiple completions and logging the results.

```typescript
import { createEvaluator } from "evalz";
import OpenAI from "openai";

const oai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

// Create score-based evaluator (0-1 scale)
const relevanceEvaluator = createEvaluator({
  client: oai,
  model: "gpt-4-turbo",
  evaluationDescription: "Rate the relevance of the completion to the prompt on a scale of 0 to 1",
  resultsType: "score",
  messages: [
    {
      role: "system",
      content: "You are an expert evaluator. Consider accuracy, completeness, and relevance."
    }
  ]
});

// Evaluate multiple completions
const scoreResults = await relevanceEvaluator({
  data: [
    {
      prompt: "What is TypeScript?",
      completion: "TypeScript is a strongly typed programming language that builds on JavaScript.",
      expectedCompletion: "TypeScript is a typed superset of JavaScript."
    },
    {
      prompt: "Explain async/await",
      completion: "Async/await is syntactic sugar for promises in JavaScript.",
      expectedCompletion: "Async/await provides a cleaner way to work with asynchronous code."
    }
  ]
});

console.log('Overall score:', scoreResults.scoreResults.value);
// 0.85
console.log('Individual results:', scoreResults.results);
// [{ value: 0.9, reasoning: "..." }, { value: 0.8, reasoning: "..." }]

// Create binary evaluator (true/false)
const factualEvaluator = createEvaluator({
  client: oai,
  model: "gpt-4-turbo",
  evaluationDescription: "Determine if the completion is factually correct",
  resultsType: "binary"
});

const binaryResults = await factualEvaluator({
  data: [
    {
      prompt: "What is the capital of France?",
      completion: "The capital of France is Paris."
    },
    {
      prompt: "What is 2+2?",
      completion: "2+2 equals 5."
    }
  ]
});

console.log('True count:', binaryResults.binaryResults.trueCount);
// 1
console.log('False count:', binaryResults.binaryResults.falseCount);
// 1

```

--------------------------------

### Define Context Evaluator Types

Source: https://github.com/hack-dance/island-ai/blob/main/apps/www/src/content/docs/evalz/getting-started.mdx

Defines the TypeScript type for different context evaluator types: 'entities-recall', 'precision', 'recall', and 'relevance'. These types specify the focus of the evaluation.

```typescript
type ContextEvaluatorType = "entities-recall" | "precision" | "recall" | "relevance";
```