### JavaScript Pipeline Example

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Demonstrates building and running a question generation pipeline using the llm-core library in JavaScript. It shows context creation, defining pipeline steps, and assembling the pipeline.

```javascript
import { pipeline } from "@jasonnathan/llm-core";

// 1) Build your context once
const ctx = {
  logger: console,
  pipeline: {},
  state: { history: [] },
};

// 2) Define your steps as curried functions
const collectContentStep = (ctx) => async (docs) => {
  ctx.logger.info("Collecting content…");
  return [
    ...docs,
    { source: "doc1.md", content: "Pipelines are great.", questions: [] },
    { source: "doc2.md", content: "They’re easy to use.", questions: [] },
  ];
};

const generateQuestionsStep = (ctx) => async (docs) => {
  ctx.logger.info("Generating questions…");
  return docs.map((doc) => ({
    ...doc,
    questions: [`What is the main point of ${doc.source}?`],
  }));
};

// 3) Assemble and run
const questionPipeline = pipeline(ctx)
  .addStep(collectContentStep)
  .addStep(generateQuestionsStep);

;(async () => {
  const result = await questionPipeline.run([]);
  console.log(JSON.stringify(result, null, 2));
})();

```

--------------------------------

### TypeScript Pipeline Example

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Illustrates constructing and executing a question generation pipeline with the llm-core library in TypeScript. This includes defining context interfaces, pipeline steps, and running the pipeline.

```typescript
import { pipeline, PipelineStep } from "@jasonnathan/llm-core";

interface QuestionDoc {
  source: string;
  content: string;
  questions: string[];
}

// 1) Define your application context shape
type Ctx = {
  logger: Console;
  pipeline: {
    retries?: number;
    timeout?: number;
    cache?: Map<any, unknown>;
    stopCondition?: (doc: QuestionDoc[]) => boolean;
  };
  state: { history: Array<{ step: number; doc: QuestionDoc[] }> };
};

const ctx: Ctx = {
  // your own fields
  logger: console,

  // pipeline helper slots (empty for defaults)
  pipeline: {},

  // internal state for stream/resume
  state: { history: [] },
};

// 2) Define steps; each is (ctx) => (doc) => T | PipelineOutcome<T>
const collectContentStep: PipelineStep<QuestionDoc[], QuestionDoc[], Ctx> = (ctx) => async (docs) => {
  ctx.logger.info("Collecting content…");
  return [
    ...docs,
    { source: "doc1.md", content: "Pipelines are great.", questions: [] },
    { source: "doc2.md", content: "They’re easy to use.", questions: [] },
  ];
};

const generateQuestionsStep: PipelineStep<QuestionDoc[], QuestionDoc[], Ctx> = (ctx) => async (docs) => {
  ctx.logger.info("Generating questions…");
  return docs.map((doc) => ({
    ...doc,
    questions: [`What is the main point of ${doc.source}?`],
  }));
};

// 3) Build and run the pipeline
const questionPipeline = pipeline<Ctx, QuestionDoc[]>(ctx)
  .addStep(collectContentStep)
  .addStep(generateQuestionsStep);

async function main() {
  const initial: QuestionDoc[] = [];
  const result = await questionPipeline.run(initial);
  console.log(JSON.stringify(result, null, 2));
}

main();

```

--------------------------------

### Install @pipewrk/llm-core using Bun

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

This snippet shows how to install the @pipewrk/llm-core library using the Bun package manager. It's a straightforward command for adding the dependency to your project.

```bash
bun install @jasonnathan/llm-core
```

--------------------------------

### Example Pipeline Steps: Trim and Require Approval

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Provides two example pipeline steps: 'trim' to remove whitespace from a message's text using a logger from the context, and 'requireApproval' to pause the pipeline if a message is not approved.

```typescript
interface MsgCtx {
  logger: Console;
}
interface Message {
  text: string;
  approved?: boolean;
}

// Trim whitespace
const trim: PipelineStep<Message, Message, MsgCtx> = (ctx) => (doc) => {
  ctx.logger.info("Trimming text...");
  return { ...doc, text: doc.text.trim() };
};

// Require approval
const requireApproval: PipelineStep<Message, Message> = () => (doc) => {
  if (!doc.approved) {
    return {
      done: false,
      reason: "approval",
      payload: doc,
    } as PipelineOutcome<Message>;
  }
  return doc;
};
```

--------------------------------

### Build Complex LLM Pipeline with Utilities

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

Demonstrates building a sophisticated LLM pipeline that incorporates retries, timeouts, caching, and tap operations. This example defines pipeline steps, a context object, and constructs the pipeline using builder methods and utility functions like `withRetry`, `withCache`, and `withTimeout`. Requires specific types from `@jasonnathan/llm-core`.

```typescript
type Ctx = Logger & { pipeline?: { retries?: number; timeout?: number; cache?: Map<unknown,unknown> } };

const stepFetch: PipelineStep<Input, Mid, Ctx> = /* … */;
const stepProcess: PipelineStep<Mid, Mid, Ctx> = /* … */;
const stepFinal: PipelineStep<Mid, Final, Ctx> = /* … */;

const ctx: Ctx = Object.assign(new Logger('./run.md'), {
  pipeline: { retries: 2, timeout: 1500, cache: new Map() }
});

const p = pipeline<Ctx, Input>(ctx)
  .addStep(withRetry(withErrorHandling(stepFetch)))      // I: Input → O: Mid
  .addStep(withCache(withTimeout(stepProcess), m => m.key)) // Mid → Mid
  .addStep(tap<Mid, Ctx>((c, d) => c.info?.(`mid: ${d.key}`)))
  .addStep(stepFinal);                                   // Mid → Final

const result = await p.run(initialInput);
```

--------------------------------

### Install @pipewrk/llm-core using npm

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

This snippet demonstrates how to install the @pipewrk/llm-core library using the npm package manager. This is a common way to add the dependency to your Node.js project.

```bash
npm install @jasonnathan/llm-core
```

--------------------------------

### TypeScript Ollama Service Example

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Shows how to create and use an Ollama service with typed JSON responses in TypeScript. It demonstrates configuring the Ollama context and calling a helper function to generate a greeting.

```typescript
import { createOllamaContext, createOllamaService } from "@jasonnathan/llm-core";

const ctx = createOllamaContext({ model: "llama3:8b-instruct-q8_0" });
const ollama = createOllamaService(ctx);

async function getGreeting() {
  const response = await ollama.generatePromptAndSend<{ greeting: string }>
    (
      "You are a friendly assistant.",
      "Provide a JSON greeting: {greeting}",
      { schema: { type: "object", properties: { greeting: { type: "string" } }, required: ["greeting"] } }
    );
  console.log(response.greeting);
}

```

--------------------------------

### Split Markdown into Logical Segments using markdownSplitter

Source: https://github.com/pipewrk/llm-core/blob/main/CHUNKER.md

The `markdownSplitter` utility function parses a markdown string and splits it into logical segments based on its syntax. It can perform basic splitting or group content under headings. It requires the markdown content as input and an optional configuration object.

```typescript
import { markdownSplitter } from '@jasonnathan/llm-core';
import fs from 'fs/promises';

async function main() {
  const markdownContent = await fs.readFile('my-doc.md', 'utf-8');
  
  // Basic splitting
  const segments = markdownSplitter(markdownContent);

  // Group content under headings
  const headingGroups = markdownSplitter(markdownContent, { useHeadingsOnly: true });

  console.log(headingGroups);
}

main();
```

--------------------------------

### Chunk Markdown Content using cosineDropChunker

Source: https://github.com/pipewrk/llm-core/blob/main/CHUNKER.md

This function demonstrates how to chunk markdown content using `cosineDropChunker`. It first reads the markdown file, then calls `cosineDropChunker` with a context object and chunking options. The `type` option is set to 'markdown', indicating that `markdownSplitter` will be used internally. Chunking parameters like `breakPercentile`, `minChunkSize`, `maxChunkSize`, and `overlapSize` can be configured.

```typescript
import { cosineDropChunker } from '@jasonnathan/llm-core';
import { createOllamaContext, embedTexts } from '@jasonnathan/llm-core';
import fs from 'fs/promises';

// 1) Build an embedding context (Ollama example)
const svc = createOllamaContext({ ollama: { model: 'all-minilm:l6-v2' } });
const embed = (texts: string[]) => embedTexts(svc, texts);

// 2) Build chunker context
const ctx = { logger: console, embed, pipeline: { retries: 0, timeout: 0 } };

async function chunkMarkdown() {
  const markdownContent = await fs.readFile('README.md', 'utf-8');

  const chunks = await cosineDropChunker(ctx, markdownContent, {
    type: 'markdown',
    breakPercentile: 95, // Split only at the most significant topic changes
    minChunkSize: 300,
    maxChunkSize: 2000,
    overlapSize: 1,      // Overlap by one sentence/segment
  });

  console.log(`Generated ${chunks.length} chunks.`);
}
```

--------------------------------

### TypeScript Type Definitions for Embeddings and Chunker Context

Source: https://github.com/pipewrk/llm-core/blob/main/CHUNKER.md

Defines the necessary TypeScript types for embedding functions and the context object used in the chunking process. The `EmbedFunction` handles text embedding, returning a promise of number arrays. `ChunkerContext` includes optional logging and pipeline settings, and crucially, the `embed` function.

```typescript
type EmbedFunction = (texts: string[]) => Promise<number[][]>;

type ChunkerContext = {
  logger?: { info?: (s: string) => void; warn?: (s: string) => void; error?: (e: unknown) => void };
  pipeline?: { retries?: number; timeout?: number };
  embed: EmbedFunction;
};
```

--------------------------------

### Generate Typed JSON Response with Ollama API in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Demonstrates using the `generatePromptAndSend` function to get a typed JSON response from Ollama. It defines a `UserProfile` interface and a schema to ensure the output is correctly structured and parsed, with error handling for failed requests.

```typescript
import { createOllamaContext, generatePromptAndSend } from '@jasonnathan/llm-core';

interface UserProfile {
  name: string;
  email: string;
  age: number;
}

const ctx = createOllamaContext({ model: 'llama3:8b-instruct-q8_0' });

async function main() {
  const systemPrompt = "You are a data extraction expert. Generate a JSON object from the user's text.";
  const userPrompt = "Create a profile for John Doe. His email is john.doe@example.com and he is 30 years old.";

  const options = {
    schema: {
      type: "object",
      properties: {
        name: { type: "string" },
        email: { type: "string" },
        age: { type: "number" },
      },
      required: ["name", "email", "age"],
    },
  };

  try {
    const profile = await generatePromptAndSend<UserProfile>(
      ctx,
      systemPrompt,
      userPrompt,
      options
    );

    console.log(profile);
    // Expected Output:
    // {
    //   name: "John Doe",
    //   email: "john.doe@example.com",
    //   age: 30
    // }
  } catch (error) {
    console.error("Failed to generate profile:", error);
  }
}

main();
```

--------------------------------

### Chunk Plain Text Content using cosineDropChunker

Source: https://github.com/pipewrk/llm-core/blob/main/CHUNKER.md

This function shows how to chunk unstructured plain text using `cosineDropChunker`. It defines sample text content and then calls `cosineDropChunker` with a context object and specific chunking options. The `type` is set to 'text', meaning the content will be split by sentences. Parameters like `breakPercentile` and `bufferSize` control the chunking behavior.

```typescript
import { cosineDropChunker } from '@jasonnathan/llm-core';
import { createOllamaContext, embedTexts } from '@jasonnathan/llm-core';

// 1) Build an embedding context (Ollama example)
const svc = createOllamaContext({ ollama: { model: 'all-minilm:l6-v2' } });
const embed = (texts: string[]) => embedTexts(svc, texts);

// 2) Build chunker context
const ctx = { logger: console, embed, pipeline: { retries: 0, timeout: 0 } };

async function chunkText() {
  const textContent = "This is the first sentence. This is the second one. A third sentence provides new context and talks about something different.";

  const chunks = await cosineDropChunker(ctx, textContent, {
    type: 'text',
    breakPercentile: 90,
    bufferSize: 2, // Use a window of 2 sentences for similarity checks
  });

  console.log(chunks);
}
```

--------------------------------

### Pipeline Factory and Usage

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Demonstrates how to create a pipeline, add steps, and run or stream documents through it.

```APIDOC
## Pipeline Creation and Execution

### Description
This section covers the creation of pipelines using the `pipeline` factory and adding steps to them, along with methods for running and streaming documents.

### `pipeline<C, T>(ctx: C): Pipeline<C, T>`

#### Description
Creates a new pipeline instance bound to a specific context.

#### Parameters
- **ctx** (C): The context object to be shared across all pipeline steps.

#### Returns
- A new `Pipeline` instance.

#### Example
```typescript
const myCtx = {
  logger: console,
  pipeline: {},
  state: { history: [] },
};

const p = pipeline(myCtx);
```

### `addStep(step: PipelineStep<T, N, C>) → Pipeline<C, T, N>`

#### Description
Appends a single step to the pipeline's sequence.

#### Parameters
- **step** (PipelineStep<T, N, C>): The step to add to the pipeline.

#### Returns
- The modified `Pipeline` instance with the new step added.

#### Example
```typescript
p.addStep(trimStep)
 .addStep(validateStep)
 .addStep(transformStep);
```

### Running and Streaming

#### `run(doc: T): Promise<O>`

##### Description
Processes the entire pipeline for a given document from start to finish.

##### Parameters
- **doc** (T): The initial document to process.

##### Returns
- A Promise resolving to the final processed document.

##### Example
```typescript
p.run({ text: " hello " }).then((final) => console.log(final));
```

#### `stream(doc: T): AsyncIterable<PipelineEvent>`

##### Description
Returns an async iterable that yields events as the pipeline processes the document. Allows for observing progress and handling pauses.

##### Parameters
- **doc** (T): The initial document to process.

##### Returns
- An AsyncIterable yielding pipeline events.

##### Example
```typescript
for await (const evt of p.stream({ text: " hello " })) {
  if (evt.type === "progress") {
    ctx.logger.info(`progress at step #${evt.step + 1}`);
  }
  if (evt.type === "pause") {
    ctx.logger.warn(`paused at step #${evt.step + 1}: ${evt.info.reason}`);
    await p.next(evt.resume.doc, evt.resume);
  }
}
```

#### `next(doc: T, resume?: PipelineResumeState<T>) → Promise<PipelineEvent>`

##### Description
Processes the next step in the pipeline or resumes from a paused state.

##### Parameters
- **doc** (T): The document to process.
- **resume** (PipelineResumeState<T>, optional): The state to resume from if the pipeline was previously paused.

##### Returns
- A Promise resolving to the next pipeline event.

##### Example
```typescript
// Assuming 'evt' is a pause event from stream()
await p.next(evt.resume.doc, evt.resume);
```
```

--------------------------------

### Run Project Tests using Bun

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Executes the test suite for the project using Bun.

```bash
bun test
```

--------------------------------

### Create and Run a Pipeline

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Demonstrates how to create a pipeline with a specific context and message type, add steps like 'trim' and 'requireApproval', and then run the pipeline end-to-end on a document. It also shows how to stream the pipeline's execution and handle pauses.

```typescript
const ctx: MsgCtx = { logger: console };
const p = pipeline<MsgCtx, Message>(ctx).addStep(trim).addStep(requireApproval);

// Run end‑to‑end
p.run({ text: " hello " }).then((final) => console.log(final));

// Stream and handle pauses (event-based)
for await (const evt of p.stream({ text: " hello " })) {
  if (evt.type === "progress") {
    ctx.logger.info(`progress at step #${evt.step + 1}`);
  }
  if (evt.type === "pause") {
    ctx.logger.warn(`paused at step #${evt.step + 1}: ${evt.info.reason}`);
    // Optionally resume immediately
    await p.next(evt.resume.doc, evt.resume);
  }
}
```

--------------------------------

### Release and Publish Project using Bun and standard-version

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Creates a new release and publishes the project to the NPM registry using standard-version. Requires correctly configured .npmrc and .env files. Supports minor and patch releases.

```bash
# For a minor release
bun --env-file=.env release:minor

# For a patch release
bun --env-file=.env release:patch
```

--------------------------------

### Build Project using Bun

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Builds the project from source using Bun. This process includes cleaning the distribution directory, performing a strict TypeScript typecheck, emitting ESM output and declaration files with tsup, and regenerating the package.json exports map.

```bash
bun run build
```

--------------------------------

### Pipeline Factory Method

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Shows how to create a new pipeline instance by calling the `pipeline` factory function with a context object. This initializes a pipeline that can then have steps added to it.

```typescript
const myCtx = {
  logger: console,
  pipeline: {},
  state: { history: [] },
};

const p = pipeline(myCtx);
```

--------------------------------

### Create Sequential Steps with `withSequence` in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `withSequence` helper runs a series of identity pipeline steps sequentially. It short-circuits if any step pauses. An optional `stopCondition` can be provided either globally or per-context to halt the sequence early based on the document's state.

```typescript
type Ctx = Logger & { pipeline?: { stopCondition?: (doc: T) => boolean } };

const s1: PipelineStep<T, T, Ctx> = /* … */;
const s2: PipelineStep<T, T, Ctx> = /* … */;

const seq = withSequence<T, Ctx>([s1, s2]);

// Optional per‑context condition
ctx.pipeline = { stopCondition: d => d.ready === true };
```

--------------------------------

### Configure LLM Services via Environment Variables

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

This code block illustrates how to set up environment variables for configuring LLM services like OpenAI and Ollama. It includes essential variables such as API keys and endpoints, crucial for connecting to these services.

```env
# For OpenAI
OPENAI_API_KEY="sk-..."
OPENAI_ENDPOINT="https://api.openai.com"

# For Ollama
OLLAMA_ENDPOINT="http://localhost:11434"
```

--------------------------------

### Create Multi-Strategy Step - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Composes multiple steps (strategies) into a single step that attempts them in order. If an optional early stop condition is met, execution halts. This is useful for trying different approaches sequentially.

```typescript
import { withMultiStrategy } from "@jasonnathan/llm-core";

const multi = withMultiStrategy([
  stepA,
  stepB,
  stepC,
]);

const p = pipeline(ctx).addStep(multi);

```

--------------------------------

### Select First Successful Strategy with `withAlternatives` in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `withAlternatives` helper tries multiple pipeline steps on the same input and stops as soon as one produces an accepted output. Pauses from any step propagate immediately. Acceptance is determined by a provided `stopCondition` function.

```typescript
type Ctx = Logger & { pipeline?: { stopCondition?: (out: O) => boolean } };

const stratA: PipelineStep<I, O, Ctx> = /* … */;
const stratB: PipelineStep<I, O, Ctx> = /* … */;

// Stop when output has at least one chunk
const accept = (out: O) => (out as any).chunks?.length > 0;

const choose = withAlternatives<I, O, Ctx>([stratA, stratB], accept);
```

--------------------------------

### Run Pipeline Sequentially - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Executes all steps in the pipeline sequentially. Resolves with the final document state. If any step returns a 'pause' outcome (done: false), `run()` resolves immediately with the last successfully processed document.

```typescript
const finalDoc = await p.run(initialDoc);

```

--------------------------------

### Environment Variables for Ollama Service

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Sets up essential environment variables for the Ollama service to connect to the API endpoint. It includes the Ollama endpoint URL and an optional API key.

```env
OLLAMA_ENDPOINT="http://localhost:11434"
# OLLAMA_API_KEY="your-api-key" # Optional
```

--------------------------------

### Compare Swift Code Changes in HTML Table (Good Practice)

Source: https://github.com/pipewrk/llm-core/blob/main/src/tests/fixtures/md-doc.md

Illustrates how to present side-by-side comparisons of Swift code changes within an HTML table. Using markdown code blocks ensures that both 'before' and 'after' code snippets maintain their formatting and syntax highlighting.

```swift
struct Hello {
   public var test: String = "World" // original
}
```

```swift
struct Hello {
   public var test: String = "Universe" // changed
}
```

--------------------------------

### Listen to Pipeline Events with `eventsFromPipeline` in JavaScript/TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `eventsFromPipeline` function wraps a pipeline to provide an `EventEmitter` instance. This emitter allows listeners to track pipeline progress, pauses, completion, and errors through strongly-typed events, facilitating UI updates or monitoring.

```javascript
import { eventsFromPipeline } from "@jasonnathan/llm-core";

const emitter = eventsFromPipeline(p, initialDoc);emitter.on("progress", ({ step, doc }) =>
  console.log("step", step, doc)
);emitter.on("pause", ({ info }) => console.log("paused because", info.reason));emitter.on("done", () => console.log("finished"));
```

--------------------------------

### Add Step to Pipeline

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Illustrates how to append a new step to an existing pipeline using the `addStep` method. This method is chainable, allowing multiple steps to be added sequentially.

```typescript
p.addStep(trimStep)
 .addStep(validateStep)
 .addStep(transformStep);
```

--------------------------------

### Display JSON Response using HTML code Element (No Highlighting)

Source: https://github.com/pipewrk/llm-core/blob/main/src/tests/fixtures/md-doc.md

Demonstrates embedding a JSON code block using a simple HTML `<code>` tag within a table. This method is noted as ineffective for syntax highlighting, resulting in plain text presentation of the code.

```json
{
  "id": 10,
  "username": "marcoeidinger",
  "email": "alan@alan.com",
  "created_at": "2021-02-097T20:45:26.433Z",
  "updated_at": "2015-02-10T19:27:16.540Z"
}
```

--------------------------------

### Initialize Ollama Context in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Creates a context object for interacting with the Ollama API, specifying the model, endpoint, API key, pipeline policies (retries and timeout), and an optional logger. Defaults are used if environment variables are set.

```typescript
import { createOllamaContext } from '@jasonnathan/llm-core';

const ctx = createOllamaContext({
  model: 'llama3:8b-instruct-q8_0',
  // endpoint?: defaults to OLLAMA_ENDPOINT env
  // apiKey?: defaults to OLLAMA_API_KEY env (optional)
  pipeline: { retries: 2, timeout: 12_000 },
  // logger?: your ILogger
});
```

--------------------------------

### Advance Pipeline One Step - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Advances the pipeline by exactly one step. This method can also resume from a paused state. It returns a `StreamEvent` or a completion object, without requiring manual generator management, making it suitable for UI or CLI applications.

```typescript
const result = await p.next(initialDoc);

```

--------------------------------

### Convert Pipeline to Node.js Transform Stream

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

Converts a given pipeline (`p`) into a Node.js `Transform` stream. This stream processes each incoming object through the pipeline. It handles pauses by triggering an optional `onPause` handler and resuming from the correct step. Requires `@jasonnathan/llm-core`.

```typescript
import { pipelineToTransform } from "@jasonnathan/llm-core";
import { createReadStream, createWriteStream } from "fs";

const transform = pipelineToTransform(p, async (pause) => {
  if (pause.reason === "rateLimit") {
    await wait(pause.payload.wait);
  }
});

createReadStream("in.ndjson", { encoding: "utf8", objectMode: true })
  .pipe(transform)
  .pipe(createWriteStream("out.ndjson"));
```

--------------------------------

### Define Pipeline Resume State - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Defines the state required to resume a paused pipeline. It includes the index of the next step to execute and the document state at the point of pause.

```typescript
export type ResumeState<T> = { nextStep: number; doc: T };

```

--------------------------------

### Cache Pipeline Step Results in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `withCache` helper caches the results of successful pipeline steps (those that do not pause) using a provided key function. The cache is stored in `ctx.pipeline?.cache`. This optimizes performance by avoiding recomputation for identical inputs.

```typescript
type Ctx = Logger & { pipeline?: { cache?: Map<unknown, unknown> } };

const sExpensive: PipelineStep<T, T, Ctx> = (ctx) => async (doc) => {
  return await heavyCompute(doc);
};

const sCached = withCache(sExpensive, d => d.id);
// ctx.pipeline.cache = new Map()
```

--------------------------------

### PipelineStep Type Definition

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Defines the structure of a pipeline step, which is a curried function that takes a context and returns a transformation function on a document. Steps can optionally signal a pause.

```APIDOC
## PipelineStep Type

### Description
Defines a pipeline step as a curried function.

### Type Definition
```typescript
export type PipelineStep<I, O, C = unknown> =
  (ctx: C) =>
    (doc: I) =>
      O
      | PipelineOutcome<O>
      | Promise<O | PipelineOutcome<O>>;
```

### Parameters
- **I**: Input document type.
- **O**: Output document type.
- **C**: Context type (defaults to unknown).

### Usage
Steps can synchronously or asynchronously transform the document. To pause execution, return an object conforming to `PipelineOutcome<O>` with `done: false`.
```

--------------------------------

### Cosine Drop Chunker for Markdown Semantic Splitting

Source: https://github.com/pipewrk/llm-core/blob/main/README.md

Splits markdown text into semantically similar chunks using the cosineDropChunker. It requires context including embedding functions and logging. The chunking process is configurable via the 'type' and 'breakPercentile' options.

```typescript
import { cosineDropChunker, createOllamaContext, embedTexts } from "@jasonnathan/llm-core";

const svc = createOllamaContext({ model: "all-minilm:l6-v2" });
const embed = (texts: string[]) => embedTexts(svc, texts);
const ctx = { embed, logger: console, pipeline: { retries: 0, timeout: 0 } };

async function chunkMyMarkdown() {
  const markdown = "# Title\n\nThis is the first paragraph. A second paragraph discusses a new topic.";
  const chunks = await cosineDropChunker(ctx as any, markdown, { type: "markdown", breakPercentile: 95 });
  console.log(chunks);
}
```

--------------------------------

### Display JSON Response with HTML pre and lang Attribute

Source: https://github.com/pipewrk/llm-core/blob/main/src/tests/fixtures/md-doc.md

Shows how to use HTML `pre` tags with a `lang` attribute to define a JSON code block within a table. This approach is recommended for ensuring syntax highlighting and proper rendering of structured data in markdown environments.

```json
{
  "id": 10,
  "username": "alanpartridge",
  "email": "alan@alan.com",
  "password_hash": "$2a$10$uhUIUmVWVnrBWx9rrDWhS.CPCWCZsyqqa8./whhfzBZydX7yvahHS",
  "password_salt": "$2a$10$uhUIUmVWVnrBWx9rrDWhS.",
  "created_at": "2015-02-14T20:45:26.433Z",
  "updated_at": "2015-02-14T20:45:26.540Z"
}
```

--------------------------------

### Stream Pipeline Events - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Iterates through the pipeline step-by-step, yielding an event after each step. Events can be 'progress' (normal document), 'pause' (pipeline halted with pause outcome), or 'done' (pipeline finished). Useful for human-in-the-loop interactions, back-pressure management, or progress reporting.

```typescript
for await (const evt of p.stream(initialDoc)) {
  if (evt.type === "pause") {
    // handle evt.info, then resume…
  } else if (evt.type === "progress") {
    console.log(evt.doc);
  }
}

```

--------------------------------

### Compose Pipeline Steps with `pipeSteps` in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `pipeSteps` function provides a functional way to compose pipeline steps from left to right, similar to Unix pipes. It supports short-circuiting on pauses and integrates well with other helpers like `withErrorHandling` and `withTimeout`.

```typescript
const s = pipeSteps<T, Logger>(
  tap((ctx, d) => ctx.info?.('start')),
  withErrorHandling(someIdentityStep),
  withTimeout(anotherIdentityStep as PipelineStep<T, T, Logger>)
);
```

--------------------------------

### PipelineOutcome Type Definition

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Represents the outcome of a pipeline step, which can either signal a successful completion or a pause in execution.

```APIDOC
## PipelineOutcome Type

### Description
Signals a pause or an early completion from within a pipeline step.

### Type Definition
```typescript
export type PipelineOutcome<T> =
  | { done: false; reason: string; payload: T }  // pause, with a reason & document
  | { done: true;  value: T };                  // an early “done” that still produces a new doc
```

### Usage
- `{ done: false, reason: string, payload: T }`: Indicates a pause in the pipeline execution, providing a reason and the current document state.
- `{ done: true, value: T }`: Indicates an early successful completion of the pipeline, returning a final value.
```

--------------------------------

### generatePromptAndSend<T>()

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Primary method for sending requests to the Ollama chat API and receiving structured, typed JSON responses.

```APIDOC
## `generatePromptAndSend<T>()`

This is the primary method for sending a request to the Ollama chat API and receiving a structured JSON response.

### Description

Initiates a request to the Ollama API to generate content based on system and user prompts. It supports requesting structured JSON output by providing a JSON schema in the options. The function handles parsing, validation, and retries based on the provided context and custom validation logic.

### Method

POST (internally calls Ollama API)

### Endpoint

`/api/chat` (internally called by the helper)

### Parameters

#### Path Parameters

None

#### Query Parameters

None

#### Request Body

- **ctx** (OllamaContext) - Required - The Ollama context object containing model, endpoint, and pipeline policy information.
- **systemPrompt** (string) - Required - The system prompt defining the LLM's role and instructions.
- **userPrompt** (string) - Required - The user's query or the content to be processed.
- **options** (object) - Optional - Configuration options for the request.
  - **schema** (object) - Optional - A JSON schema defining the expected structure of the response. If provided, the service will attempt to parse and validate the LLM's output against this schema.
- **customCheck** (function) - Optional - A function `(response: T) => T | boolean` used to perform custom validation on the parsed response. If this function returns `false`, the request will be retried.

### Request Example

```typescript
import { createOllamaContext, generatePromptAndSend } from '@jasonnathan/llm-core';

interface UserProfile {
  name: string;
  email: string;
  age: number;
}

const ctx = createOllamaContext({ model: 'llama3:8b-instruct-q8_0' });

async function main() {
  const systemPrompt = "You are a data extraction expert. Generate a JSON object from the user's text.";
  const userPrompt = "Create a profile for John Doe. His email is john.doe@example.com and he is 30 years old.";

  const options = {
    schema: {
      type: "object",
      properties: {
        name: { type: "string" },
        email: { type: "string" },
        age: { type: "number" },
      },
      required: ["name", "email", "age"],
    },
  };

  try {
    const profile = await generatePromptAndSend<UserProfile>(
      ctx,
      systemPrompt,
      userPrompt,
      options
    );

    console.log(profile);
  } catch (error) {
    console.error("Failed to generate profile:", error);
  }
}

main();
```

### Response

#### Success Response (200)

- **T** (generic type) - The parsed and validated response, conforming to the generic type `T` provided, or the structure defined by the `schema` if provided.

#### Response Example

```json
{
  "name": "John Doe",
  "email": "john.doe@example.com",
  "age": 30
}
```
```

--------------------------------

### Define Pipeline Stream Events - TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Defines the possible events emitted during pipeline streaming. These include 'progress' for normal document flow, 'pause' for halted execution with details, and 'done' for pipeline completion. Each event contains relevant data like the step number, document state, and resume information.

```typescript
export type StreamEvent<T> = 
  | { type: 'progress'; step: number; doc: T; resume: ResumeState<T> } 
  | { type: 'pause';    step: number; doc: T; info: Extract<PipelineOutcome<T>, { done: false }>; resume: ResumeState<T> } 
  | { type: 'done' };

```

--------------------------------

### embedTexts()

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Generates vector embeddings for an array of text inputs using the specified Ollama model and context.

```APIDOC
## `embedTexts()`

This function generates vector embeddings for an array of text inputs using your context.

### Description

Takes an array of strings and uses the configured Ollama model (ideally one trained for embeddings) to generate corresponding vector representations (embeddings). It adheres to the pipeline policies defined in the context for retries and timeouts.

### Method

POST (internally calls Ollama API's embedding endpoint)

### Endpoint

`/api/embeddings` (internally called by the helper)

### Parameters

#### Path Parameters

None

#### Query Parameters

None

#### Request Body

- **ctx** (OllamaContext) - Required - The Ollama context object containing model and pipeline information.
- **inputs** (array of strings) - Required - An array of strings for which to generate embeddings.

### Request Example

```typescript
import { createOllamaContext, embedTexts } from '@jasonnathan/llm-core';

// Use a model specifically trained for embeddings
const ctx = createOllamaContext({ model: 'all-minilm:l6-v2' });

async function main() {
  const texts = [
    "This is the first sentence.",
    "This is the second one.",
  ];

  try {
    const embeddings = await embedTexts(ctx, texts);
    console.log("Generated embeddings:", embeddings.length);
    console.log("Dimension:", embeddings[0].length);
  } catch (error) {
    console.error("Failed to generate embeddings:", error);
  }
}

main();
```

### Response

#### Success Response (200)

- **embeddings** (array of arrays of numbers) - An array where each inner array represents the vector embedding for the corresponding input text.

#### Response Example

```json
[
  [0.123, -0.456, ...],
  [-0.789, 0.101, ...]
]
```
```

--------------------------------

### Typed JSON Responses via LLM

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

This section details how the Ollama service generates typed JSON responses from LLM interactions, including a sequence diagram illustrating the flow and error handling with retries.

```APIDOC
## Typed JSON Responses via LLM Visual Flow

This endpoint facilitates the generation of typed JSON responses by interacting with the Ollama API.

### Description

The service receives a prompt and a schema, sends a request to the Ollama API's `/api/chat` endpoint with `format: 'json'`, parses the raw JSON response, and validates it against the provided schema. If validation fails, it retries the request based on the context's pipeline policy.

### Method

POST

### Endpoint

`/api/chat`

### Parameters

#### Query Parameters

- **format** (string) - Required - Specifies the desired output format, e.g., `'json'`.

#### Request Body

- **model** (string) - Required - The Ollama model to use.
- **messages** (array of objects) - Required - The conversation history, e.g., `[{ role: 'system', content: '...' }, { role: 'user', content: '...' }]`.
- **stream** (boolean) - Optional - Whether to stream the response.
- **options** (object) - Optional - Additional options for the generation, including `schema` for structured output.
  - **schema** (object) - Optional - A JSON schema to enforce the structure of the generated JSON response.

### Request Example

```json
{
  "model": "llama3:8b-instruct-q8_0",
  "messages": [
    {
      "role": "system",
      "content": "You are a data extraction expert. Generate a JSON object from the user's text."
    },
    {
      "role": "user",
      "content": "Create a profile for John Doe. His email is john.doe@example.com and he is 30 years old."
    }
  ],
  "options": {
    "format": "json",
    "schema": {
      "type": "object",
      "properties": {
        "name": {"type": "string"},
        "email": {"type": "string"},
        "age": {"type": "number"}
      },
      "required": ["name", "email", "age"]
    }
  }
}
```

### Response

#### Success Response (200)

- **content** (object) - The parsed and validated JSON response adhering to the provided schema.

#### Response Example

```json
{
  "name": "John Doe",
  "email": "john.doe@example.com",
  "age": 30
}
```
```

--------------------------------

### Implement Retries for Pipeline Steps in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `withRetry` helper allows a pipeline step to be retried if it results in a pause with the reason 'error'. The number of retries is determined by `ctx.pipeline?.retries`. This is crucial for handling transient failures.

```typescript
type Ctx = Logger & { pipeline?: { retries?: number } };

const sFlaky: PipelineStep<T, T, Ctx> = (ctx) => async (doc) => {
  // may pause with reason:'error'
  return maybeFlaky(doc);
};

const sRetried = withRetry(sFlaky);
// set per-run:
const ctx: Ctx = Object.assign(new Logger('./log.md'), { pipeline: { retries: 2 } });
```

--------------------------------

### Display JSON HTTP Response in HTML Table (Good Practice)

Source: https://github.com/pipewrk/llm-core/blob/main/src/tests/fixtures/md-doc.md

Demonstrates embedding a JSON code block within an HTML table to represent an HTTP response. This method ensures proper syntax highlighting and layout on platforms like GitHub. It relies on markdown's ability to interpret code blocks within table cells.

```json
{
  "id": 10,
  "username": "marcoeidinger",
  "created_at": "2021-02-097T20:45:26.433Z",
  "updated_at": "2015-02-10T19:27:16.540Z"
}
```

--------------------------------

### Wrap Step with Error Handling in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `withErrorHandling` helper wraps a pipeline step to catch any thrown errors. Instead of crashing, it returns a pause object `{ done: false, reason: 'error' }`. This is useful for making steps more resilient.

```typescript
const sFetch: PipelineStep<Input, Output, Logger> = (ctx) => async (doc) => {
  // may throw
  return fetchAndParse(doc);
};

const sSafe = withErrorHandling(sFetch);
// type: PipelineStep<Input, Output, Logger>
```

--------------------------------

### PipelineOutcome Type Definition

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Defines the structure for PipelineOutcome, used to signal a pause or an early completion within a pipeline step. It can either indicate a pause with a reason and payload, or an early 'done' state with a final value.

```typescript
export type PipelineOutcome<T> =
  | { done: false; reason: string; payload: T }  // pause, with a reason & document
  | { done: true;  value: T };                  // an early “done” that still produces a new doc
```

--------------------------------

### Generate Text Embeddings with Ollama in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/OLLAMA_SERVICE.md

Illustrates how to use the `embedTexts` function to generate vector embeddings for an array of strings using a specified Ollama model optimized for embeddings. Includes error handling for the embedding process.

```typescript
import { createOllamaContext, embedTexts } from '@jasonnathan/llm-core';

// Use a model specifically trained for embeddings
const ctx = createOllamaContext({ model: 'all-minilm:l6-v2' });

async function main() {
  const texts = [
    "This is the first sentence.",
    "This is the second one.",
  ];

  try {
    const embeddings = await embedTexts(ctx, texts);
    console.log("Generated embeddings:", embeddings.length); // Output: 2
    console.log("Dimension:", embeddings[0].length);
  } catch (error) {
    console.error("Failed to generate embeddings:", error);
  }
}

main();
```

--------------------------------

### Define Pipeline Step Type

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE.md

Defines the type for a pipeline step, which is a curried function. It takes a context and returns a transformation function that operates on a document. This function can return the transformed document, a PipelineOutcome to pause execution, or a Promise for asynchronous operations.

```typescript
export type PipelineStep<I, O, C = unknown> =
  (ctx: C) =>
    (doc: I) =>
      O
      | PipelineOutcome<O>
      | Promise<O | PipelineOutcome<O>>;
```

--------------------------------

### Implement Side-Effecting Tap in TypeScript

Source: https://github.com/pipewrk/llm-core/blob/main/PIPELINE_HELPERS.md

The `tap` helper allows for executing a side-effecting function during pipeline execution without altering the document. It forwards the document unchanged, making it useful for logging or debugging.

```typescript
const sTap = tap<T, Logger>((ctx, doc) => ctx.info?.(`seen ${doc.id}`));
// type: PipelineStep<T, T, Logger>
```