### Install and Start WebLLM Example

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/cache-usage/README.md

Installs dependencies and starts the WebLLM example application. This is a common setup step for running the provided examples.

```bash
npm install
npm start
```

--------------------------------

### Run Example Applications

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Commands to install dependencies and start a specific example application.

```bash
cd examples/<example>
npm install
npm run start
```

--------------------------------

### Test Local Code Changes

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/developer/building_from_source.rst

To test your modifications, navigate to an example's directory, update its package.json to reference your local build, and then install and start the example.

```bash
cd examples/<example>
# Modify package.json as described
npm install
npm start
```

--------------------------------

### Install and Start WebLLM Demo

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/abort-reload/README.md

Use these commands to install dependencies and launch the development server for the demo application.

```bash
npm install
npm start
```

--------------------------------

### Clone and Install WebLLM

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Initial setup commands to clone the repository and install dependencies.

```bash
git clone https://github.com/mlc-ai/web-llm.git
cd web-llm
npm install
```

--------------------------------

### Clean and Reinstall for Local Testing

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Use this command to perform a clean installation and start the example when local changes are not detected.

```bash
cd examples/get-started
rm -rf node_modules dist package-lock.json .parcel-cache
npm install
npm start
```

--------------------------------

### Start Demo Application

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/multi-round-chat/README.md

Execute this command after installing dependencies to launch the WebLLM demo application.

```bash
npm start
```

--------------------------------

### Serve documentation locally

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/README.md

Navigate to the build directory and start a local HTTP server to view the documentation.

```bash
cd _build/html
python3 -m http.server
```

--------------------------------

### Install documentation dependencies

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/README.md

Install the required Python packages listed in requirements.txt.

```bash
pip3 install -r requirements.txt
```

--------------------------------

### Install and Build WebLLM Package

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Run these commands to install dependencies and build the WebLLM package from source.

```bash
npm install
npm run build
```

--------------------------------

### Build Documentation

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Install documentation dependencies and generate the HTML site using Sphinx.

```bash
cd docs
pip3 install -r requirements.txt
make html
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/developer/building_from_source.rst

Run this command after cloning the repository to install all necessary Node.js dependencies.

```bash
npm install
```

--------------------------------

### Build the Web-LLM Project

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/developer/building_from_source.rst

Execute this command to compile and build the project after installing dependencies.

```bash
npm run build
```

--------------------------------

### Run the development server

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/next-simple-chat/README.md

Commands to start the local development environment using various package managers.

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
```

--------------------------------

### Link Local WebLLM Package in Example

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Modify the example's package.json to link to the local WebLLM package for testing changes.

```bash
cd examples/get-started
npm install
npm start
```

--------------------------------

### Verify WebLLM Installation

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/get_started.rst

Run this script after installation to confirm that WebLLM has been loaded successfully into your project.

```javascript
import { CreateMLCEngine } from "@mlc-ai/web-llm";
console.log("WebLLM loaded successfully!");
```

--------------------------------

### Install WebLLM using npm, yarn, or pnpm

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/get_started.rst

Use your preferred package manager to install the WebLLM package. This is the standard method for Node.js projects.

```bash
# npm
npm install @mlc-ai/web-llm
# yarn
yarn add @mlc-ai/web-llm
# pnpm
pnpm install @mlc-ai/web-llm
```

--------------------------------

### Build the WebLLM Chrome Extension

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/chrome-extension/README.md

Install dependencies and compile the extension source code into the dist directory.

```bash
npm install
npm run build
```

--------------------------------

### Web Worker Handler Setup

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Set up a Web Worker script to handle heavy computation. This script imports and instantiates `WebWorkerMLCEngineHandler` to manage cross-thread communication and process requests.

```typescript
// worker.ts
import { WebWorkerMLCEngineHandler } from "@mlc-ai/web-llm";

const handler = new WebWorkerMLCEngineHandler();
self.onmessage = (msg: MessageEvent) => {
    handler.onmessage(msg);
};
```

--------------------------------

### Configure Cross-Origin Storage Cache for WebLLM

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Enable the 'cross-origin' cache backend to share model artifacts across different origins, provided the Cross-Origin Storage browser extension is installed. Falls back to default cache if unavailable.

```typescript
import { AppConfig, CreateMLCEngine, prebuiltAppConfig } from "@mlc-ai/web-llm";

const appConfig: AppConfig = {
    ...prebuiltAppConfig,
    cacheBackend: "cross-origin",
};

const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct-q4f32_1-MLC", {
    appConfig,
});
```

--------------------------------

### Service Worker Handler Setup

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

This code initializes a `ServiceWorkerMLCEngineHandler` within a Service Worker. The handler is created upon activation of the service worker to manage WebLLM operations.

```typescript
// sw.ts
import { ServiceWorkerMLCEngineHandler } from "@mlc-ai/web-llm";

let handler: ServiceWorkerMLCEngineHandler;

self.addEventListener("activate", function (event) {
  handler = new ServiceWorkerMLCEngineHandler();
  console.log("Service Worker is ready");
});
```

--------------------------------

### Service Worker Handler Setup

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Configure a Service Worker script to handle computation, enabling model persistence across page refreshes. This involves importing and instantiating `ServiceWorkerMLCEngineHandler`.

```typescript
// sw.ts
import { ServiceWorkerMLCEngineHandler } from "@mlc-ai/web-llm";

self.addEventListener("activate", () => {
    const handler = new ServiceWorkerMLCEngineHandler();
    console.log("Service Worker activated!");
});
```

--------------------------------

### Get GPU Vendor Information

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Retrieves the vendor name of the GPU used for computations. Useful for understanding hardware capabilities.

```typescript
const gpuVendor = await engine.getGPUVendor();
console.log(`GPU Vendor: ${gpuVendor}`);
```

--------------------------------

### Get Maximum Storage Buffer Binding Size

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Returns the maximum storage buffer size supported by the GPU in bytes. Important for models requiring significant memory.

```typescript
const maxBufferSize = await engine.getMaxStorageBufferBindingSize();
console.log(`Max Storage Buffer Binding Size: ${maxBufferSize}`);
```

--------------------------------

### Build documentation with Make

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/README.md

Generate the HTML documentation files using the provided Makefile.

```bash
make html
```

--------------------------------

### Build, Lint, and Test

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Standard commands for building the project, running linting checks, and executing the test suite.

```bash
npm run build
npm run lint
npm test
```

--------------------------------

### Build WebLLM Package

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Run this command to build the WebLLM package after preparing dependencies.

```bash
npm run build
```

--------------------------------

### Prepare TVMjs Dependencies

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Execute this script to set up the necessary dependencies for building TVMjs.

```shell
./scripts/prep_deps.sh
```

--------------------------------

### Initialize MLCEngine

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/basic_usage.rst

Demonstrates both the factory function approach and direct class instantiation for setting up the engine.

```typescript
import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm";

    // Initialize with a progress callback
    const initProgressCallback = (progress) => {
        console.log("Model loading progress:", progress);
    };

   // Using CreateMLCEngine
   const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct", { initProgressCallback });

   // Direct instantiation
   const engineInstance = new MLCEngine({ initProgressCallback });
   await engineInstance.reload("Llama-3.1-8B-Instruct");
```

--------------------------------

### Create MLCEngine with Progress Callback

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Create an MLCEngine instance and load a model, providing a callback to track loading progress. This is the recommended way to initialize the engine.

```typescript
import { CreateMLCEngine } from "@mlc-ai/web-llm";

// Callback function to update model loading progress
const initProgressCallback = (initProgress) => {
  console.log(initProgress);
};
const selectedModel = "Llama-3.1-8B-Instruct-q4f32_1-MLC";

const engine = await CreateMLCEngine(
  selectedModel,
  { initProgressCallback: initProgressCallback }, // engineConfig
);
```

--------------------------------

### Separate Engine Initialization and Model Loading

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/basic_usage.rst

Shows how to instantiate the engine synchronously and load the model asynchronously as separate steps.

```typescript
import { MLCEngine } from "@mlc-ai/web-llm";

    // This is a synchronous call that returns immediately
    const engine = new MLCEngine({
        initProgressCallback: initProgressCallback
    });

    // This is an asynchronous call and can take a long time to finish
    await engine.reload(selectedModel);
```

--------------------------------

### Initialize MLCEngine

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Create an instance of the MLCEngine with optional configuration for app settings and progress tracking.

```typescript
const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct", {
    appConfig: { /* app-specific config */ },
    initProgressCallback: (progress) => console.log(progress),
});
```

--------------------------------

### CreateMLCEngine

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Initializes the MLCEngine with a specific model and optional configuration.

```APIDOC
## CreateMLCEngine

### Description
Initializes the MLCEngine instance for model operations.

### Parameters
#### Request Body
- **modelId** (string) - Required - The ID of the model to load.
- **config** (MLCEngineConfig) - Optional - Configuration object including appConfig, initProgressCallback, and logitProcessorRegistry.

### Request Example
```typescript
const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct", {
    appConfig: { /* app-specific config */ },
    initProgressCallback: (progress) => console.log(progress),
});
```
```

--------------------------------

### Clone Web-LLM Repository

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/developer/building_from_source.rst

Use this command to clone the official Web-LLM repository to your local machine.

```bash
git clone https://github.com/mlc-ai/web-llm.git
cd web-llm
```

--------------------------------

### Configure Cache Backend for MLCEngine

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Create an MLCEngine using a specific cache backend, such as 'cross-origin'. Ensure the necessary extensions or support are available for the chosen backend.

```typescript
import { CreateMLCEngine, prebuiltAppConfig } from "@mlc-ai/web-llm";

const appConfig = { ...prebuiltAppConfig, cacheBackend: "cross-origin" };
const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct-q4f32_1-MLC", {
  appConfig,
});
```

--------------------------------

### Import WebLLM using CDN

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/get_started.rst

Import WebLLM directly via a CDN URL. This method is useful for online IDEs and quick local experiments without a package manager.

```javascript
import * as webllm from "https://esm.run/@mlc-ai/web-llm";
```

--------------------------------

### Format Code

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Run the project's auto-formatting tools to resolve lint or style failures.

```bash
npm run format
```

--------------------------------

### Utility and GPU Methods

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Methods for managing chat state and retrieving GPU hardware information.

```APIDOC
## MLCEngine.getMessage

### Description
Retrieves the current output message from the specified model.

## MLCEngine.resetChat

### Description
Resets the chat history and optionally retains usage statistics.

## MLCEngine.getGPUVendor

### Description
Retrieves the vendor name of the GPU used for computations.

### Response
- **Returns** (string) - The GPU vendor name (e.g., "Intel", "NVIDIA").

## MLCEngine.getMaxStorageBufferBindingSize

### Description
Returns the maximum storage buffer size supported by the GPU.

### Response
- **Returns** (number) - The maximum size in bytes.
```

--------------------------------

### Dynamically Import WebLLM via CDN

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Dynamically import WebLLM using a CDN URL, useful for asynchronous loading.

```javascript
const webllm = await import("https://esm.run/@mlc-ai/web-llm");
```

--------------------------------

### Initialize MLCEngine Separately

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Initialize the MLCEngine synchronously and then reload the model asynchronously. This allows for more control over the initialization process.

```typescript
import { MLCEngine } from "@mlc-ai/web-llm";

// This is a synchronous call that returns immediately
const engine = new MLCEngine({
  initProgressCallback: initProgressCallback,
});

// This is an asynchronous call and can take a long time to finish
await engine.reload(selectedModel);
```

--------------------------------

### Create Web Worker Engine in Main Thread

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

In the main application thread, create a `WebWorkerMLCEngine` to offload heavy computations to a worker. This engine implements the `MLCEngineInterface` and communicates with the worker handler.

```typescript
// main.ts
import { CreateWebWorkerMLCEngine } from "@mlc-ai/web-llm";

async function main() {
  // Use a WebWorkerMLCEngine instead of MLCEngine here
  const engine = await CreateWebWorkerMLCEngine(
    new Worker(new URL("./worker.ts", import.meta.url), {
      type: "module",
    }),
    selectedModel,
    { initProgressCallback }, // engineConfig
  );

  // everything else remains the same
}
```

--------------------------------

### Load Custom Model with Overrides

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

This TypeScript snippet demonstrates how to load a custom model using `CreateMLCEngine`. It shows how to specify custom model artifacts (weights and WASM library) via `appConfig` and override default chat options.

```typescript
import { CreateMLCEngine } from "@mlc-ai/web-llm";

async main() {
  const appConfig = {
    "model_list": [
      {
        "model": "/url/to/my/llama",
        "model_id": "MyLlama-3b-v1-q4f32_0",
        "model_lib": "/url/to/myllama3b.wasm",
      }
    ],
  };
  // override default
  const chatOpts = {
    "repetition_penalty": 1.01
  };

  // load a prebuilt model
  // with a chat option override and app config
  // under the hood, it will load the model from myLlamaUrl
  // and cache it in the browser cache
  // The chat will also load the model library from "/url/to/myllama3b.wasm",
  // assuming that it is compatible to the model in myLlamaUrl.
  const engine = await CreateMLCEngine(
    "MyLlama-3b-v1-q4f32_0",
    { appConfig }, // engineConfig
    chatOpts,
  );
}
```

--------------------------------

### Configure WebLLM with Model Integrity Verification

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Set up the WebLLM engine with model configuration, including SRI hashes for integrity verification of model artifacts. The `onFailure` option can be set to 'error' or 'warn'.

```typescript
import { CreateMLCEngine } from "@mlc-ai/web-llm";

const appConfig = {
  model_list: [
    {
      model: "https://huggingface.co/mlc-ai/Llama-3.2-1B-Instruct-q4f16_1-MLC",
      model_id: "Llama-3.2-1B-Instruct-q4f16_1-MLC",
      model_lib:
        "https://raw.githubusercontent.com/user/model-libs/main/model.wasm",
      integrity: {
        config: "sha256-<base64-hash-of-mlc-chat-config.json>",
        model_lib: "sha256-<base64-hash-of-wasm-file>",
        tokenizer: {
          "tokenizer.json": "sha256-<base64-hash-of-tokenizer.json>",
        },
        onFailure: "error", // "error" (default) throws IntegrityError, "warn" logs and continues
      },
    },
  ],
};

const engine = await CreateMLCEngine("Llama-3.2-1B-Instruct-q4f16_1-MLC", {
  appConfig,
});
```

--------------------------------

### Node.js Script for File Copying (Windows)

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat-upload/README.md

This Node.js script is used on Windows to copy configuration files, ensuring cross-platform compatibility for the application.

```javascript
const fs = require("fs");
// Copy file
fs.copyFileSync("src/gh-config.js", "src/app-config.js");
```

--------------------------------

### Web Worker Engine Initialization and Usage

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Initialize and use a `WebWorkerMLCEngine` in the main script to offload computation to a worker thread. This engine acts as a proxy, forwarding calls to the worker for processing.

```typescript
import { CreateWebWorkerMLCEngine } from "@mlc-ai/web-llm";

async function runWorker() {
    const engine = await CreateWebWorkerMLCEngine(
        new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }),
        "Llama-3.1-8B-Instruct"
    );

    const messages = [{ role: "user", content: "How does WebLLM use workers?" }];
    const reply = await engine.chat.completions.create({ messages });
    console.log(reply.choices[0].message.content);
}

runWorker();
```

--------------------------------

### Create Service Worker Engine in Main Thread

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

In the main application, register the service worker and then create a `ServiceWorkerMLCEngine` using `CreateServiceWorkerMLCEngine`. This allows WebLLM to run in the background, persisting the model across page visits.

```typescript
// main.ts
import {
  MLCEngineInterface,
  CreateServiceWorkerMLCEngine,
} from "@mlc-ai/web-llm";

if ("serviceWorker" in navigator) {
  navigator.serviceWorker.register(
    new URL("sw.ts", import.meta.url), // worker script
    { type: "module" },
  );
}

const engine: MLCEngineInterface = await CreateServiceWorkerMLCEngine(
  selectedModel,
  { initProgressCallback }, // engineConfig
);
```

--------------------------------

### Import WebLLM into a project (all or specific)

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/get_started.rst

Import the WebLLM library into your JavaScript project. You can import all modules or specific functions like CreateMLCEngine.

```javascript
// Import everything
import * as webllm from "@mlc-ai/web-llm";
```

```javascript
// Or only import what you need
import { CreateMLCEngine } from "@mlc-ai/web-llm";
```

--------------------------------

### MLCEngine.reload

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Reloads the engine with a new model and optional chat configuration overrides.

```APIDOC
## MLCEngine.reload

### Description
Reloads the model and applies specific chat configuration overrides.

### Parameters
#### Request Body
- **modelId** (string) - Required - The ID of the model to reload.
- **chatOpts** (Partial<ChatConfig>) - Optional - Overrides for baseline model configuration.

### Request Example
```typescript
await engine.reload("Llama-3.1-8B-Instruct", {
    temperature: 0.7,
    repetition_penalty: 1.1,
    context_window_size: 4096,
});
```
```

--------------------------------

### Load Multiple Models with Different Chat Options

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Loads specified models into the engine, allowing for different chat configurations per model. Uses MLCEngineConfig during initialization.

```typescript
await engine.reload(["Llama-3.1-8B", "Gemma-2B"], [
    { temperature: 0.7 },
    { top_p: 0.9 },
]);
```

--------------------------------

### Model Management Methods

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Methods for loading and unloading models within the MLCEngine.

```APIDOC
## MLCEngine.reload

### Description
Loads the specified model(s) into the engine using MLCEngineConfig.

### Parameters
- **modelId** (string | string[]) - Required - Identifier(s) for the model(s) to load.
- **chatOpts** (ChatOptions | ChatOptions[]) - Optional - Configuration for generation.

### Request Example
await engine.reload(["Llama-3.1-8B", "Gemma-2B"], [{ temperature: 0.7 }, { top_p: 0.9 }]);

## MLCEngine.unload

### Description
Unloads all loaded models and clears their associated configurations.

### Request Example
await engine.unload();
```

--------------------------------

### Service Worker Engine Initialization

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Instantiate a `ServiceWorkerMLCEngine` in the main page script to utilize the service worker for computation. This engine acts as a proxy, forwarding requests to the service worker thread.

```typescript
// main.ts
import { MLCEngineInterface, CreateServiceWorkerMLCEngine } from "@mlc-ai/web-llm";

if ("serviceWorker" in navigator) {
    navigator.serviceWorker.register(
        new URL("sw.ts", import.meta.url),  // worker script
        { type: "module" },
    );
}

const engine: MLCEngineInterface =
    await CreateServiceWorkerMLCEngine(
        selectedModel,
        { initProgressCallback }, // engineConfig
    );
```

--------------------------------

### Configure Local WebLLM Core Package

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/embeddings/README.md

Modify the WebLLM dependencies to point to a local file path for development. This is recommended only when hacking the WebLLM core package.

```bash
"file:../.."
```

--------------------------------

### Clone TVMjs Repository

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Clone the TVMjs repository if the TVM_SOURCE_DIR environment variable is not defined.

```shell
git clone https://github.com/mlc-ai/relax 3rdparty/tvm-unity --recursive
```

--------------------------------

### Update package.json Scripts for Windows

Source: https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat-upload/README.md

Modify the scripts section in package.json to use the Node.js script for file copying on Windows systems.

```json
"scripts": {
    "start": "node copy-config.js && parcel src/llm_chat.html --port 8888",
    "mlc-local": "node copy-config.js && parcel src/llm_chat.html --port 8888",
    "build": "node copy-config.js && parcel build src/llm_chat.html --dist-dir lib --no-content-hash"
},
```

--------------------------------

### Perform Chat Completion

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/basic_usage.rst

Uses the OpenAI-style chat API to generate responses from the initialized engine.

```typescript
const messages = [
        { role: "system", content: "You are a helpful AI assistant." },
        { role: "user", content: "Hello!" }
    ];

    const reply = await engine.chat.completions.create({
        messages,
    });

    console.log(reply.choices[0].message);
    console.log(reply.usage);
```

--------------------------------

### POST /chat/completions

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Generates chat-based completions using a specified request configuration.

```APIDOC
## POST /chat/completions

### Description
Generates chat-based completions using a specified request configuration.

### Parameters
- **request** (ChatCompletionRequest) - Required - A ChatCompletionRequest instance containing messages, stream settings, and logit_bias.

### Request Example
{
  "messages": [
    { "role": "system", "content": "You are a helpful AI assistant." },
    { "role": "user", "content": "What is WebLLM?" }
  ],
  "temperature": 0.8,
  "stream": false
}
```

--------------------------------

### Generate SRI Hashes using OpenSSL

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Commands to generate SHA-256, SHA-384, and SHA-512 hashes for files, which can then be used for Subresource Integrity verification. These commands require a Unix-like shell.

```bash
# SHA-256
openssl dgst -sha256 -binary <file> | openssl base64 -A | sed 's/^/sha256-/'
# SHA-384
openssl dgst -sha384 -binary <file> | openssl base64 -A | sed 's/^/sha384-/'
# SHA-512
openssl dgst -sha512 -binary <file> | openssl base64 -A | sed 's/^/sha512-/'
```

--------------------------------

### Chat Completion with Streaming

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Initiates a chat completion request and enables streaming of responses. Useful for real-time conversational agents.

```typescript
const response = await engine.chatCompletion({
    messages: [
        { role: "user", content: "Tell me about WebLLM." },
    ],
    stream: true,
});
```

--------------------------------

### Configure IndexedDB Cache for WebLLM

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Set the cache backend to 'indexeddb' to use IndexedDB for storing model artifacts. This ensures faster subsequent model loads.

```typescript
import { AppConfig, CreateMLCEngine, prebuiltAppConfig } from "@mlc-ai/web-llm";

const appConfig: AppConfig = {
    ...prebuiltAppConfig,
    cacheBackend: "indexeddb",
};

const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct-q4f32_1-MLC", {
    appConfig,
});
```

--------------------------------

### Perform Chat Completion

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Execute a chat completion request using specific generation parameters like temperature and top_p.

```typescript
const messages = [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Explain WebLLM." },
];

const response = await engine.chatCompletion({
    messages,
    top_p: 0.9,
    temperature: 0.8,
    max_tokens: 150,
});
```

--------------------------------

### Perform Chat Completion

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Invoke chat completions using the engine's chat.completions.create method with a list of messages. The 'model' parameter is ignored here.

```typescript
const messages = [
  { role: "system", content: "You are a helpful AI assistant." },
  { role: "user", content: "Hello!" },
];

const reply = await engine.chat.completions.create({
  messages,
});
console.log(reply.choices[0].message);
console.log(reply.usage);
```

--------------------------------

### MLCEngine.chatCompletion

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Performs a chat completion task using the provided messages and generation configuration.

```APIDOC
## MLCEngine.chatCompletion

### Description
Generates a chat response based on the provided messages and generation settings.

### Parameters
#### Request Body
- **messages** (Array) - Required - List of message objects with role and content.
- **GenerationConfig** (Object) - Optional - Includes top_p, temperature, max_tokens, and other sampling parameters.

### Request Example
```typescript
const response = await engine.chatCompletion({
    messages: [{ role: "user", content: "Explain WebLLM." }],
    top_p: 0.9,
    temperature: 0.8,
    max_tokens: 150,
});
```
```

--------------------------------

### Run Individual Tests

Source: https://github.com/mlc-ai/web-llm/blob/main/CONTRIBUTING.md

Execute a specific test file using Jest without coverage reporting.

```bash
npx jest --coverage=false tests/<file>.test.ts
```

--------------------------------

### Modify WebLLM Core Package Dependency

Source: https://github.com/mlc-ai/web-llm/blob/main/tests/scripts/sanity_checks/README.md

Change the WebLLM dependency to "file:../.." to use a local build. This is recommended only when hacking the WebLLM core package.

```bash
npm install -D "@webgpu/types"
cd ../..
npm run build
cd ../sanity-checks
npm install
```

--------------------------------

### Create Chat Completion (Non-Streaming)

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Generates a chat-based completion using a request configuration. Set stream to false for a single response.

```typescript
const response = await engine.chat.completions.create({
    messages: [
        { role: "system", content: "You are a helpful AI assistant." },
        { role: "user", content: "What is WebLLM?" },
    ],
    temperature: 0.8,
    stream: false,
});
```

--------------------------------

### Import WebLLM Module

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Import the WebLLM module into your TypeScript code. You can import everything or specific functions.

```typescript
// Import everything
import * as webllm from "@mlc-ai/web-llm";
// Or only import what you need
import { CreateMLCEngine } from "@mlc-ai/web-llm";
```

--------------------------------

### Unload All Models

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Clears all loaded models and their configurations from the engine. Use this to free up resources.

```typescript
await engine.unload();
```

--------------------------------

### Reload Model Configuration

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/api_reference.rst

Update the model configuration by reloading with specific chat options that override defaults.

```typescript
await engine.reload("Llama-3.1-8B-Instruct", {
    temperature: 0.7,
    repetition_penalty: 1.1,
    context_window_size: 4096,
});
```

--------------------------------

### Perform Streaming Chat Completion

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/basic_usage.rst

Enables streaming responses by setting the stream parameter to true and iterating over the resulting AsyncGenerator.

```typescript
const messages = [
        { role: "system", content: "You are a helpful AI assistant." },
        { role: "user", content: "Hello!" },
    ]

    // chunks is an AsyncGenerator object
    const chunks = await engine.chat.completions.create({
        messages,
        temperature: 1,
        stream: true, // <-- Enable streaming
        stream_options: { include_usage: true },
    });

    let reply = "";
    for await (const chunk of chunks) {
        reply += chunk.choices[0]?.delta.content || "";
        console.log(reply);
        if (chunk.usage) {
            console.log(chunk.usage); // only last chunk has usage
        }
    }

    const fullReply = await engine.getMessage();
    console.log(fullReply);
```

--------------------------------

### WebLLM Project Citation

Source: https://github.com/mlc-ai/web-llm/blob/main/README.md

Use this citation when referencing the WebLLM project in academic or research work. It includes authors, title, year, and publication details.

```bibtex
@misc{ruan2026webllmhighperformanceinbrowserllm,
      title={WebLLM: A High-Performance In-Browser LLM Inference Engine},
      author={Charlie F. Ruan and Yucheng Qin and Akaash R. Parthasarathy and Xun Zhou and Ruihang Lai and Hongyi Jin and Yixin Dong and Bohan Hou and Meng-Shiun Yu and Yiyan Zhai and Sudeep Agarwal and Hangrui Cao and Siyuan Feng and Tianqi Chen},
      year={2026},
      eprint={2412.15803},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2412.15803},
}
```

--------------------------------

### Rerun GPU Tests

Source: https://github.com/mlc-ai/web-llm/blob/main/tests/scripts/sanity_checks/sanity_checks.html

Reloads the current module to rerun tests. This is useful for iterative development and debugging.

```javascript
import "./sanity_checks.ts";
document.getElementById("run-tests").onclick = () => {
  // Reload the module to rerun tests
  window.location.reload();
};
```

--------------------------------

### Handle Video Playback and Intersection Observation with JavaScript

Source: https://github.com/mlc-ai/web-llm/blob/main/site/_includes/hero.html

This JavaScript code snippet handles video playback, pausing it when it's out of view and resuming when it comes back into view using the Intersection Observer API. It requires jQuery for DOM manipulation.

```javascript
(function() { function handlerIn(e) { $(this).addClass("expanded"); } function handlerOut(e) { $(this).removeClass("expanded"); } $(".chat-link").hover(handlerIn, handlerOut); $(".github-link").hover(handlerIn, handlerOut); var video = $("video")[0]; video.play().then((_) => { let observer = new IntersectionObserver( (entries) => { entries.forEach((entry) => { if ( entry.intersectionRatio !== 1 && !video.paused ) { video.pause(); } else if (video.paused) { video.play(); } }); }, { threshold: 0.2 } ); observer.observe(video); }); })()
```

--------------------------------

### Customize Token Generation with Logit Bias

Source: https://github.com/mlc-ai/web-llm/blob/main/docs/user/advanced_usage.rst

Control token likelihood during generation by modifying 'logit_bias' in GenerationConfig. Use large negative values to prevent specific tokens from being generated.

```typescript
const messages = [
    { role: "user", content: "Describe WebLLM in detail." },
];

const response = await engine.chatCompletion({
    messages,
    logit_bias: { "50256": -100 }, // Example: Prevent specific token generation
});
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.