### Prefill Assistant Response Start for Custom Formatting Source: https://context7.com/webmachinelearning/prompt-api/llms.txt This code demonstrates how to guide the language model's response format by prefilling the beginning of the assistant's message. Examples include forcing TOML, JSON, or controlling the conversational style with introductory phrases. ```javascript const session = await LanguageModel.create(); // Force TOML output format const characterSheet = await session.prompt([ { role: "user", content: "Create a character sheet for a gnome barbarian" }, { role: "assistant", content: "```toml\n", prefix: true } ]); console.log(characterSheet); // Continues from the prefix: name = "Grimbold"... // Force JSON format without schema const jsonData = await session.prompt([ { role: "user", content: "List three programming languages" }, { role: "assistant", content: '{"languages": [', prefix: true } ]); console.log(jsonData); // "Python", "JavaScript", "Rust"] // Guide conversational style const response = await session.prompt([ { role: "user", content: "Explain quantum computing" }, { role: "assistant", content: "Let me explain using an analogy:\n\n", prefix: true } ]); ``` -------------------------------- ### N-shot Prompting with LanguageModel in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to provide examples of user/assistant interactions to the LanguageModel by adding entries to the 'initialPrompts' array. This allows for few-shot learning where the model learns from provided examples. The 'clone()' method is used for efficiency when creating new sessions. ```javascript const session = await LanguageModel.create({ initialPrompts: [ { role: "system", content: "Predict up to 5 emojis as a response to a comment. Output emojis, comma-separated." }, { role: "user", content: "This is amazing!" }, { role: "assistant", content: "❤️, ➕" }, { role: "user", content: "LGTM" }, { role: "assistant", content: "👍, 🚢" } ] }); // Clone an existing session for efficiency, instead of recreating one each time. async function predictEmoji(comment) { const freshSession = await session.clone(); return await freshSession.prompt(comment); } const result1 = await predictEmoji("Back to the drawing board"); const result2 = await predictEmoji("This code is so good you should get promoted"); ``` -------------------------------- ### Interact with a Language Model Session Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Shows a basic example of creating a language model session with initial prompts and then interacting with it using the prompt() method to get responses. This demonstrates the conversational nature of the session. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a friendly, helpful assistant specialized in clothing choices." }] }); const result = await session.prompt(` What should I wear today? It's sunny and I'm unsure between a t-shirt and a polo. `); console.log(result); const result2 = await session.prompt(` That sounds great, but oh no, it's actually going to rain! New advice?? `); ``` -------------------------------- ### Monitor Language Model Download Progress Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This code example shows how to monitor the download progress when creating a language model that requires downloads. It utilizes a `monitor` callback within the `LanguageModel.create()` options, which listens for 'downloadprogress' events. The example logs the download percentage to the console, allowing for UI elements like progress bars. It also notes that download failures will reject the `create()` promise with a `NetworkError`. ```javascript const session = await LanguageModel.create({ monitor(m) { m.addEventListener("downloadprogress", e => { console.log(`Downloaded ${e.loaded * 100}%`); }); } }); ``` -------------------------------- ### System Prompts and N-Shot Learning for Language Models (JavaScript) Source: https://context7.com/webmachinelearning/prompt-api/llms.txt Configures a language model session using system prompts to define roles and demonstrates n-shot learning by providing examples for specialized behavior like emoji prediction. ```javascript // System prompt defines the assistant's role const session = await LanguageModel.create({ initialPrompts: [ { role: "system", content: "You are a helpful coding assistant specialized in JavaScript. Provide concise, practical solutions." } ] }); const answer = await session.prompt("How do I debounce a function?"); console.log(answer); // N-shot prompting with examples const emojiSession = await LanguageModel.create({ initialPrompts: [ { role: "system", content: "Predict up to 5 emojis as a response to a comment. Output emojis, comma-separated." }, { role: "user", content: "This is amazing!" }, { role: "assistant", content: "❤️, 🎉, 👏, 🔥, ✨" }, { role: "user", content: "LGTM" }, { role: "assistant", content: "👍, 🚢, ✅" }, { role: "user", content: "Needs more work" }, { role: "assistant", content: "🔧, 💪, 🛠️" } ] }); // Clone for efficiency when reusing the same context async function predictEmoji(comment) { const freshSession = await emojiSession.clone(); const result = await freshSession.prompt(comment); freshSession.destroy(); return result; } console.log(await predictEmoji("Mind blown!")); // Might return: 🤯, 💥, 🧠 ``` -------------------------------- ### Configure LanguageModel for Specific Output Language (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Example of initializing a LanguageModel session specifying expected output languages, particularly for a French chatbot. This helps in setting up the session and downloading necessary resources for the specified output language. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: `You are a helpful, harmless French chatbot.` }], expectedInputs: [ { type: "text", languages: ["en" /* for the system prompt */, "fr"] } ], expectedOutputs: [ { type: "text", languages: ["fr"] } ] }); ``` -------------------------------- ### Constrain Assistant Response with Prefix (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This example demonstrates how to use the `prefix: true` option to guide the language model's response format. It's useful for prefilling parts of an 'assistant'-role message, ensuring the model starts with a specific structure, like TOML. Using `prefix` on non-final assistant messages will result in a 'SyntaxError'. ```javascript const followup = await session.prompt([ { role: "user", content: "I'm nervous about my presentation tomorrow" }, { role: "assistant", content: "Presentations are tough!" } ]); // `followup` might be something like "Here are some tips for staying calm.", or // "I remember my first presentation, I was nervous too!" or... const characterSheet = await session.prompt([ { role: "user", content: "Create a TOML character sheet for a gnome barbarian" }, { role: "assistant", content: "```toml\n", prefix: true } ]); ``` -------------------------------- ### Measure Input Usage for Response Constraint Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This example demonstrates how to measure the input token usage that will be consumed by including a response constraint (like a schema or RegExp) in the prompt. This allows developers to anticipate and manage token limits before sending the actual prompt. ```javascript session.measureInputUsage(promptMessage, { responseConstraint: schema }); ``` -------------------------------- ### Constraining Responses with Prefix Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to use the `prefix: true` option on an assistant role message to guide the language model towards a specific response format. ```APIDOC ## POST /session/prompt ### Description This endpoint allows you to constrain the model's response by providing a prefix for the assistant's message. This is useful for guiding the model towards specific output formats, such as TOML. ### Method POST ### Endpoint `/session/prompt` ### Parameters #### Request Body - **messages** (array) - Required - An array of message objects, where each object has a `role` and `content`. To prefix a response, set `prefix: true` on the trailing `"assistant"`-role message. - **role** (string) - Required - The role of the message sender (`"user"`, `"assistant"`, or `"system"`). - **content** (string or array) - Required - The content of the message. Can be a string or an array of objects for multimodal input. - **prefix** (boolean) - Optional - If true, this message content will be used as a prefix for the assistant's response. ### Request Example ```json { "messages": [ { "role": "user", "content": "Create a TOML character sheet for a gnome barbarian" }, { "role": "assistant", "content": "```toml\n", "prefix": true } ] } ``` ### Response #### Success Response (200) - **content** (string) - The generated response from the language model, prefixed as specified. #### Response Example ```json { "content": "```toml\n[character]\nname = \"Grunk\"\nclass = \"Barbarian\"\nrace = \"Gnome\"\n..." } ``` ### Error Handling - **SyntaxError DOMException**: Thrown if `prefix` is used in any message besides a final `"assistant"`-role one. ``` -------------------------------- ### Tool Use with Function Calling in JavaScript Source: https://context7.com/webmachinelearning/prompt-api/llms.txt This example illustrates how to enable the language model to invoke JavaScript functions, such as fetching weather or stock prices. It defines tool schemas and their execution logic, allowing the model to call these tools to access external data and respond to queries. ```javascript const session = await LanguageModel.create({ initialPrompts: [ { role: "system", content: "You are a helpful assistant that can check weather and stock prices." } ], tools: [ { name: "getWeather", description: "Get current weather for a location.", inputSchema: { type: "object", properties: { location: { type: "string", description: "City name or zip code" } }, required: ["location"] }, async execute({ location }) { const response = await fetch(`https://api.weather.example/current?loc=${location}`); const data = await response.json(); return JSON.stringify({ temperature: data.temp, condition: data.condition, humidity: data.humidity }); } }, { name: "getStockPrice", description: "Get current stock price for a ticker symbol.", inputSchema: { type: "object", properties: { symbol: { type: "string", description: "Stock ticker symbol" } }, required: ["symbol"] }, async execute({ symbol }) { const response = await fetch(`https://api.stocks.example/quote?symbol=${symbol}`); const data = await response.json(); return JSON.stringify({ symbol, price: data.price, change: data.change }); } } ] }); // Model can call tools concurrently when needed const result = await session.prompt( "What's the weather in Seattle, Tokyo, and Berlin? Which city is warmest?" ); console.log(result); // The model internally calls getWeather() three times concurrently const stockResult = await session.prompt( "Compare the stock prices of AAPL and GOOGL" ); console.log(stockResult); // Calls getStockPrice() twice and provides comparison ``` -------------------------------- ### Prompt with RegExp Constraint Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This example shows how to use a regular expression to validate the format of the model's string output. It prompts the model to generate a fictional email address and ensures the response conforms to the provided email pattern. Errors occur if the output does not match the RegExp. ```javascript const emailRegExp = /^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/; const emailAddress = await session.prompt( `Create a fictional email address for ${characterName}.`, { responseConstraint: emailRegExp } ); console.log(emailAddress); ``` -------------------------------- ### Structured Output with JSON Schema in JavaScript Source: https://context7.com/webmachinelearning/prompt-api/llms.txt This snippet shows how to constrain model responses to valid JSON that matches a provided schema. It includes an example for a sentiment analysis schema and a RegExp for email validation. Error handling for invalid JSON or unsupported schema is demonstrated. ```javascript const schema = { type: "object", required: ["sentiment", "score", "topics"], additionalProperties: false, properties: { sentiment: { type: "string", enum: ["positive", "negative", "neutral"] }, score: { type: "number", minimum: -1, maximum: 1 }, topics: { type: "array", items: { type: "string" } } } }; const session = await LanguageModel.create(); try { const result = await session.prompt( "Analyze this review: The product quality is excellent but shipping was delayed.", { responseConstraint: schema } ); const analysis = JSON.parse(result); console.log(analysis); // { sentiment: "positive", score: 0.6, topics: ["quality", "shipping"] } } catch (e) { if (e.name === "SyntaxError") { console.error("Model couldn't produce valid JSON"); } else if (e.name === "NotSupportedError") { console.error("JSON schema not supported"); } } // Using RegExp for structured output const emailRegExp = /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/; const email = await session.prompt( "Create a fictional email for character John Smith from Acme Corp.", { responseConstraint: emailRegExp } ); console.log(email); // john.smith@acmecorp.com ``` -------------------------------- ### Configure Model Parameters and Multi-Language Support in JavaScript Source: https://context7.com/webmachinelearning/prompt-api/llms.txt Shows how to retrieve available model parameters like temperature and topK, and create sessions with custom configurations for creativity or determinism. It also demonstrates setting up multi-language support for inputs and outputs and checking language availability. ```javascript // Get available parameters const params = await LanguageModel.params(); if (params) { console.log(`Temperature range: ${params.defaultTemperature} - ${params.maxTemperature}`); console.log(`TopK range: ${params.defaultTopK} - ${params.maxTopK}`); // Create session with custom parameters const creativeSession = await LanguageModel.create({ temperature: params.maxTemperature, // Maximum creativity topK: params.maxTopK // Maximum diversity }); const conservativeSession = await LanguageModel.create({ temperature: 0, // Deterministic topK: 1 // Always pick most likely token }); } else { console.log("Language model not available"); } // Multi-language support const translatorSession = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a translator between English, French, and Spanish." }], expectedInputs: [ { type: "text", languages: ["en", "fr", "es"] } ], expectedOutputs: [ { type: "text", languages: ["en", "fr", "es"] } ] }); // Browser downloads necessary language support or fails fast const translation = await translatorSession.prompt( "Translate to French: The weather is beautiful today" ); // Check availability for specific languages before creating const availability = await LanguageModel.availability({ expectedInputs: [ { type: "text", languages: ["ja", "ko"] }, { type: "audio", languages: ["ja"] } ] }); if (availability === "unavailable") { console.log("Japanese/Korean not supported"); } // OCR with language hint const ocrSession = await LanguageModel.create({ expectedInputs: [ { type: "image", languages: ["de"] } // Optimize for German text in images ] }); ``` -------------------------------- ### System Prompts with WebMachineLearning API Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Illustrates how to configure a language model session with a system prompt to provide context for future interactions. This uses the `initialPrompts` option and the '{ role, content }' format. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "Pretend to be an eloquent hamster." }] }); console.log(await session.prompt("What is your favorite food?")); ``` -------------------------------- ### Configure LanguageModel with Multiple Input Types and Languages (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Shows how to configure `LanguageModel.create()` with multiple input types (text, audio, image) and their respective expected languages. This allows for flexible input handling and ensures necessary resources are downloaded. ```javascript const session = await LanguageModel.create({ expectedInputs: [ // Be sure to download any material necessary for English and Japanese text // prompts, or fail-fast if the model cannot support that. { type: "text", languages: ["en", "ja"] }, // `languages` omitted: audio input processing will be best-effort based on // the base model's capability. { type: "audio" }, // Be sure to download any material necessary for OCRing French text in // images, or fail-fast if the model cannot support that. { type: "image", languages: ["fr"] } ] }); ``` -------------------------------- ### Configure LanguageModel with Expected Input and Output Languages (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to initialize a LanguageModel session specifying expected input and output languages for text. This allows the implementation to download necessary supporting materials and reject unsupported languages early. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: ` You are a foreign-language tutor for Japanese. The user is Korean. If necessary, either you or the user might "break character" and ask for or give clarification in Korean. But by default, prefer speaking in Japanese, and return to the Japanese conversation once any sidebars are concluded. ` }], expectedInputs: [{ type: "text", languages: ["en" /* for the system prompt */, "ja", "ko"] }], // See below section expectedOutputs: [{ type: "text", languages: ["ja", "ko"] }], }); ``` -------------------------------- ### Create and Prompt Language Model Session (JavaScript) Source: https://context7.com/webmachinelearning/prompt-api/llms.txt Initializes a language model session and demonstrates both complete and streaming responses to text prompts. Includes session cleanup. ```javascript const session = await LanguageModel.create(); // Simple prompt - returns complete response const result = await session.prompt("Write me a poem about the ocean."); console.log(result); // Streaming response - useful for long outputs const stream = session.promptStreaming("Write me a detailed analysis of climate change."); for await (const chunk of stream) { console.log(chunk); // Prints chunks as they arrive } // Clean up when done session.destroy(); ``` -------------------------------- ### Zero-shot Prompting with WebMachineLearning API Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to create a language model session and perform zero-shot prompting using a single string. It shows both waiting for the entire result and streaming the result in chunks. ```javascript const session = await LanguageModel.create(); // Prompt the model and wait for the whole result to come back. const result = await session.prompt("Write me a poem."); console.log(result); // Prompt the model and stream the result: const stream = session.promptStreaming("Write me an extra-long poem."); for await (const chunk of stream) { console.log(chunk); } ``` -------------------------------- ### Create Session with Multimodal Inputs (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates creating a language model session that expects both audio and image inputs. This ensures necessary resources are pre-loaded and validates model compatibility. The `expectedInputs` array specifies the types of multimodal data the session will handle. ```javascript const session = await LanguageModel.create({ // { type: "text" } is not necessary to include explicitly, unless // you also want to include expected input languages for text. expectedInputs: [ { type: "audio" }, { type: "image" } ] }); ``` -------------------------------- ### Check Language Model Availability Before Creation Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This snippet demonstrates how to use the `LanguageModel.availability()` method to check if a language model can be created with specific options before attempting to create it. It handles different availability states like 'unavailable', 'downloadable', and 'available', providing conditional logic for user feedback and session creation. This method helps prevent failures and inform users about potential download requirements. ```javascript const options = { expectedInputs: [ { type: "text", languages: ["en", "es"] }, { type: "audio", languages: ["en", "es"] } ], temperature: 2 }; const availability = await LanguageModel.availability(options); if (availability !== "unavailable") { if (availability !== "available") { console.log("Sit tight, we need to do some downloading..."); } const session = await LanguageModel.create(options); // ... Use session ... } else { // Either the API overall, or the expected languages and temperature setting, is not available. console.error("No language model for us :("); } ``` -------------------------------- ### Configure Session Parameters with Temperature and Top-K Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to create a language model session with custom temperature and top-K values. It also shows how to retrieve default and maximum parameter values using LanguageModel.params() and conditionally set parameters based on task requirements. Error handling for out-of-range values is also described. ```javascript const customSession = await LanguageModel.create({ temperature: 0.8, topK: 10 }); const params = await LanguageModel.params(); const conditionalSession = await LanguageModel.create({ temperature: isCreativeTask ? params.defaultTemperature * 1.1 : params.defaultTemperature * 0.8, topK: isGeneratingIdeas ? params.maxTopK : params.defaultTopK }); ``` -------------------------------- ### Prompt with Text and Audio Input (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Illustrates sending a prompt that includes text and an audio input to the language model. The audio data is captured using `captureMicrophoneInput` and then passed as a `Blob` within the prompt's content array. This is for responding to a critique. ```javascript const audioBlob = await captureMicrophoneInput({ seconds: 10 }); const response2 = await session.prompt([ { role: "user", content: [ { type: "text", value: "My response to your critique:" }, { type: "audio", value: audioBlob } ] } ]); ``` -------------------------------- ### Check LanguageModel Availability Before Session Creation (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Utilizes the `LanguageModel.availability()` method to check if a given `expectedInputs` configuration is supported by the browser before initiating session creation. Returns 'unavailable', 'downloadable', 'downloading', or 'available'. ```javascript const availability = await LanguageModel.availability({ expectedInputs: [ { type: "text", languages: ["en", "ja"] }, { type: "audio", languages: ["en", "ja"] } ] }); // `availability` will be one of "unavailable", "downloadable", "downloading", or "available". ``` -------------------------------- ### Check Language Model Availability and Handle Downloads (JavaScript) Source: https://context7.com/webmachinelearning/prompt-api/llms.txt Checks if a language model is available for specific input types and language codes, handling potential downloads and providing progress monitoring. ```javascript // Check availability with specific options const options = { expectedInputs: [ { type: "text", languages: ["en", "es"] }, { type: "audio", languages: ["en"] } ], temperature: 1.5 }; const availability = await LanguageModel.availability(options); if (availability === "unavailable") { console.error("Language model not supported with these options"); // Fall back to cloud API or disable feature } else if (availability === "downloadable" || availability === "downloading") { console.log("Model needs to be downloaded..."); // Create with progress monitoring const session = await LanguageModel.create({ ...options, monitor(m) { m.addEventListener("downloadprogress", e => { const percent = (e.loaded * 100).toFixed(1); console.log(`Download progress: ${percent}%`); updateProgressBar(percent); // Assume updateProgressBar is defined elsewhere }); } }); console.log("Model ready!"); } else { // availability === "available" const session = await LanguageModel.create(options); // Use immediately } ``` -------------------------------- ### Send Multimodal Inputs (Images, Audio) with Text Source: https://context7.com/webmachinelearning/prompt-api/llms.txt This code demonstrates how to send text, images, and audio clips together in a single prompt to the language model. It covers analyzing images and transcribing/summarizing audio, showcasing different ways to structure multimodal messages. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are an art critic and audio transcription expert." }], expectedInputs: [ { type: "text" }, { type: "image" }, { type: "audio" } ] }); // Analyze images const referenceImage = await (await fetch("/reference-artwork.jpg")).blob(); const userDrawing = document.querySelector("#canvas"); const artCritique = await session.prompt([{ role: "user", content: [ { type: "text", value: "Compare my drawing to the reference. Give constructive feedback:" }, { type: "image", value: referenceImage }, { type: "image", value: userDrawing } ] }]); console.log(artCritique); // Process audio const audioBlob = await recordMicrophone({ seconds: 10 }); const transcription = await session.prompt([{ role: "user", content: [ { type: "text", value: "Transcribe this audio and summarize the main points:" }, { type: "audio", value: audioBlob } ] }]); console.log(transcription); // Multiple user messages vs single multimodal message // Single message with multiple modalities: await session.prompt([{ role: "user", content: [ { type: "text", value: "Describe this image:" }, { type: "image", value: imageBlob } ] }]); // vs multiple messages (different semantic meaning): await session.prompt([ { role: "user", content: "Here's an image for context:" }, { role: "user", content: [{ type: "image", value: imageBlob }] }, { role: "user", content: "Now answer this question about it..." } ]); ``` -------------------------------- ### Maintain Session State with Cloning Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Explains how to create a persistent language model session and then clone it to maintain multiple independent continuations of the same prompt. Cloning allows for parallel exploration of different conversational paths. The clone operation can be aborted using an AbortSignal. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a friendly, helpful assistant specialized in clothing choices." }] }); const session2 = await session.clone(); const controller = new AbortController(); const session3 = await session.clone({ signal: controller.signal }); ``` -------------------------------- ### Multi-Message Prompt with Images (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates structuring a prompt using multiple messages, each containing an image. This differs from a single-message multimodal prompt and is used to illustrate how different message structures can be interpreted differently by the model, specifically in a scenario involving departmental strategies. ```javascript const response = await session.prompt([ { role: "user", content: "Your compromise just made the discussion more heated. The two departments drew up posters to illustrate their strategies' advantages:" }, { role: "user", content: [{ type: "image", value: brochureFromTheMarketingDepartment }] }, { role: "user", content: [{ type: "image", value: brochureFromTheFinanceDepartment }] } ]); ``` -------------------------------- ### Tool Use Integration with LanguageModel in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Illustrates how to enable the LanguageModel to use external tools via the 'tools' option. It defines a 'getWeather' tool with a schema and an 'execute' function. The model can then invoke this tool when needed, and the result is sent back to the model for response composition. ```javascript const session = await LanguageModel.create({ initialPrompts: [ { role: "system", content: `You are a helpful assistant. You can use tools to help the user.` } ], tools: [ { name: "getWeather", description: "Get the weather in a location.", inputSchema: { type: "object", properties: { location: { type: "string", description: "The city to check for the weather condition.", }, }, required: ["location"], }, async execute({ location }) { const res = await fetch("https://weatherapi.example/?location=" + location); // Returns the result as a JSON string. return JSON.stringify(await res.json()); }, } ] }); const result = await session.prompt("What is the weather in Seattle?"); ``` -------------------------------- ### Display Token Usage in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Logs the current token usage and the total available token quota for a language model session. This helps developers monitor progress towards the context window limit. ```javascript console.log(`${session.inputUsage} tokens used, out of ${session.inputQuota} tokens available.`); ``` -------------------------------- ### Customizing Prompt Roles with Arrays in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Shows how to use arrays of objects with '{ role, content }' format in the 'prompt()' method. This enables providing multiple user or assistant messages before a new assistant response, offering more control over the conversational flow. System prompts have specific placement restrictions. ```javascript const multiUserSession = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a mediator in a discussion between two departments." }] }); const result = await multiUserSession.prompt([ { role: "user", content: "Marketing: We need more budget for advertising campaigns." }, { role: "user", content: "Finance: We need to cut costs and advertising is on the list." }, { role: "assistant", content: "Let's explore a compromise that satisfies both departments." } ]); // `result` will contain a compromise proposal from the assistant. ``` -------------------------------- ### Abort Operations and Error Handling in JavaScript Source: https://context7.com/webmachinelearning/prompt-api/llms.txt Demonstrates how to abort specific prompts or streaming operations using AbortController. It also shows how to handle various error conditions, including QuotaExceededError, NotSupportedError, and NetworkError, when creating language model sessions. ```javascript // Abort specific prompt const controller = new AbortController(); const stopButton = document.querySelector("#stop-btn"); stopButton.addEventListener("click", () => controller.abort()); try { const result = await session.prompt( "Write a very long essay about world history.", { signal: controller.signal } ); } catch (e) { if (e.name === "AbortError") { console.log("User cancelled the request"); } } // Abort during streaming const streamController = new AbortController(); try { const stream = session.promptStreaming( "Generate a long story", { signal: streamController.signal } ); let chunkCount = 0; for await (const chunk of stream) { console.log(chunk); if (++chunkCount > 10) { streamController.abort(); // Stop after 10 chunks } } } catch (e) { if (e.name === "AbortError") { console.log("Streaming aborted"); } } // Destroy entire session const sessionController = new AbortController(); const session = await LanguageModel.create({ signal: sessionController.signal }); // Later, clean up all resources sessionController.abort(); // Rejects create() if pending, stops all prompts // Handle various errors try { const session = await LanguageModel.create({ initialPrompts: [ { role: "system", content: "A".repeat(1000000) } // Too large ] }); } catch (e) { if (e.name === "QuotaExceededError") { console.error(`Requested ${e.requested} tokens but quota is ${e.quota}`); } else if (e.name === "NotSupportedError") { console.error("This configuration is not supported"); } else if (e.name === "NetworkError") { console.error("Failed to download required model"); } } ``` -------------------------------- ### Prompt with Text and Image Inputs (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Shows how to send a prompt containing text and multiple image inputs to a language model session. The `content` property of a message is an array, where each element specifies the `type` and `value` of the input. This is used for detailed artistic critique. ```javascript const referenceImage = await (await fetch("/reference-image.jpeg")).blob(); const userDrawnImage = document.querySelector("canvas"); const response1 = await session.prompt([ { role: "user", content: [ { type: "text", value: "Give a helpful artistic critique of how well the second image matches the first:" }, { type: "image", value: referenceImage }, { type: "image", value: userDrawnImage } ] } ]); console.log(response1); ``` -------------------------------- ### Concurrent Tool Calls with LanguageModel in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Explains that the language model may call tools multiple times concurrently, including multiple instances of the same tool or different tools. The model waits for all tool call results, similar to Promise.all(), before finalizing its response. ```javascript const result = await session.prompt("Which of these locations currently has the highest temperature? Seattle, Tokyo, Berlin"); ``` -------------------------------- ### Abort Prompt with AbortSignal (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Demonstrates how to abort a specific prompt call in the WebMachineLearning API using an AbortSignal. This requires creating an AbortController, associating its signal with the prompt, and calling abort() on the controller to stop the operation. The prompt operation will be rejected with an 'AbortError' DOMException if aborted while queued or in progress. ```javascript const controller = new AbortController(); stopButton.onclick = () => controller.abort(); const result = await session.prompt("Write me a poem", { signal: controller.signal }); ``` -------------------------------- ### Prompt with JSON Schema Constraint Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This snippet demonstrates how to use a JSON schema to constrain the model's response, ensuring it's a valid JSON object with a specific structure and data types. It handles the prompt, constraint application, and parsing of the resulting JSON. Errors can occur if the schema is unsupported or the output is non-compliant. ```javascript const schema = { type: "object", required: ["rating"], additionalProperties: false, properties: { rating: { type: "number", minimum: 0, maximum: 5, }, }, }; // Prompt the model and wait for the JSON response to come back. const result = await session.prompt("Summarize this feedback into a rating between 0-5: " + "The food was delicious, service was excellent, will recommend.", { responseConstraint: schema } ); const { rating } = JSON.parse(result); console.log(rating); ``` -------------------------------- ### Maintain Conversation Context and Track Token Usage Source: https://context7.com/webmachinelearning/prompt-api/llms.txt This code illustrates how to maintain conversation history (context) across multiple prompts and monitor token usage against a defined quota. It includes handling context window overflow and appending messages without immediate responses. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a travel planning assistant." }], temperature: 0.8, topK: 20 }); // First interaction const response1 = await session.prompt( "I want to visit Japan for 2 weeks. What should I see?" ); console.log(response1); // Context is maintained const response2 = await session.prompt( "What's the best time of year for cherry blossoms?" ); console.log(response2); // Model remembers we're discussing Japan // Monitor token usage console.log(`Used ${session.inputUsage} of ${session.inputQuota} tokens`); // Check before adding more const nextPrompt = "Tell me about transportation options"; const cost = await session.measureInputUsage(nextPrompt); if (session.inputUsage + cost > session.inputQuota) { console.warn("This will cause overflow and lose early messages"); } // Handle quota overflow session.addEventListener("quotaoverflow", () => { console.log("Context window overflowed - oldest messages removed"); }); await session.prompt(nextPrompt); // Append messages without getting a response (preprocessing) const imageUpload = document.querySelector("#file-upload"); imageUpload.addEventListener("change", async (e) => { await session.append([{ role: "user", content: [ { type: "text", value: `Travel photo from: ${e.target.dataset.location}` }, { type: "image", value: e.target.files[0] } ] }]); console.log("Image added to context"); }); // Later, ask about all uploaded images const summary = await session.prompt( "Based on all the photos I shared, create a day-by-day itinerary." ); ``` -------------------------------- ### Prompt with Schema and Omitted Constraint Input Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This snippet illustrates how to use a JSON schema constraint while preventing the schema itself from being included in the input token count. It requires manually adding guidance within the prompt string to ensure the model understands the desired output format. This is useful for managing input token limits. ```javascript const result = await session.prompt(` Summarize this feedback into a rating between 0-5, only outputting a JSON object { rating }, with a single property whose value is a number: The food was delicious, service was excellent, will recommend. `, { responseConstraint: schema, omitResponseConstraintInput: true }); ``` -------------------------------- ### Appending Messages without Prompting Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Explains how to use the `append()` method to add messages to a session ahead of time, allowing the model to pre-process them before a final prompt. ```APIDOC ## POST /session/append ### Description This endpoint allows you to append messages to an existing session without immediately prompting the model for a response. This is useful for pre-processing data, especially for multimodal inputs, giving the model a head-start. ### Method POST ### Endpoint `/session/append` ### Parameters #### Request Body - **messages** (array) - Required - An array of message objects to append to the session. Accepts the same format as the `prompt()` method. - **role** (string) - Required - The role of the message sender (`"user"`, `"assistant"`, or `"system"`). - **content** (string or array) - Required - The content of the message. Can be a string or an array of objects for multimodal input (e.g., text and image). ### Request Example ```javascript const session = await LanguageModel.create({...}); // Assuming session is already created fileUpload.onchange = async (e) => { await session.append([ { "role": "user", "content": [ { "type": "text", "value": `Here's one image. Notes: ${fileNotesInput.value}` }, { "type": "image", "value": fileUpload.files[0] } ] } ]); }; ``` ### Response - The promise returned by `append()` will fulfill once the prompt has been validated, processed, and appended to the session. ### Error Handling - The promise returned by `append()` will reject if the prompt cannot be appended (e.g., too big, invalid modalities for the session, etc.). - **Overflow**: If `append()` causes tokenization context window overflow, it will evict the oldest non-system prompts from the session and fire the `"quotaoverflow"` event. ``` -------------------------------- ### Append Messages to Session Asynchronously (JavaScript) Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This code snippet illustrates the use of the `append()` method to add messages to a language model session without immediately prompting for a response. This is beneficial for pre-processing inputs, especially multimodal ones, allowing the model to begin processing while the user prepares the final prompt. The `append()` method accepts the same message format as `prompt()` and rejects on failure or fulfills upon successful validation and appending. ```javascript const session = await LanguageModel.create({ initialPrompts: [{ role: "system", content: "You are a skilled analyst who correlates patterns across multiple images." }], expectedInputs: [{ type: "image" }] }); fileUpload.onchange = async (e) => { await session.append([ { role: "user", content: [ { type: "text", value: `Here's one image. Notes: ${fileNotesInput.value}` }, { type: "image", value: fileUpload.files[0] } ] } ]); }; analyzeButton.onclick = async (e) => { analysisResult.textContent = await session.prompt(userQuestionInput.value); }; ``` -------------------------------- ### Allowing Cross-Origin Iframes with Permissions Policy Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md This HTML snippet demonstrates how to grant access to the language model API for a cross-origin iframe using the Permissions Policy 'allow' attribute. It specifies that the 'language-model' feature should be allowed for the iframe content. ```html ``` -------------------------------- ### Destroy Language Model Session to Free Memory Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Illustrates how to explicitly destroy a language model session to free up memory. This can be done by passing an AbortSignal to the create() method or by calling the session.destroy() method. Destroying a session rejects ongoing prompts and allows the user agent to unload the model. ```javascript const controller = new AbortController(); stopButton.onclick = () => controller.abort(); const session = await LanguageModel.create({ signal: controller.signal }); // Alternatively: // stopButton.onclick = () => session.destroy(); ``` -------------------------------- ### Listen for Quota Overflow Events in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Adds an event listener to detect when the context window quota has been exceeded. This event fires when inputs might be dropped to accommodate new prompts. ```javascript session.addEventListener("quotaoverflow", () => { console.log("We've gone past the quota, and some inputs will be dropped!"); }); ``` -------------------------------- ### Add Contributor via GitHub Username Source: https://github.com/webmachinelearning/prompt-api/blob/main/CONTRIBUTING.md This command is used within a pull request comment to add a GitHub username as a contributor to the work. Each username should be on a new line, prefixed with '+@'. ```text +@github_username ``` -------------------------------- ### Measure Input Token Usage in JavaScript Source: https://github.com/webmachinelearning/prompt-api/blob/main/README.md Measures the token count for a given prompt without executing it. Supports string and multimodal input arrays. The `AbortSignal` can be used to cancel the measurement. ```javascript const stringUsage = await session.measureInputUsage(promptString); ``` ```javascript const audioUsage = await session.measureInputUsage([ { role: "user", content: [ { type: "text", value: "My response to your critique:" }, { type: "audio", value: audioBlob } ] } ]); ``` ```javascript session.measureInputUsage(promptString, { signal }); ``` -------------------------------- ### Remove Contributor via GitHub Username Source: https://github.com/webmachinelearning/prompt-api/blob/main/CONTRIBUTING.md This command is used within a pull request comment to remove a GitHub username from the list of contributors. Each username should be on a new line, prefixed with '-@'. This can also be used to remove oneself if the contribution was not designed by the commenter. ```text -@github_username ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.