### Install LLM-JSON Extractor SDK Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/README.md Installs the @solvers-hub/llm-json package using npm. This is the first step to integrate the SDK into your project. ```bash npm install @solvers-hub/llm-json ``` -------------------------------- ### Install LLM-JSON SDK Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Installs the LLM-JSON SDK using npm. This is the first step to using the library in your TypeScript or JavaScript project. ```bash npm install @solvers-hub/llm-json ``` -------------------------------- ### Integrate Zod Schemas with LLM-JSON Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md Demonstrates how to use Zod schemas with llm-json for validation by converting Zod schemas to JSON Schema using `zod-to-json-schema`. It covers installation, schema conversion, and processing LLM output with both llm-json and Zod validation. ```bash npm install zod zod-to-json-schema ``` ```typescript import { LlmJson } from '@solvers-hub/llm-json'; import { z } from 'zod'; import { zodToJsonSchema } from 'zod-to-json-schema'; // Define Zod schema const productSchema = z.object({ productId: z.string(), quantity: z.number().min(0) }); // Convert to JSON Schema const productJsonSchema = zodToJsonSchema(productSchema); // Use with llm-json const llmJson = new LlmJson({ attemptCorrection: true, schemas: [{ name: 'product', schema: productJsonSchema }] }); // Process LLM output const input = 'Found the product: {"productId": "abc-123", "quantity": 10}'; const result = llmJson.extract(input); // Validate and get type-safe result if (result.validatedJson?.[0]?.isValid) { // Double validation: llm-json validated structure, now Zod validates types const validatedProduct = productSchema.parse(result.validatedJson[0].json); console.log(validatedProduct); // Type: { productId: string; quantity: number } } ``` ```typescript // Zod const schema = z.object({ name: z.string(), age: z.number().min(0), role: z.enum(['admin', 'user']).optional() }); // JSON Schema equivalent const jsonSchema = { type: 'object', properties: { name: { type: 'string' }, age: { type: 'number', minimum: 0 }, role: { type: 'string', enum: ['admin', 'user'] } }, required: ['name', 'age'] // role is optional }; ``` -------------------------------- ### Detect JSON Object Start Regex Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This regex pattern detects the start of a JSON object, specifically looking for '{' followed by characters that are not '}' and ending with a colon ':'. This helps identify the beginning of a JSON object structure. ```regex /\{[^}]*:/i ``` -------------------------------- ### Integrate Zod Schemas with llm-json using zod-to-json-schema Source: https://context7.com/solvers-hub/llm-json/llms.txt This snippet demonstrates how to convert Zod schemas into JSON Schemas compatible with llm-json. It uses the `zod-to-json-schema` library to bridge the gap, allowing for type-safe extraction and validation pipelines. Ensure `zod` and `zod-to-json-schema` are installed. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; import { z } from 'zod'; import { zodToJsonSchema } from 'zod-to-json-schema'; // Define Zod schema const productSchema = z.object({ productId: z.string(), name: z.string(), quantity: z.number().min(0), price: z.number().positive() }); type Product = z.infer; // Convert to JSON Schema for llm-json const llmJson = new LlmJson({ attemptCorrection: true, schemas: [{ name: 'product', schema: zodToJsonSchema(productSchema) }] }); const llmOutput = 'Found product: {"productId": "abc-123", "name": "Widget", "quantity": 10, "price": 29.99}'; const result = llmJson.extract(llmOutput); if (result.validatedJson?.[0]?.isValid) { // Type-safe parsing after JSON Schema validation const product: Product = productSchema.parse(result.validatedJson[0].json); console.log(`Product: ${product.name}, Qty: ${product.quantity}, Price: $${product.price}`); // Output: Product: Widget, Qty: 10, Price: $29.99 } ``` -------------------------------- ### Detect JSON Array with Object Regex Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This regex pattern identifies the start of an array that contains JSON objects. It looks for an opening bracket '[' followed by characters that are not ']' and then an opening curly brace '{'. ```regex /\[[^\]]*\{/i ``` -------------------------------- ### Detect Markdown JSON Code Block Regex Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This regex pattern detects markdown code fences specifically for JSON. The 'i' flag makes the match case-insensitive, so it will find ```json or ```JSON. ```regex /```json/i ``` -------------------------------- ### Implement Two-Stage JSON Parsing Strategy Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript class implements a two-stage parsing strategy using the LlmJson library. It first attempts a fast, no-correction parse and falls back to a slower, correction-enabled parse if necessary. This optimizes performance by handling well-formed JSON quickly while still accommodating malformed JSON. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; class TwoStageParser { private fastParser: LlmJson; private fallbackParser: LlmJson; constructor() { // Stage 1: Fast path (no correction) this.fastParser = new LlmJson({ attemptCorrection: false }); // Stage 2: Fallback path (with correction) this.fallbackParser = new LlmJson({ attemptCorrection: true }); } parse(input: string): any[] { // STAGE 1: Fast path const fastResult = this.fastParser.extract(input); // Determine if we need Stage 2 if (this.shouldUseFallback(input, fastResult)) { // STAGE 2: Fallback with correction console.log('Using fallback parser...'); const fallbackResult = this.fallbackParser.extract(input); return fallbackResult.json; } return fastResult.json; } private shouldUseFallback(input: string, fastResult: any): boolean { // If fast path succeeded, no fallback needed if (fastResult.json.length > 0) { return false; } // Check if input contains JSON-like patterns const hasJsonPattern = this.detectJsonPattern(input); // If no JSON patterns, input genuinely has no JSON // Fast path correctly returned empty result if (!hasJsonPattern) { return false; } // Input has JSON patterns but extraction failed // Likely malformed - trigger fallback return true; } private detectJsonPattern(input: string): boolean { const patterns = [ /\{[^}]*:/, // Object with property /\[[^\]]*\{/, // Array containing object /```json/i, // Markdown code block /"[^"]+"\s*:\s*/, // Quoted property name ]; return patterns.some(pattern => pattern.test(input)); } } // Usage const parser = new TwoStageParser(); // Well-formed JSON: Stage 1 succeeds (fast) const result1 = parser.parse('{"name": "test"}'); // No JSON: Stage 1 succeeds (fast) const result2 = parser.parse('Just plain text'); // Malformed JSON: Stage 1 fails, Stage 2 fixes (slower) const result3 = parser.parse('{name: "test"}'); // Missing quotes on key ``` -------------------------------- ### Implement Streaming JSON Handler in TypeScript Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript class, `StreamingJsonHandler`, buffers incoming string chunks and extracts complete JSON objects. It uses a depth counter and string/escape sequence tracking to accurately identify JSON boundaries. Dependencies include the `@solvers-hub/llm-json` library. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; class StreamingJsonHandler { private buffer: string = ''; private llmJson: LlmJson; constructor() { this.llmJson = new LlmJson({ attemptCorrection: true }); } processChunk(chunk: string): any[] { this.buffer += chunk; // Try to extract complete JSON objects const { extracted, remaining } = this.extractCompleteJson(this.buffer); this.buffer = remaining; return extracted; } private extractCompleteJson(text: string): { extracted: any[]; remaining: string } { let depth = 0; let start = -1; let inString = false; let escapeNext = false; const extracted: any[] = []; for (let i = 0; i < text.length; i++) { const char = text[i]; // Handle escape sequences if (escapeNext) { escapeNext = false; continue; } if (char === '\\') { escapeNext = true; continue; } // Track string boundaries if (char === '"') { inString = !inString; continue; } // Only count braces outside strings if (!inString) { if (char === '{') { if (depth === 0) start = i; depth++; } else if (char === '}') { depth--; // Found complete JSON object if (depth === 0 && start !== -1) { const jsonStr = text.substring(start, i + 1); const result = this.llmJson.extract(jsonStr); extracted.push(...result.json); start = -1; } } } } // Return extracted objects and remaining incomplete data const lastComplete = start === -1 ? text.length : start; return { extracted, remaining: text.substring(lastComplete) }; } finalize(): any[] { // Process any remaining buffer if (this.buffer.trim()) { const result = this.llmJson.extract(this.buffer); this.buffer = ''; return result.json; } return []; } } // Usage const handler = new StreamingJsonHandler(); // Simulate streaming chunks const chunks = ['{ "key": "val', 'ue1", "anoth', 'erKey": 123 }']; chunks.forEach(chunk => { const extracted = handler.processChunk(chunk); if (extracted.length > 0) { console.log('Extracted:', extracted); } }); const remaining = handler.finalize(); console.log('Final:', remaining); ``` -------------------------------- ### Extract and Validate JSON with Nested Schema using LlmJson Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript code snippet shows how to define a nested schema for a playlist, initialize the LlmJson class with this schema and correction enabled, and then extract and validate a JSON string. It demonstrates how to access validation results, identify specific errors using instance paths, and filter for valid data when validation fails. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; // Define nested schema const playlistSchema = { name: 'playlist', schema: { type: 'object', properties: { playlistId: { type: 'string' }, name: { type: 'string' }, tracks: { type: 'array', items: { type: 'object', properties: { id: { type: 'string' }, title: { type: 'string' }, artist: { type: 'string' }, duration: { type: 'number', exclusiveMinimum: 0 } }, required: ['id', 'title', 'artist', 'duration'] } } }, required: ['playlistId', 'name', 'tracks'] } }; const llmJson = new LlmJson({ attemptCorrection: true, schemas: [playlistSchema] }); // LLM output with one track missing 'duration' const llmOutput = `{ "playlistId": "pl-001", "name": "My Favorites", "tracks": [ { "id": "t1", "title": "Song One", "artist": "Artist A", "duration": 180 }, { "id": "t2", "title": "Song Two", "artist": "Artist B" }, { "id": "t3", "title": "Song Three", "artist": "Artist C", "duration": 240 } ] }`; const result = llmJson.extract(llmOutput); // Step 1: Check validation status const validation = result.validatedJson?.[0]; if (!validation?.isValid) { console.log('Validation failed!'); // Step 2: Identify which specific item failed validation.validationErrors?.forEach(error => { // Error path format: "/tracks/1/duration" // This means: array "tracks", index 1, field "duration" console.log('Instance Path:', error.instancePath); console.log('Message:', error.message); // Parse the path to extract array index const match = error.instancePath?.match(/\/tracks\/(\d+)/); if (match) { const trackIndex = parseInt(match[1]); console.log(`Failed track index: ${trackIndex}`); console.log(`Failed track data:`, result.json[0].tracks[trackIndex]); } }); // Step 3: Retrieve top-level data (always available in result.json) const playlist = result.json[0]; console.log('Playlist ID:', playlist.playlistId); // Still accessible! console.log('Playlist Name:', playlist.name); // Still accessible! // Step 4: Extract only valid tracks const validTracks = playlist.tracks.filter((track: any) => { return track.id && track.title && track.artist && typeof track.duration === 'number'; }); console.log('Valid tracks:', validTracks); } ``` -------------------------------- ### Parse and Validate Log Entries with LlmJson and Zod Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript code defines a LogProcessor class that leverages LlmJson and Zod to parse and categorize log entries. It converts Zod schemas to JSON schemas for LlmJson, extracts JSON from log strings, and uses Zod's parse method for type-safe validation and routing. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; import { z } from 'zod'; import { zodToJsonSchema } from 'zod-to-json-schema'; // Define Zod schemas const errorLogSchema = z.object({ level: z.literal('error'), message: z.string(), timestamp: z.string(), stackTrace: z.string().optional() }); const infoLogSchema = z.object({ level: z.literal('info'), message: z.string(), timestamp: z.string(), metadata: z.record(z.any()).optional() }); class LogProcessor { private llmJson: LlmJson; private errorSchema: z.ZodType; private infoSchema: z.ZodType; constructor(schemas: { errorLog: z.ZodType; infoLog: z.ZodType }) { this.errorSchema = schemas.errorLog; this.infoSchema = schemas.infoLog; // Convert Zod to JSON Schema this.llmJson = new LlmJson({ attemptCorrection: true, schemas: [ { name: 'errorLog', schema: zodToJsonSchema(schemas.errorLog) }, { name: 'infoLog', schema: zodToJsonSchema(schemas.infoLog) } ] }); } process(logEntry: string): { errors: any[]; infos: any[] } { const result = this.llmJson.extract(logEntry); const errors: any[] = []; const infos: any[] = []; // Process each validated JSON object result.validatedJson?.forEach(validated => { if (validated.isValid) { // Route to appropriate array based on matched schema if (validated.matchedSchema === 'errorLog') { errors.push(this.errorSchema.parse(validated.json)); } else if (validated.matchedSchema === 'infoLog') { infos.push(this.infoSchema.parse(validated.json)); } } }); return { errors, infos }; } } // Usage const processor = new LogProcessor({ errorLog: errorLogSchema, infoLog: infoLogSchema }); const logs = ` {"level": "error", "message": "DB error", "timestamp": "2024-01-15T10:30:00Z"} {"level": "info", "message": "Server started", "timestamp": "2024-01-15T10:00:00Z"} {"level": "error", "message": "API timeout", "timestamp": "2024-01-15T10:35:00Z"} `; const { errors, infos } = processor.process(logs); console.log('Errors:', errors); // 2 error logs console.log('Infos:', infos); // 1 info log ``` -------------------------------- ### Diagnose LLM JSON Extraction Results Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript function helps diagnose the outcome of an LLM JSON extraction attempt. It categorizes the result into 'success', 'no_json', or 'parse_failure' based on whether JSON was extracted, if JSON-like patterns were detected in the input, and if the initial extraction failed despite the presence of such patterns. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; function diagnoseExtraction(input: string, result: any): { status: 'success' | 'no_json' | 'parse_failure'; reason: string; } { // Case 1: Extraction succeeded if (result.json.length > 0) { return { status: 'success', reason: `Extracted ${result.json.length} JSON object(s)` }; } // Case 2 vs 3: Detect JSON-like patterns const jsonPatterns = [ /\{[^}]*:/, // Object pattern /\[[^\]]*\{/, // Array with objects /```json/i, // Code block /"[^"]+"\s*:\s*/, // Quoted property ]; const hasJsonLikeContent = jsonPatterns.some(p => p.test(input)); if (!hasJsonLikeContent) { return { status: 'no_json', reason: 'Input contains no JSON-like patterns' }; } return { status: 'parse_failure', reason: 'Input has JSON-like patterns but extraction failed' }; } // Usage const llmJson = new LlmJson({ attemptCorrection: false }); // Test 1: Success const test1 = llmJson.extract('{"name": "test"}'); console.log(diagnoseExtraction('{"name": "test"}', test1)); // { status: 'success', reason: 'Extracted 1 JSON object(s)' } // Test 2: No JSON const test2 = llmJson.extract('This is just plain text'); console.log(diagnoseExtraction('This is just plain text', test2)); // { status: 'no_json', reason: 'Input contains no JSON-like patterns' } // Test 3: Parse failure const test3 = llmJson.extract('{name: "test"}'); // Malformed console.log(diagnoseExtraction('{name: "test"}', test3)); // { status: 'parse_failure', reason: 'Input has JSON-like patterns but extraction failed' } ``` -------------------------------- ### Initialize Theme and Show App - JavaScript Source: https://github.com/solvers-hub/llm-json/blob/main/docs/interfaces/SchemaDefinition.html This snippet initializes the application's theme by reading from local storage and then shows the main application content. It uses a timeout to ensure the DOM is ready before displaying the app, with a fallback to remove the display property. ```javascript document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os"; document.body.style.display="none"; setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500) ``` -------------------------------- ### Initialize Theme and Show App - JavaScript Source: https://github.com/solvers-hub/llm-json/blob/main/docs/interfaces/ValidationResult.html Initializes the application theme from local storage and controls the initial display of the body. It uses a timeout to ensure the application is ready before showing the page, preventing a flash of unstyled content. ```javascript document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os"; document.body.style.display="none"; setTimeout(() => app ? app.showPage() : document.body.style.removeProperty("display"), 500) ``` -------------------------------- ### Initialize LlmJson Constructor Source: https://context7.com/solvers-hub/llm-json/llms.txt Create an instance of the LlmJson class to manage JSON extraction settings. You can enable automatic correction and provide JSON Schema definitions for validation. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; // Basic instance with correction enabled const llmJson = new LlmJson({ attemptCorrection: true }); // Instance with schema validation const llmJsonWithSchemas = new LlmJson({ attemptCorrection: true, schemas: [ { name: 'person', schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'integer' } }, required: ['name', 'age'] } } ] }); // Singleton pattern (optional) const instance = LlmJson.getInstance({ attemptCorrection: true }); ``` -------------------------------- ### Basic JSON Extraction with LLM-JSON Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Demonstrates the basic usage of the LlmJson class to extract text and JSON objects from an LLM output string. It shows how to instantiate the class and use the `extract` method. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const llmOutput = `Here's some text followed by JSON: { "name": "John", "age": 30, "skills": ["JavaScript", "TypeScript", "React"] }`; const llmJson = new LlmJson({ attemptCorrection: true }); const { text, json } = llmJson.extract(llmOutput); console.log(text); // ['Here\'s some text followed by JSON:'] console.log(json); // [{ name: 'John', age: 30, skills: ['JavaScript', 'TypeScript', 'React'] }] ``` -------------------------------- ### Processing Multiple Schema Types Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Demonstrates how to configure a processor to handle multiple distinct schema types simultaneously, routing validated results based on the matched schema name. ```typescript class LogProcessor { private llmJson: LlmJson; constructor(schemas: { errorLog: any; infoLog: any }) { this.llmJson = new LlmJson({ attemptCorrection: true, schemas: [ { name: 'errorLog', schema: schemas.errorLog }, { name: 'infoLog', schema: schemas.infoLog } ] }); } process(logEntry: string): { errors: any[]; infos: any[] } { const result = this.llmJson.extract(logEntry); const errors: any[] = []; const infos: any[] = []; result.validatedJson?.forEach(validated => { if (validated.isValid) { if (validated.matchedSchema === 'errorLog') { errors.push(validated.json); } else if (validated.matchedSchema === 'infoLog') { infos.push(validated.json); } } }); return { errors, infos }; } } ``` -------------------------------- ### Detect Quoted JSON Property Regex Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This regex pattern identifies a JSON property that is enclosed in double quotes. It looks for a sequence of characters within double quotes, followed by optional whitespace and a colon. ```regex /"[^"]+"\s*:\s*/i ``` -------------------------------- ### Integrate with Zod schemas Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Demonstrates how to use Zod for schema definition and validation by converting Zod schemas to JSON Schema format for the LlmJson library. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; import { z } from 'zod'; import { zodToJsonSchema } from 'zod-to-json-schema'; const productSchema = z.object({ productId: z.string(), quantity: z.number().min(0) }); const llmJson = new LlmJson({ attemptCorrection: true, schemas: [{ name: 'product', schema: zodToJsonSchema(productSchema) }] }); const result = llmJson.extract('Found product: {"productId": "abc-123", "quantity": 10}'); if (result.validatedJson[0]?.isValid) { const validatedProduct = productSchema.parse(result.validatedJson[0].json); console.log(validatedProduct); } ``` -------------------------------- ### Process Multiple Schema Types with LlmJson (TypeScript) Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Builds a typed processor for different data types by initializing LlmJson with multiple schemas. The process method takes a log entry string and returns a structured object containing extracted errors and info logs based on the provided schemas. ```typescript class LogProcessor { private llmJson: LlmJson; constructor(schemas: { errorLog: any; infoLog: any }) { this.llmJson = new LlmJson({ attemptCorrection: true, schemas: [ { name: 'errorLog', schema: schemas.errorLog }, { name: 'infoLog', schema: schemas.infoLog } ] }); } process(logEntry: string): { errors: any[]; infos: any[] } { const result = this.llmJson.extract(logEntry); const errors: any[] = []; const infos: any[] = []; result.validatedJson?.forEach(validated => { if (validated.isValid) { if (validated.matchedSchema === 'errorLog') { errors.push(validated.json); } else if (validated.matchedSchema === 'infoLog') { infos.push(validated.json); } } }); return { errors, infos }; } } ``` -------------------------------- ### Handling Streaming and Chunked JSON Input Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Manages real-time LLM streaming by buffering incoming chunks and detecting complete JSON objects using brace-depth tracking. This ensures that only valid, complete JSON fragments are passed to the parser. ```typescript class StreamingHandler { private buffer = ''; private llmJson = new LlmJson({ attemptCorrection: true }); processChunk(chunk: string) { this.buffer += chunk; const complete = this.findCompleteJson(this.buffer); if (complete) { const result = this.llmJson.extract(complete.json); this.buffer = complete.remaining; return result.json; } return []; } private findCompleteJson(text: string): { json: string; remaining: string } | null { let depth = 0; let start = text.indexOf('{'); if (start === -1) return null; for (let i = start; i < text.length; i++) { if (text[i] === '{') depth++; if (text[i] === '}') depth--; if (depth === 0) { return { json: text.substring(start, i + 1), remaining: text.substring(i + 1) }; } } return null; } } ``` -------------------------------- ### Handling Streaming and Chunked JSON Input Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Provides a buffer-based handler for real-time LLM streaming. It tracks brace depth to detect complete JSON objects within a stream of incoming text chunks. ```typescript class StreamingHandler { private buffer = ''; private llmJson = new LlmJson({ attemptCorrection: true }); processChunk(chunk: string) { this.buffer += chunk; const complete = this.findCompleteJson(this.buffer); if (complete) { const result = this.llmJson.extract(complete.json); this.buffer = complete.remaining; return result.json; } return []; } private findCompleteJson(text: string): { json: string; remaining: string } | null { let depth = 0; let start = text.indexOf('{'); if (start === -1) return null; for (let i = start; i < text.length; i++) { if (text[i] === '{') depth++; if (text[i] === '}') depth--; if (depth === 0) { return { json: text.substring(start, i + 1), remaining: text.substring(i + 1) }; } } return null; } } ``` -------------------------------- ### Implementing Two-Stage Parsing for Performance Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Implements a high-throughput parsing strategy that attempts a fast, non-corrective parse first. If the fast path fails but the input appears to be JSON, it falls back to a more expensive corrective parser. ```typescript class OptimizedParser { private fastParser = new LlmJson({ attemptCorrection: false }); private fallbackParser = new LlmJson({ attemptCorrection: true }); parse(input: string) { const result = this.fastParser.extract(input); if (result.json.length === 0 && this.hasJsonPattern(input)) { return this.fallbackParser.extract(input); } return result; } private hasJsonPattern(input: string): boolean { return /{[^}]*:/.test(input) || /\[[^\]]*{/.test(input); } } ``` -------------------------------- ### Expected Validated JSON Structure for Playlist Schema Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/COMPREHENSIVE_GUIDE.md This TypeScript interface defines the expected structure of the `validatedJson` object returned by the `LlmJson.extract` method when processing the playlist schema and encountering validation errors. It includes the raw extracted JSON, the matched schema name, a boolean indicating validity, and an array of detailed validation errors. ```typescript { json: { /* the full extracted object */ }, matchedSchema: 'playlist', isValid: false, validationErrors: [ { instancePath: '/tracks/1/duration', schemaPath: '#/properties/tracks/items/required', message: "must have required property 'duration'" } ] } ``` -------------------------------- ### Basic JSON Extraction with LLM-JSON Extractor Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/README.md Demonstrates the basic usage of the LlmJson class to extract JSON data from a string. It initializes the class with correction enabled and calls the extract method, logging the separated text and JSON. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const llmOutput = `Here's some text followed by JSON: { "name": "John", "age": 30, "skills": ["JavaScript", "TypeScript", "React"] }`; const llmJson = new LlmJson({ attemptCorrection: true }); const { text, json } = llmJson.extract(llmOutput); console.log(text); // ['Here\'s some text followed by JSON:'] console.log(json); // [{ name: 'John', age: 30, skills: ['JavaScript', 'TypeScript', 'React'] }] ``` -------------------------------- ### Handling Detailed Validation Errors Source: https://context7.com/solvers-hub/llm-json/llms.txt Illustrates how to inspect validation errors when extracted JSON fails to match a schema. It uses the instancePath property to pinpoint exactly which field caused the validation failure, allowing for granular error handling. ```typescript const result = llmJson.extract(input); const validation = result.validatedJson?.[0]; if (!validation?.isValid) { validation?.validationErrors?.forEach(error => { console.log('Path:', error.instancePath); console.log('Message:', error.message); }); } ``` -------------------------------- ### Implement Two-Stage Parsing for Optimized Throughput Source: https://context7.com/solvers-hub/llm-json/llms.txt This pattern optimizes high-throughput pipelines by first attempting a fast parse without correction, falling back to a correction-enabled parse only when necessary. This balances performance and reliability. The `hasJsonPattern` helper function is crucial for deciding when to trigger the fallback. ```typescript import { LlmJson, ExtractResult } from '@solvers-hub/llm-json'; class OptimizedParser { private fastParser = new LlmJson({ attemptCorrection: false }); private fallbackParser = new LlmJson({ attemptCorrection: true }); parse(input: string): ExtractResult { // Stage 1: Fast path without correction const result = this.fastParser.extract(input); // Only use Stage 2 if Stage 1 failed AND input contains JSON patterns if (result.json.length === 0 && this.hasJsonPattern(input)) { return this.fallbackParser.extract(input); } return result; } private hasJsonPattern(input: string): boolean { return /\{[^}]*:/.test(input) || /\[[^\]]*\{/.test(input); } } // Usage const parser = new OptimizedParser(); // Well-formed JSON - uses fast path const cleanResult = parser.parse('{"name": "John", "age": 30}'); console.log(cleanResult.json); // [{ name: "John", age: 30 }] // Malformed JSON - triggers correction fallback const malformedResult = parser.parse('{name: "John", age: 30,}'); console.log(malformedResult.json); // [{ name: "John", age: 30 }] ``` -------------------------------- ### Extract JSON arrays and objects using extractAll() Source: https://context7.com/solvers-hub/llm-json/llms.txt The extractAll method is designed for scenarios where the LLM output contains standalone JSON arrays or a mix of arrays and objects at the root level. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const llmJson = new LlmJson({ attemptCorrection: true }); // Extract standalone arrays const arrayInput = `Here is a list of names: [\"John\", \"Jane\", \"Bob\"]`; const arrayResult = llmJson.extractAll(arrayInput); console.log(arrayResult.json); // Mixed objects and arrays const mixedInput = `First object: {\"name\": \"John\", \"age\": 30}\n\nArray of numbers: [1, 2, 3, 4, 5]\n\nSecond object: {\"city\": \"New York\", \"country\": \"USA\"}`; const mixedResult = llmJson.extractAll(mixedInput); console.log(mixedResult.json); // Complex arrays with nested objects const complexInput = `Here's an array of users:\n[\n { \"name\": \"John Doe\", \"age\": 30, \"skills\": [\"JavaScript\"] },\n { \"name\": \"Jane Smith\", \"age\": 28, \"skills\": [\"Python\", \"ML\"] }\n]`; const complexResult = llmJson.extractAll(complexInput); console.log(complexResult.json[0]); ``` -------------------------------- ### Integrate LLM-JSON with Zod Schemas Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Demonstrates how to use Zod schemas with the LLM-JSON SDK by converting Zod schemas to JSON schemas using `zod-to-json-schema`. This allows for type-safe validation of extracted JSON. ```bash npm install zod zod-to-json-schema ``` ```typescript import { LlmJson } from '@solvers-hub/llm-json'; import { z } from 'zod'; import { zodToJsonSchema } from 'zod-to-json-schema'; // Define your Zod schema const productSchema = z.object({ productId: z.string(), quantity: z.number().min(0) }); // Convert to JSON Schema const llmJson = new LlmJson({ attemptCorrection: true, schemas: [{ name: 'product', schema: zodToJsonSchema(productSchema) }] }); // Process LLM output const result = llmJson.extract('Found product: {"productId": "abc-123", "quantity": 10}'); // Get validated result if (result.validatedJson[0]?.isValid) { const validatedProduct = productSchema.parse(result.validatedJson[0].json); console.log(validatedProduct); // Type-safe! } ``` -------------------------------- ### Handling Nested Validation Errors in LLM-JSON Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Demonstrates how to traverse validation errors in complex nested schemas. It uses instance paths to identify specific array indices that failed validation, allowing for targeted error reporting and data recovery. ```javascript const playlistSchema = { name: 'playlist', schema: { type: 'object', properties: { playlistId: { type: 'string' }, tracks: { type: 'array', items: { type: 'object', properties: { id: { type: 'string' }, title: { type: 'string' }, duration: { type: 'number' } }, required: ['id', 'title', 'duration'] } } }, required: ['playlistId', 'tracks'] }}; const result = llmJson.extract(llmOutput); if (!result.validatedJson[0]?.isValid) { result.validatedJson[0].validationErrors?.forEach(error => { console.log('Path:', error.instancePath); console.log('Message:', error.message); const match = error.instancePath?.match(/\/tracks\/(\d+)/); if (match) { const failedIndex = parseInt(match[1]); console.log('Failed track:', result.json[0].tracks[failedIndex]); } }); } ``` -------------------------------- ### Create LogProcessors for Categorizing Mixed Schema Types Source: https://context7.com/solvers-hub/llm-json/llms.txt This pattern enables the creation of typed processors that categorize extracted JSON objects based on matching schemas. It's ideal for handling diverse data like mixed log entries or multi-format LLM outputs. The `LogProcessor` class uses `llm-json` with multiple schemas to differentiate and sort incoming data. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; class LogProcessor { private llmJson: LlmJson; constructor() { this.llmJson = new LlmJson({ attemptCorrection: true, schemas: [ { name: 'errorLog', schema: { type: 'object', properties: { level: { type: 'string', enum: ['error'] }, message: { type: 'string' }, timestamp: { type: 'string' } }, required: ['level', 'message', 'timestamp'] } }, { name: 'infoLog', schema: { type: 'object', properties: { level: { type: 'string', enum: ['info'] }, message: { type: 'string' }, timestamp: { type: 'string' } }, required: ['level', 'message', 'timestamp'] } } ] }); } process(logEntry: string): { errors: any[]; infos: any[] } { const result = this.llmJson.extract(logEntry); const errors: any[] = []; const infos: any[] = []; result.validatedJson?.forEach(validated => { if (validated.isValid) { if (validated.matchedSchema === 'errorLog') { errors.push(validated.json); } else if (validated.matchedSchema === 'infoLog') { infos.push(validated.json); } } }); return { errors, infos }; } } // Usage const processor = new LogProcessor(); const logs = ` {"level": "error", "message": "DB connection failed", "timestamp": "2024-01-15T10:30:00Z"} {"level": "info", "message": "Server started", "timestamp": "2024-01-15T10:00:00Z"} {"level": "error", "message": "API timeout", "timestamp": "2024-01-15T10:35:00Z"} `; const { errors, infos } = processor.process(logs); console.log('Errors:', errors.length); // 2 console.log('Infos:', infos.length); // 1 ``` -------------------------------- ### Extract JSON from LLM output Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Demonstrates basic usage of the LlmJson class to extract and correct JSON objects from a string containing mixed text and JSON. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const llmOutput = `Here's some text followed by JSON:{ "name": "John", "age": 30, "skills": ["JavaScript", "TypeScript", "React"] }`; const llmJson = new LlmJson({ attemptCorrection: true }); const { text, json } = llmJson.extract(llmOutput); console.log(text); console.log(json); ``` -------------------------------- ### Interface: JsonBlock Source: https://github.com/solvers-hub/llm-json/blob/main/docs-md/interfaces/JsonBlock.md Represents the structure of a JSON block detected within an input string, providing metadata about its location and validity. ```APIDOC ## Interface: JsonBlock ### Description Represents a JSON block identified within a text stream. It includes the raw string, the parsed object, and metadata regarding its position and whether any auto-correction was applied. ### Properties - **startIndex** (number) - Required - The starting index of the JSON block in the input string. - **endIndex** (number) - Required - The ending index of the JSON block in the input string. - **raw** (string) - Required - The original, raw JSON string extracted from the input. - **parsed** (any) - Optional - The resulting object after parsing the raw JSON string. - **wasCorrected** (boolean) - Optional - Indicates if the JSON content required correction to be successfully parsed. ### Usage Example { "startIndex": 0, "endIndex": 25, "raw": "{\"key\": \"value\"}", "parsed": { "key": "value" }, "wasCorrected": false } ``` -------------------------------- ### Optimizing Parsing Performance with Two-Stage Strategy Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Implements a high-throughput parsing pattern that attempts a fast, non-corrective parse first, falling back to a corrective parser only when necessary. This reduces latency for well-formed inputs. ```typescript class OptimizedParser { private fastParser = new LlmJson({ attemptCorrection: false }); private fallbackParser = new LlmJson({ attemptCorrection: true }); parse(input: string) { const result = this.fastParser.extract(input); if (result.json.length === 0 && this.hasJsonPattern(input)) { return this.fallbackParser.extract(input); } return result; } private hasJsonPattern(input: string): boolean { return /\{[^}]*:/.test(input) || /\[[^\]]*\{/.test(input); } } ``` -------------------------------- ### JSON Schema Validation with LLM-JSON Source: https://github.com/solvers-hub/llm-json/blob/main/README.md Shows how to use the LlmJson class with predefined JSON schemas to validate extracted JSON data. It illustrates how to pass schemas during instantiation and access validation results. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const schemas = [ { name: 'person', schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'integer' } }, required: ['name', 'age'] } } ]; const llmJson = new LlmJson({ attemptCorrection: true, schemas }); const llmOutput = `Here's a person: {"name": "John", "age": 30} And some other data: {"title": "Meeting notes"}`; const result = llmJson.extract(llmOutput); // Note: All extracted JSON objects are included in the json array console.log(result.json); // [ // { name: 'John', age: 30 }, // { title: 'Meeting notes' } // ] // The validatedJson array includes validation results for each JSON object console.log(result.validatedJson); // [ // { // json: { name: 'John', age: 30 }, // matchedSchema: 'person', // isValid: true // }, // { // json: { title: 'Meeting notes' }, // matchedSchema: null, // isValid: false, // validationErrors: [...] // Validation errors // } // ] ``` -------------------------------- ### Validating Extracted JSON against Schemas Source: https://context7.com/solvers-hub/llm-json/llms.txt Shows how to define and apply JSON schemas to extracted data. The library validates each extracted object and returns a validatedJson array containing validity status and matched schema information. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const schemas = [ { name: 'person', schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'integer' } }, required: ['name', 'age'] } } ]; const llmJson = new LlmJson({ attemptCorrection: true, schemas }); const input = 'Person: {"name": "John", "age": 32}'; const result = llmJson.extract(input); console.log(result.validatedJson); ``` -------------------------------- ### Validate extracted JSON against schemas Source: https://github.com/solvers-hub/llm-json/blob/main/docs/index.html Shows how to provide a list of JSON schemas to the LlmJson instance to validate extracted objects and receive detailed validation results. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const schemas = [{ name: 'person', schema: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'integer' } }, required: ['name', 'age'] } }]; const llmJson = new LlmJson({ attemptCorrection: true, schemas }); const llmOutput = `Here's a person: {"name": "John", "age": 30}And some other data: {"title": "Meeting notes"}`; const result = llmJson.extract(llmOutput); console.log(result.json); console.log(result.validatedJson); ``` -------------------------------- ### Correcting Malformed JSON with TypeScript Source: https://context7.com/solvers-hub/llm-json/llms.txt Demonstrates how to enable the attemptCorrection feature to automatically fix common LLM-generated JSON errors like unquoted keys, trailing commas, and single quotes. This ensures that otherwise invalid JSON strings can be successfully parsed and extracted. ```typescript import { LlmJson } from '@solvers-hub/llm-json'; const llmJson = new LlmJson({ attemptCorrection: true }); const malformedInput = `Here is some data: { name: "John", // Unquoted key age: 30, 'city': "New York", // Single quotes skills: ["JS", "TS",], // Trailing comma preferences: { theme: "dark", notifications: true, // Trailing comma } }`; const result = llmJson.extract(malformedInput); console.log(result.json[0]); ```