# Prism - Laravel LLM Integration Package Prism is a powerful Laravel package for integrating Large Language Models (LLMs) into your applications. It provides a fluent, expressive interface for generating text, handling multi-step conversations, working with tools, and utilizing various AI providers including OpenAI, Anthropic, Gemini, Mistral, Groq, and Ollama. Prism abstracts away the complexities of different AI provider APIs, allowing developers to focus on building innovative AI features. The package supports text generation, structured output, embeddings, image generation, audio processing (text-to-speech and speech-to-text), streaming responses, and multi-modal inputs including images, documents, audio, and video. It draws inspiration from the Vercel AI SDK and adapts its powerful concepts to the Laravel ecosystem with first-class testing support. ## Installation Install Prism via Composer and publish the configuration file. ```bash # Install the package composer require prism-php/prism # Publish the configuration file php artisan vendor:publish --tag=prism-config ``` ## Basic Text Generation Generate text responses from LLMs with a simple, fluent API. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022') ->withSystemPrompt('You are an expert mathematician who explains concepts simply.') ->withPrompt('Explain the Pythagorean theorem.') ->asText(); echo $response->text; // Access token usage echo "Prompt tokens: {$response->usage->promptTokens}"; echo "Completion tokens: {$response->usage->completionTokens}"; // Check finish reason echo $response->finishReason->name; // "Stop" ``` ## Text Generation with Conversations Build multi-turn conversations using message chains to maintain context. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\ValueObjects\Messages\UserMessage; use Prism\Prism\ValueObjects\Messages\AssistantMessage; $response = Prism::text() ->using(Provider::OpenAI, 'gpt-4') ->withMessages([ new UserMessage('What is JSON?'), new AssistantMessage('JSON is a lightweight data format...'), new UserMessage('Can you show me an example?') ]) ->asText(); echo $response->text; ``` ## Multi-Modal Input (Images, Documents, Audio, Video) Process images, documents, audio, and video files alongside text prompts for rich multi-modal analysis. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\ValueObjects\Media\Image; use Prism\Prism\ValueObjects\Media\Document; use Prism\Prism\ValueObjects\Media\Audio; use Prism\Prism\ValueObjects\Media\Video; // Analyze an image $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022') ->withPrompt( 'What objects do you see in this image?', [Image::fromLocalPath('/path/to/image.jpg')] ) ->asText(); // Process a PDF document $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022') ->withPrompt( 'Summarize the key points from this document', [Document::fromLocalPath('/path/to/document.pdf')] ) ->asText(); // Analyze audio content $response = Prism::text() ->using(Provider::Gemini, 'gemini-1.5-flash') ->withPrompt( 'What is being discussed in this audio?', [Audio::fromLocalPath('/path/to/audio.mp3')] ) ->asText(); // Process video content $response = Prism::text() ->using(Provider::Gemini, 'gemini-1.5-flash') ->withPrompt( 'Describe what happens in this video', [Video::fromUrl('https://example.com/video.mp4')] ) ->asText(); // Multiple media types in one prompt $response = Prism::text() ->using(Provider::Gemini, 'gemini-1.5-flash') ->withPrompt( 'Compare this image with the information in this document', [ Image::fromLocalPath('/path/to/chart.png'), Document::fromLocalPath('/path/to/report.pdf') ] ) ->asText(); ``` ## Structured Output Get AI responses formatted according to a defined schema, perfect for APIs and data processing. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\Schema\ObjectSchema; use Prism\Prism\Schema\StringSchema; use Prism\Prism\Schema\NumberSchema; use Prism\Prism\Schema\ArraySchema; $schema = new ObjectSchema( name: 'movie_review', description: 'A structured movie review', properties: [ new StringSchema('title', 'The movie title'), new StringSchema('rating', 'Rating out of 5 stars'), new StringSchema('summary', 'Brief review summary'), new ArraySchema( name: 'pros', description: 'List of positive aspects', items: new StringSchema('pro', 'A positive aspect') ), ], requiredFields: ['title', 'rating', 'summary', 'pros'] ); $response = Prism::structured() ->using(Provider::OpenAI, 'gpt-4o') ->withSchema($schema) ->withPrompt('Review the movie Inception') ->asStructured(); // Access structured data as PHP array $review = $response->structured; echo $review['title']; // "Inception" echo $review['rating']; // "5 stars" echo $review['summary']; // "A mind-bending masterpiece..." foreach ($review['pros'] as $pro) { echo "- {$pro}\n"; } ``` ## Tools and Function Calling Extend AI capabilities by defining custom tools that can interact with external services and your application logic. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Facades\Tool; use Prism\Prism\Enums\Provider; // Create a weather tool $weatherTool = Tool::as('weather') ->for('Get current weather conditions') ->withStringParameter('city', 'The city to get weather for') ->using(function (string $city): string { // Your weather API logic here return "The weather in {$city} is sunny and 72°F."; }); // Create a search tool $searchTool = Tool::as('search') ->for('Search for current information') ->withStringParameter('query', 'The search query') ->using(function (string $query): string { // Your search implementation return "Search results for: {$query}"; }); $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest') ->withMaxSteps(3) // Required: allow multiple steps for tool usage ->withPrompt('What is the weather like in Paris and what are the latest news about it?') ->withTools([$weatherTool, $searchTool]) ->asText(); echo $response->text; // Inspect tool usage foreach ($response->toolResults as $toolResult) { echo "Tool: {$toolResult->toolName}\n"; echo "Result: {$toolResult->result}\n"; } ``` ## Tools with Complex Parameters Define tools with object parameters, arrays, and enums for complex data structures. ```php use Prism\Prism\Facades\Tool; use Prism\Prism\Schema\ObjectSchema; use Prism\Prism\Schema\StringSchema; use Prism\Prism\Schema\NumberSchema; use Prism\Prism\Schema\ArraySchema; // Tool with object parameter $tool = Tool::as('update_user') ->for('Update a user profile') ->withObjectParameter( 'user', 'The user profile data', [ new StringSchema('name', 'User\'s full name'), new NumberSchema('age', 'User\'s age'), new StringSchema('email', 'User\'s email address') ], requiredFields: ['name', 'email'] ) ->using(function (array $user): string { return "Updated user profile for: {$user['name']}"; }); // Tool with array parameter $tagsTool = Tool::as('process_tags') ->for('Process a list of tags') ->withArrayParameter( 'tags', 'List of tags to process', new StringSchema('tag', 'A single tag') ) ->using(function (array $tags): string { return "Processing tags: " . implode(', ', $tags); }); // Tool with enum parameter $statusTool = Tool::as('set_status') ->for('Set the status') ->withEnumParameter( 'status', 'The new status', ['draft', 'published', 'archived'] ) ->using(function (string $status): string { return "Status set to: {$status}"; }); ``` ## Concurrent Tool Execution Run multiple tools in parallel for I/O-bound operations to reduce total wait time. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Facades\Tool; use Illuminate\Support\Facades\Http; $weatherTool = Tool::as('weather') ->for('Get current weather conditions') ->withStringParameter('city', 'The city to get weather for') ->using(function (string $city): string { return Http::get("https://api.weather.com/{$city}")->json('conditions'); }) ->concurrent(); // Mark as safe for parallel execution $stockTool = Tool::as('stock_price') ->for('Get current stock price') ->withStringParameter('symbol', 'The stock ticker symbol') ->using(function (string $symbol): string { return Http::get("https://api.stocks.com/{$symbol}")->json('price'); }) ->concurrent(); // Mark as safe for parallel execution // Both tools execute in parallel instead of sequentially $response = Prism::text() ->using('anthropic', 'claude-3-5-sonnet-latest') ->withMaxSteps(3) ->withPrompt('What is the weather in NYC and the current price of AAPL?') ->withTools([$weatherTool, $stockTool]) ->asText(); ``` ## Streaming Responses - Server-Sent Events Stream AI responses in real-time to web interfaces using Server-Sent Events. ```php // Laravel Route Route::get('/chat', function () { return Prism::text() ->using('anthropic', 'claude-3-7-sonnet') ->withPrompt(request('message')) ->asEventStreamResponse(); }); ``` ```javascript // JavaScript client const eventSource = new EventSource('/chat?message=Hello'); eventSource.addEventListener('text_delta', (event) => { const data = JSON.parse(event.data); document.getElementById('output').textContent += data.delta; }); eventSource.addEventListener('stream_end', (event) => { const data = JSON.parse(event.data); console.log('Stream ended:', data.finish_reason); eventSource.close(); }); ``` ## Streaming with Vercel AI SDK Integration Use the Data Protocol adapter for compatibility with Vercel AI SDK UI components. ```php // Laravel Route Route::post('/api/chat', function () { return Prism::text() ->using('openai', 'gpt-4') ->withPrompt(request('message')) ->asDataStreamResponse(); }); ``` ```javascript // React component with AI SDK 5.0 import { useChat } from '@ai-sdk/react'; import { useState } from 'react'; export default function Chat() { const [input, setInput] = useState(''); const { messages, sendMessage, status } = useChat({ transport: { api: '/api/chat' }, }); const handleSubmit = (e) => { e.preventDefault(); if (input.trim() && status === 'ready') { sendMessage(input); setInput(''); } }; return (
{messages.map(m => (
{m.role}: {m.parts.filter(p => p.type === 'text').map(p => p.text).join('')}
))}
setInput(e.target.value)} />
); } ``` ## Streaming with WebSocket Broadcasting Process AI requests in background jobs and broadcast results via WebSockets for real-time multi-user applications. ```php // Job class for background processing namespace App\Jobs; use Illuminate\Broadcasting\Channel; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Foundation\Bus\Dispatchable; use Illuminate\Queue\InteractsWithQueue; use Illuminate\Queue\SerializesModels; use Prism\Prism\Facades\Prism; class ProcessAiStreamJob implements ShouldQueue { use Dispatchable, InteractsWithQueue, SerializesModels; public function __construct( public string $message, public string $channel, public string $model = 'claude-3-7-sonnet' ) {} public function handle(): void { Prism::text() ->using('anthropic', $this->model) ->withPrompt($this->message) ->asBroadcast(new Channel($this->channel)); } } // Controller Route::post('/chat-broadcast', function () { $sessionId = request('session_id') ?? 'session_' . uniqid(); ProcessAiStreamJob::dispatch( request('message'), "chat.{$sessionId}" ); return response()->json(['status' => 'processing', 'session_id' => $sessionId]); }); ``` ## Streaming Response Callbacks Handle completion events for persisting conversations, tracking analytics, or logging. ```php use Prism\Prism\Text\PendingRequest; use Prism\Prism\Text\Response; use Illuminate\Support\Collection; use Prism\Prism\Streaming\Events\TextDeltaEvent; // Callback for non-streaming requests $response = Prism::text() ->using('anthropic', 'claude-3-7-sonnet') ->withPrompt(request('message')) ->asText(function (PendingRequest $request, Response $response) use ($conversationId) { ConversationMessage::create([ 'conversation_id' => $conversationId, 'role' => 'assistant', 'content' => $response->text, 'tool_calls' => $response->toolCalls, ]); }); // Callback for streaming responses return Prism::text() ->using('anthropic', 'claude-3-7-sonnet') ->withPrompt(request('message')) ->asEventStreamResponse(function (PendingRequest $request, Collection $events) use ($conversationId) { $fullText = $events ->filter(fn ($event) => $event instanceof TextDeltaEvent) ->map(fn ($event) => $event->delta) ->join(''); ConversationMessage::create([ 'conversation_id' => $conversationId, 'role' => 'assistant', 'content' => $fullText, ]); }); ``` ## Embeddings Generation Generate vector embeddings for semantic search, recommendations, and similarity matching. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\ValueObjects\Media\Image; // Single text embedding $response = Prism::embeddings() ->using(Provider::OpenAI, 'text-embedding-3-large') ->fromInput('Your text goes here') ->asEmbeddings(); $embeddings = $response->embeddings[0]->embedding; echo "Token usage: {$response->usage->tokens}"; // Multiple embeddings in one request $response = Prism::embeddings() ->using(Provider::OpenAI, 'text-embedding-3-large') ->fromInput('First text') ->fromInput('Second text') ->fromArray(['Third', 'Fourth']) ->asEmbeddings(); foreach ($response->embeddings as $embedding) { $vector = $embedding->embedding; // Store in vector database (Milvus, Qdrant, pgvector, etc.) } // Embedding from file $response = Prism::embeddings() ->using(Provider::OpenAI, 'text-embedding-3-large') ->fromFile('/path/to/document.txt') ->asEmbeddings(); // Multimodal embeddings (images) $response = Prism::embeddings() ->using('provider', 'model') ->fromImage(Image::fromLocalPath('/path/to/product.jpg')) ->asEmbeddings(); ``` ## Image Generation Generate images from text prompts using AI-powered models. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\ValueObjects\Media\Image; // Basic image generation $response = Prism::image() ->using('openai', 'dall-e-3') ->withPrompt('A cute baby sea otter floating on its back in calm blue water') ->generate(); $image = $response->firstImage(); echo $image->url; // With provider options $response = Prism::image() ->using('openai', 'dall-e-3') ->withPrompt('A beautiful sunset over mountains') ->withProviderOptions([ 'size' => '1792x1024', 'quality' => 'hd', 'style' => 'vivid', ]) ->generate(); // GPT-Image-1 (returns base64) $response = Prism::image() ->using('openai', 'gpt-image-1') ->withPrompt('Abstract geometric patterns') ->withProviderOptions([ 'size' => '1024x1024', 'quality' => 'high', 'output_format' => 'png', ]) ->generate(); $image = $response->firstImage(); file_put_contents('generated.png', base64_decode($image->base64)); // Image editing with mask $response = Prism::image() ->using('openai', 'gpt-image-1') ->withPrompt('Replace the sky with a starry night', [ Image::fromLocalPath('landscape.png'), ]) ->withProviderOptions([ 'mask' => Image::fromLocalPath('sky-mask.png'), ]) ->generate(); ``` ## Audio Processing - Text-to-Speech and Speech-to-Text Convert text to speech and transcribe audio to text. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\ValueObjects\Media\Audio; // Text-to-Speech $response = Prism::audio() ->using(Provider::OpenAI, 'tts-1') ->withInput('Hello, this is a test of text-to-speech functionality.') ->withVoice('alloy') ->asAudio(); $audio = $response->audio; if ($audio->hasBase64()) { file_put_contents('output.mp3', base64_decode($audio->base64)); } // High-quality TTS with options $response = Prism::audio() ->using('openai', 'tts-1-hd') ->withInput('Welcome to our application!') ->withVoice('nova') ->withProviderOptions([ 'response_format' => 'mp3', 'speed' => 1.0, ]) ->asAudio(); // Speech-to-Text $audioFile = Audio::fromPath('/path/to/audio.mp3'); $response = Prism::audio() ->using(Provider::OpenAI, 'whisper-1') ->withInput($audioFile) ->asText(); echo "Transcription: {$response->text}"; // STT with language hint and verbose output $response = Prism::audio() ->using('openai', 'whisper-1') ->withInput(Audio::fromUrl('https://example.com/speech.wav')) ->withProviderOptions([ 'language' => 'en', 'response_format' => 'verbose_json', ]) ->asText(); // Access segment timestamps foreach ($response->additionalContent['segments'] as $segment) { echo "{$segment['start']}s - {$segment['end']}s: {$segment['text']}\n"; } ``` ## Structured Output with Tools Combine tools with structured output to gather data and return formatted results. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Facades\Tool; use Prism\Prism\Schema\ObjectSchema; use Prism\Prism\Schema\StringSchema; $schema = new ObjectSchema( name: 'weather_analysis', description: 'Analysis of weather conditions', properties: [ new StringSchema('summary', 'Summary of the weather'), new StringSchema('recommendation', 'Recommendation based on weather'), ], requiredFields: ['summary', 'recommendation'] ); $weatherTool = Tool::as('get_weather') ->for('Get current weather for a location') ->withStringParameter('location', 'The city and state') ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny"); $response = Prism::structured() ->using('anthropic', 'claude-3-5-sonnet-latest') ->withSchema($schema) ->withTools([$weatherTool]) ->withMaxSteps(3) // Required for tool + structured output ->withPrompt('What is the weather in San Francisco and should I wear a coat?') ->asStructured(); dump($response->structured); // ['summary' => 'Currently sunny and 72°F...', 'recommendation' => 'No coat needed...'] // Access tool execution details foreach ($response->toolCalls as $toolCall) { echo "Called: {$toolCall->name}\n"; } ``` ## Provider Tools (Built-in Provider Capabilities) Use built-in capabilities offered by AI providers like code execution and web search. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Facades\Tool; use Prism\Prism\ValueObjects\ProviderTool; // Using provider tools $response = Prism::text() ->using('anthropic', 'claude-3-5-sonnet-latest') ->withPrompt('Calculate the fibonacci sequence up to 100') ->withProviderTools([ new ProviderTool(type: 'code_execution_20250522', name: 'code_execution') ]) ->asText(); // Combining provider tools with custom tools $customTool = Tool::as('database_lookup') ->for('Look up user information') ->withStringParameter('user_id', 'The user ID to look up') ->using(fn (string $userId): string => "User data for ID: {$userId}"); $response = Prism::text() ->using('anthropic', 'claude-3-5-sonnet-latest') ->withMaxSteps(5) ->withPrompt('Look up user 123 and calculate their usage statistics') ->withTools([$customTool]) ->withProviderTools([ new ProviderTool(type: 'code_execution_20250522', name: 'code_execution') ]) ->asText(); ``` ## Error Handling Handle potential errors in AI generations gracefully. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\Exceptions\PrismException; use Throwable; try { $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022') ->withPrompt('Generate text...') ->withClientOptions(['timeout' => 30]) ->withClientRetry(3, 100) // Retry 3 times with 100ms delay ->asText(); echo $response->text; } catch (PrismException $e) { Log::error('Prism-specific error:', ['error' => $e->getMessage()]); // Handle API errors, rate limits, etc. } catch (Throwable $e) { Log::error('Generic error:', ['error' => $e->getMessage()]); } ``` ## Testing with Fakes Test AI integrations using Prism's built-in fake implementation. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; use Prism\Prism\Testing\TextResponseFake; use Prism\Prism\Testing\StructuredResponseFake; use Prism\Prism\Testing\EmbeddingsResponseFake; use Prism\Prism\ValueObjects\Usage; use Prism\Prism\ValueObjects\Embedding; use Prism\Prism\ValueObjects\EmbeddingsUsage; // Test text generation it('can generate text', function () { $fakeResponse = TextResponseFake::make() ->withText('Hello, I am Claude!') ->withUsage(new Usage(10, 20)); Prism::fake([$fakeResponse]); $response = Prism::text() ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest') ->withPrompt('Who are you?') ->asText(); expect($response->text)->toBe('Hello, I am Claude!'); }); // Test structured output it('can generate structured response', function () { $fakeResponse = StructuredResponseFake::make() ->withStructured(['name' => 'Alice', 'bio' => 'Developer']); Prism::fake([$fakeResponse]); $response = Prism::structured() ->using('anthropic', 'claude-3-sonnet') ->withPrompt('Generate a user profile') ->withSchema($schema) ->asStructured(); expect($response->structured['name'])->toBe('Alice'); }); // Test embeddings it('can generate embeddings', function () { $fakeResponse = EmbeddingsResponseFake::make() ->withEmbeddings([Embedding::fromArray(array_fill(0, 1536, 0.1))]) ->withUsage(new EmbeddingsUsage(10)); Prism::fake([$fakeResponse]); $response = Prism::embeddings() ->using(Provider::OpenAI, 'text-embedding-3-small') ->fromInput('Test content') ->asEmbeddings(); expect($response->embeddings)->toHaveCount(1); }); // Assert request details $fake = Prism::fake([$fakeResponse]); // ... make requests ... $fake->assertPrompt('Who are you?'); $fake->assertCallCount(1); $fake->assertRequest(function ($requests) { expect($requests[0]->provider())->toBe('anthropic'); }); ``` ## Configuration Override Override provider configuration at runtime for multi-tenant applications. ```php use Prism\Prism\Facades\Prism; use Prism\Prism\Enums\Provider; // Override via third parameter of using() $response = Prism::text() ->using(Provider::OpenAI, 'gpt-4o', [ 'api_key' => 'user-specific-api-key', 'url' => 'https://custom-endpoint.com' ]) ->withPrompt('Explain quantum computing.') ->asText(); // Or via usingProviderConfig() $response = Prism::text() ->using(Provider::OpenAI, 'gpt-4o') ->usingProviderConfig([ 'api_key' => 'user-specific-api-key', 'organization' => 'user-org-id', ]) ->withPrompt('Explain quantum computing.') ->asText(); ``` ## Summary Prism provides a comprehensive solution for Laravel developers who need to integrate AI capabilities into their applications. The primary use cases include chatbots and conversational AI, content generation and summarization, semantic search and recommendations using embeddings, document analysis with multi-modal inputs, automated workflows with tool calling, image generation and editing, and audio processing for voice interfaces. The unified provider interface allows seamless switching between AI providers without code changes. Integration patterns commonly used with Prism include building API endpoints that stream responses to frontend applications, background job processing for long-running AI tasks with WebSocket broadcasting, combining tools with structured output for data extraction pipelines, multi-modal analysis workflows processing documents, images, and audio, and test-driven development using the comprehensive fake system. The fluent API design feels native to Laravel developers, and the package handles the complexities of different provider APIs, rate limiting, retries, and streaming protocols transparently.