### Install fastapi-ai-sdk Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Install the fastapi-ai-sdk package using pip. This command fetches and installs the library and its dependencies. ```bash pip install fastapi-ai-sdk ``` -------------------------------- ### Install Documentation Dependencies (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Installs the necessary Python packages required for building the project's documentation, typically using pip and requirements files. ```bash pip install -e ".[docs]" ``` -------------------------------- ### Install Development Dependencies for FastAPI AI SDK Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Commands to install development dependencies for the project, including the necessary packages for testing and linting. It uses `pip` and assumes the project is cloned locally. ```bash # Install dev dependencies pip install -e ".[dev]" ``` -------------------------------- ### Comprehensive AI Stream Example Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt An example showcasing the `AIStreamBuilder` to construct complex AI streams, including reasoning, tool calls, text responses, and structured data. ```APIDOC ## POST /api/advanced ### Description Demonstrates a comprehensive AI stream with reasoning, tool calls, text responses, and structured data transmission. ### Method POST ### Endpoint /api/advanced ### Parameters #### Request Body - **query** (str) - Required - The user's query. ### Request Example ```json { "query": "What's the weather like?" } ``` ### Response #### Success Response (200) - **stream** (SSE Stream) - A Server-Sent Events stream containing various event types (reasoning, tool_call, text, data). #### Response Example ```text data: {"id": "custom_msg_123", "event": "text", "data": {"value": "Analyzing your query and determining the best approach..."}} data: {"id": "tool_call_abc123", "event": "tool_call", "data": {"name": "get_weather", "arguments": {"city": "San Francisco"}, "tool_call_id": "call_abc123"}} data: {"id": "txt_001", "event": "text", "data": {"value": "Based on the weather data, it's a beautiful sunny day!"}} data: {"id": "weather_data", "event": "data", "data": {"weather": {"temperature": 72, "condition": "sunny", "humidity": 65, "timestamp": "2025-12-16T10:30:00Z"}}} ``` ``` -------------------------------- ### Create and Activate Virtual Environment (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Steps to create a Python virtual environment and activate it. This isolates project dependencies and ensures a clean development setup. Activation commands differ for Windows and Unix-based systems. ```bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate ``` -------------------------------- ### Install Project in Development Mode (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Installs the FastAPI AI SDK package in editable mode along with its development dependencies. This allows for immediate testing and modification of the source code without reinstallation. ```bash pip install -e ".[dev]" ``` -------------------------------- ### LLM Integration with Streaming Support - Python Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt Demonstrates how to integrate real LLM providers with streaming support. This example uses a mock LLM for demonstration but can be replaced with actual API calls to services like OpenAI or Anthropic. It handles text streaming and can optionally include reasoning steps for specific model types. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint from pydantic import BaseModel from typing import List import asyncio app = FastAPI() class Message(BaseModel): role: str content: str class ChatRequest(BaseModel): messages: List[Message] model: str = "gpt-4" temperature: float = 0.7 async def mock_llm_stream(messages: List[Message], model: str): """Mock LLM streaming - replace with OpenAI/Anthropic/etc.""" response = "This is a mock streaming response from the LLM. " response += "In production, this would connect to OpenAI, Anthropic, or other LLM APIs. " response += "The stream would yield tokens as they arrive from the LLM provider." words = response.split() for word in words: yield word + " " await asyncio.sleep(0.05) @app.post("/api/chat/llm") @ai_endpoint() async def chat_with_llm(request: ChatRequest): """Stream responses from LLM providers.""" builder = AIStreamBuilder() builder.start() # Show reasoning for o1-style models if request.model.startswith("o1"): builder.reasoning("Processing request and formulating response...") # Start text streaming text_id = "response_text" from fastapi_ai_sdk.models import TextStartEvent, TextDeltaEvent, TextEndEvent builder.add_event(TextStartEvent(id=text_id)) # Stream from LLM async for chunk in mock_llm_stream(request.messages, request.model): builder.add_event(TextDeltaEvent(id=text_id, delta=chunk)) builder.add_event(TextEndEvent(id=text_id)) builder.finish() return builder ``` -------------------------------- ### Generate Custom Event Streams with FastAPI AI SDK Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This snippet demonstrates how to generate custom event streams by directly controlling event timing and types using the FastAPI AI SDK. It includes examples of starting and ending message streams, showing reasoning processes, and streaming text responses word by word. Dependencies include FastAPI, fastapi_ai_sdk, and asyncio. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStream, create_ai_stream_response from fastapi_ai_sdk.models import ( StartEvent, TextStartEvent, TextDeltaEvent, TextEndEvent, FinishEvent, ReasoningStartEvent, ReasoningDeltaEvent, ReasoningEndEvent ) import asyncio app = FastAPI() @app.get("/api/custom-stream") async def custom_stream(): """Fine-grained control over event generation and timing.""" async def event_generator(): # Start the message stream yield StartEvent(message_id="msg_custom") # Show reasoning process reasoning_id = "reasoning_001" yield ReasoningStartEvent(id=reasoning_id) reasoning_text = "Let me think about this carefully..." for word in reasoning_text.split(): yield ReasoningDeltaEvent(id=reasoning_id, delta=word + " ") await asyncio.sleep(0.1) # Simulate thinking delay yield ReasoningEndEvent(id=reasoning_id) # Stream text response word by word text_id = "text_001" yield TextStartEvent(id=text_id) response_words = ["Hello", "from", "FastAPI", "AI", "SDK"] for word in response_words: yield TextDeltaEvent(id=text_id, delta=word + " ") await asyncio.sleep(0.2) yield TextEndEvent(id=text_id) yield FinishEvent() stream = AIStream(event_generator()) return create_ai_stream_response(stream) ``` -------------------------------- ### Example Commit Message Structure (Git) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Demonstrates the recommended format for Git commit messages, including a concise subject line and a detailed body explaining the changes and referencing issues. Adheres to conventional commit guidelines. ```git Add support for custom event types - Implement DataEvent.create() factory method - Add validation for event type prefixes - Update tests to cover new functionality Fixes #123 ``` -------------------------------- ### Example Test Case Structure (Python) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Illustrates a basic structure for a pytest test case within a class. It shows how to define a test method with a descriptive name and a docstring explaining its purpose. ```python class TestAIStreamBuilder: def test_text_method_adds_three_events(self): """Test that text() method adds start, delta, and end events.""" # Test implementation ``` -------------------------------- ### Text Streaming with Chunking Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt Example of streaming long-form text content in configurable chunks for smooth animations, simulating a typewriter effect. ```APIDOC ## POST /api/story ### Description Streams long-form content with a custom chunk size, creating a typewriter effect for the response. ### Method POST ### Endpoint /api/story ### Parameters #### Request Body - **prompt** (str) - Required - The prompt to generate the story. ### Request Example ```json { "prompt": "Tell me a story about a FastAPI application." } ``` ### Response #### Success Response (200) - **stream** (SSE Stream) - A Server-Sent Events stream containing text chunks. #### Response Example ```text data: {"id": "story_chunk_1", "event": "text", "data": {"value": "Once upon a time in a digital realm, there lived a FastAPI" }} data: {"id": "story_chunk_2", "event": "text", "data": {"value": "application that could stream responses to eager frontends. The" }} ``` ``` -------------------------------- ### Advanced FastAPI Chat with Reasoning and Tool Calls Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md An advanced example showcasing a FastAPI endpoint that integrates AI reasoning and tool calls using the AIStreamBuilder and @ai_endpoint decorator. It demonstrates a more complex interaction flow. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint # Assume get_weather_data is defined elsewhere and returns weather info async def get_weather_data(city: str): # Placeholder for actual data fetching return {"temperature": 25, "condition": "cloudy"} app = FastAPI() @app.post("/api/advanced-chat") @ai_endpoint() async def advanced_chat(query: str): builder = AIStreamBuilder() # Start with reasoning builder.reasoning("Analyzing your query...") # Make a tool call weather_data = await get_weather_data("Berlin") builder.tool_call( "get_weather", input_data={"city": "Berlin"}, output_data=weather_data ) # Stream the response builder.text(f"Based on the weather data: {weather_data}") return builder ``` -------------------------------- ### Basic FastAPI Chat Endpoint with AI SDK Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Demonstrates a basic FastAPI endpoint that uses the AIStreamBuilder to stream a text response. This serves as a starting point for AI-powered chat applications. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint app = FastAPI() @app.post("/api/chat") @ai_endpoint() async def chat(message: str): """Simple chat endpoint that streams a response.""" builder = AIStreamBuilder() builder.text(f"You said: {message}") return builder ``` -------------------------------- ### Text Streaming with Chunking in FastAPI Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This example demonstrates streaming long-form text content in configurable chunks to achieve typewriter effects in the frontend. It also shows how to send multiple distinct text parts within a single stream. ```python from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint from fastapi import FastAPI app = FastAPI() @app.post("/api/story") @ai_endpoint() async def generate_story(prompt: str): """Stream long-form content with custom chunk size.""" builder = AIStreamBuilder() story = """Once upon a time in a digital realm, there lived a FastAPI application that could stream responses to eager frontends. The application used the powerful FastAPI AI SDK to communicate seamlessly with Next.js applications built on the Vercel AI SDK. Together, they created magical user experiences with real-time streaming responses.""" # Stream in 50-character chunks for smooth animation builder.text(story, chunk_size=50) return builder @app.post("/api/multi-part") @ai_endpoint() async def multi_part_response(message: str): """Multiple text parts in a single stream.""" builder = AIStreamBuilder() builder.text("Here's the first part of my response.", text_id="part_1") builder.text("And here's the second part.", text_id="part_2") builder.text("Finally, the conclusion.", text_id="part_3") return builder ``` -------------------------------- ### Chat Endpoint with @ai_endpoint Decorator Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt Automatically converts FastAPI endpoints into AI SDK-compatible streaming responses with lifecycle management. This example demonstrates a basic chat endpoint. ```APIDOC ## POST /api/chat ### Description Basic chat endpoint that echoes the user's message and provides a canned response, streamed automatically. ### Method POST ### Endpoint /api/chat ### Parameters #### Request Body - **message** (str) - Required - The message from the user. ### Request Example ```json { "message": "Hello, AI!" } ``` ### Response #### Success Response (200) - **stream** (SSE Stream) - A Server-Sent Events stream containing text chunks. #### Response Example ```text data: {"id": "some_id_1", "event": "text", "data": {"value": "You said: Hello, AI!"}} data: {"id": "some_id_2", "event": "text", "data": {"value": "This is my response with automatic start/finish events."}} ``` ``` -------------------------------- ### Lint Code with flake8 (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Runs the flake8 linter on the specified directories and files to check for style guide violations and potential errors. This helps enforce code quality standards. ```bash flake8 fastapi_ai_sdk tests examples ``` -------------------------------- ### Structured Data and File References - Python Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt Enables sending structured data and file references alongside text responses. This example showcases how to include file URLs, source URLs, document references, and arbitrary JSON data within the AI stream response using the FastAPI AI SDK. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint from fastapi_ai_sdk.models import FileEvent, SourceURLEvent, SourceDocumentEvent app = FastAPI() @app.post("/api/multimodal") @ai_endpoint() async def multimodal_response(query: str, image_url: str = None): """Send structured data and file references.""" builder = AIStreamBuilder() # Add file reference if provided if image_url: builder.add_event(FileEvent( url=image_url, media_type="image/png" )) # Add source URL references builder.add_event(SourceURLEvent( source_id="source_1", url="https://fastapi.tiangolo.com/tutorial/" )) # Add document reference builder.add_event(SourceDocumentEvent( source_id="doc_1", media_type="application/pdf", title="FastAPI AI SDK Documentation" )) # Add text response builder.text("I've analyzed the image and relevant documentation.") # Add structured analysis data builder.data("analysis", { "confidence": 0.95, "detected_objects": ["person", "laptop", "desk"], "scene_description": "A developer working on a computer", "recommended_actions": [ "Consider ergonomic setup", "Ensure good lighting", "Take regular breaks" ] }) # Add metadata builder.data("metadata", { "model_version": "v1.0.0", "processing_time_ms": 234, "tokens_used": 150 }) return builder ``` -------------------------------- ### Run Tests Matching a Pattern with Pytest (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Runs tests whose names match a given pattern, for example, all tests containing 'test_stream'. This allows for selective execution of relevant tests. ```bash pytest -k "test_stream" ``` -------------------------------- ### GET /api/generate - Custom Async Generators Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md This endpoint demonstrates how to create custom asynchronous generators for streaming AI responses. It yields different event types like StartEvent, TextDeltaEvent, and FinishEvent to manage the streaming process. ```APIDOC ## GET /api/generate ### Description This endpoint allows for custom asynchronous generators to stream AI responses. It supports yielding different event types to manage the lifecycle of the streamed data. ### Method GET ### Endpoint /api/generate ### Parameters None ### Request Body None ### Request Example None ### Response #### Success Response (200) - **event_stream** (async generator) - A stream of events (StartEvent, TextDeltaEvent, FinishEvent) representing the AI response. #### Response Example ```json { "event_type": "StartEvent", "message_id": "gen_1" } { "event_type": "TextDeltaEvent", "id": "txt_1", "delta": "Hello" } { "event_type": "TextDeltaEvent", "id": "txt_1", "delta": " " } // ... more text deltas { "event_type": "FinishEvent" } ``` ``` -------------------------------- ### GET /api/simple-stream - Simple Text Streaming Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This endpoint utilizes the `@streaming_endpoint` decorator to stream plain text responses. It allows for configuration of chunk size and delay to control the streaming behavior, creating effects like a typewriter. ```APIDOC ## GET /api/simple-stream ### Description Streams plain text in chunks with configurable delay and chunk size, suitable for simulating typing or gradual content delivery. ### Method GET ### Endpoint /api/simple-stream ### Parameters None ### Response #### Success Response (200) - Streams plain text content chunk by chunk. ### Response Example ``` This is a si ent. ``` ``` -------------------------------- ### Build Documentation with Make (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Navigates to the docs directory and builds the HTML documentation using the make utility. This command is used when documentation content has been updated. ```bash cd docs make html ``` -------------------------------- ### Clone Repository and Navigate (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Instructions to clone the FastAPI AI SDK repository and navigate into the project directory using git and bash commands. This is a prerequisite for setting up the development environment. ```bash git clone https://github.com/your-username/fastapi-ai-sdk.git cd fastapi-ai-sdk ``` -------------------------------- ### Create and Activate Python Virtual Environment Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Steps to create a Python virtual environment using `venv` and activate it. This isolates project dependencies. The activation command differs for Windows and Unix-like systems. ```bash # Create virtual environment python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate ``` -------------------------------- ### Run Linting with Black, isort, flake8, and mypy Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Commands to run code style checks and type checking tools: `black` for formatting, `isort` for import sorting, `flake8` for linting, and `mypy` for static type analysis. These tools ensure code quality and consistency. ```bash # Run linting black fastapi_ai_sdk tests isort fastapi_ai_sdk tests flake8 fastapi_ai_sdk tests mypy fastapi_ai_sdk ``` -------------------------------- ### Implement Tool-Calling with FastAPI AI SDK Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This snippet demonstrates how to implement tool-calling capabilities with structured input/output handling using the FastAPI AI SDK. It shows an AI assistant that uses tools to answer questions, including reasoning, tool execution, adding tool calls to the stream, and incorporating responses based on tool output. Dependencies include FastAPI, fastapi_ai_sdk, pydantic, and asyncio. ```python from fastapi import FastAPI, HTTPException from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint, tool_endpoint from pydantic import BaseModel app = FastAPI() class WeatherRequest(BaseModel): city: str units: str = "celsius" async def fetch_weather(city: str, units: str = "celsius"): """Mock weather API call.""" return { "city": city, "temperature": 22, "condition": "partly cloudy", "units": units } @app.post("/api/assistant-with-tools") @ai_endpoint() async def assistant_with_tools(message: str): """AI assistant that uses tools to answer questions.""" builder = AIStreamBuilder() # Show reasoning builder.reasoning("I'll check the weather for you...") # Execute tool weather = await fetch_weather("London") # Add tool call to stream builder.tool_call( tool_name="get_weather", input_data={"city": "London", "units": "celsius"}, output_data=weather, stream_input=False # Set True to stream input character by character ) # Add response based on tool output builder.text( f"The weather in {weather['city']} is {weather['condition']} " f"with a temperature of {weather['temperature']}°{weather['units'][0].upper()}." ) # Add structured data for frontend builder.data("weather_result", weather) return builder @app.post("/api/tools/weather") @tool_endpoint("get_weather") async def weather_tool(request: WeatherRequest): """Standalone tool endpoint with automatic formatting.""" try: weather_data = await fetch_weather(request.city, request.units) return weather_data except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` -------------------------------- ### Run FastAPI AI SDK Tests with Coverage Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Instructions for running the test suite using `pytest`, including generating a coverage report that shows missing lines. This helps in assessing the thoroughness of the tests. ```bash # Run tests with coverage pytest --cov=fastapi_ai_sdk --cov-report=term-missing ``` -------------------------------- ### Clone FastAPI AI SDK Repository Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Command to clone the FastAPI AI SDK project from GitHub to your local machine. This is the first step in setting up the development environment. ```bash # Clone the repository git clone https://github.com/doganarif/fastapi-ai-sdk.git cd fastapi-ai-sdk ``` -------------------------------- ### Using AIStreamBuilder for Different Event Types Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Illustrates how to use the AIStreamBuilder to construct a stream with various Vercel AI SDK event types, including text, reasoning, structured data, and tool calls. ```python from fastapi_ai_sdk import AIStreamBuilder # Create a builder builder = AIStreamBuilder(message_id="optional_id") # Add different types of content builder.start() # Start the stream builder.text("Here's some text") # Add text content builder.reasoning("Let me think about this...") # Add reasoning builder.data("weather", {"temperature": 20, "city": "Berlin"}) # Add structured data builder.tool_call( # Add tool usage "get_weather", input_data={"city": "Berlin"}, output_data={"temperature": 20} ) builder.finish() # End the stream # Build and return the stream return builder.build() ``` -------------------------------- ### Assistant with Tools API Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt An AI assistant endpoint that uses tools to answer questions, demonstrating structured input/output handling. ```APIDOC ## POST /api/assistant-with-tools ### Description AI assistant that uses tools to answer questions. It can demonstrate reasoning, execute tools, and provide responses based on tool output. ### Method POST ### Endpoint /api/assistant-with-tools ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **message** (string) - Required - The user's message or question for the assistant. ### Request Example ```json { "message": "What is the weather like?" } ``` ### Response #### Success Response (200) Returns a streaming response containing reasoning, tool calls, text responses, and structured data. #### Response Example ``` { "type": "reasoning", "content": "I\'ll check the weather for you..." } ``` ``` { "type": "tool_code", "tool_name": "get_weather", "input_data": { "city": "London", "units": "celsius" } } ``` ``` { "type": "tool_response", "tool_name": "get_weather", "output_data": { "city": "London", "temperature": 22, "condition": "partly cloudy", "units": "celsius" } } ``` ``` { "type": "text", "content": "The weather in London is partly cloudy with a temperature of 22°C." } ``` ``` { "type": "data", "key": "weather_result", "value": { "city": "London", "temperature": 22, "condition": "partly cloudy", "units": "celsius" } } ``` ``` { "type": "finish" } ``` ``` -------------------------------- ### Run Specific FastAPI AI SDK Test File Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Command to execute a specific test file within the project using `pytest`. Useful for focusing on a particular module or feature during development. ```bash # Run specific test file pytest tests/test_models.py ``` -------------------------------- ### Run All Tests with Pytest (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Executes all tests defined in the project using the pytest framework. This is a fundamental step to ensure code correctness before and after making changes. ```bash pytest ``` -------------------------------- ### Run FastAPI AI SDK Tests with Verbose Output Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Executes the test suite with verbose output enabled using `pytest -v`. This provides detailed information about each test being run, which can be helpful for debugging. ```bash # Run with verbose output pytest -v ``` -------------------------------- ### Run Tests with Coverage (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Runs tests using pytest and generates a coverage report, highlighting lines of code that are not executed by the tests. This helps in identifying areas that need more test coverage. ```bash pytest --cov=fastapi_ai_sdk --cov-report=term-missing ``` -------------------------------- ### POST /api/chat/llm - LLM Integration Pattern Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This endpoint demonstrates how to integrate with LLM providers, supporting streaming responses. It allows for chat-like interactions where the LLM's output is streamed back to the client. ```APIDOC ## POST /api/chat/llm ### Description Stream responses from LLM providers. This endpoint integrates real LLM providers with streaming support, allowing for dynamic and interactive responses. ### Method POST ### Endpoint /api/chat/llm ### Parameters #### Request Body - **messages** (List[Message]) - Required - A list of message objects, each with a 'role' (e.g., 'user', 'assistant') and 'content'. - **model** (str) - Optional - The LLM model to use. Defaults to "gpt-4". - **temperature** (float) - Optional - Controls randomness. Defaults to 0.7. ### Request Example ```json { "messages": [ { "role": "user", "content": "Explain the concept of streaming in LLMs." } ], "model": "gpt-4", "temperature": 0.7 } ``` ### Response #### Success Response (200) Returns an `AIStreamBuilder` object which is a stream of events, including text chunks, reasoning, and metadata. #### Response Example ```json // This is a stream, not a single JSON response. The client will receive // Server-Sent Events (SSE) or similar streaming format. // Example event structure: // data: {"event": "text_delta", "id": "response_text", "delta": "This is a "} // data: {"event": "text_delta", "id": "response_text", "delta": "mock "} // data: {"event": "text_end", "id": "response_text"} // data: {"event": "finish"} ``` ``` -------------------------------- ### Custom Async Generators for Streaming Responses in FastAPI Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Demonstrates how to create a custom asynchronous generator to stream events (StartEvent, TextDeltaEvent, FinishEvent) using `create_ai_stream_response`. This is useful for real-time AI responses. Requires `fastapi_ai_sdk` and `asyncio`. ```python from fastapi_ai_sdk import create_ai_stream_response import asyncio from fastapi_ai_sdk.models import StartEvent, TextDeltaEvent, FinishEvent # Assuming 'app' is your FastAPI instance # from fastapi import FastAPI # app = FastAPI() @app.get("/api/generate") async def generate(): async def event_generator(): yield StartEvent(message_id="gen_1") for word in ["Hello", " ", "from", " ", "FastAPI"]: yield TextDeltaEvent(id="txt_1", delta=word) await asyncio.sleep(0.1) yield FinishEvent() return create_ai_stream_response(event_generator()) ``` -------------------------------- ### Graceful Error Handling in AI Streams with FastAPI Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt Demonstrates how to use AIStreamBuilder to handle and stream errors gracefully within AI responses. It includes simulating potential failure points and providing user-friendly error messages while allowing the stream to continue. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint import asyncio app = FastAPI() @app.post("/api/safe-assistant") @ai_endpoint() async def safe_assistant(query: str): """Demonstrate error handling in streams.""" builder = AIStreamBuilder() try: # Simulate some processing builder.reasoning("Processing your request...") # Simulate potential failure point if "error" in query.lower(): raise ValueError("Simulated error for demonstration") # Successful response builder.text(f"Successfully processed: {query}") except ValueError as e: # Add error event to stream builder.error(f"Processing error: {str(e)}") builder.text("I encountered an error but can continue the conversation.") except Exception as e: # Catch-all error handling builder.error(f"Unexpected error: {str(e)}") return builder @app.post("/api/resilient-stream") @ai_endpoint() async def resilient_stream(message: str): """Continue streaming even after partial failures.""" builder = AIStreamBuilder() builder.text("Starting to process your request...") # Step 1 - always succeeds builder.data("step1", {"status": "complete"}) # Step 2 - might fail try: # Risky operation result = await some_risky_operation(message) builder.data("step2", {"status": "complete", "result": result}) except Exception as e: builder.error(f"Step 2 failed: {str(e)}") builder.data("step2", {"status": "failed", "error": str(e)}) # Step 3 - continues regardless builder.text("Completed processing with partial results.") return builder async def some_risky_operation(msg: str): """Simulated operation that might fail.""" await asyncio.sleep(0.1) if len(msg) > 100: raise ValueError("Message too long") return {"processed": True} ``` -------------------------------- ### Chunked Text Streaming with AIStreamBuilder in FastAPI Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Shows how to use `AIStreamBuilder` to stream a generated text in chunks of a specified size. This method is helpful for managing large text outputs and providing a smoother user experience. Requires `fastapi_ai_sdk` and a function `generate_long_story`. ```python from fastapi_ai_sdk import ai_endpoint, AIStreamBuilder # Assuming 'app' is your FastAPI instance # from fastapi import FastAPI # app = FastAPI() # Placeholder for your story generation logic # async def generate_long_story(prompt: str) -> str: # return "This is a very long story that needs to be streamed..." @app.post("/api/story") @ai_endpoint() async def generate_story(prompt: str): builder = AIStreamBuilder() story = await generate_long_story(prompt) # Your story generation logic # Stream with custom chunk size builder.text(story, chunk_size=50) # Streams in 50-character chunks return builder ``` -------------------------------- ### Format Code with Black (Bash) Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/CONTRIBUTING.md Applies code formatting to the specified directories and files using the Black formatter. This ensures consistent code style across the project. ```bash black fastapi_ai_sdk tests examples ``` -------------------------------- ### FastAPI streaming_endpoint Decorator for Simple Streaming Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Demonstrates the @streaming_endpoint decorator for simple text streaming in FastAPI. It allows defining chunk size and delay for controlled data streaming. ```python from fastapi import FastAPI from fastapi_ai_sdk import streaming_endpoint app = FastAPI() @app.get("/stream") @streaming_endpoint(chunk_size=10, delay=0.1) async def stream(): return "This text will be streamed chunk by chunk" ``` -------------------------------- ### POST /api/multimodal - Multimodal Response Pattern Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This endpoint allows sending structured data and file references alongside text responses. It's designed for scenarios requiring rich outputs that include files, URLs, and structured analysis. ```APIDOC ## POST /api/multimodal ### Description Send structured data and file references alongside text responses. This endpoint enables the integration of various data types, including images, documents, and structured analysis results, within a single response stream. ### Method POST ### Endpoint /api/multimodal ### Parameters #### Query Parameters - **query** (str) - Required - The user's query or prompt. - **image_url** (str) - Optional - A URL pointing to an image to be analyzed or included in the response. ### Request Example ```http POST /api/multimodal?query=Analyze%20this%20image%20and%20provide%20recommendations HTTP/1.1 Host: example.com Content-Type: application/json { "image_url": "https://example.com/path/to/your/image.png" } ``` *Note: The above is an example of how query parameters might be sent. The actual request structure depends on the FastAPI implementation.* ### Response #### Success Response (200) Returns an `AIStreamBuilder` object which is a stream of events, including file references, source URLs, document references, text responses, and structured data (key-value pairs). #### Response Example ```json // This is a stream, not a single JSON response. The client will receive // Server-Sent Events (SSE) or similar streaming format. // Example event structures: // data: {"event": "file", "url": "https://example.com/path/to/your/image.png", "media_type": "image/png"} // data: {"event": "source_url", "source_id": "source_1", "url": "https://fastapi.tiangolo.com/tutorial/"} // data: {"event": "text", "content": "I've analyzed the image and relevant documentation.", "data": null} // data: {"event": "data", "key": "analysis", "value": {"confidence": 0.95, "detected_objects": ["person", "laptop", "desk"], "scene_description": "A developer working on a computer", "recommended_actions": ["Consider ergonomic setup", "Ensure good lighting", "Take regular breaks"]}} // data: {"event": "data", "key": "metadata", "value": {"model_version": "v1.0.0", "processing_time_ms": 234, "tokens_used": 150}} // data: {"event": "finish"} ``` ``` -------------------------------- ### FastAPI ai_endpoint Decorator for AI SDK Handling Source: https://github.com/doganarif/fastapi-ai-sdk/blob/main/README.md Shows the usage of the @ai_endpoint decorator in FastAPI, which automatically handles AI SDK response formatting and streaming. This simplifies the creation of AI-enabled endpoints. ```python from fastapi import FastAPI from fastapi_ai_sdk import AIStreamBuilder, ai_endpoint app = FastAPI() @app.post("/chat") @ai_endpoint() async def chat(message: str): builder = AIStreamBuilder() builder.text(f"Response: {message}") return builder ``` -------------------------------- ### Next.js Chat Page with FastAPI AI SDK Source: https://context7.com/doganarif/fastapi-ai-sdk/llms.txt This Next.js component (`app/chat/page.tsx`) utilizes the Vercel AI SDK to create a real-time chat interface. It connects to a FastAPI backend at 'http://localhost:8000/api/chat', displays streaming messages, handles user input, and shows loading and error states. Dependencies include '@ai-sdk/react'. ```tsx 'use client'; import { useChat } from '@ai-sdk/react'; import { useState } from 'react'; export default function ChatPage() { const { messages, input, handleInputChange, handleSubmit, isLoading, error } = useChat({ api: 'http://localhost:8000/api/chat', onError: (error) => { console.error('Chat error:', error); }, }); return (
{message.role}
{message.content}
{/* Display structured data if present */} {message.data && ({JSON.stringify(message.data, null, 2)}