### Example AI Model Formatting Instructions Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/structured-output/converters.html Illustrates example formatting instructions provided to an AI Model to guide its output structure and compliance. ```text Your response should be in JSON format. The data structure for the JSON should match this Java class: java.util.HashMap Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation. ``` -------------------------------- ### Start Milvus with Docker Compose Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/milvus.html Use this command to start the Milvus environment from the specified directory. Ensure Docker is installed and running. ```bash docker-compose up ``` -------------------------------- ### Complete Example: Working with Existing Index Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/azure.html Demonstrates configuring and using an Azure Vector Store with an existing Azure AI Search index that uses custom field names. Includes client setup, builder configuration, and a similarity search example. ```java @Configuration public class VectorStoreConfig { @Bean public SearchIndexClient searchIndexClient() { return new SearchIndexClientBuilder() .endpoint(System.getenv("AZURE_AI_SEARCH_ENDPOINT")) .credential(new AzureKeyCredential(System.getenv("AZURE_AI_SEARCH_API_KEY"))) .buildClient(); } @Bean public VectorStore vectorStore(SearchIndexClient searchIndexClient, EmbeddingModel embeddingModel) { return AzureVectorStore.builder(searchIndexClient, embeddingModel) .indexName("production-documents-index") .initializeSchema(false) // Use existing index // Map to existing index field names .contentFieldName("document_text") .embeddingFieldName("text_vector") .metadataFieldName("document_metadata") // Define filterable metadata fields from existing schema .filterMetadataFields(List.of( MetadataField.text("department"), MetadataField.int64("year"), MetadataField.date("created_date"))) .defaultTopK(10) .defaultSimilarityThreshold(0.75) .build(); } } ``` ```java // Search using the existing index with custom field names List results = vectorStore.similaritySearch( SearchRequest.builder() .query("artificial intelligence") .topK(5) .filterExpression("department == 'Engineering' && year >= 2023") .build()); // The results contain documents with text from the 'document_text' field results.forEach(doc -> System.out.println(doc.getText())); ``` -------------------------------- ### Complete Example: Spring AI with Citations Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/anthropic-chat.html A full example demonstrating the creation of a citation document, calling the model, and processing the response citations. ```java // Create a citation document AnthropicCitationDocument document = AnthropicCitationDocument.builder() .plainText("Spring AI is an application framework for AI engineering. " + "It provides a Spring-friendly API for developing AI applications. " + "The framework includes abstractions for chat models, embedding models, " + "and vector databases.") .title("Spring AI Overview") .citationsEnabled(true) .build(); // Call the model with the document ChatResponse response = chatModel.call( new Prompt( "What is Spring AI?", AnthropicChatOptions.builder() .model("claude-sonnet-4-20250514") .maxTokens(1024) .citationDocuments(document) .build() ) ); // Display the response System.out.println("Response: " + response.getResult().getOutput().getText()); System.out.println("\nCitations:"); // Process citations List citations = (List) response.getMetadata().get("citations"); if (citations != null && !citations.isEmpty()) { for (int i = 0; i < citations.size(); i++) { Citation citation = citations.get(i); System.out.println("\n[" + (i + 1) + "] " + citation.getDocumentTitle()); System.out.println(" Location: " + citation.getLocationDescription()); System.out.println(" Text: " + citation.getCitedText()); } } else { System.out.println("No citations were provided in the response."); } ``` -------------------------------- ### Simple Prompt Template Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/concepts.html This is a basic example of a prompt template. It uses placeholders for dynamic content insertion. ```text Tell me a {adjective} joke about {content}. ``` -------------------------------- ### Resource URI Completion Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-server.html This example shows how to use the `@McpComplete` annotation with the `uri` attribute to complete configuration keys within a URI template. ```APIDOC ## Resource URI Completion ```java @McpComplete(uri = "config://{key}") public List completeConfigKey(String prefix) { return configKeys.stream() .filter(key -> key.startsWith(prefix)) .limit(10) .toList(); } ``` ``` -------------------------------- ### vLLM Server Configuration Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/openai-chat.html This example demonstrates configuring vLLM-specific sampling parameters like `top_k`, `top_p`, `repetition_penalty`, and `min_p` via Spring Boot properties for a Llama model. ```properties spring.ai.openai.base-url=http://localhost:8000 spring.ai.openai.chat.model=meta-llama/Llama-3-70B-Instruct spring.ai.openai.chat.extra-body.top_k=40 spring.ai.openai.chat.extra-body.top_p=0.95 spring.ai.openai.chat.extra-body.repetition_penalty=1.05 spring.ai.openai.chat.extra-body.min_p=0.05 ``` -------------------------------- ### Prompt Argument Completion Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-server.html This example demonstrates how to use the `@McpComplete` annotation with the `prompt` attribute to complete city names based on a prefix. ```APIDOC ## Prompt Argument Completion ```java @Component public class CompletionProvider { @McpComplete(prompt = "city-search") public List completeCityName(String prefix) { return cities.stream() .filter(city -> city.toLowerCase().startsWith(prefix.toLowerCase())) .limit(10) .toList(); } } ``` ``` -------------------------------- ### Few-Shot Prompting for JSON Parsing Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/prompt-engineering-patterns.html Employ for tasks requiring specific output formats, like JSON. Providing examples guides the model to understand complex patterns or edge cases. The quality and diversity of examples are crucial for performance. ```java public void pt_one_shot_few_shots(ChatClient chatClient) { String pizzaOrder = chatClient.prompt(""" Parse a customer's pizza order into valid JSON EXAMPLE 1: I want a small pizza with cheese, tomato sauce, and pepperoni. JSON Response: ``` { "size": "small", "type": "normal", "ingredients": ["cheese", "tomato sauce", "pepperoni"] } ``` EXAMPLE 2: Can I get a large pizza with tomato sauce, basil and mozzarella. JSON Response: ``` { "size": "large", "type": "normal", "ingredients": ["tomato sauce", "basil", "mozzarella"] } ``` Now, I would like a large pizza, with the first half cheese and mozzarella. And the other tomato sauce, ham and pineapple. ") .options(ChatOptions.builder() .model("claude-sonnet-4-6") .temperature(0.1) .maxTokens(250) .build()) .call() .content(); } ``` -------------------------------- ### Example Usage with PromptTemplate Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/structured-output/converters.html Demonstrates how to use a StructuredOutputConverter with a PromptTemplate to include formatting instructions in the AI model's prompt. ```APIDOC ## Example Usage with PromptTemplate ### Description This example shows how the `FormatProvider`'s format instructions are typically appended to the user input using a `PromptTemplate`. ### Code Example ```java StructuredOutputConverter outputConverter = ... // Initialize your converter String userInputTemplate = """ ... user text input .... {format} """; // user input with a "format" placeholder. Prompt prompt = new Prompt( PromptTemplate.builder() .template(this.userInputTemplate) .variables(Map.of(..., "format", this.outputConverter.getFormat())) // replace the "format" placeholder with the converter's format. .build().createMessage() ); ``` ``` -------------------------------- ### Streamable-HTTP Connection Configuration Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-client-boot-starter-docs.html This YAML configuration example shows how to define multiple Streamable-HTTP connections for the MCP client, specifying different URLs and custom endpoints. ```yaml spring: ai: mcp: client: streamable-http: connections: server1: url: http://localhost:8080 server2: url: http://otherserver:8081 endpoint: /custom-sse ``` -------------------------------- ### Token Limit Usage Examples Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/openai-chat.html Examples demonstrating how to use `maxTokens` and `maxCompletionTokens` with different model types. ```APIDOC ## Token Limit Usage Examples ### For non-reasoning models (gpt-4o, gpt-3.5-turbo): ```java ChatResponse response = chatModel.call( new Prompt( "Explain quantum computing in simple terms.", OpenAiChatOptions.builder() .model("gpt-4o") .maxTokens(150) // Use maxTokens for non-reasoning models .build() )); ``` ### For reasoning models (o1, o3 series): ```java ChatResponse response = chatModel.call( new Prompt( "Solve this complex math problem step by step: ...", OpenAiChatOptions.builder() .model("o1-preview") .maxCompletionTokens(1000) // Use maxCompletionTokens for reasoning models .build() )); ``` ### Builder Pattern Validation Example: ```java // This will automatically clear maxTokens and use maxCompletionTokens OpenAiChatOptions options = OpenAiChatOptions.builder() .maxTokens(100) // Set first .maxCompletionTokens(200) // This clears maxTokens and logs a warning .build(); // Result: maxTokens = null, maxCompletionTokens = 200 ``` ``` -------------------------------- ### Orchestrator-Workers Workflow Usage Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/effective-agents.html Use this example to demonstrate how to initialize and run the Orchestrator-Workers workflow with a specific task description. Ensure the ChatClient is properly initialized. ```java ChatClient chatClient = // ... initialize chat client OrchestratorWorkersWorkflow workflow = new OrchestratorWorkersWorkflow(chatClient); WorkerResponse response = workflow.process( "Generate both technical and user-friendly documentation for a REST API endpoint" ); System.out.println("Analysis: " + response.analysis()); System.out.println("Worker Outputs: " + response.workerResponses()); ``` -------------------------------- ### Implementing Completion Capabilities Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-stateless-server-boot-starter-docs.html Provide completion suggestions to clients by registering SyncCompletionSpecification beans. This example defines a basic completion specification. ```java @Bean public List myCompletions() { var completion = new McpStatelessServerFeatures.SyncCompletionSpecification( new McpSchema.PromptReference( "ref/prompt", "code-completion", "Provides code completion suggestions"), (exchange, request) -> { // Implementation that returns completion suggestions return new McpSchema.CompleteResult(List.of("python", "pytorch", "pyside"), 10, true); } ); return List.of(completion); } ``` -------------------------------- ### Create and Use ChatClient Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chatclient.html Demonstrates how to inject and use an autoconfigured ChatClient.Builder to get a String response from an AI model. ```java @RestController class MyController { private final ChatClient chatClient; public MyController(ChatClient.Builder chatClientBuilder) { this.chatClient = chatClientBuilder.build(); } @GetMapping("/ai") String generation(String userInput) { return this.chatClient.prompt() .user(userInput) .call() .content(); } } ``` -------------------------------- ### Defining Static and Dynamic Resources Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-stateless-server-boot-starter-docs.html Expose resources to clients by defining SyncResourceSpecification beans. This example shows how to create a system information resource. ```java @Bean public List myResources(...) { var systemInfoResource = new McpSchema.Resource(...); var resourceSpecification = new McpStatelessServerFeatures.SyncResourceSpecification(systemInfoResource, (context, request) -> { try { var systemInfo = Map.of(...); String jsonContent = new JsonMapper().writeValueAsString(systemInfo); return new McpSchema.ReadResourceResult( List.of(new McpSchema.TextResourceContents(request.uri(), "application/json", jsonContent))); } catch (Exception e) { throw new RuntimeException("Failed to generate system info", e); } }); return List.of(resourceSpecification); } ``` -------------------------------- ### Simple MCP Server Implementation Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/guides/getting-started-mcp.html Define an MCP server using the @Service and @McpTool annotations. This example shows how to expose a 'getTemperature' tool. ```java @Service public class WeatherService { @McpTool(description = "Get current temperature for a location") public String getTemperature( @McpToolParam(description = "City name", required = true) String city) { return String.format("Current temperature in %s: 22°C", city); } } ``` -------------------------------- ### Exposing Prompt Templates Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-stateless-server-boot-starter-docs.html Define and expose prompt templates to clients by registering SyncPromptSpecification beans. This example demonstrates a greeting prompt with an argument. ```java @Bean public List myPrompts() { var prompt = new McpSchema.Prompt("greeting", "A friendly greeting prompt", List.of(new McpSchema.PromptArgument("name", "The name to greet", true))); var promptSpecification = new McpStatelessServerFeatures.SyncPromptSpecification(prompt, (context, getPromptRequest) -> { String nameArgument = (String) getPromptRequest.arguments().get("name"); if (nameArgument == null) { nameArgument = "friend"; } var userMessage = new PromptMessage(Role.USER, TextContent.builder("Hello " + nameArgument + "! How can I assist you today?").build()); return GetPromptResult.builder(List.of(userMessage)).description("A personalized greeting message").build(); }); return List.of(promptSpecification); } ``` -------------------------------- ### Simple MCP Client Implementation Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/guides/getting-started-mcp.html Implement an MCP client using CommandLineRunner to interact with an MCP server. This example calls the 'getTemperature' tool. ```java @Bean public CommandLineRunner demo(ChatClient chatClient, ToolCallbackProvider mcpTools) { return args -> { String response = chatClient .prompt("What's the weather like in Paris?") .tools(mcpTools) .call() .content(); System.out.println(response); }; } ``` -------------------------------- ### New JSON Format for Usage Tokens Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Example of the updated JSON format for usage tokens, using 'completionTokens' instead of 'generationTokens'. ```json { "promptTokens": 100, "completionTokens": 50, "totalTokens": 150 } ``` -------------------------------- ### Manual Configuration of Tool Search Advisor Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/tools/tool-search-tool.html This example demonstrates manual wiring for the Tool Search Advisor. It covers configuring a ToolIndex, building the advisor with custom parameters like maxResults, and registering it with the ChatClient. Note that tools are indexed but not sent to the LLM upfront. A session ID must be supplied via the advisor context for requests. ```java // 1. Configure a ToolIndex (semantic, keyword, or regex) @Bean ToolIndex toolIndex(VectorStore vectorStore) { return new VectorToolIndex(vectorStore); } // 2. Build the advisor var toolSearchAdvisor = ToolSearchToolCallingAdvisor.builder() .toolIndex(toolIndex) .maxResults(5) .build(); // 3. Register with ChatClient — tools are indexed but NOT sent to the LLM up front ChatClient chatClient = ChatClient.builder(chatModel) .defaultTools(new MyTools()) .defaultAdvisors(toolSearchAdvisor) .build(); // 4. Make a request — supply a session ID via the advisor context String answer = chatClient.prompt("Help me plan what to wear today in Amsterdam") .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "user-42-session")) .call() .content(); ``` -------------------------------- ### Chroma Cloud Configuration Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html Configure connection details, API key, tenant, database, and collection for Chroma Cloud. Set initialize-schema to true to automatically create the collection if it doesn't exist. ```properties # Chroma Cloud connection spring.ai.vectorstore.chroma.client.host=api.trychroma.com spring.ai.vectorstore.chroma.client.port=443 spring.ai.vectorstore.chroma.client.key-token= # Chroma Cloud tenant and database (required) spring.ai.vectorstore.chroma.tenant-name= spring.ai.vectorstore.chroma.database-name= # Collection configuration spring.ai.vectorstore.chroma.collection-name=my-collection spring.ai.vectorstore.chroma.initialize-schema=true ``` -------------------------------- ### Evaluator-Optimizer Workflow Usage Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/effective-agents.html This example shows how to use the Evaluator-Optimizer workflow. Initialize the ChatClient and the workflow, then call the loop method with the task to get a refined solution and its evolution. ```java ChatClient chatClient = // ... initialize chat client EvaluatorOptimizerWorkflow workflow = new EvaluatorOptimizerWorkflow(chatClient); RefinedResponse response = workflow.loop( "Create a Java class implementing a thread-safe counter" ); System.out.println("Final Solution: " + response.solution()); System.out.println("Evolution: " + response.chainOfThought()); ``` -------------------------------- ### Run Weaviate in Docker Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/weaviate.html Starts a local Weaviate instance accessible at localhost:8080. Ensure Docker is installed and running. ```bash docker run -it --rm --name weaviate \ -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \ -e PERSISTENCE_DATA_PATH=/var/lib/weaviate \ -e QUERY_DEFAULTS_LIMIT=25 \ -e DEFAULT_VECTORIZER_MODULE=none \ -e CLUSTER_HOSTNAME=node1 \ -p 8080:8080 \ semitechnologies/weaviate:1.31.22 ``` -------------------------------- ### Tool Calling in Spring AI M8 (Correct Usage) Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html This example shows the correct way to register tool specifications and callbacks in M8 using the `toolSpecifications()` method. This replaces the deprecated `tools()` method. ```java // This works in M8 ChatClient chatClient = new OpenAiChatClient(api) .toolSpecifications(List.of( new Tool("get_current_weather", "Get the current weather in a given location", new ToolSpecification.ToolParameter("location", "The city and state, e.g. San Francisco, CA", true)) )) .toolCallbacks(List.of( new ToolCallback("get_current_weather", (toolName, params) -> { // Weather retrieval logic return Map.of("temperature", 72, "unit", "fahrenheit", "description", "Sunny"); }) )); Copied! ``` -------------------------------- ### Add OpenAI Dependency Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Include this Maven dependency to use Spring AI with OpenAI. Verify the version aligns with your project setup. ```xml org.springframework.experimental.ai spring-ai-openai-spring-boot-starter 0.7.1-SNAPSHOT ``` -------------------------------- ### FileDocumentWriter Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html Demonstrates how to initialize and use FileDocumentWriter to write documents to a file. It includes options for document markers, metadata handling, and appending to the file. ```java List documents = // initialize your documents FileDocumentWriter writer = new FileDocumentWriter("output.txt", true, MetadataMode.ALL, true); writer.accept(documents); ``` -------------------------------- ### Tool Calling in Spring AI M7 (Fails in M8) Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html This example demonstrates code that worked in M7 but will silently fail in M8 due to the deprecation of the `tools()` method. Use `toolSpecifications()` instead. ```java // This worked in M7 but silently fails in M8 ChatClient chatClient = new OpenAiChatClient(api) .tools(List.of( new Tool("get_current_weather", "Get the current weather in a given location", new ToolSpecification.ToolParameter("location", "The city and state, e.g. San Francisco, CA", true)) )) .toolCallbacks(List.of( new ToolCallback("get_current_weather", (toolName, params) -> { // Weather retrieval logic return Map.of("temperature", 72, "unit", "fahrenheit", "description", "Sunny"); }) )); Copied! ``` -------------------------------- ### Chain of Thought (CoT) Few-Shot Prompting Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/prompt-engineering-patterns.html Improve reasoning accuracy by providing examples of step-by-step problem-solving before the actual query. This few-shot approach guides the model's response format and logic. ```java public void pt_chain_of_thought_singleshot_fewshots(ChatClient chatClient) { String output = chatClient .prompt(""" Q: When my brother was 2 years old, I was double his age. Now I am 40 years old. How old is my brother? Let's think step by step. A: When my brother was 2 years, I was 2 * 2 = 4 years old. That's an age difference of 2 years and I am older. Now I am 40 years old, so my brother is 40 - 2 = 38 years old. The answer is 38. Q: When I was 3 years old, my partner was 3 times my age. Now, I am 20 years old. How old is my partner? Let's think step by step. A: """) .call() .content(); } ``` -------------------------------- ### Prompt Caching with Cost Tracking Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html Demonstrates how to enable prompt caching with the SYSTEM_ONLY strategy and track cache-related token usage for both cache creation and reads. The first request creates the cache, while the second hits the cache. ```java String largeSystemPrompt = "You are an expert software architect specializing in distributed systems..."; // (Ensure this is 1024+ tokens for cache effectiveness) // First request - creates cache ChatResponse firstResponse = chatModel.call( new Prompt( List.of( new SystemMessage(largeSystemPrompt), new UserMessage("What is microservices architecture?") ), BedrockChatOptions.builder() .model("us.anthropic.claude-haiku-4-5-20251001-v1:0") .cacheOptions(BedrockCacheOptions.builder() .strategy(BedrockCacheStrategy.SYSTEM_ONLY) .build()) .maxTokens(500) .build() ) ); // Access cache-related token usage from metadata Integer cacheWrite1 = (Integer) firstResponse.getMetadata() .getMetadata() .get("cacheWriteInputTokens"); Integer cacheRead1 = (Integer) firstResponse.getMetadata() .getMetadata() .get("cacheReadInputTokens"); System.out.println("Cache creation tokens: " + cacheWrite1); System.out.println("Cache read tokens: " + cacheRead1); // Second request with same system prompt - reads from cache ChatResponse secondResponse = chatModel.call( new Prompt( List.of( new SystemMessage(largeSystemPrompt), // Same prompt - cache hit new UserMessage("What are the benefits of event sourcing?") ), BedrockChatOptions.builder() .model("us.anthropic.claude-haiku-4-5-20251001-v1:0") .cacheOptions(BedrockCacheOptions.builder() .strategy(BedrockCacheStrategy.SYSTEM_ONLY) .build()) .maxTokens(500) .build() ) ); Integer cacheWrite2 = (Integer) secondResponse.getMetadata() .getMetadata() .get("cacheWriteInputTokens"); Integer cacheRead2 = (Integer) secondResponse.getMetadata() .getMetadata() .get("cacheReadInputTokens"); System.out.println("Cache creation tokens: " + cacheWrite2); // Should be 0 System.out.println("Cache read tokens: " + cacheRead2); // Should be > 0 ``` -------------------------------- ### FileDocumentWriter Example Usage Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html Provides a code example demonstrating how to use the FileDocumentWriter to process and write documents. ```APIDOC ## Example ```java List documents = // initialize your documents FileDocumentWriter writer = new FileDocumentWriter("output.txt", true, MetadataMode.ALL, true); writer.accept(documents); Copied! ``` This will write all documents to "output.txt", including document markers, using all available metadata, and appending to the file if it already exists. ``` -------------------------------- ### Customize Tool Execution Eligibility with ToolExecutionEligibilityChecker Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Use ToolExecutionEligibilityChecker to customize when the tool-call loop iterates. This example adds a check for the finish reason. ```java ToolCallingAdvisor advisor = ToolCallingAdvisor.builder() .toolExecutionEligibilityChecker(response -> response != null && response.hasToolCalls() && !"stop".equals(response.getResult().getMetadata().getFinishReason())) .build(); ``` -------------------------------- ### Vector Store Delete Operation Before M6 Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Example of checking the Optional return value from the delete operation in VectorStore before version 1.0.0-M6. ```java Optional result = vectorStore.delete(ids); if (result.isPresent() && result.get()) { // handle successful deletion } ``` -------------------------------- ### Configure EmbeddingModel Bean Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html Example of configuring an EmbeddingModel bean, required for the vector store. This example uses OpenAI. ```java @Bean public EmbeddingModel embeddingModel() { // Can be any other EmbeddingModel implementation. return new OpenAiEmbeddingModel(OpenAiEmbeddingOptions.builder().apiKey(System.getenv("OPENAI_API_KEY")).build()); } ``` -------------------------------- ### Use Chroma Vector Store: Add and Search Documents Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html Example of auto-wiring the Chroma Vector Store, adding documents, and performing a similarity search. ```java @Autowired VectorStore vectorStore; // ... List documents = List.of( new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")), new Document("The World is Big and Salvation Lurks Around the Corner"), new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2"))); // Add the documents vectorStore.add(documents); // Retrieve documents similar to a query List results = this.vectorStore.similaritySearch(SearchRequest.builder().query("Spring").topK(5).build()); ``` -------------------------------- ### Vector Store Delete Operation In M6 and Later Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Example of the void delete operation in VectorStore in version 1.0.0-M6 and later. Deletion is successful if no exception is thrown. ```java vectorStore.delete(ids); // deletion successful if no exception is thrown ``` -------------------------------- ### Custom StringTemplate Renderer Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/prompt.html Example of configuring PromptTemplate with a custom StringTemplate renderer using '<' and '>' as delimiters for template variables. ```java PromptTemplate promptTemplate = PromptTemplate.builder() .renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build()) .template(""" Tell me the names of 5 movies whose soundtrack was composed by . """) .build(); String prompt = promptTemplate.render(Map.of("composer", "John Williams")); ``` -------------------------------- ### Auto-wire and Use QdrantVectorStore Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/qdrant.html Demonstrates how to auto-wire the QdrantVectorStore and use it to add documents and perform similarity searches. Ensure an EmbeddingModel bean is configured. ```java @Autowired VectorStore vectorStore; // ... List documents = List.of( new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")), new Document("The World is Big and Salvation Lurks Around the Corner"), new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")) ); // Add the documents to Qdrant vectorStore.add(documents); // Retrieve documents similar to a query List results = vectorStore.similaritySearch(SearchRequest.builder().query("Spring").topK(5).build()); ``` -------------------------------- ### Combine User Text with Video Input Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html This example demonstrates sending a text prompt and a video file to a multimodal model. The video should be in a supported format like MP4. ```java String response = ChatClient.create(chatModel) .prompt() .user(u -> u.text("Explain what do you see in this video?") .media(Media.Format.VIDEO_MP4, new ClassPathResource("/test.video.mp4"))) .call() .content(); logger.info(response); ``` -------------------------------- ### Migrate FunctionCallback to ToolCallback API Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html Spring AI 2.0 removes the deprecated `FunctionCallback` API entirely, standardizing on `ToolCallback`. This example shows the conversion of a function-style callback. ```java // Before — function-style callback FunctionCallback.builder() .function("getCurrentWeather", new MockWeatherService()) .description("Get the weather in location") .inputType(MockWeatherService.Request.class) .build(); // After — explicit ToolCallback via FunctionToolCallback FunctionToolCallback.builder("getCurrentWeather", new MockWeatherService()) .description("Get the weather in location") .inputType(MockWeatherService.Request.class) .build(); ``` -------------------------------- ### Getting the Full Response Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/structured-output.html Use `.responseEntity(…​)` to get both the parsed entity and the underlying `ChatResponse` object, which contains metadata like token usage. ```APIDOC ## Getting the Full Response `.entity(…​)` returns only the parsed object. If you also need the underlying `ChatResponse` — for token usage, observability metadata, or anything beyond the entity — use `.responseEntity(…​)`: ```java ResponseEntity result = chatClient.prompt() .user("Generate the filmography for a random actor.") .call() .responseEntity(ActorsFilms.class); ActorsFilms films = result.entity(); ChatResponse raw = result.response(); long totalTokens = raw.getMetadata().getUsage().getTotalTokens(); ``` It has the same overload set as `.entity(…​)` — `Class`, `ParameterizedTypeReference`, custom `StructuredOutputConverter`, and the `EntityParamSpec` consumer all apply. ``` -------------------------------- ### MCP Client Handler Examples Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-client-boot-starter-docs.html Demonstrates how to use various MCP annotations like @McpLogging, @McpSampling, @McpProgress, and @McpToolListChanged to handle server notifications and requests. ```java @Component public class McpClientHandlers { @McpLogging(clients = "server1") public void handleLoggingMessage(LoggingMessageNotification notification) { System.out.println("Received log: " + notification.level() + " - " + notification.data()); } @McpSampling(clients = "server1") public CreateMessageResult handleSamplingRequest(CreateMessageRequest request) { // Process the request and generate a response String response = generateLLMResponse(request); return CreateMessageResult.builder(Role.ASSISTANT, response, "gpt-4") .build(); } @McpProgress(clients = "server1") public void handleProgressNotification(ProgressNotification notification) { double percentage = notification.progress() * 100; System.out.println(String.format("Progress: %.2f%% - %s", percentage, notification.message())); } @McpToolListChanged(clients = "server1") public void handleToolListChanged(List updatedTools) { System.out.println("Tool list updated: " + updatedTools.size() + " tools available"); // Update local tool registry toolRegistry.updateTools(updatedTools); } } ``` -------------------------------- ### Add Standard MCP Client Dependency Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-client-boot-starter-docs.html Include this dependency in your project to enable the standard MCP client starter. This starter supports STDIO, SSE, Streamable-HTTP, and Stateless Streamable-HTTP transports. ```xml org.springframework.ai spring-ai-starter-mcp-client ``` -------------------------------- ### Configure Redis Vector Store with application.yml Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/redis.html Example configuration for Redis connection URL, schema initialization, index name, and key prefix using application.yml. ```yaml spring: data: redis: url: ai: vectorstore: redis: initialize-schema: true index-name: custom-index prefix: custom-prefix ``` -------------------------------- ### MCP Client Application Setup Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-examples.html Sets up the Spring Boot application to use MCP tools and handlers. It configures a ChatClient with default tools and demonstrates a user interaction with an LLM, including a complex prompt. ```java package com.example.mcp; import org.springframework.boot.CommandLineRunner; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Bean; import io.spring.ai.chat.ChatClient; import io.spring.ai.openai.OpenAiChatModel; import io.spring.ai.mcp.tool.ToolCallbackProvider; @SpringBootApplication public class McpClientApplication { public static void main(String[] args) { SpringApplication.run(McpClientApplication.class, args).close(); } @Bean public CommandLineRunner predefinedQuestions(OpenAiChatModel openAiChatModel, ToolCallbackProvider mcpToolProvider) { return args -> { ChatClient chatClient = ChatClient.builder(openAiChatModel) .defaultTools(mcpToolProvider) .build(); String userQuestion = """ What is the weather in Amsterdam right now? Please incorporate all creative responses from all LLM providers. After the other providers add a poem that synthesizes the poems from all the other providers. """ ; System.out.println("> USER: " + userQuestion); System.out.println("> ASSISTANT: " + chatClient.prompt(userQuestion).call().content()); }; } } ``` -------------------------------- ### JsonReader get(String pointer) Method Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html Details the `get(String pointer)` method of the JsonReader, which allows for retrieving specific parts of a JSON document using JSON Pointers. ```APIDOC ## JsonReader get(String pointer) ### Description This method allows you to use a JSON Pointer to retrieve a specific part of the JSON document, enabling the extraction of nested data. ### Method Signature ```java public List get(String pointer) ``` ### Parameters * `pointer`: A JSON Pointer string (as defined in RFC 6901) to locate the desired element within the JSON structure. ### Return Value * Returns a `List` containing the documents parsed from the JSON element located by the pointer. ### Behavior * The method uses the provided JSON Pointer to navigate to a specific location in the JSON structure. * If the pointer is valid and points to an existing element: * For a JSON object: it returns a list with a single Document. * For a JSON array: it returns a list of Documents, one for each element in the array. * If the pointer is invalid or points to a non-existent element, it throws an `IllegalArgumentException`. ### Example ```java // Assuming 'resource' is a valid Spring Resource pointing to a JSON file JsonReader jsonReader = new JsonReader(resource, "description"); List documents = this.jsonReader.get("/store/books/0"); ``` ``` -------------------------------- ### Add Qdrant Starter Dependency (Maven) Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/qdrant.html Include the Qdrant starter dependency in your Maven pom.xml to enable auto-configuration for the Qdrant Vector Store. ```xml org.springframework.ai spring-ai-starter-vector-store-qdrant ``` -------------------------------- ### Keyword Metadata Enrichment Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html A complete example demonstrating the initialization and usage of `KeywordMetadataEnricher` with both default and custom templates, including applying it to documents and printing extracted keywords. Requires a `ChatModel`. ```java ChatModel chatModel = // initialize your chat model KeywordMetadataEnricher enricher = KeywordMetadataEnricher.builder(chatModel) .keywordCount(5) .build(); // Or use custom templates KeywordMetadataEnricher enricher = KeywordMetadataEnricher.builder(chatModel) .keywordsTemplate(new PromptTemplate("Extract 5 important keywords from the following text and separate them with commas:\n{context_str}")) .build(); Document doc = new Document("This is a document about artificial intelligence and its applications in modern technology."); List enrichedDocs = enricher.apply(List.of(this.doc)); Document enrichedDoc = this.enrichedDocs.get(0); String keywords = (String) this.enrichedDoc.getMetadata().get("excerpt_keywords"); System.out.println("Extracted keywords: " + keywords); ``` -------------------------------- ### Create Qdrant Client Bean Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/qdrant.html Manually configure and create a `QdrantClient` bean. Ensure you replace placeholders like ``, ``, ``, and `` with your actual Qdrant connection details. ```java @Bean public QdrantClient qdrantClient() { QdrantGrpcClient.Builder grpcClientBuilder = QdrantGrpcClient.newBuilder( "", , ); grpcClientBuilder.withApiKey(""); return new QdrantClient(grpcClientBuilder.build()); } ``` -------------------------------- ### Install PGvector Extensions and Create Table Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/pgvector.html Manually install required PostgreSQL extensions and create the `vector_store` table with an index for storing embeddings. Ensure the embedding dimension matches your model. ```sql CREATE EXTENSION IF NOT EXISTS vector; CREATE EXTENSION IF NOT EXISTS hstore; CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; CREATE TABLE IF NOT EXISTS vector_store ( id uuid DEFAULT uuid_generate_v4() PRIMARY KEY, content text, metadata json, embedding vector(1536) // 1536 is the default embedding dimension ); CREATE INDEX ON vector_store USING HNSW (embedding vector_cosine_ops); ``` -------------------------------- ### Basic PromptTemplate Usage Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/prompt.html Create a simple prompt template with placeholders and generate a prompt instance. ```java PromptTemplate promptTemplate = new PromptTemplate("Tell me a {adjective} joke about {topic}"); Prompt prompt = promptTemplate.create(Map.of("adjective", adjective, "topic", topic)); return chatModel.call(prompt).getResult(); ``` -------------------------------- ### Elasticsearch Filter Expression Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/elasticsearch.html Illustrates a portable filter expression that will be converted into Elasticsearch's proprietary filter format. This example shows a common filtering pattern for author and article type. ```string author in ['john', 'jill'] && 'article_type' == 'blog' ``` -------------------------------- ### Basic OAuth 2.0 Setup for MCP Server Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-security.html Configure basic OAuth 2.0 security for the MCP server using Spring Security. This setup enforces authentication on every request and requires the issuer URI. ```java @Configuration @EnableWebSecurity class McpServerConfiguration { @Value("${spring.security.oauth2.resourceserver.jwt.issuer-uri}") private String issuerUrl; @Bean SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception { return http // Enforce authentication with token on EVERY request .authorizeHttpRequests(auth -> auth.anyRequest().authenticated()) // Configure OAuth2 on the MCP server .with( McpServerOAuth2Configurer.mcpServerOAuth2(), (mcpAuthorization) -> { // REQUIRED: the issuerURI mcpAuthorization.authorizationServer(issuerUrl); // OPTIONAL: enforce the `aud` claim in the JWT token. // Not all authorization servers support resource indicators, // so it may be absent. Defaults to `false`. // See RFC 8707 Resource Indicators for OAuth 2.0 // https://www.rfc-editor.org/rfc/rfc8707.html mcpAuthorization.validateAudienceClaim(true); } ) .build(); } } ``` -------------------------------- ### Custom ToolExecutionEligibilityChecker Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/tools/tool-calling-advisor.html Example of overriding the default ToolExecutionEligibilityChecker for provider-specific logic. ```APIDOC ## Custom ToolExecutionEligibilityChecker ### Description Allows customization of the condition that triggers the next tool-call iteration. ### `ToolExecutionEligibilityChecker` Interface ```java @FunctionalInterface public interface ToolExecutionEligibilityChecker { boolean isToolCallResponse(@Nullable ChatResponse chatResponse); } ``` ### Request Example ```java ToolExecutionEligibilityChecker strictChecker = response -> response != null && response.hasToolCalls() && "tool_calls".equals(response.getMetadata().getFinishReason()); var advisor = ToolCallingAdvisor.builder() .toolExecutionEligibilityChecker(strictChecker) .build(); ``` ``` -------------------------------- ### TextToSpeechPrompt Example Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/audio/speech.html Encapsulates the input text and options for a text-to-speech request. ```java TextToSpeechPrompt prompt = new TextToSpeechPrompt( "Hello, this is a text-to-speech example.", options ); ``` -------------------------------- ### Manual OpenAI Speech Model Configuration Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/audio/speech/openai-speech.html Manually configure and instantiate an `OpenAiAudioSpeechModel`. This example shows setting the API key, defining speech options, and making a text-to-speech call, including accessing response metadata and output bytes. ```java var openAiAudioApi = new OpenAiAudioApi() .apiKey(System.getenv("OPENAI_API_KEY")) .build(); var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi); var speechOptions = OpenAiAudioSpeechOptions.builder() .responseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3) .speed(1.0) .model(OpenAiAudioApi.TtsModel.GPT_4_O_MINI_TTS.value) .build(); var speechPrompt = new TextToSpeechPrompt("Hello, this is a text-to-speech example.", speechOptions); TextToSpeechResponse response = openAiAudioSpeechModel.call(speechPrompt); // Accessing metadata (rate limit info) OpenAiAudioSpeechResponseMetadata metadata = (OpenAiAudioSpeechResponseMetadata) response.getMetadata(); byte[] responseAsBytes = response.getResult().getOutput(); ``` -------------------------------- ### Implement Streaming Advisor with Flux Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/advisors.html This example demonstrates how to implement a streaming advisor using Project Reactor's Flux and Mono. It shows the structure for processing a request before passing it to the next advisor in the chain and handling the response afterwards. Use this pattern for advisors that need to process data streams. ```java @Override public Flux adviseStream(ChatClientRequest chatClientRequest, StreamAdvisorChain chain) { return Mono.just(chatClientRequest) .publishOn(Schedulers.boundedElastic()) .map(request -> { // This can be executed by blocking and non-blocking Threads. // Advisor before next section }) .flatMapMany(request -> chain.nextStream(request)) .map(response -> { // Advisor after next section }); } ``` -------------------------------- ### Get All Voices Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/audio/speech/elevenlabs-speech.html Retrieves a list of all available voices from the ElevenLabs API. ```APIDOC ## Get All Voices ### Description Retrieves a list of all available voices. ### Method GET ### Endpoint /voices ### Parameters None ### Request Example ```java ResponseEntity voicesResponse = voicesApi.getVoices(); List voices = voicesResponse.getBody().voices(); ``` ### Response #### Success Response (200) - **voices** (List) - A list of available voices. #### Response Example ```json { "voices": [ { "voiceId": "21m00Tcm4T8xjW01u80U", "name": "Rachel", "samples": [], "category": "generated", "fineTuning": { "is_allowed_to_fine_tune": true, "language": "en" }, "labels": { "accent": "american", "age": "young", "gender": "female", "use_case": "narration" } } ] } ``` ``` -------------------------------- ### Get Voice Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/audio/speech/elevenlabs-speech.html Retrieves metadata about a specific voice using its voice ID. ```APIDOC ## Get Voice ### Description Returns metadata about a specific voice. ### Method GET ### Endpoint /voices/{voice_id} ### Parameters #### Path Parameters - **voiceId** (String) - Required - The ID of the voice to retrieve metadata for. ### Request Example ```java ResponseEntity voiceDetailsResponse = voicesApi.getVoice(voiceId); ElevenLabsVoicesApi.Voice voiceDetails = voiceDetailsResponse.getBody(); ``` ### Response #### Success Response (200) - **voiceId** (String) - The ID of the voice. - **name** (String) - The name of the voice. - **samples** (List) - A list of voice samples. - **category** (String) - The category of the voice. - **fineTuning** (Object) - Fine-tuning information. - **labels** (Object) - Labels associated with the voice. #### Response Example ```json { "voiceId": "21m00Tcm4T8xjW01u80U", "name": "Rachel", "samples": [], "category": "generated", "fineTuning": { "is_allowed_to_fine_tune": true, "language": "en" }, "labels": { "accent": "american", "age": "young", "gender": "female", "use_case": "narration" } } ``` ``` -------------------------------- ### Basic Milvus Vector Store Usage Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/milvus.html Demonstrates adding documents and performing similarity searches using the auto-configured Milvus Vector Store. ```java @Autowired VectorStore vectorStore; // ... List documents = List.of( new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")), new Document("The World is Big and Salvation Lurks Around the Corner"), new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2"))); // Add the documents to Milvus Vector Store vectorStore.add(documents); // Retrieve documents similar to a query List results = this.vectorStore.similaritySearch(SearchRequest.builder().query("Spring").topK(5).build()); ```