Spring AI

https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/
Admin
Spring AI streamlines the development of AI-powered applications by providing portable APIs for...
Tokens:432,203
Snippets:3,596
Trust Score:10
Update:1 month ago
Show doc for...
Tokens:
Raw
### Example AI Formatting Instructions

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/structured-output/converters.html

A sample set of instructions used to guide an AI model to produce RFC8259 compliant JSON matching a specific Java class.

```text
  Your response should be in JSON format.
  The data structure for the JSON should match this Java class: java.util.HashMap
  Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
```

--------------------------------

### Implement Role Prompting with Style Instructions (pt_role_prompting_2)

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/prompt-engineering-patterns.html

Enhances the role prompting technique by requesting the travel guide to provide suggestions in a humorous style. This example shows how adding style instructions to a system prompt can affect the model's output, achieving a consistent tone for engaging interactions.

```java
// Implementation of Section 2.3.2: Role prompting with style instructions
public void pt_role_prompting_2(ChatClient chatClient) {
    String humorousTravelSuggestions = chatClient
            .prompt()
            .system("""
                    I want you to act as a travel guide. I will write to you about
                    my location and you will suggest 3 places to visit near me in
                    a humorous style.
                    """)
            .user("""
                    My suggestion: "I am in Amsterdam and I want to visit only museums."
                    Travel Suggestions:
                    """)
            .call()
            .content();
}
```

--------------------------------

### Start Milvus with Docker Compose

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/milvus.html

Run from the src/test/resources/ folder to start the Milvus store. After startup, connect to the vector store at http://localhost:19530 or management at http://localhost:9001 (user: minioadmin, pass: minioadmin).

```bash
docker-compose up

```

--------------------------------

### Create OpenAiAudioTranscriptionModel

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/audio/transcriptions/openai-transcriptions.html

Create an OpenAiAudioTranscriptionModel with options, then call it with an audio file to get a transcription response. The example uses an API key from the environment variable OPENAI_API_KEY, sets response format to TEXT, and temperature to 0.

```java
var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .responseFormat(AudioResponseFormat.TEXT)
    .temperature(0f)
    .build();

var openAiAudioTranscriptionModel = OpenAiAudioTranscriptionModel.builder()
    .options(transcriptionOptions)
    .build();

var audioFile = new FileSystemResource("/path/to/your/resource/speech/jfk.flac");

AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
AudioTranscriptionResponse response = openAiAudioTranscriptionModel.call(transcriptionRequest);
```

--------------------------------

### Complete example: citation document, call, and processing

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/anthropic-chat.html

A self-contained example showing how to create a citation document, call the model with it, and process the citations in the response, including a check for null or empty citations.

```java
// Create a citation document
AnthropicCitationDocument document = AnthropicCitationDocument.builder()
    .plainText("Spring AI is an application framework for AI engineering. " +
               "It provides a Spring-friendly API for developing AI applications. " +
               "The framework includes abstractions for chat models, embedding models, " +
               "and vector databases.")
    .title("Spring AI Overview")
    .citationsEnabled(true)
    .build();

// Call the model with the document
ChatResponse response = chatModel.call(
    new Prompt(
        "What is Spring AI?",
        AnthropicChatOptions.builder()
            .model("claude-sonnet-4-20250514")
            .maxTokens(1024)
            .citationDocuments(document)
            .build()
    )
);

// Display the response
System.out.println("Response: " + response.getResult().getOutput().getText());
System.out.println("\nCitations:");

// Process citations
List<Citation> citations = (List<Citation>) response.getMetadata().get("citations");

if (citations != null && !citations.isEmpty()) {
    for (int i = 0; i < citations.size(); i++) {
        Citation citation = citations.get(i);
        System.out.println("\n[" + (i + 1) + "] " + citation.getDocumentTitle());
        System.out.println("    Location: " + citation.getLocationDescription());
        System.out.println("    Text: " + citation.getCitedText());
    }
} else {
    System.out.println("No citations were provided in the response.");
}
```

--------------------------------

### Example Output

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/guides/llm-as-judge.html

Example console output showing the evaluation retry process with tool calls and feedback.

```text
REQUEST: [{"role":"user","content":"What is current weather in Paris?"}]

>>> Tool Call responseTemp: -255
Evaluation failed on attempt 1, evaluation: The response contains unrealistic temperature data, feedback: The temperature of -255°C is physically impossible and indicates a data error.

>>> Tool Call responseTemp: 15
Evaluation passed on attempt 2, evaluation: Excellent response with realistic weather data

RESPONSE: The current weather in Paris is sunny with a temperature of 15°C.
```

--------------------------------

### ChatClient Video Response Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html

Example output generated by the video analysis prompt.

```text
The video shows a group of baby chickens, also known as chicks, huddled together on a surface
...
```

--------------------------------

### KeywordMetadataEnricher Usage Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html

A complete example showing initialization, configuration with custom templates, and retrieving the 'excerpt_keywords' metadata field.

```java
ChatModel chatModel = // initialize your chat model
KeywordMetadataEnricher enricher = KeywordMetadataEnricher.builder(chatModel)
                .keywordCount(5)
                .build();

// Or use custom templates
KeywordMetadataEnricher enricher = KeywordMetadataEnricher.builder(chatModel)
                .keywordsTemplate(new PromptTemplate("Extract 5 important keywords from the following text and separate them with commas:\n{context_str}"))
                .build();

Document doc = new Document("This is a document about artificial intelligence and its applications in modern technology.");

List<Document> enrichedDocs = enricher.apply(List.of(this.doc));

Document enrichedDoc = this.enrichedDocs.get(0);
String keywords = (String) this.enrichedDoc.getMetadata().get("excerpt_keywords");
System.out.println("Extracted keywords: " + keywords);
```

--------------------------------

### Streamable HTTP transport connection configuration example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-client-boot-starter-docs.html

Example YAML configuration for Streamable HTTP transport connections, showing default and custom endpoint usage.

```yaml
spring:
  ai:
    mcp:
      client:
        streamable-http:
          connections:
            server1:
              url: http://localhost:8080
            server2:
              url: http://otherserver:8081
              endpoint: /custom-sse

```

--------------------------------

### Deprecated tools() method example (M7)

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Example of tool registration using the deprecated `tools()` method that worked in M7 but silently fails in M8. Use this to identify code that needs updating.

```java
// This worked in M7 but silently fails in M8
ChatClient chatClient = new OpenAiChatClient(api)
    .tools(List.of(
        new Tool("get_current_weather", "Get the current weather in a given location",
            new ToolSpecification.ToolParameter("location", "The city and state, e.g. San Francisco, CA", true))
    ))
    .toolCallbacks(List.of(
        new ToolCallback("get_current_weather", (toolName, params) -> {
            // Weather retrieval logic
            return Map.of("temperature", 72, "unit", "fahrenheit", "description", "Sunny");
        })
    ));
```

--------------------------------

### Custom SimpleLoggerAdvisor example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chatclient.html

Example usage of the SimpleLoggerAdvisor constructor with custom request and response formatting. Be cautious about logging sensitive information in production.

```java
SimpleLoggerAdvisor customLogger = new SimpleLoggerAdvisor(
    request -> "Custom request: " + request.prompt().getUserMessage(),
    response -> "Custom response: " + response.getResult(),
    0
);
```

--------------------------------

### SSE transport connection configuration example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-client-boot-starter-docs.html

Example YAML configuration for SSE transport connections, showing simple, custom endpoint, complex URL with path and token, and SSE endpoint with query parameters.

```yaml
spring:
  ai:
    mcp:
      client:
        sse:
          connections:
            # Simple configuration using default /sse endpoint
            server1:
              url: http://localhost:8080
            # Custom SSE endpoint
            server2:
              url: http://otherserver:8081
              sse-endpoint: /custom-sse
            # Complex URL with path and token (like MCP Hub)
            mcp-hub:
              url: http://localhost:3000
              sse-endpoint: /mcp-hub/sse/cf9ec4527e3c4a2cbb149a85ea45ab01
            # SSE endpoint with query parameters
            api-server:
              url: https://api.example.com
              sse-endpoint: /v1/mcp/events?token=abc123&format=json

```

--------------------------------

### Using S3VectorStore in application

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/s3-vector-store.html

Example of auto-wiring S3VectorStore, adding documents, and performing a similarity search.

```java
@Autowired VectorStore vectorStore;

// ...

List <Document> documents = List.of(
    new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
    new Document("The World is Big and Salvation Lurks Around the Corner"),
    new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

// Add the documents to S3 Vector Store Bucket
vectorStore.add(documents);

// Retrieve documents similar to a query
List<Document> results = this.vectorStore.similaritySearch(SearchRequest.builder().query("Spring").topK(5).build());
```

--------------------------------

### ChatClient.builder() with ToolCallingAdvisor

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Example of using the new ChatClient.builder() overload with a custom ToolCallingAdvisor.

```java
ChatClient chatClient = ChatClient
    .builder(chatModel, observationRegistry, null, null,
            ToolCallingAdvisor.builder().toolCallingManager(myToolCallingManager))
    .build();
Copied!
```

--------------------------------

### New JSON Usage Format

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Example of the updated JSON structure for Usage, using completionTokens instead of generationTokens.

```json
{
  "promptTokens": 100,
  "completionTokens": 50,
  "totalTokens": 150
}
```

--------------------------------

### Replace internalCall and internalStream with call and stream

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Migration example showing replacement of internalCall and internalStream with call and stream. The internal methods are now private and direct calls will fail to compile.

```java
// Before
ChatResponse response = model.internalCall(prompt, previousChatResponse);
Flux<ChatResponse> responseFlux = model.internalStream(prompt, previousChatResponse);

// After
ChatResponse response = model.call(prompt);
Flux<ChatResponse> responseFlux = model.stream(prompt);
```

--------------------------------

### Registering ToolCallback Bean with tools()

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Example of registering a ToolCallback bean directly with the tools() method.

```java
chatClient.prompt("What's the weather like in Copenhagen?")
    .tools(currentWeather)
    .call().content();
Copied!
```

--------------------------------

### Migration: ToolSpec Consumer API Removal

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Migration example showing the removal of ToolSpec consumer API and the new tools() and toolContext() methods.

```java
// Before
chatClient.prompt()
    .tools(t -> t.callbacks(myCallback).context("tenantId", "acme"))
    .call().content();

// After
chatClient.prompt()
    .tools(myCallback)
    .toolContext(Map.of("tenantId", "acme"))
    .call().content();
Copied!
```

--------------------------------

### ChatClient Options: Pass ChatOptions.Builder (Before and After Example)

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Documents the breaking change in ChatClient.options()/.defaultOptions(): they now require a ChatOptions.Builder. Before shows the old working pattern that no longer compiles; after shows the new passing directly. The builder is merged with the model defaults before the first advisor call.

```java
// Before — passing a built ChatOptions instance (no longer compiles with ChatClient)
ChatOptions opts = AnthropicChatOptions.builder()
    .maxTokens(100)
    .temperature(0.7)
    .build();
String response = chatClient.prompt("Tell me a joke")
    .options(opts)
    .call().content();

// After — pass the builder directly
String response = chatClient.prompt("Tell me a joke")
    .options(AnthropicChatOptions.builder()
        .maxTokens(100)
        .temperature(0.7))
    .call().content();
```

--------------------------------

### Migrate ModelOptionsUtils JSON methods to JsonHelper

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Migration example for replacing removed ModelOptionsUtils JSON methods with JsonHelper and JacksonUtils equivalents. The before block shows the old API; the after block shows the new usage with a static JsonHelper instance.

```java
// Before
import org.springframework.ai.model.ModelOptionsUtils;

Map<String, Object> map = ModelOptionsUtils.jsonToMap(jsonString);
String json = ModelOptionsUtils.toJsonString(myObject);
JsonMapper mapper = ModelOptionsUtils.JSON_MAPPER;

// After
import org.springframework.ai.util.JsonHelper;
import org.springframework.ai.util.JacksonUtils;

private static final JsonHelper jsonHelper = new JsonHelper();

Map<String, Object> map = jsonHelper.fromJsonToMap(jsonString);
String json = jsonHelper.toJson(myObject);
JsonMapper mapper = JacksonUtils.getDefaultJsonMapper();
```

--------------------------------

### Migration: SpringBeanToolCallbackResolver Removal

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Migration example showing how to replace bare functional beans with explicit ToolCallback beans.

```java
// Before — bare Function bean resolved by name at runtime
@Bean
@Description("Get the weather in location")
Function<WeatherRequest, WeatherResponse> currentWeather() {
    return weatherService::getWeather;
}

// After — explicit ToolCallback bean
@Bean
ToolCallback currentWeather() {
    return FunctionToolCallback.builder("currentWeather", weatherService::getWeather)
        .description("Get the weather in location")
        .inputType(WeatherRequest.class)
        .build();
}
Copied!
```

--------------------------------

### JsonReader get() with JSON Pointer example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html

Example of using the get(String pointer) method to retrieve a specific element from a JSON document. The pointer "/store/books/0" navigates to the first book in the store array.

```java
JsonReader jsonReader = new JsonReader(resource, "description");
List<Document> documents = this.jsonReader.get("/store/books/0");
```

--------------------------------

### Complete Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/guides/llm-as-judge.html

Complete Spring Boot application integrating SelfRefineEvaluationAdvisor with Anthropic for generation and Ollama for evaluation, including a weather tool that returns randomized temperatures.

```java
@SpringBootApplication
public class EvaluationAdvisorDemoApplication {

    @Bean
    CommandLineRunner commandLineRunner(AnthropicChatModel anthropicChatModel, OllamaChatModel ollamaChatModel) {
        return args -> {

            ChatClient chatClient = ChatClient.builder(anthropicChatModel)
                    .defaultTools(new MyTools())
                    .defaultAdvisors(

                        SelfRefineEvaluationAdvisor.builder()
                            .chatClientBuilder(ChatClient.builder(ollamaChatModel)) // Separate model for evaluation
                            .maxRepeatAttempts(15)

            var answer = chatClient
                .prompt("What is current weather in Paris?")
                .call()
                .content();

            System.out.println(answer);
        };
    }

    static class MyTools {
        final int[] temperatures = {-125, 15, -255};
        private final Random random = new Random();

        @Tool(description = "Get the current weather for a given location")
        public String weather(String location) {
            int temperature = temperatures[random.nextInt(temperatures.length)];
            System.out.println(">>> Tool Call responseTemp: " + temperature);
            return "The current weather in " + location + " is sunny with a temperature of " + temperature + "°C.";
        }
    }
}
```

--------------------------------

### Get Full ChatResponse with .responseEntity()

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/structured-output.html

Use .responseEntity() to get both the parsed entity and the underlying ChatResponse. The example shows retrieving the entity, the raw response, and the total token usage from the response metadata.

```java
ResponseEntity<ChatResponse, ActorsFilms> result = chatClient.prompt()
    .user("Generate the filmography for a random actor.")
    .call()
    .responseEntity(ActorsFilms.class);

ActorsFilms films = result.entity();
ChatResponse raw = result.response();
long totalTokens = raw.getMetadata().getUsage().getTotalTokens();
```

--------------------------------

### SYSTEM_AND_TOOLS Cache Strategy Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html

Demonstrates how changing system prompts or tools affects cache hits and misses using the SYSTEM_AND_TOOLS strategy.

```java
// Request 1: Cache both tools and system
ChatResponse r1 = chatModel.call(
    new Prompt(
        List.of(new SystemMessage("System prompt"), new UserMessage("Question")),
        BedrockChatOptions.builder()
            .cacheOptions(BedrockCacheOptions.builder()
                .strategy(BedrockCacheStrategy.SYSTEM_AND_TOOLS)
                .build())
            .toolCallbacks(tools)
            .build()
    )
);
// Result: Both caches created

// Request 2: Change only system prompt (tools same)
ChatResponse r2 = chatModel.call(
    new Prompt(
        List.of(new SystemMessage("DIFFERENT system prompt"), new UserMessage("Question")),
        BedrockChatOptions.builder()
            .cacheOptions(BedrockCacheOptions.builder()
                .strategy(BedrockCacheStrategy.SYSTEM_AND_TOOLS)
                .build())
            .toolCallbacks(tools) // SAME tools
            .build()
    )
);
// Result: Tools cache HIT (reused), system cache MISS (recreated)

// Request 3: Change tools (system same as Request 2)
ChatResponse r3 = chatModel.call(
    new Prompt(
        List.of(new SystemMessage("DIFFERENT system prompt"), new UserMessage("Question")),
        BedrockChatOptions.builder()
            .cacheOptions(BedrockCacheOptions.builder()
                .strategy(BedrockCacheStrategy.SYSTEM_AND_TOOLS)
                .build())
            .toolCallbacks(newTools) // DIFFERENT tools
            .build()
    )
);
// Result: BOTH caches MISS (tools change invalidates everything downstream)
```

--------------------------------

### Configure QdrantVectorStore via application.yml

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/qdrant.html

Provides a sample application.yml configuration for connecting to a Qdrant instance, including host, port, API key, collection name, and schema initialization.

```yaml
spring:
  ai:
    vectorstore:
      qdrant:
        host: <qdrant host>
        port: <qdrant grpc port>
        api-key: <qdrant api key>
        collection-name: <collection name>
        content-field-name: <content field name>
        use-tls: false
        initialize-schema: true
```

--------------------------------

### CallToolRequest with progress token

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-special-params.html

Shows how to use CallToolRequest together with @McpProgressToken and McpSyncServerExchange to send progress notifications while processing dynamic arguments. The example sends progress at the start and end of processing.

```java
@McpTool(name = "flexible-with-progress", description = "Flexible tool with progress")
public CallToolResult flexibleWithProgress(
        @McpProgressToken String progressToken,
        CallToolRequest request,
        McpSyncServerExchange exchange) {

    Map<String, Object> args = request.arguments();

    if (progressToken != null) {
        exchange.progressNotification(ProgressNotification.builder(progressToken, 0.0)
            .total(1.0).message("Processing dynamic request").build());
    }

    // Process dynamic arguments
    String result = processDynamicArgs(args);

    if (progressToken != null) {
        exchange.progressNotification(ProgressNotification.builder(progressToken, 1.0)
            .total(1.0).message("Complete").build());
    }

    return CallToolResult.builder()
        .addTextContent(result)
        .build();
}
```

--------------------------------

### Stateless Server Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-examples.html

A Java class demonstrating stateless MCP tools, resources, and prompts using Spring AI annotations.

```java
@Component
public class StatelessTools {

    // Simple stateless tool
    @McpTool(name = "format-text", description = "Format text")
    public String formatText(
            @McpToolParam(description = "Text to format", required = true) String text,
            @McpToolParam(description = "Format type", required = true) String format) {

        return switch (format.toLowerCase()) {
            case "uppercase" -> text.toUpperCase();
            case "lowercase" -> text.toLowerCase();
            case "title" -> toTitleCase(text);
            case "reverse" -> new StringBuilder(text).reverse().toString();
            default -> text;
        };
    }

    // Stateless with transport context
    @McpTool(name = "validate-json", description = "Validate JSON")
    public CallToolResult validateJson(
            McpTransportContext context,
            @McpToolParam(description = "JSON string", required = true) String json) {

        try {
            JsonMapper mapper = new JsonMapper();
            mapper.readTree(json);

            return CallToolResult.builder()
                .addTextContent("Valid JSON")
                .structuredContent(Map.of("valid", true))
                .build();
        } catch (JacksonException e) {
            return CallToolResult.builder()
                .addTextContent("Invalid JSON: " + e.getMessage())
                .structuredContent(Map.of("valid", false, "error", e.getMessage()))
                .build();
        }
    }

    @McpResource(uri = "static://{path}", name = "Static Resource")
    public String getStaticResource(String path) {
        // Simple stateless resource
        return loadStaticContent(path);
    }

    @McpPrompt(name = "template", description = "Template prompt")
    public GetPromptResult templatePrompt(
            @McpArg(name = "template", required = true) String templateName,
            @McpArg(name = "variables", required = false) String variables) {

        String template = loadTemplate(templateName);
        if (variables != null) {
            template = substituteVariables(template, variables);
        }

        return GetPromptResult.builder(List.of(new PromptMessage(Role.USER, TextContent.builder(template).build())))
            .description("Template: " + templateName)
            .build();
    }
}

```

--------------------------------

### After: Prompt-level options must be full or null

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/anthropic-migration.html

Shows the new behavior where prompt-level options are used as-is, so they must be complete or null. The example uses chatModel.getOptions().mutate() to start from the model defaults.

```java
// After: prompt-level options must be "full", or null.
Prompt prompt = new Prompt(
    "Tell me a joke",
    chatModel.getOptions().mutate().maxTokens(2048).build());
chatModel.call(prompt);
Copied!
```

--------------------------------

### Migrate @McpTool checked exception to RuntimeException

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

For @McpTool methods that previously relied on checked exceptions being converted to error results, wrap the exception in a RuntimeException to convey the error to the model. The example shows before and after versions of a lookupOrder method.

```java
// Before — checked exception was silently converted to an error result
@McpTool(description = "Look up an order")
public String lookupOrder(String orderId) throws NotFoundException {
    throw new NotFoundException("Order not found: " + orderId);
}

// After — wrap in RuntimeException to convey error to model,
//         or remove the checked-exception declaration and let it propagate
@McpTool(description = "Look up an order")
public String lookupOrder(String orderId) {
    throw new RuntimeException("Order not found: " + orderId); // reaches model as error result
}
Copied!
```

--------------------------------

### Build a Prompt with SystemPromptTemplate and UserMessage

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/prompt.html

Shows how to create a system message with placeholders using SystemPromptTemplate and combine it with a user message to form a Prompt, then call the ChatModel to get a list of generations.

```java
String userText = """
    Tell me about three famous pirates from the Golden Age of Piracy and why they did.
    Write at least a sentence for each pirate.
    """;

Message userMessage = new UserMessage(userText);

String systemText = """
  You are a helpful AI assistant that helps people find information.
  Your name is {name}
  You should reply to the user's request with your name and also in the style of a {voice}.
  """;

SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemText);
Message systemMessage = systemPromptTemplate.createMessage(Map.of("name", name, "voice", voice));

Prompt prompt = new Prompt(List.of(userMessage, systemMessage));

List<Generation> response = chatModel.call(prompt).getResults();
```

--------------------------------

### Get cumulative usage from ChatClient tool-calling loop

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/usage-handling.html

In a multi-step flow like a tool-calling loop, `getUsage()` returns cumulative token usage across all model calls. The example shows a `ChatClient` prompt with a tool, then retrieves the total token count. The cumulative total is computed by `org.springframework.ai.support.UsageCalculator`.

```java
ChatResponse response = chatClient.prompt("What is the weather in Paris?")
        .tools(new WeatherTools())
        .call()
        .chatResponse();

// Cumulative across all model calls in the tool-calling loop
Usage usage = response.getMetadata().getUsage();
int totalTokens = usage.getTotalTokens();
```

--------------------------------

### Migrate HttpClient transport customizer to McpClientCustomizer

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Migrate HttpClient transport customization from McpSyncHttpClientRequestCustomizer / McpAsyncHttpClientRequestCustomizer beans to McpClientCustomizer<HttpClientSseClientTransport.Builder>. The target builder type depends on the transport (SSE vs. Streamable HTTP); this example shows the SSE transport.

```java
// Before
@Bean
public McpSyncHttpClientRequestCustomizer myRequestCustomizer() {
    return requestBuilder -> requestBuilder.header("Authorization", "Bearer token");
}

// After
@Bean
public McpClientCustomizer<HttpClientSseClientTransport.Builder> mySseTransportCustomizer() {
    return (name, builder) -> builder.httpRequestCustomizer(
        req -> req.header("Authorization", "Bearer token")
    );
}
```

--------------------------------

### Spring AI Integration with MCP Tools

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-examples.html

Example showing MCP tools integrated with Spring AI's function calling. Includes a ChatController that uses SyncMcpToolCallbackProvider to get MCP tools as Spring AI function callbacks, and a WeatherTools component with @McpTool annotations for weather and forecast.

```java
@RestController
@RequestMapping("/chat")
public class ChatController {

    private final ChatModel chatModel;
    private final SyncMcpToolCallbackProvider toolCallbackProvider;

    public ChatController(ChatModel chatModel,
                          SyncMcpToolCallbackProvider toolCallbackProvider) {
        this.chatModel = chatModel;
        this.toolCallbackProvider = toolCallbackProvider;
    }

    @PostMapping
    public ChatResponse chat(@RequestBody ChatRequest request) {
        // Get MCP tools as Spring AI function callbacks
        ToolCallback[] mcpTools = toolCallbackProvider.getToolCallbacks();

        // Create prompt with MCP tools
        Prompt prompt = new Prompt(
            request.getMessage(),
            ChatOptionsBuilder.builder()
                .withTools(mcpTools)
                .build()
        );

        // Call chat model with MCP tools available
        return chatModel.call(prompt);
    }
}

@Component
public class WeatherTools {

    @McpTool(name = "get-weather", description = "Get current weather")
    public WeatherInfo getWeather(
            @McpToolParam(description = "City name", required = true) String city,
            @McpToolParam(description = "Units (metric/imperial)", required = false) String units) {

        String unit = units != null ? units : "metric";

        // Call weather API
        return weatherService.getCurrentWeather(city, unit);
    }

    @McpTool(name = "get-forecast", description = "Get weather forecast")
    public ForecastInfo getForecast(
            @McpToolParam(description = "City name", required = true) String city,
            @McpToolParam(description = "Days (1-7)", required = false) Integer days) {

        int forecastDays = days != null ? days : 3;

        return weatherService.getForecast(city, forecastDays);
    }
}
Copied!

```

--------------------------------

### Prompt caching with cost tracking using BedrockCacheOptions

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html

Complete example of prompt caching with cost tracking using BedrockChatOptions and BedrockCacheOptions. It sends two requests with the same system prompt to demonstrate cache creation and cache hit, then reads cacheWriteInputTokens and cacheReadInputTokens from the metadata Map. The system prompt must be 1024+ tokens for cache effectiveness.

```java
// Create system content that will be reused multiple times
String largeSystemPrompt = "You are an expert software architect specializing in distributed systems...";
// (Ensure this is 1024+ tokens for cache effectiveness)

// First request - creates cache
ChatResponse firstResponse = chatModel.call(
    new Prompt(
        List.of(
            new SystemMessage(largeSystemPrompt),
            new UserMessage("What is microservices architecture?")
        ),
        BedrockChatOptions.builder()
            .model("us.anthropic.claude-haiku-4-5-20251001-v1:0")
            .cacheOptions(BedrockCacheOptions.builder()
                .strategy(BedrockCacheStrategy.SYSTEM_ONLY)
                .build())
            .maxTokens(500)
            .build()
    )
);

// Access cache-related token usage from metadata
Integer cacheWrite1 = (Integer) firstResponse.getMetadata()
    .getMetadata()
    .get("cacheWriteInputTokens");
Integer cacheRead1 = (Integer) firstResponse.getMetadata()
    .getMetadata()
    .get("cacheReadInputTokens");

System.out.println("Cache creation tokens: " + cacheWrite1);
System.out.println("Cache read tokens: " + cacheRead1);

// Second request with same system prompt - reads from cache
ChatResponse secondResponse = chatModel.call(
    new Prompt(
        List.of(
            new SystemMessage(largeSystemPrompt),  // Same prompt - cache hit
            new UserMessage("What are the benefits of event sourcing?")
        ),
        BedrockChatOptions.builder()
            .model("us.anthropic.claude-haiku-4-5-20251001-v1:0")
            .cacheOptions(BedrockCacheOptions.builder()
                .strategy(BedrockCacheStrategy.SYSTEM_ONLY)
                .build())
            .maxTokens(500)
            .build()
    )
);

Integer cacheWrite2 = (Integer) secondResponse.getMetadata()
    .getMetadata()
    .get("cacheWriteInputTokens");
Integer cacheRead2 = (Integer) secondResponse.getMetadata()
    .getMetadata()
    .get("cacheReadInputTokens");

System.out.println("Cache creation tokens: " + cacheWrite2); // Should be 0
System.out.println("Cache read tokens: " + cacheRead2);      // Should be > 0
Copied!
```

--------------------------------

### Create documents

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html

In your main code, create some documents. The example shows a list of Document objects with optional metadata.

```java
List<Document> documents = List.of(
 new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
 new Document("The World is Big and Salvation Lurks Around the Corner"),
 new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));
```

--------------------------------

### PagePdfDocumentReader Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html

Example implementation of PagePdfDocumentReader to extract text from a PDF file.

```java
@Component
public class MyPagePdfDocumentReader {

	List<Document> getDocsFromPdf() {

		PagePdfDocumentReader pdfReader = new PagePdfDocumentReader("classpath:/sample1.pdf",
				PdfDocumentReaderConfig.builder()
					.withPageTopMargin(0)
					.withPageExtractedTextFormatter(ExtractedTextFormatter.builder()
						.withNumberOfTopTextLinesToDelete(0)
						.build())
					.withPagesPerDocument(1)
					.build());

		return pdfReader.read();
    }

}
```

--------------------------------

### Standard STDIO Server Configuration (spring-ai-starter-mcp-server)

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-stdio-sse-server-boot-starter-docs.html

Configure a standard STDIO MCP server using the spring-ai-starter-mcp-server starter. Sets the server name, version, and synchronous type.

```yaml
# Using spring-ai-starter-mcp-server
spring:
  ai:
    mcp:
      server:
        name: stdio-mcp-server
        version: 1.0.0
        type: SYNC

```

--------------------------------

### Sample Output from MCP Client

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-annotations-examples.html

Example output when running the MCP client, showing user prompt and assistant responses from multiple LLM providers, including weather data.

```text
> USER: What is the weather in Amsterdam right now?
Please incorporate all creative responses from all LLM providers.
After the other providers add a poem that synthesizes the poems from all the other providers.

> ASSISTANT:
OpenAI poem about the weather:
**Amsterdam's Winter Whisper**
*Temperature: 4.2°C*

In Amsterdam's embrace, where canals reflect the sky,
A gentle chill of 4.2 degrees drifts by...

Anthropic poem about the weather:
**Canal-Side Contemplation**
*Current conditions: 4.2°C*

Along the waterways where bicycles rest,
The winter air puts Amsterdam to test...

Weather Data:
{
  "current": {
    "time": "2025-01-23T11:00",
    "interval": 900,
    "temperature_2m": 4.2
  }
}
Copied!

```

--------------------------------

### ChatClient Document Response Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html

Example output generated by the document summarization prompt.

```text
**Introduction:**
- Spring AI is designed to simplify the development of applications with artificial intelligence (AI) capabilities, aiming to avoid unnecessary complexity.
...
```

--------------------------------

### ChatClient Image Response Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/chat/bedrock-converse.html

Example output generated by the image analysis prompt.

```text
The image shows a close-up view of a wire fruit basket containing several pieces of fruit.
...
```

--------------------------------

### Run OpenRewrite M3MigrateMcpClientCustomizer recipe

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/upgrade-notes.html

Run the OpenRewrite recipe M3MigrateMcpClientCustomizer to automate import and type changes for the MCP client customizer migration. Use the provided configLocation and set -Dmaven.compiler.failOnError=false to avoid build failures during migration.

```bash
mvn org.openrewrite.maven:rewrite-maven-plugin:6.32.0:run \
  -Drewrite.configLocation=https://raw.githubusercontent.com/spring-projects/spring-ai/refs/heads/main/src/rewrite/migrate-to-2-0-0-M3.yaml \
  -Drewrite.activeRecipes=org.springframework.ai.migration.M3MigrateMcpClientCustomizer \
  -Dmaven.compiler.failOnError=false
```

--------------------------------

### ParagraphPdfDocumentReader Example

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/etl-pipeline.html

Example implementation of ParagraphPdfDocumentReader to split a PDF into paragraphs using the PDF catalog.

```java
@Component
public class MyPagePdfDocumentReader {

	List<Document> getDocsFromPdfWithCatalog() {

        ParagraphPdfDocumentReader pdfReader = new ParagraphPdfDocumentReader("classpath:/sample1.pdf",
                PdfDocumentReaderConfig.builder()
                    .withPageTopMargin(0)
                    .withPageExtractedTextFormatter(ExtractedTextFormatter.builder()
                        .withNumberOfTopTextLinesToDelete(0)
                        .build())
                    .withPagesPerDocument(1)
                    .build());

	    return pdfReader.read();
    }
}
```

--------------------------------

### Chroma Cloud configuration properties

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html

Example application.properties for connecting to Chroma Cloud. Requires the host api.trychroma.com, port 443, an API key via key-token, and matching tenant and database names. Set initialize-schema=true to auto-create the collection.

```properties
# Chroma Cloud connection
spring.ai.vectorstore.chroma.client.host=api.trychroma.com
spring.ai.vectorstore.chroma.client.port=443
spring.ai.vectorstore.chroma.client.key-token=<your-chroma-cloud-api-key>

# Chroma Cloud tenant and database (required)
spring.ai.vectorstore.chroma.tenant-name=<your-tenant-id>
spring.ai.vectorstore.chroma.database-name=<your-database-name>

# Collection configuration
spring.ai.vectorstore.chroma.collection-name=my-collection
spring.ai.vectorstore.chroma.initialize-schema=true
```

--------------------------------

### WebMVC Server Configuration (spring-ai-starter-mcp-server-webmvc)

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/mcp/mcp-stdio-sse-server-boot-starter-docs.html

Configure a WebMVC MCP server using the spring-ai-starter-mcp-server-webmvc starter. Includes instructions, capabilities, and SSE properties.

```yaml
# Using spring-ai-starter-mcp-server-webmvc
spring:
  ai:
    mcp:
      server:
        name: webmvc-mcp-server
        version: 1.0.0
        type: SYNC
        instructions: "This server provides weather information tools and resources"
        capabilities:
          tool: true
          resource: true
          prompt: true
          completion: true
        # sse properties
        sse-message-endpoint: /mcp/messages
        keep-alive-interval: 30s

```

--------------------------------

### Run Chroma locally with Docker

Source: https://docs.spring.io/spring-ai/reference/2.0-SNAPSHOT/api/vectordbs/chroma.html

Starts a chroma store at localhost:8000/api/v1 using the Docker image ghcr.io/chroma-core/chroma:1.0.0.

```bash
docker run -it --rm --name chroma -p 8000:8000 ghcr.io/chroma-core/chroma:1.0.0
```
Tokens:
Raw