SAP Cloud SDK for AI Java

https://sap.github.io/ai-sdk/docs/java/
Admin
The SAP Cloud SDK for AI Java enables developers to seamlessly integrate AI capabilities, such as...
Tokens:58,531
Snippets:526
Trust Score:8.3
Update:1 month ago
Show doc for...
Tokens:
Raw
### Example deployment response for gpt-realtime model

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/realtime

Example response from the AI Core /deployments endpoint for a deployed gpt-realtime model. Shows the deployment URL (wss://) and configuration details needed to connect to the Realtime API.

```json
{

      "id": "d123456abcdefg",

      "deploymentUrl": "wss://realtime.ai.region.aws.ml.hana.ondemand.com/v2/inference/deployments/d123456abcdefg",

      "configurationId": "12345-123-123-123-123456abcdefg",

      "configurationName": "gpt-realtime",

      "scenarioId": "foundation-models",

      "status": "RUNNING",

      "statusMessage": null,

      "targetStatus": "RUNNING",

      "lastOperation": "CREATE",

      "latestRunningConfigurationId": "12345-123-123-123-123456abcdefg",

      "ttl": null,

      "details": {

        "scaling": {

          "backendDetails": {}

        },

        "resources": {

          "backendDetails": {

            "model": {

              "name": "gpt-realtime",

              "version": "latest"

            }

          }

        }

      },

      "createdAt": "2024-07-03T12:44:22Z",

      "modifiedAt": "2024-07-16T12:44:19Z",

      "submissionTime": "2024-07-03T12:44:51Z",

      "startTime": "2024-07-03T12:45:56Z",

      "completionTime": null

    }
```

--------------------------------

### Example AI Core configuration JSON

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/ai-core-deployment

Example configuration from the AI Core /configuration endpoint. It shows the structure of a configuration object, including executableId, parameterBindings, and scenarioId.

```json
{

  "createdAt": "2024-07-03T12:44:08Z",

  "executableId": "azure-openai",

  "id": "12345-123-123-123-123456abcdefg",

  "inputArtifactBindings": [],

  "name": "gpt-4o-mini",

  "parameterBindings": [

    {

      "key": "modelName",

      "value": "gpt-4o-mini"

    },

    {

      "key": "modelVersion",

      "value": "latest"

    }

  ],

  "scenarioId": "foundation-models"

}
```

--------------------------------

### Create a response with reasoning

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

This example demonstrates how to create a response with a specific reasoning effort level and a concise reasoning summary.

```APIDOC
## Create a response with reasoning

### Description
This creates a response with a specific reasoning effort level and a concise reasoning summary.

### Method
create

### Endpoint
AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses().create(params)

### Parameters
#### Request Body (ResponseCreateParams)
- **input** (String) - Required - The user input message.
- **reasoning** (Reasoning) - Required - The reasoning configuration, including effort and summary.
- **store** (Boolean) - Optional - Whether to store the response.

### Request Example
java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var reasoning = Reasoning.builder()
    .effort(ReasoningEffort.MEDIUM)
    .summary(Reasoning.Summary.CONCISE)
    .build();

var params = ResponseCreateParams.builder()
    .input("Explain quantum entanglement.")
    .reasoning(reasoning)
    .store(false)
    .build();

var response = responseClient.create(params);


### Response
#### Success Response
- **response** (Response) - The created response object.

#### Response Example
java
var response = responseClient.create(params);

```

--------------------------------

### Create a response with a prompt cache key

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

This example demonstrates how to initialize the response client and configure a prompt cache key within the ResponseCreateParams to optimize cache reuse.

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();



// Routes requests sharing this key to the same backend, improving prompt cache hit rate on shared prefixes

var params =

    ResponseCreateParams.builder()

        .input("Hi, how are you?")

        .promptCacheKey("my-cache-key")

        .store(false)

        .build();



var response = responseClient.create(params);
```

--------------------------------

### Create OrchestrationClient and configure GPT-4o

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Initializes an OrchestrationClient and an OrchestrationModuleConfig with the GPT-4o model. Use this as the starting point for chat completion tasks.

```java
var client = new OrchestrationClient();



var config = new OrchestrationModuleConfig()

        .withLlmConfig(OrchestrationAiModel.GPT_4O);
```

--------------------------------

### Create a response with structured output

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

This example demonstrates how to create a response with structured JSON output conforming to a given schema.

```APIDOC
## Create a response with structured output

### Description
This creates a response with structured JSON output conforming to a given schema.

### Method
create

### Endpoint
AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses().create(params)

### Parameters
#### Request Body (ResponseCreateParams)
- **input** (String) - Required - The user input message.
- **text** (ResponseTextConfig) - Required - The text configuration, including the output format.
- **store** (Boolean) - Optional - Whether to store the response.

### Request Example
java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

// JSON Schema the model's output must conform to
var schema = ResponseFormatTextJsonSchemaConfig.Schema.builder()
    .putAdditionalProperty("type", JsonValue.from("object"))
    .putAdditionalProperty("properties", JsonValue.from(Map.of(
        "name", Map.of("type", "string"),
        "age", Map.of("type", "integer"))))
    .putAdditionalProperty("required", JsonValue.from(List.of("name", "age")))
    .putAdditionalProperty("additionalProperties", JsonValue.from(false))
    .build();

// setting strict to true guarantees the returned JSON conforms exactly to the schema
var format = ResponseFormatTextJsonSchemaConfig.builder()
    .name("person")
    .schema(schema)
    .strict(true)
    .build();

var textConfig = ResponseTextConfig.builder().format(format).build();

var params = ResponseCreateParams.builder()
    .input("Extract the person: John is 30 years old.")
    .text(textConfig)
    .store(false)
    .build();

var response = responseClient.create(params);


### Response
#### Success Response
- **response** (Response) - The created response object.

#### Response Example
java
var response = responseClient.create(params);

```

--------------------------------

### Chat Completion with OpenAiChatModel

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/openai

Initialize the OpenAI client for GPT_4O_MINI and use it with the OpenAiChatModel to send a prompt and get a response.

```java
OpenAiClient openAiClient = OpenAiClient.forModel(OpenAiModel.GPT_4O_MINI);

ChatModel client = new OpenAiChatModel(openAiClient);



Prompt prompt = new Prompt("What is the capital of France?");

ChatResponse response = client.call(prompt);
```

--------------------------------

### Deployment of SAP RPT Model

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/sap-rpt/table-completion

Example output from the AI Core /deployments endpoint showing a running SAP RPT model. Provides the deployment URL and model details.

```json
{

  "id": "c4dfbfd3bet3d4f1",

  "createdAt": "2026-01-14T06:29:33Z",

  "modifiedAt": "2026-01-14T06:29:33Z",

  "status": "RUNNING",

  "details": {

    "scaling": {

      "backendDetails": {},

      "backend_details": {}

    },

    "resources": {

      "backendDetails": {

        "model": {

          "name": "sap-rpt-1-small",

          "version": "latest"

        }

      },

      "backend_details": {

        "model": {

          "name": "sap-rpt-1-small",

          "version": "latest"

        }

      }

    }

  },

  "scenarioId": "foundation-models",

  "configurationId": "54fe1b6a-2d4b-4e2f-9f4b-5c8e9f1a2b3c",

  "latestRunningConfigurationId": "54fe1b6a-2d4b-4e2f-9f4b-5c8e9f1a2b3c",

  "lastOperation": "CREATE",

  "targetStatus": "RUNNING",

  "submissionTime": "2026-01-14T06:30:08Z",

  "startTime": "2026-01-14T06:31:44Z",

  "configurationName": "sap-rpt-1-small",

  "deploymentUrl": "https://api.ai.intprod-eu12.eu-central-1.aws.ml.hana.ondemand.com/v2/inference/deployments/c4dfbfd3bet3d4f1"

}
```

--------------------------------

### Add custom headers with OpenAIClient.withHeader

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/openai

Use the `.withHeader` method on `OpenAIClient` to add custom headers to LLM calls. The example creates a client with a header, wraps it in `OpenAiChatModel`, and sends a prompt to get a response.

```java
var clientWithHeader = new OpenAIClient().withHeader("foo", "bar");

ChatModel client = new OpenAiChatModel(clientWithHeader);



Prompt prompt = new Prompt("What is the capital of France?");

ChatResponse response = client.call(prompt);
```

--------------------------------

### Create a Batch Job with BatchesApi.createBatch

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/batch

Submits a new batch job by specifying the input file, output directory, and model. The example uses the 'azure-openai' provider with model 'gpt-4.1' and sets the type to LLM_NATIVE.

```java
BatchesApi client = new BatchesApi();



BatchCreateResponse response = client.createBatch(

    "my-resource-group",

    BatchCreateRequest.create()

        .type(TypeEnum.LLM_NATIVE)

        .input(BatchCreateRequestInput.create().uri("ai://s3secret/input-batch.jsonl"))

        .output(BatchCreateRequestOutput.create().uri("ai://s3secret/batch-output/"))

        .spec(BatchCreateRequestSpec.create().provider("azure-openai").model("gpt-4.1")));
```

--------------------------------

### List Prompt Templates with PromptClient

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/prompt-registry

Retrieves a list of prompt templates using the PromptClient. The listPromptTemplates() method returns a PromptTemplateListResponse. A complete example is available in the PromptRegistryController.java file in the Spring Boot application.

```java
PromptClient client = new PromptClient();



PromptTemplateListResponse templates = client.listPromptTemplates();
```

--------------------------------

### Create a response with tool calling

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

This example demonstrates how to create a response with a function tool that the model may call. It builds a FunctionTool with a JSON schema for parameters and uses ToolChoiceOptions.AUTO to let the model decide whether to invoke the tool.

```APIDOC
## Create a response with tool calling

### Description
This creates a response with a function tool that the model may call.

### Method
create

### Endpoint
AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses().create(params)

### Parameters
#### Request Body (ResponseCreateParams)
- **input** (String) - Required - The user input message.
- **tool** (FunctionTool) - Required - The function tool to be added.
- **toolChoice** (ToolChoiceOptions) - Optional - Specifies whether the model should call the tool. Use AUTO to let the model decide.
- **store** (Boolean) - Optional - Whether to store the response.

### Request Example
java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

// JSON Schema describing the function's arguments
var parameters = FunctionTool.Parameters.builder()
    .putAdditionalProperty("type", JsonValue.from("object"))
    .putAdditionalProperty("properties", JsonValue.from(Map.of(
        "location", Map.of("type", "string", "description", "City name, e.g. Berlin"),
        "unit", Map.of("type", "string", "enum", List.of("celsius", "fahrenheit"), "description", "Temperature unit"))))
    .putAdditionalProperty("required", JsonValue.from(List.of("location", "unit")))
    .putAdditionalProperty("additionalProperties", JsonValue.from(false))
    .build();

var tool = FunctionTool.builder()
    .name("get_weather")
    .description("Retrieves current weather for a given location.")
    .parameters(parameters)
    .strict(true)
    .build();

var params = ResponseCreateParams.builder()
    .input("What's the weather in Berlin?")
    .addTool(tool)
    .toolChoice(ToolChoiceOptions.AUTO)
    .store(false)
    .build();

var response = responseClient.create(params);


### Response
#### Success Response
- **response** (Response) - The created response object.

#### Response Example
java
var response = responseClient.create(params);

```

--------------------------------

### Spring service setup with chat memory and tools

Source: https://sap.github.io/ai-sdk/docs/java/tutorials/agentic-workflows

Initial Spring service skeleton for an agentic workflow. Configures chat memory with InMemoryChatMemory and MessageChatMemoryAdvisor, adds mocked tools via ToolCallbacks.from, and returns null as a placeholder for the workflow.

```java
@Service

public class SpringAiAgenticWorkflowService {

  private final ChatModel client = new OrchestrationChatModel();

  private final OrchestrationModuleConfig config =

      new OrchestrationModuleConfig().withLlmConfig(GPT_4O_MINI);



  public ChatResponse runAgent(String userInput) {



    //    Configure chat memory

    var memory = new InMemoryChatMemory();

    var advisor = new MessageChatMemoryAdvisor(memory);

    var cl = ChatClient.builder(client).defaultAdvisors(advisor).build();



    //    Add (mocked) tools

    var options = new OrchestrationChatOptions(config);

    options.setToolCallbacks(

        List.of(ToolCallbacks.from(new WeatherMethod(), new RestaurantMethod())));

    options.setInternalToolExecutionEnabled(true);



    //    The actual agentic workflow will be added here.



    return null;

  }

}
```

--------------------------------

### Configure MCP client for file system server

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/orchestration

Configure an MCP client for the file system MCP server. Requires npx to be installed. The client type is SYNC and uses the current working directory.

```yaml
spring:

  ai:

    mcp:

      client:

        # Type of client: SYNC or ASYNC

        type: SYNC

        stdio:

          connections:

            # Configuration for file system access, uses the current working directory

            localFS:

              command: npx

              args:

                - '-y'

                - '@modelcontextprotocol/server-filesystem'

                - '.'

              env:

                DEBUG: 'true'
```

--------------------------------

### Enable reasoning with withReasoningEffort and read getReasoningText

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Configure reasoning effort on the model via withReasoningEffort and read the reasoning text from the response with getReasoningText. The example uses OrchestrationAiModel.CLAUDE_4_5_SONNET with ReasoningEffort.MEDIUM. Not all models support the full range of effort levels; unsupported levels are mapped to the closest equivalent.

```java
var reasoningConfig =

    config.withLlmConfig(

        OrchestrationAiModel.CLAUDE_4_5_SONNET.withReasoningEffort(ReasoningEffort.MEDIUM));

var prompt =

    new OrchestrationPrompt("Think carefully and explain step by step: Why is the sky blue?");



var response = client.chatCompletion(prompt, reasoningConfig);



String answer = response.getContent();

String reasoning = response.getReasoningText();
```

--------------------------------

### Example deployed model from AI Core /deployments endpoint

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/embedding

Shows the JSON response from the AI Core /deployments endpoint for a deployed OpenAI model. It includes the deployment URL, configuration, and model details.

```json
{

  "id": "d123456abcdefg",

  "deploymentUrl": "https://api.ai.region.aws.ml.hana.ondemand.com/v2/inference/deployments/d123456abcdefg",

  "configurationId": "12345-123-123-123-123456abcdefg",

  "configurationName": "gpt-4o-mini",

  "scenarioId": "foundation-models",

  "status": "RUNNING",

  "statusMessage": null,

  "targetStatus": "RUNNING",

  "lastOperation": "CREATE",

  "latestRunningConfigurationId": "12345-123-123-123-123456abcdefg",

  "ttl": null,

  "details": {

    "scaling": {

      "backendDetails": {}

    },

    "resources": {

      "backendDetails": {

        "model": {

          "name": "gpt-4o-mini",

          "version": "latest"

        }

      }

    }

  },

  "createdAt": "2024-07-03T12:44:22Z",

  "modifiedAt": "2024-07-16T12:44:19Z",

  "submissionTime": "2024-07-03T12:44:51Z",

  "startTime": "2024-07-03T12:45:56Z",

  "completionTime": null

}
```

--------------------------------

### Example deployment response from AI Core /deployments endpoint

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/embedding

Shows the JSON structure returned by the AI Core /deployments endpoint for a deployed orchestration model. It includes fields such as id, deploymentUrl, configurationId, and status.

```json
{

  "id": "d123456abcdefg",

  "deploymentUrl": "https://api.ai.intprod-eu12.eu-central-1.aws.ml.hana.ondemand.com/v2/inference/deployments/d123456abcdefg",

  "configurationId": "12345-123-123-123-123456abcdefg",

  "configurationName": "orchestration",

  "scenarioId": "orchestration",

  "status": "RUNNING",

  "statusMessage": null,

  "targetStatus": "RUNNING",

  "lastOperation": "CREATE",

  "latestRunningConfigurationId": "12345-123-123-123-123456abcdefg",

  "ttl": null,

  "createdAt": "2024-08-05T16:17:29Z",

  "modifiedAt": "2024-08-06T06:32:50Z",

  "submissionTime": "2024-08-05T16:17:40Z",

  "startTime": "2024-08-05T16:18:41Z",

  "completionTime": null

}
```

--------------------------------

### Multi-turn reasoning with messageHistory and getAllMessages

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Keep reasoning enabled across turns by feeding the previous response's getAllMessages() into the follow-up prompt as message history. The example shows a first prompt, a follow-up prompt with messageHistory, and reading the final answer and reasoning text.

```java
var firstResponse = client.chatCompletion(new OrchestrationPrompt("What is 6 times 7?"), reasoningConfig);



var followUp =

    new OrchestrationPrompt("Are you sure?").messageHistory(firstResponse.getAllMessages());

var followUpResponse = client.chatCompletion(followUp, reasoningConfig);



String answer = followUpResponse.getContent();

String reasoning = followUpResponse.getReasoningText();
```

--------------------------------

### Verify orchestration deployment via /lm/deployments API

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Example response from the `/lm/deployments` API of AI Core, showing a running orchestration deployment. Use this to verify that your custom resource group has an active orchestration deployment before using the Orchestration module.

```json
{

  "id": "d123456abcdefg",

  "deploymentUrl": "https://api.ai.intprod-eu12.eu-central-1.aws.ml.hana.ondemand.com/v2/inference/deployments/d123456abcdefg",

  "configurationId": "12345-123-123-123-123456abcdefg",

  "configurationName": "orchestration",

  "scenarioId": "orchestration",

  "status": "RUNNING",

  "statusMessage": null,

  "targetStatus": "RUNNING",

  "lastOperation": "CREATE",

  "latestRunningConfigurationId": "12345-123-123-123-123456abcdefg",

  "ttl": null,

  "createdAt": "2024-08-05T16:17:29Z",

  "modifiedAt": "2024-08-06T06:32:50Z",

  "submissionTime": "2024-08-05T16:17:40Z",

  "startTime": "2024-08-05T16:18:41Z",

  "completionTime": null
}
```

--------------------------------

### Grounding via Orchestration: Configure and use grounding in chat completion

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/document-grounding

Use the grounding service via orchestration. The example configures a document grounding filter with a maximum chunk count of 3, sets up placeholders for input and output, and adds the filter to the grounding module config. It then creates an orchestration module config with GPT-4O and the grounding config, and sends a chat completion request with a prompt that embeds the grounding results.

```java
OrchestrationClient client;



var databaseFilter =

    DocumentGroundingFilter.create()

        .dataRepositoryType(VECTOR)

        .searchConfig(GroundingFilterSearchConfiguration.create().maxChunkCount(3));

var groundingConfigConfig =

    GroundingModuleConfigConfig.create()

        .placeholders(

            GroundingModuleConfigConfigPlaceholders.create().input("query").output("results"))

        .addFiltersItem(databaseFilter);

var groundingConfig =

    GroundingModuleConfig.create()

        .type(DOCUMENT_GROUNDING_SERVICE)

        .config(groundingConfigConfig);

var configWithGrounding =

    new OrchestrationModuleConfig().withLlmConfig(GPT_4O).withGroundingConfig(groundingConfig);



var inputParams = Map.of("query", "What is SAP Cloud SDK for AI?");



var prompt =

    new OrchestrationPrompt(

        inputParams,

        Message.system("Context message with embedded grounding results. {{?results}}"));



OrchestrationChatResponse response = client.chatCompletion(prompt, configWithGrounding);
```

--------------------------------

### Pipeline API: Read, create, and get pipeline status

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/document-grounding

Shows how to use the GroundingClient pipelines API to list all pipelines, create a new MSSharePoint pipeline, and retrieve its status. Requires a resource group ID and a pipeline secret.

```java
var api = new GroundingClient().pipelines();

var resourceGroupId = "default";



// get all pipelines

GetPipelines pipelines = api.getAllPipelines(resourceGroupId);



// create new pipeline (MSSharePoint example)

var pipelineSecret = "my-secret-name";

var sharePointSite = SharePointSite.create().name("my-site");

var sharePointConfig = SharePointConfig.create().site(sharePointSite);

var config =

    MSSharePointConfiguration.create().destination(pipelineSecret).sharePoint(sharePointConfig);

var request =

    MSSharePointPipelineCreateRequest.create()

        .type(MS_SHARE_POINT)

        ._configuration(config);

PipelineId pipeline = api.createPipeline(resourceGroupId, request);



// get pipeline by id, returns a GetPipeline (MSSharePointPipelineGetResponse, S3PipelineGetResponse, or SFTPPipelineGetResponse)

var status = (MSSharePointPipelineGetResponse) = api.getPipelineById(resourceGroupId, pipeline.getPipelineId());
```

--------------------------------

### Parse Prompt Template by ID (Tenant Scope)

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/prompt-registry

Fills a prompt template by its ID using the tenant scope approach. The parsePromptTemplateById() method takes the template ID, resource group ID, resource group scope, and a substitution request with input parameters. A complete example is available in the PromptRegistryController.java file.

```java
PromptClient client = new PromptClient();



PromptTemplateSubstitutionResponse substitution = client.parsePromptTemplateById(

    UUID.fromString("template id"),

    null,

    null,

    false,

    PromptTemplateSubstitutionRequest.create()

        .inputParams(Map.of("inputExample", "I love football")));
```

--------------------------------

### Get a Batch Job by ID with BatchesApi.getBatchById

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/batch

Retrieves the details and status of a specific batch job by its ID. The example uses 'your-batch-job-id' as a placeholder for the actual UUID.

```java
BatchesApi client = new BatchesApi();



BatchDetailResponse response = client.getBatchById(

    "my-resource-group",

    UUID.fromString("your-batch-job-id"));
```

--------------------------------

### Instantiate AI Core Clients

Source: https://sap.github.io/ai-sdk/docs/java/connecting-to-ai-core

Demonstrates how to create specific clients like OpenAiClient, OrchestrationClient, or DeploymentApi using the initialized AiCoreService.

```java
// To create an OpenAiClient:

var openAiDestination = aiCoreService.getInferenceDestination().forModel(OpenAiModel.TEXT_EMBEDDING_3_LARGE);

var client = OpenAiClient.withCustomDestination(openAiDestination);



// To create an OrchestrationClient:

var orchestrationDestination = aiCoreService.getInferenceDestination().forScenario("orchestration");

var client = new OrchestrationClient(orchestrationDestination);



// To create any of the com.sap.ai.sdk.core.client.* classes (e.g. DeploymentApi):

new DeploymentApi(aiCoreService);
```

--------------------------------

### Create AI Core Clients from AiCoreService

Source: https://sap.github.io/ai-sdk/docs/java/guides/connecting-to-ai-core

Demonstrates how to instantiate specific AI clients like OpenAiClient, OrchestrationClient, or DeploymentApi using the initialized AiCoreService.

```java
// To create an OpenAiClient:  

var openAiDestination = aiCoreService.getInferenceDestination().forModel(OpenAiModel.TEXT_EMBEDDING_3_LARGE);  

var client = OpenAiClient.withCustomDestination(openAiDestination);  

  

// To create an OrchestrationClient:  

var orchestrationDestination = aiCoreService.getInferenceDestination().forScenario("orchestration");  

var client = new OrchestrationClient(orchestrationDestination);  

  

// To create any of the com.sap.ai.sdk.core.client.* classes (e.g. DeploymentApi):  

new DeploymentApi(aiCoreService);  
```

--------------------------------

### Add Maven dependency for document-grounding

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/document-grounding

Add this Maven dependency to your pom.xml to use the SAP Document Grounding module. The version is specified by the ${ai-sdk.version} property. See the linked example pom in the Spring Boot application for a complete setup.

```xml
<dependency>

    <groupId>com.sap.ai.sdk</groupId>

    <artifactId>document-grounding</artifactId>

    <version>${ai-sdk.version}</version>

</dependency>
```

--------------------------------

### Create a response with truncation

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

This example demonstrates how to create a response with truncation enabled so long conversations are automatically trimmed.

```APIDOC
## Create a response with truncation

### Description
This creates a response with truncation enabled so long conversations are automatically trimmed.

### Method
create

### Endpoint
AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses().create(params)

### Parameters
#### Request Body (ResponseCreateParams)
- **input** (String) - Required - The user input message.
- **previousResponseId** (String) - Optional - The ID of the previous response for conversation context.
- **truncation** (ResponseCreateParams.Truncation) - Required - The truncation strategy. AUTO drops older items from the middle if the conversation exceeds the context window.
- **store** (Boolean) - Optional - Whether to store the response.

### Request Example
java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseCreateParams.builder()
    .input("Can you elaborate on your last point?")
    .previousResponseId("previous-response-id")
    .truncation(ResponseCreateParams.Truncation.AUTO)
    .store(true)
    .build();

var response = responseClient.create(params);


### Response
#### Success Response
- **response** (Response) - The created response object.

#### Response Example
java
var response = responseClient.create(params);

```

--------------------------------

### Initialize Orchestration Client

Source: https://sap.github.io/ai-sdk/docs/java/getting-started

Create an instance of the OrchestrationClient to begin interacting with the service.

```java
var client = new OrchestrationClient();
```

--------------------------------

### Create a prompt template with tenant scope

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/prompt-registry

Creates a reusable prompt template with placeholders and defaults, then sends a request to create or update it. The response is a PromptTemplatePostResponse.

```java
PromptClient client = new PromptClient();



var spec =

    PromptTemplateSpec.create()

        .template(

            SingleChatTemplate.create()

                .role("system")

                .content(

                    "You classify input text into the two following categories: {{?categories}}"),

            SingleChatTemplate.create().role("user").content("{{?inputExample}}"))

        .defaults(Map.of("categories", "Finance, Tech, Sports"));



var template = PromptTemplatePostRequest.create()

    .name("template-name")

    .version("0.0.1")

    .scenario("categorization")

    .spec(spec);



PromptTemplatePostResponse response = client.createUpdatePromptTemplate(template);
```

--------------------------------

### Create an Orchestration Configuration with OrchestrationConfigClient

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/prompt-registry

Creates a new Orchestration configuration in Prompt Registry using the OrchestrationConfigClient. Builds a prompt templating module with a user message and model details, then posts the configuration with name, version, and scenario. The response is an OrchestrationConfigPostResponse.

```java
import com.sap.ai.sdk.prompt.registry.OrchestrationConfigClient;

import com.sap.ai.sdk.prompt.registry.model.LLMModelDetails;

import com.sap.ai.sdk.prompt.registry.model.OrchestrationConfigPostRequest;

import com.sap.ai.sdk.prompt.registry.model.OrchestrationConfigPostResponse;

import com.sap.ai.sdk.prompt.registry.model.PartialModuleConfigs;

import com.sap.ai.sdk.prompt.registry.model.PromptRegistryOrchestrationConfig;

import com.sap.ai.sdk.prompt.registry.model.PromptRegistryOrchestrationConfigModules;

import com.sap.ai.sdk.prompt.registry.model.PartialPromptTemplatingModuleConfig;

import com.sap.ai.sdk.prompt.registry.model.Template;

import com.sap.ai.sdk.prompt.registry.model.UserChatMessage;

import com.sap.ai.sdk.prompt.registry.model.UserChatMessageContent;



var orchConfigClient = new OrchestrationConfigClient();

// build PromptRegistryOrchestrationConfig

var message =

    UserChatMessage.create()

        .content(new UserChatMessageContent.InnerString("message"))

        .role(UserChatMessage.RoleEnum.USER);

var promptTemplating =

    PartialPromptTemplatingModuleConfig.create()

        .model(LLMModelDetails.create().name("model-name"))

        .prompt(Template.create().template(message));

var orchestrationConfig =

    PromptRegistryOrchestrationConfig.create()

        .modules(

            PromptRegistryOrchestrationConfigModules.createInnerPartialModuleConfigs(

                PartialModuleConfigs.create().promptTemplating(promptTemplating)));

// use the config

var postRequest =

      OrchestrationConfigPostRequest.create()

          .name("name")

          .version("0.0.1")

          .scenario("scenario")

          .spec(orchestrationConfig);

OrchestrationConfigPostResponse response = orchConfigClient.createUpdateOrchestrationConfig(postRequest);
```

--------------------------------

### List Batch Jobs with BatchesApi.listBatches

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/batch

Retrieves all batch jobs for a given resource group. The example uses 'my-resource-group' as the resource group identifier.

```java
BatchesApi client = new BatchesApi();



BatchListResponse response = client.listBatches("my-resource-group");
```

--------------------------------

### Import Prompt Template at Tenant Scope

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/prompt-registry

Imports a declarative prompt template in YAML format at the tenant scope. Uses the default tenant ID and null for resource group scope. Requires a PromptClient instance and a ClassPathResource pointing to the template file.

```java
PromptClient client = new PromptClient();



File template = new ClassPathResource("prompt-template.yaml").getFile();

PromptTemplatePostResponse response = client.importPromptTemplate("default", null, template);
```

--------------------------------

### Execute chat completion with AI Launchpad configuration JSON

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Executes a chat completion using a configuration JSON copied or downloaded from AI Launchpad. Not recommended for long-term use, but useful for demos and PoCs. The JSON can be pasted directly or loaded from a file.

```java
var configJson = """

    ... paste your configuration JSON in here ...

    """;

// or load your config from a file, e.g.

// configJson = Files.readString(Paths.get("path/to/my/orchestration-config.json"));



var prompt = new OrchestrationPrompt(Map.of("your-input-parameter", "your-param-value"));



new OrchestrationClient().executeRequestFromJsonModuleConfig(prompt, configJson);
```

--------------------------------

### Chat Completion with OrchestrationChatModel

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/orchestration

Basic usage of the Orchestration client in Spring AI. Creates a ChatModel, configures the LLM, and sends a prompt to get a response.

```java
ChatModel client = new OrchestrationChatModel();

OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_4O_MINI);

OrchestrationChatOptions opts = new OrchestrationChatOptions(config);



Prompt prompt = new Prompt("What is the capital of France?", opts);

ChatResponse response = client.call(prompt);
```

--------------------------------

### Blocking Asynchronous Streaming

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/chat-completion

A blocking example that streams chat completion delta strings directly to the console using a try-with-resources block to ensure the connection closes.

```java
String msg = "Can you give me the first 100 numbers of the Fibonacci sequence?";



OpenAiClient client = OpenAiClient.forModel(GPT_4O_MINI);



// try-with-resources on stream ensures the connection will be closed

try (Stream<String> stream = client.streamChatCompletion(msg)) {

    stream.forEach(

        deltaString -> {

            System.out.print(deltaString);

            System.out.flush();

        });

}
```

--------------------------------

### Grounding via help.sap.com

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Use help.sap.com as the grounding source for SAP-specific context. The snippet creates a filter with the HELP_SAP_COM data repository type, builds a grounding configuration, and calls chatCompletion with the grounded prompt.

```java
var groundingHelpSapCom =

        DocumentGroundingFilter.create()

                .dataRepositoryType(DataRepositoryType.HELP_SAP_COM);

var groundingConfig = Grounding.create().filters(groundingHelpSapCom);

var configWithGrounding = config.withGrounding(groundingConfig);



var prompt = groundingConfig.createGroundingPrompt("What is a fuzzy search?");

var response = client.chatCompletion(prompt, configWithGrounding);
```

--------------------------------

### Create a response with reasoning using AiCoreOpenAiClient

Source: https://sap.github.io/ai-sdk/docs/java/foundation-models/openai/responses

Creates a response with a specific reasoning effort level (ReasoningEffort.MEDIUM) and a concise reasoning summary (Reasoning.Summary.CONCISE). The response is created with store(false).

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();



var reasoning =

    Reasoning.builder().effort(ReasoningEffort.MEDIUM).summary(Reasoning.Summary.CONCISE).build();



var params =

    ResponseCreateParams.builder().input("Explain quantum entanglement.").reasoning(reasoning).store(false).build();



var response = responseClient.create(params);
```

--------------------------------

### Asynchronous Streaming with streamChatCompletion

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Streams chat completion deltas asynchronously and prints them to the console. Uses try-with-resources on the stream to ensure the connection is closed. This is a blocking example for streaming and printing directly to the console.

```scala
val msg = "Can you give me the first 100 numbers of the Fibonacci sequence?";

val prompt = new OrchestrationPrompt(msg);



// try-with-resources on stream ensures the connection will be closed

try (Stream<String> stream = client.streamChatCompletion(prompt, config)) {

    stream.forEach(

        deltaString -> {

            System.out.print(deltaString);

            System.out.flush();

        });

}
```

--------------------------------

### Delete a batch job by ID using BatchesApi

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/batch

Deletes a specific batch job by its ID. Requires a resource group and a batch job ID as a UUID. Refer to BatchController.java for a complete example.

```java
BatchesApi client = new BatchesApi();



BatchDeleteResponse response = client.deleteBatch(

    "my-resource-group",

    UUID.fromString("your-batch-job-id"));
```

--------------------------------

### Run Application Locally

Source: https://sap.github.io/ai-sdk/docs/java/getting-started

Set the required AICORE_SERVICE_KEY environment variable and run the application using Maven.

```bash
cd your-spring-app/



# assuming a bash, for other shells (e.g. PowerShell) see the below documentation

export AICORE_SERVICE_KEY='{ "clientid": "...", "clientsecret": "...", "url": "...", "serviceurls": { "AI_API_URL": "..." } }'



# assuming Maven and a Spring Boot application

mvn spring-boot:run
```

--------------------------------

### Get embeddings using OrchestrationEmbeddingRequest

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/embedding

Creates an embedding request for the TEXT_EMBEDDING_3_SMALL model, optionally optimizes for document content, and retrieves the embedding vector from the response. Requires a configured client and a deployed orchestration model.

```java
var request = OrchestrationEmbeddingRequest

    .forModel(TEXT_EMBEDDING_3_SMALL)

    .forInputs("I am John Doe")

    .asDocument() // Optionally optimize for document content



OrchestrationEmbeddingResponse response = client.embed(request);

float[] embedding = response.getEmbeddingVectors().get(0);
```

--------------------------------

### Test a prompt template locally from YAML with TemplateConfig.create().fromYaml()

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Loads a prompt template from a local YAML file using TemplateConfig.create().fromYaml(). The fromYaml() method throws an exception if the YAML is invalid or does not match the Prompt Registry spec. Note that additionalFields from the Prompt Registry spec are ignored when using a local template.

```java
String promptTemplate = Files.readString(Path.of("path/to/my/prompt-template.yaml"));

var template = TemplateConfig.create().fromYaml(promptTemplate);

var configWithTemplate = config.withTemplateConfig(template);



var inputParams = Map.of("language", "German");

var prompt = new OrchestrationPrompt(inputParams);

var response = client.chatCompletion(prompt, configWithTemplate);
```

--------------------------------

### Add Maven dependency for Batch module

Source: https://sap.github.io/ai-sdk/docs/java/ai-core/batch

Add this Maven dependency to your pom.xml to use the Batch module. The version is specified by the ${ai-sdk.version} property. See the linked example pom in the Spring Boot application for reference.

```xml
<dependency>

    <groupId>com.sap.ai.sdk</groupId>

    <artifactId>batch</artifactId>

    <version>${ai-sdk.version}</version>

</dependency>
```

--------------------------------

### Initialize AI Core Service with Destination

Source: https://sap.github.io/ai-sdk/docs/java/connecting-to-ai-core

Retrieves a configured destination from the BTP Destination Service and uses it to initialize the AiCoreService.

```java
Destination destination = DestinationAccessor.getDestination("my-aicore").asHttp();

AiCoreService aiCoreService = new AiCoreService().withBaseDestination(destination);
```

--------------------------------

### Use an inline prompt template with TemplateConfig.create().withMessages()

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Uses a prompt template defined inline with a placeholder {{?language}} that is replaced by the value from inputParams. The template is configured via TemplateConfig.create().withMessages() and passed to the chat completion call.

```java
var message = Message.user("Reply with 'Orchestration Service is working!' in {{?language}}");

var templatingConfig = TemplateConfig.create().withMessages(message);

var configWithTemplate = config.withTemplateConfig(templatingConfig);



var inputParams = Map.of("language", "German");

var prompt = new OrchestrationPrompt(inputParams);



var result = client.chatCompletion(prompt, configWithTemplate);
```

--------------------------------

### Pass a tool to the model with OrchestrationChatModel

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/orchestration

Configures an OrchestrationChatModel with GPT_4O_MINI, registers the WeatherMethod tool via ToolCallbacks.from, and calls the model with a prompt asking about weather in two cities.

```java
ChatModel client = new OrchestrationChatModel();

OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_4O_MINI);

OrchestrationChatOptions opts = new OrchestrationChatOptions(config);



opts.setToolCallbacks(List.of(ToolCallbacks.from(new WeatherMethod())));



Prompt prompt = new Prompt("What is the weather in Potsdam and in Toulouse?", opts);



ChatResponse response = client.call(prompt);
```

--------------------------------

### Chain workflow with system prompts and loop

Source: https://sap.github.io/ai-sdk/docs/java/tutorials/agentic-workflows

Full implementation of a chain-like agentic workflow. Uses MessageWindowChatMemory with InMemoryChatMemoryRepository, defines three system prompts, and loops over them, feeding the previous LLM output as input to the next call. Requires the Spring service class and returns the final response text.

```java
@Service

public class SpringAiAgenticWorkflowService {

  private final ChatModel client = new OrchestrationChatModel();

  private final OrchestrationModuleConfig config =

      new OrchestrationModuleConfig().withLlmConfig(GPT_4O_MINI);



  @Nonnull

  public String runAgent(String userInput) {



    //    Configure chat memory

    val repository = new InMemoryChatMemoryRepository();

    val memory = MessageWindowChatMemory.builder().chatMemoryRepository(repository).build();

    val advisor = MessageChatMemoryAdvisor.builder(memory).build();

    var cl = ChatClient.builder(client).defaultAdvisors(advisor).build();



    //    Add (mocked) tools

    var options = new OrchestrationChatOptions(config);

    options.setToolCallbacks(

        List.of(ToolCallbacks.from(new WeatherMethod(), new RestaurantMethod())));

    options.setInternalToolExecutionEnabled(true);



    //    Prompts for the chain workflow

    List<String> systemPrompts =

        List.of(

            "You are a traveling planning agent for a single day trip. Where appropriate, use the provided tools. First, start by suggesting some restaurants for the mentioned city.",

            "Now, check the whether for the city.",

            "Finally, combine the suggested itinerary from this conversation into a short, one-sentence plan for the day trip.");



    //    Perform the chain workflow

    String responseText = userInput;



    for (String systemPrompt : systemPrompts) {



      // Combine the pre-defined prompt with the previous answer to get the new input

      String input = String.format("{%s}\n {%s}", systemPrompt, responseText);

      var prompt = new Prompt(input, options);



      // Make a call to the LLM with the new input

      var response =

          Objects.requireNonNull(

              cl.prompt(prompt)

                  .advisors(spec -> spec.param(ChatMemory.CONVERSATION_ID, "conversation id"))

                  .call()

                  .chatResponse(),

              "Chat response is null.");

      responseText = response.getResult().getOutput().getText();

    }



    return responseText;

  }

}
```

--------------------------------

### Add custom headers with OrchestrationClient.withHeader

Source: https://sap.github.io/ai-sdk/docs/java/spring-ai/orchestration

Adds a custom header ("foo": "bar") to all LLM calls made through the OrchestrationClient. The header is set on the client before creating the ChatModel, and the example shows a complete call flow with a GPT-4o-mini configuration.

```java
var clientWithHeader = new OrchestrationClient().withHeader("foo", "bar");

ChatModel client = new OrchestrationChatModel(clientWithHeader);



OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_4O_MINI);

OrchestrationChatOptions opts = new OrchestrationChatOptions(config);



Prompt prompt = new Prompt("What is the capital of France?", opts);

ChatResponse response = client.call(prompt);
```

--------------------------------

### Grounding with Vector Data Repository

Source: https://sap.github.io/ai-sdk/docs/java/orchestration/chat-completion

Use a vector data repository to provide grounding context. The snippet shows optional filters for collections and document chunks, then creates a grounding configuration and a prompt, and finally calls chatCompletion with the grounded configuration.

```java
// optional filter for collections

var documentMetadata =

    SearchDocumentKeyValueListPair.create()

        .key("my-collection")

        .value("value")

        .addSelectModeItem(SearchSelectOptionEnum.IGNORE_IF_KEY_ABSENT);

// optional filter for document chunks

var databaseFilter =

    DocumentGroundingFilter.create()

        .id("")

        .dataRepositoryType(DataRepositoryType.VECTOR)

        .addDocumentMetadataItem(documentMetadata);



var groundingConfig = Grounding.create().filter(databaseFilter);

var prompt = groundingConfig.createGroundingPrompt("What does Joule do?");

var configWithGrounding = config.withGrounding(groundingConfig);



var result = client.chatCompletion(prompt, configWithGrounding);
```
Tokens:
Raw