### Compile Java Examples with Maven

Source: https://github.com/triton-inference-server/client/blob/main/src/java/README.md

Navigate to the Java client source directory and use Maven to clean, install, and build the examples. The `-Ddir=examples` flag specifies the directory containing the examples to be built.

```bash
cd client/src/java
mvn clean install -Ddir=examples
```

--------------------------------

### Install Java Client Prerequisites

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Install Maven and a JDK for building the Java client on Ubuntu.

```bash
$ apt-get install default-jdk maven
```

--------------------------------

### Install Third-Party Include Directories

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Installs include directories for gRPC, absl, protobuf, and re2.

```cmake
install(
      DIRECTORY
        ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/grpc/include/
        ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/absl/include/
        ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/protobuf/include/
        ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/re2/include/
        DESTINATION include
    )
```

--------------------------------

### Run Go Client Example

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/go/README.rst

Clone the server repository for example models, set up the 'simple' model, launch Triton in a Docker container, and then run the Go client.

```bash
# Clone the repo containing the example model, from within client/src/grpc_generated/go/.
git clone https://github.com/triton-inference-server/server.git

# Setup "simple" model from example model_repository
cd server/docs/examples
./fetch_models.sh

# Launch (detached) Triton
docker run -d -p8000:8000 -p8001:8001 -p8002:8002 -it -v $(pwd)/model_repository:/models nvcr.io/nvidia/tritonserver:22.11-py3 tritonserver --model-store=/models

# Use client
cd ../../../
go run grpc_simple_client.go
```

--------------------------------

### Build and Run Java Example Client

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/java/README.md

Build the examples project using Maven and then run the SimpleJavaClient. Provide the host and port of the Triton Inference Server as arguments.

```bash
cd examples

mvn clean install 

mvn exec:java -Dexec.mainClass=clients.SimpleJavaClient -Dexec.args="<host> <port>"
```

--------------------------------

### Install gRPC Client Header

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Installs the gRPC client header file.

```cmake
install(
      FILES
      ${CMAKE_CURRENT_SOURCE_DIR}/grpc_client.h
      DESTINATION include
  )
```

--------------------------------

### Install Dependencies

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/javascript/README.md

Install the required Node.js dependencies for the project using npm.

```bash
npm install
```

--------------------------------

### Build Triton Clients and Examples

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Build the Triton client libraries and examples using make with parallel processing. Specify desired client types.

```bash
$ make -j$(nproc)/ cc-clients python-clients java-clients
```

--------------------------------

### Build and Run Scala Example Client

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/java/README.md

Build the examples project using Maven and then run the SimpleClient. Provide the host and port of the Triton Inference Server as arguments. This client is more comprehensive and checks server/model readiness.

```bash
mvn exec:java -Dexec.mainClass=clients.SimpleClient -Dexec.args="<host> <port>"
```

--------------------------------

### Install Triton Client with All Protocols (pip)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Use this command to install the Triton Python client library with support for both HTTP/REST and gRPC protocols.

```bash
$ pip install tritonclient[all]
```

--------------------------------

### Install Static Libraries (Windows)

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Installs static libraries (.lib) for various third-party dependencies on Windows systems.

```cmake
install(
        DIRECTORY
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/curl/lib/
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/grpc/lib/
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/protobuf/lib/
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/c-ares/lib/
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/absl/lib/
          ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/re2/lib/
        DESTINATION ${CMAKE_INSTALL_LIBDIR}
        FILES_MATCHING
          PATTERN "*\.lib"
          PATTERN "CMakeFiles" EXCLUDE
          PATTERN "cmake" EXCLUDE
          PATTERN "gens" EXCLUDE
          PATTERN "libs" EXCLUDE
          PATTERN "third_party" EXCLUDE
      )
```

--------------------------------

### Install Protocol Buffers Compiler

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Install the Protocol Buffers compiler, a prerequisite for building the Triton client. Instructions are provided for macOS and Ubuntu/Debian.

```bash
# macOS
brew install protobuf
```

```bash
# Ubuntu/Debian
apt-get install protobuf-compiler
```

--------------------------------

### Basic Example

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Demonstrates connecting to Triton, checking server health, retrieving metadata, building and executing an inference request, and processing the response.

```APIDOC
## Basic Example

```rust
use triton_client::prelude::*;

#[tokio::main]
async fn main() -> triton_client::error::Result<()> {
    // Connect to Triton
    let client = TritonClient::connect("http://localhost:8001").await?;

    // Check health
    assert!(client.is_server_live().await?);
    assert!(client.is_server_ready().await?);

    // Query server metadata
    let metadata = client.server_metadata().await?;
    println!("Server: {} v{}", metadata.name, metadata.version);

    // Build an inference request
    let input = InferInput::new("input0", vec![1, 16], DataType::Fp32)
        .with_data_f32(&[0.0; 16]);

    let request = InferRequestBuilder::new("my_model")
        .model_version("1")
        .input(input)
        .output("output0")
        .build();

    // Run inference
    let response = client.infer(request).await?;
    let output = response.output_as_f32(0)?;
    println!("Output: {:?}", output);

    Ok(())
}
```
```

--------------------------------

### Install Static Libraries (Linux/macOS)

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Installs static libraries (.a) for various third-party dependencies on non-Windows systems.

```cmake
install(
          DIRECTORY
            ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/absl/${LIB_DIR}/
            ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/re2/${LIB_DIR}/
        DESTINATION ${CMAKE_INSTALL_LIBDIR}
        FILES_MATCHING
          PATTERN "*\.a"
          PATTERN "CMakeFiles" EXCLUDE
          PATTERN "cmake" EXCLUDE
          PATTERN "gens" EXCLUDE
          PATTERN "libs" EXCLUDE
          PATTERN "third_party" EXCLUDE
      )
```

--------------------------------

### Install Triton Client with HTTP Protocol Only (pip)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Install only the HTTP/REST client library for the Triton Python client. This is useful if gRPC is not required.

```bash
$ pip install tritonclient[http]
```

--------------------------------

### Go Client Example Output

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/go/README.rst

Sample output from running the grpc_simple_client.go. It shows client flags, Triton health status, model details, and inference results.

```text
$ go run grpc_simple_client.go
  FLAGS: {simple  1 localhost:8001}
  Triton Health - Live: true
  Triton Health - Ready: true
  name:"simple"  versions:"1"  platform:"tensorflow_graphdef"  inputs:{name:"INPUT0"  datatype:"INT32"  shape:-1  shape:16}  inputs:{name:"INPUT1"  datatype:"INT32"  shape:-1  shape:16}  outputs:{name:"OUTPUT0"  datatype:"INT32"  shape:-1  shape:16}  outputs:{name:"OUTPUT1"  datatype:"INT32"  shape:-1  shape:16}
  
  Checking Inference Outputs
  --------------------------
  0 + 1 = 1
  0 - 1 = -1
  1 + 1 = 2
  1 - 1 = 0
  2 + 1 = 3
  2 - 1 = 1
  3 + 1 = 4
  3 - 1 = 2
  4 + 1 = 5
  4 - 1 = 3
  5 + 1 = 6
  5 - 1 = 4
  6 + 1 = 7
  6 - 1 = 5
  7 + 1 = 8
  7 - 1 = 6
  8 + 1 = 9
  8 - 1 = 7
  9 + 1 = 10
  9 - 1 = 8
  10 + 1 = 11
  10 - 1 = 9
  11 + 1 = 12
  11 - 1 = 10
  12 + 1 = 13
  12 - 1 = 11
  13 + 1 = 14
  13 - 1 = 12
  14 + 1 = 15
  14 - 1 = 13
  15 + 1 = 16
  15 - 1 = 14
```

--------------------------------

### Basic Triton Client Example

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Connect to a Triton server, check its health, retrieve metadata, build and execute an inference request, and process the output.

```rust
use triton_client::prelude::*;

#[tokio::main]
async fn main() -> triton_client::error::Result<()> {
    // Connect to Triton
    let client = TritonClient::connect("http://localhost:8001").await?;

    // Check health
    assert!(client.is_server_live().await?);
    assert!(client.is_server_ready().await?);

    // Query server metadata
    let metadata = client.server_metadata().await?;
    println!("Server: {} v{}", metadata.name, metadata.version);

    // Build an inference request
    let input = InferInput::new("input0", vec![1, 16], DataType::Fp32)
        .with_data_f32(&[0.0; 16]);

    let request = InferRequestBuilder::new("my_model")
        .model_version("1")
        .input(input)
        .output("output0")
        .build();

    // Run inference
    let response = client.infer(request).await?;
    let output = response.output_as_f32(0)?;
    println!("Output: {:?}", output);

    Ok(())
}
```

--------------------------------

### Copy Generated Stubs to Examples Project

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/java/README.md

Copy the generated GRPC Java stubs from the library's target directory into the examples project. This makes them available for use in the sample clients.

```bash
cd ..
cp -R library/target/generated-sources/protobuf/java/inference examples/src/main/java/inference
cp -R library/target/generated-sources/protobuf/grpc-java/inference/*.java examples/src/main/java/inference/
```

--------------------------------

### Generate Go Client Stubs

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/go/README.rst

Clone the common repository containing proto definitions and run the generation script. Ensure protoc-gen-go-grpc is installed.

```bash
# Clone the repo containing the proto definitions, from within client/src/grpc_generated/go/
git clone https://github.com/triton-inference-server/common.git

# Compiles *.proto to *.pb.go
# install protoc-gen-go-grpc with
# go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
./gen_go_stubs.sh
```

--------------------------------

### C++ HTTP Infer Client Example

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to use SSL/TLS settings and compression options for HTTP inference requests with the C++ client library.

```c++
#include <c++/library/triton_client.h>
#include <iostream>
#include <string>
#include <vector>

int
main(int argc, char** argv)
{
  if (argc != 2)
  {
    std::cerr << "Usage: " << argv[0] << " <protocol>\n";
    std::cerr << "    <protocol> - "HTTP" or "gRPC"\n";
    return 1;
  }
  std::string protocol_str = argv[1];
  std::string url;
  std::string model_name;
  std::string model_version;

  // Parse command line ...
  // ... (rest of the example code)

  std::unique_ptr<triton::client::InferenceClient> client;
  triton::client::Error err;

  // Create client
  if (protocol_str == "HTTP") {
    triton::client::HttpClientOptions http_options;
    // Example: Enable SSL/TLS
    // http_options.SetSslOptions(true);
    // Example: Enable compression
    // http_options.SetRequestCompressionAlgorithm(triton::client::CompressionType::SNAPPY);
    // http_options.SetResponseCompressionAlgorithm(triton::client::CompressionType::SNAPPY);

    err = triton::client::InferenceClient::Create(&client, url, protocol_str, "", true, http_options);
  } else if (protocol_str == "gRPC") {
    err = triton::client::InferenceClient::Create(&client, url, protocol_str, "", true);
  } else {
    std::cerr << "unknown protocol: " << protocol_str << std::endl;
    return 1;
  }

  if (!err.IsOk()) {
    std::cerr << "failed to create client: " << err << std::endl;
    return 1;
  }

  // ... (rest of the example code for inference)

  return 0;
}

```

--------------------------------

### Build Triton Client

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Build the Triton client using Cargo after installing necessary dependencies.

```bash
cargo build
```

--------------------------------

### Python HTTP Infer Client Example

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to use SSL/TLS settings and compression options for HTTP inference requests with the Python client library.

```python
import tritonclient.http as http_client

# Example: SSL/TLS options
# ssl_options = {
#     'ssl': True,
#     'ssl_options': {
#         'cert_reqs': 'CERT_REQUIRED',
#         'ca_certs': '/path/to/ca.crt'
#     }
# }
# Example: Compression options
# compression_options = {
#     'request_compression_algorithm': 'snappy',
#     'response_compression_algorithm': 'snappy'
# }

# client = http_client.InferenceServerClient(url='localhost:8000', 
#                                             verbose=False, 
#                                             **ssl_options, 
#                                             **compression_options)

# For simplicity, using default options here:
client = http_client.InferenceServerClient(url='localhost:8000', verbose=False)

# ... (rest of the example code for inference)

```

--------------------------------

### Configure Installation Prefix

Source: https://github.com/triton-inference-server/client/blob/main/CMakeLists.txt

Sets the TRITON_INSTALL_PREFIX based on whether CMAKE_INSTALL_PREFIX was initialized to its default value. This determines the final installation location for the client components.

```cmake
if (CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
  set(TRITON_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/cc-clients/install)
else()
  set(TRITON_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX})
endif()
```

--------------------------------

### Install Triton Client with HTTP and CUDA Support (pip)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Install the Triton Python client with HTTP support and CUDA shared memory utilities. The 'cuda' package must be explicitly specified if not using the '[all]' option.

```bash
$ pip install tritonclient[http, cuda]
```

--------------------------------

### Configure Java HTTP Client Build

Source: https://github.com/triton-inference-server/client/blob/main/src/java/CMakeLists.txt

Sets up the build for the Java HTTP client, including finding the Java package, setting target directories and JAR names, and defining custom commands for building and installing.

```cmake
find_package(Java REQUIRED)
include(UseJava)
set(PROJECT_TARGET_DIR "${CMAKE_CURRENT_SOURCE_DIR}/target")
set(PROJECT_JAR "java-api-0.0.1.jar")
set(EXAMPLES_PATH "src/main/java/triton/client")
add_custom_command(
    OUTPUT "${PROJECT_TARGET_DIR}/${PROJECT_JAR}"
    COMMAND mkdir -p ${CMAKE_INSTALL_PREFIX}/java
    COMMAND mvn clean install -Ddir=${PROJECT_TARGET_DIR}/examples
    COMMAND cp -r ${PROJECT_TARGET_DIR}/examples ${CMAKE_INSTALL_PREFIX}/java
    COMMAND cp -r ${PROJECT_TARGET_DIR}/${PROJECT_JAR} ${CMAKE_INSTALL_PREFIX}/java
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
    VERBATIM
)

add_custom_target(
    JavaProject ALL
    DEPENDS "${PROJECT_TARGET_DIR}/${PROJECT_JAR}"
)
```

--------------------------------

### Python AsyncIO HTTP Infer Client Example

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to perform inference using async/await syntax with the Python client library, including SSL/TLS options for asynchronous operations.

```python
import tritonclient.http.aio as http_client_aio

async def main():
    # Example: SSL/TLS options for AsyncIO
    # ssl_options = {
    #     'ssl': True,
    #     'ssl_context': ssl_context
    # }

    # For simplicity, using default options here:
    client = http_client_aio.InferenceServerClient(url='localhost:8000', verbose=False)

    # ... (rest of the example code for async inference)

if __name__ == '__main__':
    import asyncio
    asyncio.run(main())

```

--------------------------------

### Infer on Directory of Images with C++ Client

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Process all images within a directory for inferencing using the C++ image_client. This example demonstrates batching and multiple requests for different images.

```bash
$ image_client -m inception_graphdef -s INCEPTION -c 3 -b 2 qa/images
Request 0, batch size 2
Image '/opt/tritonserver/qa/images/car.jpg':
    0.819196 (818) = SPORTS CAR
    0.033457 (437) = BEACH WAGON
    0.031232 (480) = CAR WHEEL
Image '/opt/tritonserver/qa/images/mug.jpg':
    0.754130 (505) = COFFEE MUG
    0.157077 (969) = CUP
    0.002880 (968) = ESPRESSO
Request 1, batch size 2
Image '/opt/tritonserver/qa/images/vulture.jpeg':
    0.977632 (24) = VULTURE
    0.000613 (9) = HEN
    0.000560 (137) = EUROPEAN GALLINULE
Image '/opt/tritonserver/qa/images/car.jpg':
    0.819196 (818) = SPORTS CAR
    0.033457 (437) = BEACH WAGON
    0.031232 (480) = CAR WHEEL
```

--------------------------------

### ORCA Header Metrics - Text Format Example

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Example of a request header to receive ORCA metrics in text format. This format provides comma-separated key-value pairs.

```text
endpoint-load-metrics-format: text
```

--------------------------------

### Set Project Version and Options

Source: https://github.com/triton-inference-server/client/blob/main/src/java/CMakeLists.txt

Defines the project version and configures build options such as enabling Java HTTP client, examples, tests, and GPU support.

```cmake
set(TRITON_VERSION "0.0.0" CACHE STRING "Version for the clients")
option(TRITON_ENABLE_JAVA_HTTP "Enable JAVA HTTP client libraries" OFF)
option(TRITON_ENABLE_EXAMPLES "Include examples in build" OFF)
option(TRITON_ENABLE_TESTS "Include tests in build" OFF)
option(TRITON_ENABLE_GPU "Enable GPU support in libraries" OFF)
```

--------------------------------

### ORCA Header Metrics - Example Response Header (JSON)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Example of a response header containing ORCA metrics in JSON format. Metrics include utilization and request/execution rates.

```json
endpoint-load-metrics: JSON {"cpu_utilization": 0.3, "mem_utilization": 0.8, "rps_fractional": 10.0, "eps": 1, "named_metrics": {"custom-metric-util": 0.4}}
```

--------------------------------

### ORCA Header Metrics - Example Response Header (Text)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Example of a response header containing ORCA metrics in text format. Metrics include utilization and request/execution rates.

```text
endpoint-load-metrics: TEXT cpu_utilization=0.3, mem_utilization=0.8, rps_fractional=10.0, eps=1, named_metrics.custom_metric_util=0.4
```

--------------------------------

### Minimal Triton Java Inference Example

Source: https://github.com/triton-inference-server/client/blob/main/src/java/README.md

Demonstrates a basic synchronous inference request using the Triton Java API. Requires setting up input tensors with data and specifying requested outputs.

```java
package triton.client.example;

import java.util.Arrays;
import java.util.List;

import com.google.common.collect.Lists;
import triton.client.InferInput;
import triton.client.InferRequestedOutput;
import triton.client.InferResult;
import triton.client.InferenceServerClient;
import triton.client.pojo.DataType;

public class MinExample {
    public static void main(String[] args) throws Exception {
        boolean isBinary = true;
        InferInput inputIds = new InferInput("input_ids", new long[] {1L, 32}, DataType.INT32);
        int[] inputIdsData = new int[32];
        Arrays.fill(inputIdsData, 1); // fill with some data. 
        inputIds.setData(inputIdsData, isBinary);

        InferInput inputMask = new InferInput("input_mask", new long[] {1, 32}, DataType.INT32);
        int[] inputMaskData = new int[32];
        Arrays.fill(inputMaskData, 1);
        inputMask.setData(inputMaskData, isBinary);

        InferInput segmentIds = new InferInput("segment_ids", new long[] {1, 32}, DataType.INT32);
        int[] segmentIdsData = new int[32];
        Arrays.fill(segmentIdsData, 0);
        segmentIds.setData(segmentIdsData, isBinary);
        List<InferInput> inputs = Lists.newArrayList(inputIds, inputMask, segmentIds);
        List<InferRequestedOutput> outputs = Lists.newArrayList(new InferRequestedOutput("logits", isBinary));

        InferenceServerClient client = new InferenceServerClient("0.0.0.0:8000", 5000, 5000);
        InferResult result = client.infer("roberta", inputs, outputs);
        float[] logits = result.getOutputAsFloat("logits");
        System.out.println(Arrays.toString(logits));
    }
}
```

--------------------------------

### ORCA Header Metrics - JSON Format Example

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Example of a request header to receive ORCA metrics in JSON format. This format provides structured JSON encoding of the metrics.

```text
endpoint-load-metrics-format: json
```

--------------------------------

### Get Top Classifications with C++ Client

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Retrieve the top 3 classifications for an image using the -c flag with the C++ image_client. This shows more than just the most probable result.

```bash
$ image_client -m inception_graphdef -s INCEPTION -c 3 qa/images/mug.jpg
Request 0, batch size 1
Image 'qa/images/mug.jpg':
    0.754130 (505) = COFFEE MUG
    0.157077 (969) = CUP
    0.002880 (968) = ESPRESSO
```

--------------------------------

### Pull Triton Server SDK Docker Image

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Use this command to pull a Docker image containing the Triton client libraries and examples from NVIDIA GPU Cloud (NGC). Replace <xx.yy> with the desired version.

```bash
$ docker pull nvcr.io/nvidia/tritonserver:<xx.yy>-py3-sdk
```

--------------------------------

### Configure Java Client Build

Source: https://github.com/triton-inference-server/client/blob/main/CMakeLists.txt

Adds the Java clients as an external project when Java HTTP clients are enabled. This snippet configures the build with relevant CMake cache arguments, including SSL, toolchain, and installation paths.

```cmake
if(TRITON_ENABLE_JAVA_HTTP)
  ExternalProject_Add(java-clients
    PREFIX java-clients
    SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/src/java"
    BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/java-clients"
    CMAKE_CACHE_ARGS
      ${_CMAKE_ARGS_OPENSSL_ROOT_DIR}
      ${_CMAKE_ARGS_CMAKE_TOOLCHAIN_FILE}
      ${_CMAKE_ARGS_VCPKG_TARGET_TRIPLET}
      -DTRITON_REPO_ORGANIZATION:STRING=${TRITON_REPO_ORGANIZATION}
      -DTRITON_VERSION:STRING=${TRITON_VERSION}
      -DTRITON_ENABLE_JAVA_HTTP:BOOL=${TRITON_ENABLE_JAVA_HTTP}
      -DTRITON_ENABLE_EXAMPLES:BOOL=${TRITON_ENABLE_EXAMPLES}
      -DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}
      -DCMAKE_INSTALL_PREFIX:PATH=${TRITON_INSTALL_PREFIX}
    INSTALL_COMMAND ""
  )

endif() # TRITON_ENABLE_JAVA_HTTP
```

--------------------------------

### Checkout Main Branch and Prepare Build Directory

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Navigate to the repository root, checkout the main branch, and create a build directory.

```bash
$ git checkout main
$ mkdir -p build && cd build
```

--------------------------------

### Copy Proto Files

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/javascript/README.md

Create a 'proto' directory and copy all .proto files from the cloned common repository into it.

```bash
mkdir proto
cp common-repo/protobuf/*.proto ./proto/
```

--------------------------------

### Build Simple gRPC Custom Args Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_custom_args_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_custom_args_client simple_grpc_custom_args_client.cc)
  target_link_libraries(
    simple_grpc_custom_args_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_custom_args_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Configure HTTP Client Link Script

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Copies the linker script for the HTTP client library.

```cmake
configure_file(libhttpclient.ldscript libhttpclient.ldscript COPYONLY)
```

--------------------------------

### Create Static HTTP Client Library

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Builds the static version of the HTTP client library from the object library.

```cmake
add_library(
      httpclient_static STATIC
      $<TARGET_OBJECTS:http-client-library>
  )
  add_library(
      TritonClient::httpclient_static ALIAS httpclient_static
  )
```

--------------------------------

### Copy Protobuf Files

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/java/README.md

Copy the protobuf (*.proto) files from the cloned common repository to the library's src/main/proto directory.

```bash
cd library
cp ../common-repo/protobuf/*.proto src/main/proto/
```

--------------------------------

### Build Simple gRPC Custom Repeat Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_custom_repeat executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_custom_repeat simple_grpc_custom_repeat.cc)
  target_link_libraries(
    simple_grpc_custom_repeat
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_custom_repeat
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build Simple gRPC Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_infer_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_infer_client simple_grpc_infer_client.cc)
  target_link_libraries(
    simple_grpc_infer_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_infer_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build Simple gRPC Model Control Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_model_control executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_model_control simple_grpc_model_control.cc)
  target_link_libraries(
    simple_grpc_model_control
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_model_control
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build Simple gRPC Async Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_async_infer_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_async_infer_client simple_grpc_async_infer_client.cc)
  target_link_libraries(
    simple_grpc_async_infer_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_async_infer_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Download and Extract Client Libraries from GitHub

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Steps to download a tarball of client libraries from a Triton GitHub release and extract them. The libraries will be placed in lib/, headers in include/, Python wheels in python/, and Java jars in java/.

```bash
$ mkdir clients
$ cd clients
$ wget https://github.com/triton-inference-server/server/releases/download/<tarfile_path>
$ tar xzf <tarfile_name>
```

--------------------------------

### Build Simple gRPC String Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_string_infer_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_string_infer_client simple_grpc_string_infer_client.cc)
  target_link_libraries(
    simple_grpc_string_infer_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_string_infer_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### C++ GRPC Client Compression Configuration

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to configure on-wire compression for the C++ gRPC client. Supported algorithms are GRPC_COMPRESS_NONE, GRPC_COMPRESS_DEFLATE, and GRPC_COMPRESS_GZIP.

```c++
#include "triton/core/tritonbackend.h"
#include "triton/core/tritonserver.h"
#include "triton/core/tritonutils.h"
#include "triton/core/tritonclient.h"
#include "triton/core/tritonclient_grpc.h"

#include <iostream>
#include <memory>
#include <string>
#include <vector>

int main(int argc, char** argv)
{
    std::shared_ptr<triton::client::GRPCClient> client;
    triton::client::Error err;

    // Create GRPC client with compression enabled
    err = triton::client::GRPCClient::Create(&client, "localhost:8001", 
        triton::client::GRPCClientOptions::TritonClientKind::SYNC);
    if (!err.IsOk()) {
        std::cerr << "failed to create GRPCClient: " << err << std::endl;
        return 1;
    }

    // Set compression algorithm for inference requests
    std::string compression_algorithm = "gzip"; // or "deflate"
    client->SetCompression(compression_algorithm);

    std::cout << "Successfully created GRPC client with compression enabled: " << compression_algorithm << std::endl;

    // ... rest of the client logic ...

    return 0;
}
```

--------------------------------

### Build Simple gRPC Sequence Sync Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_sequence_sync_infer_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_sequence_sync_infer_client simple_grpc_sequence_sync_infer_client.cc)
  target_link_libraries(
    simple_grpc_sequence_sync_infer_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_sequence_sync_infer_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Python GRPC Client Compression Configuration

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to configure on-wire compression for the Python gRPC client. Supported algorithms are 'none', 'deflate', and 'gzip'.

```python
import tritonclient.grpc as grpcclient

# Create GRPC client
triton_client = grpcclient.InferenceServerClient(url='localhost:8001')

# Set compression algorithm for inference requests
compression_algorithm = 'gzip' # or 'deflate' or 'none'
triton_client.set_compression(compression_algorithm=compression_algorithm)

print(f'Successfully created GRPC client with compression enabled: {compression_algorithm}')

# ... rest of the client logic ...
```

--------------------------------

### Run Unit Tests

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Execute unit tests for the triton-client library. No Triton server is required for these tests.

```bash
# Unit tests (no server required)
cargo test
```

--------------------------------

### Build Image Client with OpenCV

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the image_client executable, which requires OpenCV for image processing. It links against gRPC and HTTP client libraries.

```cmake
find_package(OpenCV REQUIRED)
  add_executable(
    image_client
    image_client.cc
    $<TARGET_OBJECTS:json-utils-library>
  )
  target_include_directories(
    image_client
    PRIVATE ${OpenCV_INCLUDE_DIRS}
  )
  target_link_libraries(
    image_client
    PRIVATE
      grpcclient_static
      httpclient_static
      ${OpenCV_LIBS}
  )
  install(
    TARGETS image_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Define HTTP Client Source and Headers

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Sets variables for the source and header files used to build the HTTP client library.

```cmake
set(
      REQUEST_SRCS
      http_client.cc common.cc cencode.cc
  )

  set(
      REQUEST_HDRS
      http_client.h common.h ipc.h cencode.h
  )
```

--------------------------------

### Build Simple gRPC Health Metadata Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_health_metadata executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_health_metadata simple_grpc_health_metadata.cc)
  target_link_libraries(
    simple_grpc_health_metadata
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_health_metadata
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build Simple gRPC Sequence Stream Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_sequence_stream_infer_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_sequence_stream_infer_client simple_grpc_sequence_stream_infer_client.cc)
  target_link_libraries(
    simple_grpc_sequence_stream_infer_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_sequence_stream_infer_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build Ensemble Image Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the ensemble_image_client executable. This client is linked against gRPC and HTTP client libraries.

```cmake
add_executable(
    ensemble_image_client
    ensemble_image_client.cc
    $<TARGET_OBJECTS:json-utils-library>
  )
  target_link_libraries(
    ensemble_image_client
    PRIVATE
      grpcclient_static
      httpclient_static
  )
  install(
    TARGETS ensemble_image_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Run Triton Client Integration Tests

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Execute integration tests for the Triton client. Ensure a Triton server is running at the specified URL.

```bash
TRITON_TEST_URL=http://localhost:8001 cargo test
```

--------------------------------

### C++ GRPC Client SSL/TLS Configuration

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to configure SSL/TLS settings for the C++ gRPC client. Ensure server-side SSL/TLS is also configured.

```c++
#include "triton/core/tritonbackend.h"
#include "triton/core/tritonserver.h"
#include "triton/core/tritonutils.h"
#include "triton/core/tritonclient.h"
#include "triton/core/tritonclient_grpc.h"

#include <iostream>
#include <memory>
#include <string>
#include <vector>

int main(int argc, char** argv)
{
    std::shared_ptr<triton::client::GRPCClient> client;
    triton::client::Error err;

    // Create GRPC client with SSL options
    std::unique_ptr<triton::client::GRPCClientOptions> options;
    err = triton::client::GRPCClientOptions::Create( 
        &options, "localhost:8001", 
        triton::client::GRPCClientOptions::TritonClientKind::ASYNC, 
        true, 
        "root.pem", 
        "client.key", 
        "client.pem");

    if (!err.IsOk()) {
        std::cerr << "failed to create GRPCClientOptions: " << err << std::endl;
        return 1;
    }

    err = triton::client::GRPCClient::Create(&client, *options);
    if (!err.IsOk()) {
        std::cerr << "failed to create GRPCClient: " << err << std::endl;
        return 1;
    }

    std::cout << "Successfully created GRPC client with SSL/TLS enabled." << std::endl;

    // ... rest of the client logic ...

    return 0;
}
```

--------------------------------

### Build Simple gRPC Shared Memory Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_shm_client executable, which utilizes shared memory utilities. It is linked against the static gRPC client library.

```cmake
add_executable(
    simple_grpc_shm_client
    simple_grpc_shm_client.cc
    $<TARGET_OBJECTS:shm-utils-library>
  )
  target_link_libraries(
    simple_grpc_shm_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_shm_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Connection Options

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Illustrates how to configure connection options for the Triton client, including timeouts, message size, and keep-alive settings.

```APIDOC
## Connection Options

```rust
use std::time::Duration;
use triton_client::client::{ClientOptions, TritonClient};

let options = ClientOptions::default()
    .connect_timeout(Duration::from_secs(10))
    .request_timeout(Duration::from_secs(60))
    .max_message_size(256 * 1024 * 1024)
    .keep_alive_interval(Duration::from_secs(30))
    .keep_alive_timeout(Duration::from_secs(10));

let client = TritonClient::connect_with_options("http://localhost:8001", options).await?;
```
```

--------------------------------

### Run Javascript Client

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/javascript/README.md

Execute the Node.js client script, providing the host and port of the Triton inference server. Defaults are 'localhost' and '8001' respectively.

```bash
node client.js <host> <port>
```

--------------------------------

### Add triton-client to Cargo.toml

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Add the triton-client library and tokio to your project's Cargo.toml file to manage dependencies.

```toml
[dependencies]
triton-client = { path = "src/rust/triton-client" }
tokio = { version = "1", features = ["full"] }
```

--------------------------------

### Build Simple gRPC Keepalive Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/examples/CMakeLists.txt

Configures the build for the simple_grpc_keepalive_client executable. This client is linked against the static gRPC client library.

```cmake
add_executable(simple_grpc_keepalive_client simple_grpc_keepalive_client.cc)
  target_link_libraries(
    simple_grpc_keepalive_client
    PRIVATE
      grpcclient_static
  )
  install(
    TARGETS simple_grpc_keepalive_client
    RUNTIME DESTINATION bin
  )
```

--------------------------------

### Build HTTP Client Object Library

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Creates an object library for the HTTP client, used for both static and shared builds.

```cmake
add_library(
      http-client-library EXCLUDE_FROM_ALL OBJECT
      ${REQUEST_SRCS} ${REQUEST_HDRS}
  )
```

--------------------------------

### Create Shared HTTP Client Library

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Builds the shared version of the HTTP client library from the object library.

```cmake
add_library(
      httpclient SHARED
      $<TARGET_OBJECTS:http-client-library>
  )
  add_library(
      TritonClient::httpclient ALIAS httpclient
  )
```

--------------------------------

### Model Management

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Shows how to list available models in the Triton repository and how to load or unload models programmatically.

```APIDOC
## Model Management

```rust
// List available models
let models = client.repository_index().await?;
for model in &models {
    println!("{} v{} [{}]", model.name, model.version, model.state);
}

// Load / unload models
client.load_model("my_model").await?;
client.unload_model("my_model").await?;
```
```

--------------------------------

### Send Batch of Images with C++ Client

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Perform inferencing on a batch of images using the -b flag with the C++ image_client. The client repeats images if the batch size exceeds the number of specified images.

```bash
$ image_client -m inception_graphdef -s INCEPTION -c 3 -b 2 qa/images/mug.jpg
Request 0, batch size 2
Image 'qa/images/mug.jpg':
    0.754130 (505) = COFFEE MUG
    0.157077 (969) = CUP
    0.002880 (968) = ESPRESSO
Image 'qa/images/mug.jpg':
    0.754130 (505) = COFFEE MUG
    0.157077 (969) = CUP
    0.002880 (968) = ESPRESSO
```

--------------------------------

### Register and Use Custom Plugin

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Register a custom plugin with the `InferenceServerClient` to automatically modify headers for subsequent inference requests.

```python
from tritonclient.http import InferenceServerClient

client = InferenceServerClient(...)

# Register the plugin
my_plugin = MyPlugin()
client.register_plugin(my_plugin)

# All the method calls will update the headers according to the plugin
# implementation.
client.infer(...)
```

--------------------------------

### Link ZLIB for Static HTTP Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Links the ZLIB library if ZLIB is enabled for the static HTTP client.

```cmake
if(${TRITON_ENABLE_ZLIB})
    target_link_libraries(
      httpclient_static
      PRIVATE ZLIB::ZLIB
    )
  endif() # TRITON_ENABLE_ZLIB
```

--------------------------------

### Configure CMake Build for Triton Client

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Configure the CMake build for Triton Client, enabling various components and features. Adjust flags based on required components.

```bash
cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=$(pwd)/install \
  -DTRITON_ENABLE_CC_HTTP=ON \
  -DTRITON_ENABLE_CC_GRPC=ON \
  -DTRITON_ENABLE_PYTHON_HTTP=ON \
  -DTRITON_ENABLE_PYTHON_GRPC=ON \
  -DTRITON_ENABLE_JAVA_HTTP=ON \
  -DTRITON_ENABLE_EXAMPLES=ON \
  -DTRITON_ENABLE_TESTS=ON \
  -DTRITON_ENABLE_GPU=ON \
  -DTRITON_ENABLE_ZLIB=ON \
  ..
```

--------------------------------

### Python AsyncIO GRPC Infer Client

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to perform inference using the Python gRPC client with AsyncIO syntax. This is a beta feature and may change.

```python
import asyncio
import tritonclient.grpc as grpcclient

async def main():
    # Create an async GRPC client
    triton_client = grpcclient.InferenceServerClient(url='localhost:8001')

    # Example inference request (replace with actual model and inputs)
    model_name = 'my_model'
    inputs = [
        grpcclient.InferInput('input_name', [1, 10], 'INT32')
    ]
    inputs[0].set_data_from_numpy(np.random.rand(1, 10).astype(np.int32))

    # Perform async inference
    results = await triton_client.infer(model_name, inputs)

    print(f'Inference successful: {results}')

if __name__ == '__main__':
    asyncio.run(main())

```

--------------------------------

### Python GRPC Client SSL/TLS Configuration

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Demonstrates how to configure SSL/TLS settings for the Python gRPC client. Ensure server-side SSL/TLS is also configured.

```python
import tritonclient.grpc as grpcclient

# Create GRPC client with SSL options
triton_client = grpcclient.InferenceServerClient(
    url='localhost:8001',
    ssl=True,
    root_certificates='root.pem',
    private_key='client.key',
    certificate_chain='client.pem')

print('Successfully created GRPC client with SSL/TLS enabled.')

# ... rest of the client logic ...
```

--------------------------------

### Link Dependencies for Static HTTP Client

Source: https://github.com/triton-inference-server/client/blob/main/src/c++/library/CMakeLists.txt

Links necessary libraries (triton-common-json, libcurl, Threads) for the static HTTP client.

```cmake
target_link_libraries(
      httpclient_static
      PRIVATE triton-common-json
      PUBLIC CURL::libcurl
      PUBLIC Threads::Threads
  )
```

--------------------------------

### Compile Library to Generate GRPC Stubs

Source: https://github.com/triton-inference-server/client/blob/main/src/grpc_generated/java/README.md

Compile the library using Maven to generate the GRPC Java stubs. These generated files will be located in the target/generated-sources/protobuf/java directory.

```bash
mvn compile
```

--------------------------------

### Run Image Classification with C++ Client (GRPC)

Source: https://github.com/triton-inference-server/client/blob/main/README.md

Send an inference request using the C++ image_client with the GRPC protocol. Specify the GRPC endpoint using -u and the protocol using -i grpc.

```bash
$ image_client -i grpc -u localhost:8001 -m inception_graphdef -s INCEPTION qa/images/mug.jpg
Request 0, batch size 1
Image 'qa/images/mug.jpg':
    0.754130 (505) = COFFEE MUG
```

--------------------------------

### Declare and Fetch Third-Party Repository

Source: https://github.com/triton-inference-server/client/blob/main/CMakeLists.txt

Declares and fetches a third-party Git repository using FetchContent. This is used to manage external dependencies for the project.

```cmake
include(FetchContent)

FetchContent_Declare(
  repo-third-party
  GIT_REPOSITORY ${TRITON_REPO_ORGANIZATION}/third_party.git
  GIT_TAG ${TRITON_THIRD_PARTY_REPO_TAG}
  GIT_SHALLOW ON
)
FetchContent_MakeAvailable(repo-third-party)
```

--------------------------------

### TritonClient API Reference

Source: https://github.com/triton-inference-server/client/blob/main/src/rust/triton-client/README.md

Provides a comprehensive list of methods available on the TritonClient for interacting with the Triton Inference Server.

```APIDOC
## API Reference

### `TritonClient`

| Method | Description |
|--------|-------------|
| `connect(url)` | Connect with default options |
| `connect_with_options(url, options)` | Connect with custom options |
| `is_server_live()` | Check if the server process is running |
| `is_server_ready()` | Check if the server is ready for inference |
| `is_model_ready(name, version)` | Check if a model is ready |
| `server_metadata()` | Get server name, version, extensions |
| `model_metadata(name, version)` | Get model inputs/outputs metadata |
| `model_config(name, version)` | Get full model configuration |
| `infer(request)` | Run a single inference request |
| `infer_stream(requests)` | Run streaming inference |
| `model_statistics(name, version)` | Get inference statistics |
| `repository_index()` | List models in the repository |
| `load_model(name)` | Load a model |
| `unload_model(name)` | Unload a model |
| `system_shared_memory_status(name)` | Query system shared memory |
| `cuda_shared_memory_status(name)` | Query CUDA shared memory |
| `trace_setting(model, settings)` | Get/set trace settings |
| `log_settings(settings)` | Get/set log settings |
```