inference4j

https://inference4j.github.io/inference4j
Admin
inference4j is an inference-only AI library for Java built on ONNX Runtime that provides ergonomic,...
Tokens:55,650
Snippets:493
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Quick example of DepthAnythingEstimator

Source: https://inference4j.github.io/inference4j/use-cases/depth-estimation

A minimal example showing how to initialize the estimator and process an image file.

```java
try (var estimator = DepthAnythingEstimator.builder().build()) {
    DepthMap depth = estimator.estimate(Path.of("room.jpg"));
}
```

--------------------------------

### Full TextToSql example with streaming and GenerationResult

Source: https://inference4j.github.io/inference4j/use-cases/text-to-sql

Complete runnable example that builds a T5SqlGenerator with maxNewTokens(200), generates SQL for multiple questions, and streams tokens to stdout. Uses GenerationResult to print generated token count and duration.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.nlp.T5SqlGenerator;

public class TextToSql {
    public static void main(String[] args) {
        String schema = "CREATE TABLE employees (id INT, name VARCHAR, department VARCHAR, salary INT); " +
                         "CREATE TABLE departments (id INT, name VARCHAR, location VARCHAR)";

        try (var sqlGen = T5SqlGenerator.t5SmallAwesome()
                .maxNewTokens(200)
                .build()) {

            String[] questions = {
                "What is the average salary by department?",
                "List all employees in New York",
                "Which department has the most employees?"
            };

            for (String question : questions) {
                GenerationResult result = sqlGen.generateSql(question, schema,
                        token -> System.out.print(token));
                System.out.println();
                System.out.printf("  → %d tokens in %,d ms%n",
                        result.generatedTokens(), result.duration().toMillis());
            }
        }
    }
}

```

--------------------------------

### Quick example: transcribe audio with WhisperSpeechModel

Source: https://inference4j.github.io/inference4j/generative-ai/whisper

Minimal setup for transcribing an audio file with WhisperSpeechModel. Requires the model artifact inference4j/whisper-small-genai, which is not yet available on HuggingFace.

```java
try (var whisper = WhisperSpeechModel.builder()
        .modelId("inference4j/whisper-small-genai")
        .build()) {
    System.out.println(whisper.transcribe(Path.of("meeting.wav")).text());
}

```

--------------------------------

### Full example: OnnxTextGenerator with sampling parameters and GenerationResult

Source: https://inference4j.github.io/inference4j/generative-ai/native-text-generation

Full example using the generic builder with Qwen2 preset, configuring maxNewTokens, temperature, topK, and topP. The GenerationResult provides text(), generatedTokens(), and duration(). GPT-2 defaults to greedy decoding (temperature=0) producing repetitive text; sampling improves output for instruct models.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.nlp.OnnxTextGenerator;

public class TextGeneration {
    public static void main(String[] args) {
        try (var gen = OnnxTextGenerator.qwen2()
                .maxNewTokens(100)
                .temperature(0.8f)
                .topK(50)
                .topP(0.9f)
                .build()) {

            GenerationResult result = gen.generate("The meaning of life is");

            System.out.println(result.text());
            System.out.printf("%d tokens in %,d ms%n",
                    result.generatedTokens(), result.duration().toMillis());
        }
    }
}
```

--------------------------------

### Phi-3 formatted prompt example

Source: https://inference4j.github.io/inference4j/generative-ai/chat-templates

Example output of the Phi-3 ChatTemplate for the user message "What is Java?". Shows the formatted prompt with the user and assistant markers that the tokenizer will encode.

```text
<|user|>
What is Java?<|end|>
<|assistant|>

```

--------------------------------

### Full example with custom configuration

Source: https://inference4j.github.io/inference4j/use-cases/voice-activity-detection

A complete implementation demonstrating how to configure the detector with specific thresholds and iterate through the detected segments to print their details.

```java
import io.github.inference4j.audio.SileroVadDetector;
import io.github.inference4j.audio.VoiceSegment;
import java.nio.file.Path;

public class VoiceActivityDetection {
    public static void main(String[] args) {
        try (var vad = SileroVadDetector.builder()
                .threshold(0.7f)
                .minSpeechDuration(0.3f)
                .minSilenceDuration(0.15f)
                .build()) {

            List<VoiceSegment> segments = vad.detect(Path.of("meeting.wav"));

            for (VoiceSegment seg : segments) {
                System.out.printf("%.2fs - %.2fs (duration: %.2fs, confidence: %.2f)%n",
                    seg.start(), seg.end(), seg.duration(), seg.confidence());
            }
        }
    }
}
```

--------------------------------

### Quick example: generate description

Source: https://inference4j.github.io/inference4j/generative-ai/phi-vision

Builds a VisionLanguageModel with the Phi-3.5 Vision model and generates a description for an image. The result is printed to the console.

```java
try (var vision = VisionLanguageModel.builder()
        .model(ModelSources.phi3Vision())
        .build()) {
    GenerationResult result = vision.generate(
            new VisionInput(Path.of("cat.jpg"), "Describe this image."));
    System.out.println(result.text());
}

```

```text
The image shows a close-up of an orange tabby cat with a white collar.
The cat's eyes are green, and it has a neutral expression. The background
is blurred, with hints of a red object and a green plant.

```

--------------------------------

### Configuring Session Options

Source: https://inference4j.github.io/inference4j/guides/hardware-acceleration

Example of using the sessionOptions builder method to enable CoreML and set intra-op threads.

```java
.sessionOptions(opts -> {
    opts.addCoreML();
    opts.setIntraOpNumThreads(4);
})
```

--------------------------------

### Full example of DepthEstimation

Source: https://inference4j.github.io/inference4j/use-cases/depth-estimation

A complete implementation demonstrating depth estimation, accessing depth map metadata, and saving the result as an image.

```java
import io.github.inference4j.vision.Colormap;
import io.github.inference4j.vision.DepthAnythingEstimator;
import io.github.inference4j.vision.DepthMap;
import io.github.inference4j.vision.ImageAnnotator;
import java.nio.file.Path;

public class DepthEstimation {
    public static void main(String[] args) throws Exception {
        try (var estimator = DepthAnythingEstimator.builder().build()) {
            DepthMap depth = estimator.estimate(Path.of("cat.jpg"));

            System.out.printf("Size: %d x %d%n", depth.width(), depth.height());
            System.out.printf("Range: %.3f to %.3f%n", depth.min(), depth.max());

            // Larger values are closer to the camera
            System.out.printf("Centre depth: %.3f%n",
                depth.at(depth.width() / 2, depth.height() / 2));

            ImageAnnotator.save(depth.toImage(Colormap.TURBO), Path.of("depth.png"));
        }
    }
}
```

--------------------------------

### Full example: SpeechToText class with Wav2Vec2Recognizer

Source: https://inference4j.github.io/inference4j/use-cases/speech-to-text

Complete runnable class showing imports and the full transcription workflow. Uses try-with-resources and prints the transcribed text.

```java
import io.github.inference4j.audio.Wav2Vec2Recognizer;
import io.github.inference4j.audio.Transcription;
import java.nio.file.Path;

public class SpeechToText {
    public static void main(String[] args) {
        try (var recognizer = Wav2Vec2Recognizer.builder().build()) {
            Transcription result = recognizer.transcribe(Path.of("speech.wav"));
            System.out.println(result.text());
        }
    }
}

```

--------------------------------

### Quick example: transcribe audio.wav with Wav2Vec2Recognizer

Source: https://inference4j.github.io/inference4j/use-cases/speech-to-text

Minimal usage for transcribing a WAV file from a Path. The recognizer is closed automatically via try-with-resources.

```java
try (var recognizer = Wav2Vec2Recognizer.builder().build()) {
    System.out.println(recognizer.transcribe(Path.of("audio.wav")).text());
}

```

--------------------------------

### Quick translation with MarianMT and Flan-T5

Source: https://inference4j.github.io/inference4j/use-cases/translation

Quick examples for MarianMT (fixed language pair) and Flan-T5 (flexible) translation. The MarianMT example uses the model "inference4j/opus-mt-en-fr" and translates English to French. The Flan-T5 example uses the base model and specifies source and target languages via Language.EN and Language.FR.

```java
try (var translator = MarianTranslator.builder()
        .modelId("inference4j/opus-mt-en-fr")
        .build()) {
    String french = translator.translate("The weather is beautiful today.");
    System.out.println(french); // Le temps est beau aujourd'hui.
}

```

```java
try (var translator = FlanT5TextGenerator.flanT5Base().build()) {
    String french = translator.translate("The weather is beautiful today.",
            Language.EN, Language.FR);
    System.out.println(french);
}

```

--------------------------------

### Full text generation with GenerationResult

Source: https://inference4j.github.io/inference4j/generative-ai/text-generation

Full example showing how to configure generation with maxLength and temperature, and how to access the GenerationResult for token count and duration. Use this when you need to control generation parameters and inspect timing information.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.genai.ModelSources;
import io.github.inference4j.genai.nlp.TextGenerator;

public class TextGeneration {
    public static void main(String[] args) {
        try (var generator = TextGenerator.builder()
                .model(ModelSources.phi3Mini())
                .maxLength(200)
                .temperature(0.7)
                .build()) {

            GenerationResult result = generator.generate("What is Java in one sentence?");

            System.out.println(result.text());
            System.out.printf("%d tokens in %,d ms%n",
                    result.generatedTokens(), result.duration().toMillis());
        }
    }
}

```

--------------------------------

### Enable CoreML hardware acceleration

Source: https://inference4j.github.io/inference4j/use-cases/depth-estimation

Builds a DepthAnythingEstimator with CoreML session options for hardware acceleration, then estimates depth from an image file. The example uses the default model.

```java
try (var estimator = DepthAnythingEstimator.builder()
        .sessionOptions(opts -> opts.addCoreML())
        .build()) {
    estimator.estimate(Path.of("room.jpg"));
}

```

--------------------------------

### Custom Pipeline Builder

Source: https://inference4j.github.io/inference4j/contributing/adding-a-wrapper

Example of configuring a custom ImageTransformPipeline using the builder for models with specific normalization requirements.

```java
ImageTransformPipeline pipeline = ImageTransformPipeline.builder()
    .resize(280)
    .mean(new float[]{127f/255, 127f/255, 127f/255})
    .std(new float[]{128f/255, 128f/255, 128f/255})
    .layout(ImageLayout.NHWC)
    .build();
```

--------------------------------

### Quick start: classify an image with ClipClassifier

Source: https://inference4j.github.io/inference4j/use-cases/visual-search

Builds a ClipClassifier and classifies an image against a list of labels, printing the top result's label and confidence.

```java
try (ClipClassifier classifier = ClipClassifier.builder().build()) {
    List<Classification> results = classifier.classify(
            Path.of("photo.jpg"), List.of("cat", "dog", "bird", "car", "airplane"));
    System.out.println(results.get(0).label());      // "cat"
    System.out.println(results.get(0).confidence());  // 0.92
}
```

--------------------------------

### Quick example: Recognize entities with default model

Source: https://inference4j.github.io/inference4j/use-cases/named-entity-recognition

Builds a BertNerRecognizer with default settings and recognizes entities in a sample sentence. The output shows the recognized entities with their labels.

```java
try (var ner = BertNerRecognizer.builder().build()) {
    List<NamedEntity> entities = ner.recognize("John works at Google in London.");
    // [NamedEntity[text=John, label=PER], NamedEntity[text=Google, label=ORG],
    //  NamedEntity[text=London, label=LOC]]
}
```

--------------------------------

### ImageNet Preset Pipeline

Source: https://inference4j.github.io/inference4j/contributing/adding-a-wrapper

Example of initializing an ImageTransformPipeline using the ImageNet preset for models like ResNet or VGG.

```java
ImageTransformPipeline pipeline = ImageTransformPipeline.imagenet(224);
// mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225]
// NCHW layout, size 224×224
```

--------------------------------

### Quick example: OnnxTextGenerator presets for GPT-2, SmolLM2, TinyLlama, Qwen2

Source: https://inference4j.github.io/inference4j/generative-ai/native-text-generation

Shows one-liner usage of the named presets (gpt2, smolLM2, tinyLlama, qwen2) to generate text with default settings. Each preset is a completion or instruct model; instruct models use ChatML or Zephyr-style formatting. Set maxNewTokens to limit output length.

```java
// GPT-2 — completion model
try (var gen = OnnxTextGenerator.gpt2().maxNewTokens(50).build()) {
    System.out.println(gen.generate("Once upon a time").text());
}

// SmolLM2-360M — ChatML instruct model
try (var gen = OnnxTextGenerator.smolLM2().maxNewTokens(50).build()) {
    System.out.println(gen.generate("What is the capital of France?").text());
}

// TinyLlama-1.1B-Chat — Zephyr-style instruct model
try (var gen = OnnxTextGenerator.tinyLlama().maxNewTokens(100).build()) {
    System.out.println(gen.generate("Explain gravity").text());
}

// Qwen2.5-1.5B — ChatML instruct model
try (var gen = OnnxTextGenerator.qwen2().maxNewTokens(100).build()) {
    System.out.println(gen.generate("Explain gravity").text());
}
```

--------------------------------

### Quick text generation with TextGenerator

Source: https://inference4j.github.io/inference4j/generative-ai/text-generation

Minimal setup for generating text with the default Phi-3 model. Use this when you only need the generated text and can accept default generation settings.

```java
try (var generator = TextGenerator.builder()
        .model(ModelSources.phi3Mini())
        .build()) {
    System.out.println(generator.generate("What is Java?").text());
}

```

--------------------------------

### Full BartSummarizer example with streaming

Source: https://inference4j.github.io/inference4j/use-cases/summarization

Complete Java program demonstrating BartSummarizer with streaming output. Configures maxNewTokens to 150, summarizes a long article, prints tokens as they are generated, and reports generated token count and duration from the GenerationResult.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.nlp.BartSummarizer;

public class Summarization {
    public static void main(String[] args) {
        try (var summarizer = BartSummarizer.distilBartCnn()
                .maxNewTokens(150)
                .build()) {

            String article = """
                The Amazon rainforest, often referred to as the "lungs of the Earth",
                produces about 20% of the world's oxygen. Spanning across nine countries
                in South America, it is the largest tropical rainforest in the world,
                covering approximately 5.5 million square kilometers. The forest is home
                to an estimated 10% of all species on Earth, including over 40,000 plant
                species, 1,300 bird species, and 3,000 types of fish. Deforestation
                remains a critical threat, with an estimated 17% of the forest lost in
                the last 50 years due to logging, agriculture, and urban expansion.
                """;

            GenerationResult result = summarizer.summarize(article, token -> System.out.print(token));
            System.out.println();
            System.out.printf("%d tokens in %,d ms%n",
                    result.generatedTokens(), result.duration().toMillis());
        }
    }
}

```

--------------------------------

### Full example: WhisperTranscription class

Source: https://inference4j.github.io/inference4j/generative-ai/whisper

Complete Java class showing how to transcribe an audio file and print the resulting text. Includes the required imports and a main method.

```java
import io.github.inference4j.genai.audio.WhisperSpeechModel;
import io.github.inference4j.audio.Transcription;
import java.nio.file.Path;

public class WhisperTranscription {
    public static void main(String[] args) {
        try (var whisper = WhisperSpeechModel.builder()
                .modelId("inference4j/whisper-small-genai")
                .build()) {

            Transcription result = whisper.transcribe(Path.of("meeting.wav"));
            System.out.println(result.text());
        }
    }
}

```

--------------------------------

### Full example: print bounding boxes and confidence

Source: https://inference4j.github.io/inference4j/use-cases/text-detection

Complete Java program that detects text regions and prints each region's bounding box coordinates and confidence. Uses try-with-resources for the detector and iterates over the returned regions.

```java
import io.github.inference4j.vision.CraftTextDetector;
import io.github.inference4j.vision.TextRegion;
import java.nio.file.Path;

public class TextDetection {
    public static void main(String[] args) {
        try (var detector = CraftTextDetector.builder().build()) {
            List<TextRegion> regions = detector.detect(Path.of("document.jpg"));

            for (TextRegion r : regions) {
                System.out.printf("[%.1f, %.1f, %.1f, %.1f] (confidence=%.2f)%n",
                    r.box().x1(), r.box().y1(),
                    r.box().x2(), r.box().y2(),
                    r.confidence());
            }
        }
    }
}

```

--------------------------------

### Full example: ImageClassification main class

Source: https://inference4j.github.io/inference4j/use-cases/image-classification

Complete Java program that classifies an image with a specified topK (5) and prints each label with its confidence as a percentage. Requires imports for ResNetClassifier, Classification, and Path.

```java
import io.github.inference4j.vision.ResNetClassifier;
import io.github.inference4j.vision.Classification;
import java.nio.file.Path;

public class ImageClassification {
    public static void main(String[] args) {
        try (var classifier = ResNetClassifier.builder().build()) {
            List<Classification> results = classifier.classify(Path.of("cat.jpg"), 5);

            for (Classification c : results) {
                System.out.printf("%-30s %.2f%%%n", c.label(), c.confidence() * 100);
            }
        }
    }
}
```

--------------------------------

### Full example: Print entities with scores

Source: https://inference4j.github.io/inference4j/use-cases/named-entity-recognition

Complete Java program that imports required classes, builds a recognizer, and prints each entity's text, label, and confidence score. The output shows sample results with scores.

```java
import io.github.inference4j.nlp.BertNerRecognizer;
import io.github.inference4j.nlp.NamedEntity;
import java.util.List;

public class NerExample {
    public static void main(String[] args) {
        try (var ner = BertNerRecognizer.builder().build()) {
            String text = "Marie Curie worked at the University of Paris in France.";
            List<NamedEntity> entities = ner.recognize(text);

            for (NamedEntity entity : entities) {
                System.out.printf("%-20s → %s (%.2f%%)%n",
                    entity.text(), entity.label(), entity.score() * 100);
            }
            // Marie Curie          → PER (99.12%)
            // University of Paris  → ORG (98.45%)
            // France               → LOC (97.89%)
        }
    }
}
```

--------------------------------

### Full example: ObjectDetection main class

Source: https://inference4j.github.io/inference4j/use-cases/object-detection

Complete Java program that detects objects in an image and prints each detection's label, confidence percentage, and bounding box coordinates. Uses YoloV8Detector with default settings.

```java
import io.github.inference4j.vision.YoloV8Detector;
import io.github.inference4j.vision.Detection;
import java.nio.file.Path;

public class ObjectDetection {
    public static void main(String[] args) {
        try (var detector = YoloV8Detector.builder().build()) {
            List<Detection> detections = detector.detect(Path.of("street.jpg"));

            for (Detection d : detections) {
                System.out.printf("%-12s (%.0f%%) at [%.0f, %.0f, %.0f, %.0f]%n",
                    d.label(), d.confidence() * 100,
                    d.box().x1(), d.box().y1(), d.box().x2(), d.box().y2());
            }
        }
    }
}
```

--------------------------------

### Quick BartSummarizer example

Source: https://inference4j.github.io/inference4j/use-cases/summarization

Minimal usage of BartSummarizer with the distilBartCnn preset. Builds a summarizer, calls summarize on a string, and prints the result. The summarizer is closed automatically via try-with-resources.

```java
try (var summarizer = BartSummarizer.distilBartCnn().build()) {
    String summary = summarizer.summarize("Long article text here...");
    System.out.println(summary);
}

```

--------------------------------

### Full example: classify multiple reviews with DistilBertTextClassifier

Source: https://inference4j.github.io/inference4j/use-cases/sentiment-analysis

Complete Java program that classifies a list of reviews and prints the top label and confidence percentage for each. Uses try-with-resources to manage the classifier.

```java
import io.github.inference4j.nlp.DistilBertTextClassifier;
import io.github.inference4j.nlp.TextClassification;
import java.util.List;

public class SentimentAnalysis {
    public static void main(String[] args) {
        try (var classifier = DistilBertTextClassifier.builder().build()) {
            List<String> reviews = List.of(
                "This movie was fantastic!",
                "Terrible experience, would not recommend.",
                "It was okay, nothing special."
            );

            for (String review : reviews) {
                List<TextClassification> results = classifier.classify(review);
                TextClassification top = results.get(0);
                System.out.printf("%-45s → %s (%.2f%%)%n",
                    review, top.label(), top.confidence() * 100);
            }
        }
    }
}
```

--------------------------------

### Use a custom Depth Anything V2 model source

Source: https://inference4j.github.io/inference4j/use-cases/depth-estimation

Shows how to point the estimator at a custom model repository by setting a LocalModelSource and a modelId. The model.onnx file must be at the repository root. This example uses the 'depth-anything-v2-base' variant, which is not hosted under the inference4j org.

```java
try (var estimator = DepthAnythingEstimator.builder()
        .modelSource(new LocalModelSource(Path.of("/models")))
        .modelId("depth-anything-v2-base")
        .build()) {
    DepthMap depth = estimator.estimate(image);
}

```

--------------------------------

### Full CoeditGrammarCorrector example with GenerationResult

Source: https://inference4j.github.io/inference4j/use-cases/grammar-correction

Full example that corrects multiple sentences with CoeditGrammarCorrector.coeditBase(). Uses maxNewTokens(200), streams tokens via a callback, and prints the generated token count and duration from the returned GenerationResult.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.nlp.CoeditGrammarCorrector;

public class GrammarCorrection {
    public static void main(String[] args) {
        try (var corrector = CoeditGrammarCorrector.coeditBase()
                .maxNewTokens(200)
                .build()) {

            String[] sentences = {
                "She don't likes swimming.",
                "Me and him went to the store yesterday.",
                "The informations is very useful for we."
            };

            for (String sentence : sentences) {
                GenerationResult result = corrector.correct(sentence,
                        token -> System.out.print(token));
                System.out.println();
                System.out.printf("  → %d tokens in %,d ms%n",
                        result.generatedTokens(), result.duration().toMillis());
            }
        }
    }
}

```

--------------------------------

### Extend the base class for ResNetClassifier

Source: https://inference4j.github.io/inference4j/contributing/adding-a-wrapper

Example of extending AbstractImageClassifier to create a custom classifier wrapper.

```java
public class ResNetClassifier extends AbstractImageClassifier {
    // AbstractImageClassifier extends AbstractInferenceTask<BufferedImage, List<Classification>>
    // and implements ImageClassifier (which extends Classifier<BufferedImage, Classification>)
}
```

--------------------------------

### Pin starter to 0.10.1 for Spring Boot 3 (Gradle and Maven)

Source: https://inference4j.github.io/inference4j/guides/spring-boot

Pin the starter to the last Boot 3 release (0.10.1) while letting inference4j-core float independently. The core has no Spring dependency, so it can stay current while the starter remains pinned; the 0.10.1 starter will not receive fixes.

```groovy
implementation 'io.github.inference4j:inference4j-spring-boot-starter:0.10.1'
implementation 'io.github.inference4j:inference4j-core:${inference4jVersion}'

```

```xml
<dependency>
    <groupId>io.github.inference4j</groupId>
    <artifactId>inference4j-spring-boot-starter</artifactId>
    <version>0.10.1</version>
</dependency>
<dependency>
    <groupId>io.github.inference4j</groupId>
    <artifactId>inference4j-core</artifactId>
    <version>${inference4jVersion}</version>
</dependency>

```

--------------------------------

### Speech-to-Text with Wav2Vec2Recognizer

Source: https://inference4j.github.io/inference4j

Use for speech-to-text with Wav2Vec2Recognizer. The transcribe method takes a Path to an audio file and returns an object with a text method to get the transcription.

```java
try (var recognizer = Wav2Vec2Recognizer.builder().build()) {
    System.out.println(recognizer.transcribe(Path.of("audio.wav")).text());
}
```

--------------------------------

### BPE tokenizer encoding with BpeTokenizer.fromFiles

Source: https://inference4j.github.io/inference4j/guides/tokenizers

Creates a BPE tokenizer from vocabulary and merges files, then encodes a sample sentence. The comments show the resulting input IDs, attention mask, and padding behavior.

```java
Tokenizer tokenizer = WordPieceTokenizer.fromFiles(
        Path.of("vocab.json"), Path.of("merges.txt"));
EncodedInput encoded = tokenizer.encode("a photo of a cat");
// encoded.inputIds()      → [49406, 320, 1125, 539, 320, 2368, 49407, 0, ...]
// encoded.attentionMask() → [1, 1, 1, 1, 1, 1, 1, 0, ...]

```

--------------------------------

### Custom thresholds per-call

Source: https://inference4j.github.io/inference4j/use-cases/object-detection

Override confidence and IoU thresholds for a single detect call. The example uses stricter confidence (0.5) and tighter NMS (0.3).

```java
try (var detector = YoloV8Detector.builder().build()) {
    // Stricter confidence, tighter NMS
    List<Detection> detections = detector.detect(
        Path.of("image.jpg"), 0.5f, 0.3f);
}
```

--------------------------------

### Full example: ImageDescription class

Source: https://inference4j.github.io/inference4j/generative-ai/phi-vision

Complete Java class that describes an image and asks a follow-up question about it. Uses the VisionLanguageModel builder with the Phi-3.5 Vision model.

```java
import io.github.inference4j.generation.GenerationResult;
import io.github.inference4j.genai.ModelSources;
import io.github.inference4j.genai.vision.VisionInput;
import io.github.inference4j.genai.vision.VisionLanguageModel;
import java.nio.file.Path;

public class ImageDescription {
    public static void main(String[] args) {
        Path image = Path.of("cat.jpg");

        try (var vision = VisionLanguageModel.builder()
                .model(ModelSources.phi3Vision())
                .build()) {

            // Describe an image
            GenerationResult description = vision.generate(
                    new VisionInput(image, "Describe this image."));
            System.out.println(description.text());

            // Ask a question about it
            GenerationResult answer = vision.generate(
                    new VisionInput(image, "What colors are prominent in this image?"));
            System.out.println(answer.text());
        }
    }
}

```

--------------------------------

### Inspect available execution provider devices

Source: https://inference4j.github.io/inference4j/guides/hardware-acceleration

Iterates through all available execution provider devices and prints their name, vendor, and device information.

```java
for (OrtEpDevice d : OrtEnvironment.getEnvironment().getEpDevices()) {
    System.out.println(d.getEpName() + " (" + d.getEpVendor() + ") -> " + d.getDevice());
}
```

--------------------------------

### Quick example: ResNetClassifier

Source: https://inference4j.github.io/inference4j/use-cases/image-classification

Minimal usage of ResNetClassifier to classify an image from a file path. The default topK is 5, and the result is a list of Classification records with label and confidence.

```java
try (var classifier = ResNetClassifier.builder().build()) {
    List<Classification> results = classifier.classify(Path.of("cat.jpg"));
    // [Classification[label=tabby cat, confidence=0.87], ...]
}
```

--------------------------------

### Text Embeddings with SentenceTransformerEmbedder

Source: https://inference4j.github.io/inference4j

Use for text embeddings and semantic search with SentenceTransformerEmbedder. Specify the model ID 'inference4j/all-MiniLM-L6-v2' and call encode to get a float array embedding.

```java
try (var embedder = SentenceTransformerEmbedder.builder()
        .modelId("inference4j/all-MiniLM-L6-v2").build()) {
    float[] embedding = embedder.encode("Hello, world!");
}
```

--------------------------------

### Hardware acceleration: enable CoreML

Source: https://inference4j.github.io/inference4j/use-cases/text-detection

Enables CoreML acceleration via sessionOptions. The builder configures the ONNX Runtime session with CoreML, then detects text in an image.

```java
try (var detector = CraftTextDetector.builder()
        .sessionOptions(opts -> opts.addCoreML())
        .build()) {
    detector.detect(Path.of("document.jpg"));
}

```

--------------------------------

### Preconfigured chat template with ModelSources.phi3Mini()

Source: https://inference4j.github.io/inference4j/generative-ai/chat-templates

Shows that onnxruntime-genai models from ModelSources already have their chat template configured. No additional ChatTemplate is needed when using ModelSources.phi3Mini().

```java
// Chat template is bundled — nothing extra to configure
TextGenerator.builder()
        .model(ModelSources.phi3Mini())
        .build();

```

--------------------------------

### Sentiment Analysis with DistilBertTextClassifier

Source: https://inference4j.github.io/inference4j

Use for sentiment analysis with DistilBertTextClassifier. The classify method returns a TextClassification object with label and confidence. The example prints the result for a sample movie review.

```java
try (var classifier = DistilBertTextClassifier.builder().build()) {
    System.out.println(classifier.classify("This movie was fantastic!"));
    // [TextClassification[label=POSITIVE, confidence=0.9998]]
}
```

--------------------------------

### Audio transcription with Wav2Vec2Recognizer

Source: https://inference4j.github.io/inference4j/getting-started/quickstart

Builds a Wav2Vec2Recognizer and transcribes an audio file. The transcription text is printed.

```java
import io.github.inference4j.audio.Wav2Vec2Recognizer;

try (var recognizer = Wav2Vec2Recognizer.builder().build()) {
    System.out.println(recognizer.transcribe(Path.of("speech.wav")).text());
}

```
Tokens:
Raw