### Example Anti-Prompt Configuration

Source: https://github.com/scisharp/llamasharp/blob/master/docs/FAQ.md

This example shows the content of a prompt file and suggests setting the anti-prompt to 'User:'. This helps prevent infinite output generation in chat sessions.

```text
Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.

User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in Europe.
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
User:
```

--------------------------------

### Run LLamaSharp Kernel Memory Example

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/KernelMemorySaveAndLoad.md

Main entry point for the Kernel Memory integration example. It initializes memory, checks for existing storage, ingests documents if necessary, and then enters a chat session.

```csharp
public static async Task Run()
    {
        Console.ForegroundColor = ConsoleColor.Yellow;
        Console.WriteLine(
            """

            This program uses the Microsoft.KernelMemory package to ingest documents
            and store the embeddings as local files so they can be quickly recalled
            when this application is launched again. 

            """);

        string modelPath = UserSettings.GetModelPath();
        IKernelMemory memory = CreateMemoryWithLocalStorage(modelPath);

        Console.ForegroundColor = ConsoleColor.Yellow;
        if (StorageExists)
        {
            Console.WriteLine(
                """
                
                Kernel memory files have been located!
                Information about previously analyzed documents has been loaded.

                """);
        }
        else
        {
            Console.WriteLine(
                $"""

                 Existing kernel memory was not found.
                 Documents will be analyzed (slow) and information saved to disk.
                 Analysis will not be required the next time this program is run.
                 Press ENTER to proceed...
 
                 """);
            Console.ReadLine();
            await IngestDocuments(memory);
        }

        await AskSingleQuestion(memory, "What formats does KM support?");
        await StartUserChatSession(memory);
    }
```

--------------------------------

### Run LLamaSharp Kernel Memory Basic Example

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/KernelMemory.md

This is the main entry point for the Kernel Memory integration example. It sets up the memory, ingests documents, and handles interactive question answering. Ensure you have the necessary assets (sample-SK-Readme.pdf, sample-KM-Readme.pdf) in an 'Assets' folder.

```csharp
using LLamaSharp.KernelMemory;
using Microsoft.KernelMemory;
using Microsoft.KernelMemory.Configuration;
using System.Diagnostics;

namespace LLama.Examples.Examples
{
    public class KernelMemory
    {
        public static async Task Run()
        {
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine(
                """

                This program uses the Microsoft.KernelMemory package to ingest documents
                and answer questions about them in an interactive chat prompt.

                """);

            string modelPath = UserSettings.GetModelPath();
            IKernelMemory memory = CreateMemory(modelPath);

            string[] filesToIngest = [
                Path.GetFullPath(@"./Assets/sample-SK-Readme.pdf"),
                Path.GetFullPath(@"./Assets/sample-KM-Readme.pdf"),
            ];

            for (int i = 0; i < filesToIngest.Length; i++)
            {
                string path = filesToIngest[i];
                Stopwatch sw = Stopwatch.StartNew();
                Console.ForegroundColor = ConsoleColor.Blue;
                Console.WriteLine($"Importing {i + 1} of {filesToIngest.Length}: {path}");
                await memory.ImportDocumentAsync(path, steps: Constants.PipelineWithoutSummary);
                Console.WriteLine($"Completed in {sw.Elapsed}\n");
            }

            Console.ForegroundColor = ConsoleColor.Green;
            string question1 = "What formats does KM support";
            Console.WriteLine($"Question: {question1}");
            await AnswerQuestion(memory, question1);

            while (true)
            {
                Console.ForegroundColor = ConsoleColor.Green;
                Console.Write("Question: ");
                string question = Console.ReadLine()!;
                if (string.IsNullOrEmpty(question))
                    return;

                await AnswerQuestion(memory, question);
            }
        }

        private static IKernelMemory CreateMemory(string modelPath)
        {
            Common.InferenceParams infParams = new() { AntiPrompts = ["\n\n"] };

            LLamaSharpConfig lsConfig = new(modelPath) { DefaultInferenceParams = infParams };

            SearchClientConfig searchClientConfig = new() 
            {
                MaxMatchesCount = 1,
                AnswerTokens = 100,
            };

            TextPartitioningOptions parseOptions = new()
            {
                MaxTokensPerParagraph = 300,
                MaxTokensPerLine = 100,
                OverlappingTokens = 30
            };

            return new KernelMemoryBuilder()
                .WithLLamaSharpDefaults(lsConfig)
                .WithSearchClientConfig(searchClientConfig)
                .With(parseOptions)
                .Build();
        }

        private static async Task AnswerQuestion(IKernelMemory memory, string question)
        {
            Stopwatch sw = Stopwatch.StartNew();
            Console.ForegroundColor = ConsoleColor.DarkGray;
            Console.WriteLine($"Generating answer...");

            MemoryAnswer answer = await memory.AskAsync(question);
            Console.WriteLine($"Answer generated in {sw.Elapsed}");

            Console.ForegroundColor = ConsoleColor.Gray;
            Console.WriteLine($"Answer: {answer.Result}");
            foreach (var source in answer.RelevantSources)
            {
                Console.WriteLine($"Source: {source.SourceName}");
            }
            Console.WriteLine();
        }
    }
}
```

--------------------------------

### InstructExecutor Example Prompt (alpaca)

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Tutorials/Executors.md

An example prompt structure for the InstructExecutor, using the 'alpaca' style. This format is designed for scenarios where the LLM should follow specific instructions.

```text
// alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.
```

--------------------------------

### StatelessExecutor Example Prompt and Response

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Tutorials/Executors.md

An example demonstrating how to use the StatelessExecutor with a question-answer prompt. The executor focuses solely on the current input, unaffected by previous interactions.

```text
Q: Who is Trump? A: 
Donald J. Trump, born June 14, 1946, is an American businessman, television personality, politician and the 45th President of the United States (2017-2021). # Anexo:Torneo de Hamburgo 2022 (individual masculino)
```

--------------------------------

### Install LLamaSharp Package

Source: https://github.com/scisharp/llamasharp/blob/master/README.md

Use the Package Manager Console to install the main LLamaSharp NuGet package.

```powershell
PM> Install-Package LLamaSharp
```

--------------------------------

### InteractiveExecutor Example Prompt (chat-with-bob)

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Tutorials/Executors.md

An example prompt demonstrating the 'chat-with-bob' persona for the InteractiveExecutor. This setup is suitable for continuous dialogue where the LLM acts as a helpful assistant.

```text
// chat-with-bob

Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.

User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in Europe.
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
User:
```

--------------------------------

### DefaultSamplingPipeline Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.sampling.defaultsamplingpipeline.md

Initializes a new instance of the DefaultSamplingPipeline class. No specific setup is required beyond instantiation.

```csharp
public DefaultSamplingPipeline()
```

--------------------------------

### Get Default LLamaContextParams

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.llamacontextparams.md

Retrieves the default configuration for LLamaContextParams. This is useful for starting with a standard setup before applying custom modifications.

```csharp
LLamaContextParams Default()
```

--------------------------------

### Get Text Embeddings with LLamaSharp

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/GetEmbeddings.md

Use this example to generate embeddings from a text prompt. Ensure you have set the model path correctly. Embeddings are numerical representations that capture semantic meaning.

```cs
using LLama.Common;

namespace LLama.Examples.Examples
{
    // This example shows how to get embeddings from a text prompt.
    public class GetEmbeddings
    {
        public static void Run()
        {
            string modelPath = UserSettings.GetModelPath();

            Console.ForegroundColor = ConsoleColor.DarkGray;
            var @params = new ModelParams(modelPath) { EmbeddingMode = true };
            using var weights = LLamaWeights.LoadFromFile(@params);
            var embedder = new LLamaEmbedder(weights, @params);

            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine(
                """
                This example displays embeddings from a text prompt.
                Embeddings are numerical codes that represent information like words, images, or concepts.
                These codes capture important relationships between those objects, 
                like how similar words are in meaning or how close images are visually.
                This allows machine learning models to efficiently understand and process complex data.
                Embeddings of a text in LLM is sometimes useful, for example, to train other MLP models.
                """ // NOTE: this description was AI generated
                );

            while (true)
            {
                Console.ForegroundColor = ConsoleColor.White;
                Console.Write("Please input your text: ");
                Console.ForegroundColor = ConsoleColor.Green;
                var text = Console.ReadLine();
                Console.ForegroundColor = ConsoleColor.White;

                float[] embeddings = embedder.GetEmbeddings(text).Result;
                Console.WriteLine($"Embeddings contain {embeddings.Length:N0} floating point values:");
                Console.ForegroundColor = ConsoleColor.DarkGray;
                Console.WriteLine(string.Join(", ", embeddings.Take(20)) + ", ");
                Console.WriteLine();
            }
        }
    }
}

```

--------------------------------

### Get System Information

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.systeminfo.md

Retrieves the system information for the current machine. This method may throw a PlatformNotSupportedException if the platform is not supported.

```csharp
public static SystemInfo Get()
```

--------------------------------

### SystemInfo Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.systeminfo.md

Initializes a new instance of the SystemInfo class with specified OS platform, CUDA major version, and Vulkan version.

```csharp
public SystemInfo(OSPlatform OSPlatform, int CudaMajorVersion, string VulkanVersion)
```

--------------------------------

### BaseSamplingPipeline Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.sampling.basesamplingpipeline.md

Initializes a new instance of the BaseSamplingPipeline class, designed to wrap a llama.cpp sampler chain. No specific setup is required beyond instantiation.

```csharp
public BaseSamplingPipeline()
```

--------------------------------

### Run Coding Assistant Example in C#

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/CodingAssistant.md

This C# code sets up and runs a coding assistant using LlamaSharp. It requires a Code Llama model and allows users to input instructions for code generation. Ensure you have a compatible model like 'codellama-7b-instruct.Q4_K_S.gguf' for optimal results.

```csharp
namespace LLama.Examples.Examples
{
    using LLama.Common;
    using System;

    // This example shows how to apply code completion as a coding assistant
    internal class CodingAssistant
    {
        // Source paper with example prompts:
        // https://doi.org/10.48550/arXiv.2308.12950
        const string InstructionPrefix = "[INST]";
        const string InstructionSuffix = "[/INST]";
        const string SystemInstruction = "You're an intelligent, concise coding assistant. " + 
            "Wrap code in ``` for readability. Don't repeat yourself. " + 
            "Use best practice and good coding standards.";

        public static async Task Run()
        {
            string modelPath = UserSettings.GetModelPath();
            if (!modelPath.Contains("codellama", StringComparison.InvariantCultureIgnoreCase))
            {
                Console.ForegroundColor = ConsoleColor.Yellow;
                Console.WriteLine("WARNING: the model you selected is not a Code LLama model!");
                Console.WriteLine("For this example we specifically recommend 'codellama-7b-instruct.Q4_K_S.gguf'");
                Console.WriteLine("Press ENTER to continue...");
                Console.ReadLine();
            }

            var parameters = new ModelParams(modelPath)
            {
                ContextSize = 4096
            };
            using var model = LLamaWeights.LoadFromFile(parameters);
            using var context = model.CreateContext(parameters);
            var executor = new InstructExecutor(context, InstructionPrefix, InstructionSuffix, null);

            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("The executor has been enabled. In this example, the LLM will follow your instructions." + 
                "\nIt's a 7B Code Llama, so it's trained for programming tasks like \"Write a C# function reading " + 
                "a file name from a given URI\" or \"Write some programming interview questions\"." + 
                "\nWrite 'exit' to exit");
            Console.ForegroundColor = ConsoleColor.White;

            var inferenceParams = new InferenceParams()
            {
                Temperature = 0.8f,
                MaxTokens = -1,
            };

            string instruction = $"{SystemInstruction}\n\n";
            await Console.Out.WriteAsync("Instruction: ");
            instruction += Console.ReadLine() ?? "Ask me for instructions.";
            while (instruction != "exit")
            {

                Console.ForegroundColor = ConsoleColor.Green;
                await foreach (var text in executor.InferAsync(instruction + Environment.NewLine, inferenceParams))
                {
                    Console.Write(text);
                }
                Console.ForegroundColor = ConsoleColor.White;

                await Console.Out.WriteAsync("Instruction: ");
                instruction = Console.ReadLine() ?? "Ask me for instructions.";
            }
        }
    }
}
```

--------------------------------

### Build Native Library with CMake

Source: https://github.com/scisharp/llamasharp/blob/master/CONTRIBUTING.md

Example CMake commands to build the native library, including enabling CUBLAS and shared libraries. Ensure to replace `<Your native library path>` with the actual path.

```bash
mkdir build && cd build
cmake .. -DLLAMA_CUBLAS=ON -DBUILD_SHARED_LIBS=ON
cmake --build . --config Release
```

--------------------------------

### Run Semantic Kernel Chat Example

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/SemanticKernelChat.md

This C# code demonstrates setting up and running a chat conversation using Semantic Kernel and LlamaSharp. It includes loading the model, creating a chat history, and exchanging messages between a user and an AI assistant configured as a librarian.

```csharp
using LLama.Common;
using LLamaSharp.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.ChatCompletion;

namespace LLama.Examples.Examples
{
    public class SemanticKernelChat
    {
        public static async Task Run()
        {
            string modelPath = UserSettings.GetModelPath();

            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("This example is from: 
" +
                "https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/KernelSyntaxExamples/Example17_ChatGPT.cs");

            // Load weights into memory
            var parameters = new ModelParams(modelPath);
            using var model = LLamaWeights.LoadFromFile(parameters);
            var ex = new StatelessExecutor(model, parameters);

            var chatGPT = new LLamaSharpChatCompletion(ex);

            var chatHistory = chatGPT.CreateNewChat("This is a conversation between the " +
                "assistant and the user. 

 You are a librarian, expert about books. ");

            Console.WriteLine("Chat content:");
            Console.WriteLine("------------------------");

            chatHistory.AddUserMessage("Hi, I'm looking for book suggestions");
            await MessageOutputAsync(chatHistory);

            // First bot assistant message
            var reply = await chatGPT.GetChatMessageContentAsync(chatHistory);
            chatHistory.AddAssistantMessage(reply.Content);
            await MessageOutputAsync(chatHistory);

            // Second user message
            chatHistory.AddUserMessage("I love history and philosophy, I'd like to learn " +
                "something new about Greece, any suggestion");
            await MessageOutputAsync(chatHistory);

            // Second bot assistant message
            reply = await chatGPT.GetChatMessageContentAsync(chatHistory);
            chatHistory.AddAssistantMessage(reply.Content);
            await MessageOutputAsync(chatHistory);
        }

        /// <summary>
        /// Outputs the last message of the chat history
        /// </summary>
        private static Task MessageOutputAsync(Microsoft.SemanticKernel.ChatCompletion.ChatHistory chatHistory)
        {
            var message = chatHistory.Last();

            Console.WriteLine($"{message.Role}: {message.Content}");
            Console.WriteLine("------------------------");

            return Task.CompletedTask;
        }
    }
}
```

--------------------------------

### Install and Run xmldoc2md Tool

Source: https://github.com/scisharp/llamasharp/blob/master/docs/ContributingGuide.md

Installs the XMLDoc2Markdown tool globally and demonstrates its usage for generating markdown documentation from C# XML documentation files. Ensure the path to the tool is correct for your system.

```bash
dotnet tool install -g XMLDoc2Markdown
cd LLama/bin/Debug/net8 # change the path to your bin path.
dotnet xmldoc2md LLamaSharp.dll -o ../../../../docs/xmldocs --back-button
```

```bash
C:\Users\liu_y\.dotnet\tools\xmldoc2md.exe LLamaSharp.dll  -o ../../../../docs/xmldocs --back-button
```

--------------------------------

### Prepare Native Library Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.abstractions.inativelibrary.md

Prepares the native library file and returns its local path. If the path is relative, LLamaSharp will search in the configured directories. This method is crucial for loading and initializing native dependencies.

```csharp
IEnumerable<string> Prepare(SystemInfo systemInfo, LLamaLogCallback logCallback)
```

--------------------------------

### Implement Custom Input Transforms

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Tutorials/ChatSession.md

Provides example implementations of the `ITextTransform` interface for modifying input text. These can be chained to create complex input processing pipelines.

```cs
public class MyInputTransform1 : ITextTransform
{
    public string Transform(string text)
    {
        return $"Question: {text}\n";
    }
}

public class MyInputTransform2 : ITextTransform
{
    public string Transform(string text)
    {
        return text + "Answer: ";
    }
}

session.AddInputTransform(new MyInputTransform1()).AddInputTransform(new MyInputTransform2());
```

--------------------------------

### Compile Native Library with CUDA

Source: https://github.com/scisharp/llamasharp/blob/master/docs/ContributingGuide.md

Example CMake command to build the native library with CUDA support and shared libraries enabled. After compilation, the library files will be in the 'build/src' directory.

```bash
mkdir build && cd build
cmake .. -DGGML_CUDA=ON -DBUILD_SHARED_LIBS=ON
cmake --build . --config Release
```

--------------------------------

### Sample Data for Semantic Memory

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/SemanticKernelMemory.md

Provides a dictionary of sample GitHub file URLs and their corresponding descriptions to be used when populating semantic memory.

```csharp
private static Dictionary<string, string> SampleData()
{
    return new Dictionary<string, string>
    {
        ["https://github.com/microsoft/semantic-kernel/blob/main/README.md"]
            = "README: Installation, getting started, and how to contribute",
        ["https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/02-running-prompts-from-file.ipynb"]
            = "Jupyter notebook describing how to pass prompts from a file to a semantic skill or function",
        ["https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks//00-getting-started.ipynb"]
            = "Jupyter notebook describing how to get started with the Semantic Kernel",
        ["https://github.com/microsoft/semantic-kernel/tree/main/samples/skills/ChatSkill/ChatGPT"]
            = "Sample demonstrating how to create a chat skill interfacing with ChatGPT",
        ["https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/Memory/VolatileMemoryStore.cs"]
            = "C# class that defines a volatile embedding store",
        ["https://github.com/microsoft/semantic-kernel/blob/main/samples/dotnet/KernelHttpServer/README.md"]
            = "README: How to set up a Semantic Kernel Service API using Azure Function Runtime v4",
        ["https://github.com/microsoft/semantic-kernel/blob/main/samples/apps/chat-summary-webapp-react/README.md"]
            = "README: README associated with a sample chat summary react-based webapp"
    };
}
```

--------------------------------

### ggml_backend_buft_name

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Get the name of a buffer type.

```APIDOC
## ggml_backend_buft_name

### Description
Get the name of a buffer type.

### Method
N/A (This appears to be a C# function signature, not a REST API endpoint)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
N/A

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### ggml_backend_dev_get

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Get a backend device by index.

```APIDOC
## ggml_backend_dev_get

### Description
Get a backend device by index.

### Method
N/A (This appears to be a C# function signature, not a REST API endpoint)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
N/A

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Sample Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.sampling.isamplingpipelineextensions.md

Samples a single token from the given context at the specified position.

```APIDOC
## Sample(ISamplingPipeline, LLamaContext, Int32)

### Description
Sample a single token from the given context at the given position.

### Method
static POST

### Endpoint
/api/sampling/sample

### Parameters
#### Path Parameters
- **pipe** (ISamplingPipeline) - Required - The sampling pipeline instance.
- **ctx** (LLamaContext) - Required - The context being sampled from.
- **index** (Int32) - Required - Position to sample logits from.

### Request Example
```json
{
  "pipe": "<ISamplingPipeline object>",
  "ctx": "<LLamaContext object>",
  "index": 0
}
```

### Response
#### Success Response (200)
- **token** (LLamaToken) - The sampled token.

#### Response Example
```json
{
  "token": 12345
}
```
```

--------------------------------

### ggml_backend_dev_buffer_type

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Get the buffer type for a backend device.

```APIDOC
## ggml_backend_dev_buffer_type

### Description
Get the buffer type for a backend device.

### Method
N/A (This appears to be a C# function signature, not a REST API endpoint)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
N/A

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Implement Auto-Downloaded Libraries Wrapper

Source: https://github.com/scisharp/llamasharp/blob/master/docs/AdvancedTutorials/CustomizeNativeLibraryLoading.md

This class wraps a CUDA native library and its download settings. It prepares libraries by attempting to use default paths first, then triggering a download if necessary. Note that the Prepare method cannot be asynchronous.

```csharp
public class AutoDownloadedLibraries
{
    // Wrap a cuda native library
    public class Cuda: INativeLibrary
    {
        // the default cuda native library implementation in LLamaSHarp
        private NativeLibraryWithCuda _cudaLibrary;
        // Some download settings
        private NativeLibraryDownloadSettings _settings;

        public Cuda(NativeLibraryWithCuda cudaLibrary, NativeLibraryDownloadSettings settings)
        {
            _cudaLibrary = cudaLibrary;
            _settings = settings;
        }

        public NativeLibraryMetadata? Metadata => _cudaLibrary.Metadata;

        public IEnumerable<string> Prepare(SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback = null)
        {
            foreach(var relativePath in _cudaLibrary.Prepare(systemInfo, logCallback))
            {
                // try to use the default path first. If loaded successfully, the download will not be triggered.
                yield return relativePath;
                // download the file.
                // NOTE: be sure to complete the downloading process here. You CANNOT make `Prepare` as an async method.
                var path = NativeLibraryDownloader.DownloadLibraryFile(_settings, relativePath, logCallback).Result;
                // if the downloading is successful, return the path of the downloaded file.
                if (path is not null)
                {
                    yield return path;
                }
            }
        }
    }
}
```

--------------------------------

### Sample Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.sampling.isamplingpipeline.md

Samples a single token from the provided context at a specific index.

```APIDOC
## Method Sample

### Description
Sample a single token from the given context at the given position

### Signature
```csharp
LLamaToken Sample(SafeLLamaContextHandle ctx, int index)
```

### Parameters
#### Path Parameters
* **ctx** (SafeLLamaContextHandle) - The context being sampled from
* **index** (Int32) - Position to sample logits from

### Returns
[LLamaToken](./llama.native.llamatoken.md)
```

--------------------------------

### ggml_backend_dev_count

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Get the number of available backend devices.

```APIDOC
## ggml_backend_dev_count

### Description
Get the number of available backend devices

### Method
N/A (This appears to be a C# function signature, not a REST API endpoint)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
N/A

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### SafeLLamaContextHandle ModelHandle Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamacontexthandle.md

Gets the model which this context is using.

```csharp
public SafeLlamaModelHandle ModelHandle { get; }
```

--------------------------------

### SafeLLamaSamplerChainHandle.Sample() Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamasamplerchainhandle.md

Samples and accepts a token from the specified index of the last evaluation's logits. This is a shorthand for a sequence of operations involving getting logits, creating a token data array, applying the sampler chain, and accepting the selected token.

```csharp
public LLamaToken Sample(SafeLLamaContextHandle context, int index)
```

--------------------------------

### Initialize MTMD Context and Executor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/MtmdInteractiveModeExecute.md

Sets up the multimodal context parameters, loads the base model and context, initializes the SafeMtmdWeights with the multimodal projection, and creates an InteractiveExecutor. Ensure model and projection paths are valid.

```csharp
var mtmdParameters = MtmdContextParams.Default();

using var model = await LLamaWeights.LoadFromFileAsync(parameters);
using var context = model.CreateContext(parameters);

// Mtmd Init
using var clipModel = await SafeMtmdWeights.LoadFromFileAsync(
    multiModalProj,
    model,
    mtmdParameters);

var mediaMarker = mtmdParameters.MediaMarker
    ?? NativeApi.MtmdDefaultMarker()
    ?? "<media>";

var ex = new InteractiveExecutor(context, clipModel);
```

--------------------------------

### SafeLLamaContextHandle PoolingType Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamacontexthandle.md

Gets the pooling type for this context.

```csharp
public LLamaPoolingType PoolingType { get; }
```

--------------------------------

### LLaMA Chat Session Example

Source: https://github.com/scisharp/llamasharp/blob/master/docs/QuickStart.md

This C# code demonstrates how to set up and run an interactive chat session with an LLM using LLamaSharp. Ensure you replace '<Your Model Path>' with the actual path to your GGUF model file. Adjust 'GpuLayerCount' based on your GPU memory.

```csharp
using LLama.Common;
using LLama;

string modelPath = @"<Your Model Path>"; // change it to your own model path. 

var parameters = new ModelParams(modelPath)
{
    ContextSize = 1024, // The longest length of chat as memory.
    GpuLayerCount = 5 // How many layers to offload to GPU. Please adjust it according to your GPU memory.
};
using var model = LLamaWeights.LoadFromFile(parameters);
using var context = model.CreateContext(parameters);
var executor = new InteractiveExecutor(context);

// Add chat histories as prompt to tell AI how to act.
var chatHistory = new ChatHistory();
chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.");
chatHistory.AddMessage(AuthorRole.User, "Hello, Bob.");
chatHistory.AddMessage(AuthorRole.Assistant, "Hello. How may I help you today?");

ChatSession session = new(executor, chatHistory);

InferenceParams inferenceParams = new InferenceParams()
{
    MaxTokens = 256, // No more than 256 tokens should appear in answer. Remove it if antiprompt is enough for control.
    AntiPrompts = new List<string> { "User:" } // Stop generation once antiprompts appear.
};

Console.ForegroundColor = ConsoleColor.Yellow;
Console.Write("The chat session has started.\nUser: ");
Console.ForegroundColor = ConsoleColor.Green;
string userInput = Console.ReadLine() ?? "";

while (userInput != "exit")
{
    await foreach (
        var text
        in session.ChatAsync(
            new ChatHistory.Message(AuthorRole.User, userInput),
            inferenceParams))
    {
        Console.ForegroundColor = ConsoleColor.White;
        Console.Write(text);
    }
    Console.ForegroundColor = ConsoleColor.Green;
    userInput = Console.ReadLine() ?? "";
}

```

--------------------------------

### Conversation ID Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.batched.conversation.md

Gets the unique identifier for this conversation.

```csharp
public LLamaSeqId ConversationId { get; }
```

--------------------------------

### Basic Semantic Kernel Integration with LlamaSharp

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/SemanticKernelPrompt.md

This snippet demonstrates setting up Semantic Kernel with LlamaSharp for text completion. It requires loading the LlamaSharp model and configuring the kernel's services. Use this for basic text generation tasks.

```csharp
using LLama.Common;
using LLamaSharp.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel;
using LLamaSharp.SemanticKernel.TextCompletion;
using Microsoft.SemanticKernel.TextGeneration;
using Microsoft.Extensions.DependencyInjection;

namespace LLama.Examples.Examples
{
    // The basic example for using the semantic-kernel integration
    public class SemanticKernelPrompt
    {
        public static async Task Run()
        {
            string modelPath = UserSettings.GetModelPath();

            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("This example is from: " +
                "https://github.com/microsoft/semantic-kernel/blob/main/dotnet/README.md");

            // Load weights into memory
            var parameters = new ModelParams(modelPath);
            using var model = LLamaWeights.LoadFromFile(parameters);
            var ex = new StatelessExecutor(model, parameters);

            var builder = Kernel.CreateBuilder();
            builder.Services.AddKeyedSingleton<ITextGenerationService>("local-llama", new LLamaSharpTextCompletion(ex));

            var kernel = builder.Build();

            var prompt = @"{{$input}}

One line TLDR with the fewest words.";

            ChatRequestSettings settings = new() { MaxTokens = 100 };
            var summarize = kernel.CreateFunctionFromPrompt(prompt, settings);

            string text1 = @"
1st Law of Thermodynamics - Energy cannot be created or destroyed.
2nd Law of Thermodynamics - For a spontaneous process, the entropy of the universe increases.
3rd Law of Thermodynamics - A perfect crystal at zero Kelvin has zero entropy.";

            string text2 = @"
1. An object at rest remains at rest, and an object in motion remains in motion at constant speed and in a straight line unless acted on by an unbalanced force.
2. The acceleration of an object depends on the mass of the object and the amount of force applied.
3. Whenever one object exerts a force on another object, the second object exerts an equal and opposite on the first.";

            Console.WriteLine((await kernel.InvokeAsync(summarize, new() { ["input"] = text1 })).GetValue<string>());

            Console.WriteLine((await kernel.InvokeAsync(summarize, new() { ["input"] = text2 })).GetValue<string>());
        }
    }
}

```

--------------------------------

### llama_n_seq_max

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Gets the maximum number of sequences (n_seq_max) for the given context.

```APIDOC
## llama_n_seq_max

### Description
Get the n_seq_max for this context.

### Method
Not specified (likely a static method call in C#)

### Endpoint
Not applicable (this is a library function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```csharp
public static uint llama_n_seq_max(SafeLLamaContextHandle ctx)
```

### Response
#### Success Response (200)
Returns a UInt32 representing the maximum number of sequences.

#### Response Example
```csharp
// Example return value (actual value depends on context)
uint maxSequences = ...;
```
```

--------------------------------

### Create Kernel Memory with Local Storage

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/KernelMemorySaveAndLoad.md

Configures and builds an IKernelMemory instance using local file storage and LLamaSharp for embeddings. Ensure the model path is correctly set.

```csharp
private static IKernelMemory CreateMemoryWithLocalStorage(string modelPath)
    {
        Common.InferenceParams infParams = new() { AntiPrompts = ["\n\n"] };

        LLamaSharpConfig lsConfig = new(modelPath) { DefaultInferenceParams = infParams };

        SearchClientConfig searchClientConfig = new() 
        {
            MaxMatchesCount = 1,
            AnswerTokens = 100,
        };

        TextPartitioningOptions parseOptions = new() 
        {
            MaxTokensPerParagraph = 300,
            MaxTokensPerLine = 100,
            OverlappingTokens = 30
        };

        SimpleFileStorageConfig storageConfig = new() 
        {
            Directory = StorageFolder,
            StorageType = FileSystemTypes.Disk,
        };

        SimpleVectorDbConfig vectorDbConfig = new() 
        {
            Directory = StorageFolder,
            StorageType = FileSystemTypes.Disk,
        };

        Console.ForegroundColor = ConsoleColor.Blue;
        Console.WriteLine($"Kernel memory folder: {StorageFolder}");

        Console.ForegroundColor = ConsoleColor.DarkGray;
        return new KernelMemoryBuilder()
            .WithSimpleFileStorage(storageConfig)
            .WithSimpleVectorDb(vectorDbConfig)
            .WithLLamaSharpDefaults(lsConfig)
            .WithSearchClientConfig(searchClientConfig)
            .With(parseOptions)
            .Build();
    }
```

--------------------------------

### SafeLLamaContextHandle BatchSize Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamacontexthandle.md

Gets the maximum batch size for this context.

```csharp
public uint BatchSize { get; }
```

--------------------------------

### Prepare Native Library

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibraryfrompath.md

Prepares the native library for use based on system information and a log callback. This method returns an enumerable collection of strings, likely representing paths or configurations.

```csharp
public IEnumerable<string> Prepare(SystemInfo systemInfo, LLamaLogCallback logCallback)
```

--------------------------------

### SafeLLamaContextHandle ContextSize Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamacontexthandle.md

Gets the total number of tokens in the context.

```csharp
public uint ContextSize { get; }
```

--------------------------------

### NativeLibraryWithMacOrFallback Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibrarywithmacorfallback.md

Initializes a new instance of the NativeLibraryWithMacOrFallback class with the specified library name.

```csharp
public NativeLibraryWithMacOrFallback(NativeLibraryName libraryName)
```

--------------------------------

### LLamaWeights ParameterCount Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.llamaweights.md

Gets the total number of parameters in the model.

```csharp
public ulong ParameterCount { get; }
```

--------------------------------

### NativeLibraryWithVulkan Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibrarywithvulkan.md

Explains the constructor for the NativeLibraryWithVulkan class and its parameters.

```APIDOC
## Constructors

### NativeLibraryWithVulkan(String, NativeLibraryName, AvxLevel, Boolean)

```csharp
public NativeLibraryWithVulkan(string vulkanVersion, NativeLibraryName libraryName, AvxLevel avxLevel, bool skipCheck)
```

#### Parameters

- **vulkanVersion** (String) - Description of the Vulkan version.
- **libraryName** (NativeLibraryName) - The name of the native library.
- **avxLevel** (AvxLevel) - The AVX level supported by the library.
- **skipCheck** (Boolean) - Flag to skip library checks.
```

--------------------------------

### LLamaWeights SizeInBytes Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.llamaweights.md

Gets the size of the model weights in bytes.

```csharp
public ulong SizeInBytes { get; }
```

--------------------------------

### Chat with LLaVA for Image Description

Source: https://github.com/scisharp/llamasharp/blob/master/docs/QuickStart.md

This C# snippet demonstrates how to initialize and use LLaVA for image description tasks. It handles image loading, prompt formatting, and interactive chat. Ensure you replace placeholder paths with your actual file paths.

```csharp
using System.Text.RegularExpressions;
using LLama;
using LLama.Common;

string multiModalProj = @"<Your multi-modal proj file path>";
string modelPath = @"<Your LLaVA model file path>";
string modelImage = @"<Your image path>";
const int maxTokens = 1024; // The max tokens that could be generated.

var prompt = $"{{{modelImage}}}
USER:
Provide a full description of the image.
ASSISTANT:
";

var parameters = new ModelParams(modelPath)
{
    ContextSize = 4096,
    Seed = 1337,
};
using var model = LLamaWeights.LoadFromFile(parameters);
using var context = model.CreateContext(parameters);

// Llava Init
using var clipModel = LLavaWeights.LoadFromFile(multiModalProj);

var ex = new InteractiveExecutor(context, clipModel);

Console.ForegroundColor = ConsoleColor.Yellow;
Console.WriteLine("The executor has been enabled. In this example, the prompt is printed, the maximum tokens is set to {0} and the context size is {1}.", maxTokens, parameters.ContextSize);
Console.WriteLine("To send an image, enter its filename in curly braces, like this {c:/image.jpg}.");

var inferenceParams = new InferenceParams() { Temperature = 0.1f, AntiPrompts = new List<string> { "\nUSER:" }, MaxTokens = maxTokens };

do
{

    // Evaluate if we have images
    //
    var imageMatches = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Value);
    var imageCount = imageMatches.Count();
    var hasImages = imageCount > 0;
    byte[][] imageBytes = null;

    if (hasImages)
    {
        var imagePathsWithCurlyBraces = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Value);
        var imagePaths = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Groups[1].Value);

        try
        {
            imageBytes = imagePaths.Select(File.ReadAllBytes).ToArray();
        }
        catch (IOException exception)
        {
            Console.ForegroundColor = ConsoleColor.Red;
            Console.Write(
                "Could not load your {(imageCount == 1 ? "image" : "images")}:");
            Console.Write($"{exception.Message}");
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine("Please try again.");
            break;
        }


        int index = 0;
        foreach (var path in imagePathsWithCurlyBraces)
        {
            // First image replace to tag <image, the rest of the images delete the tag
            if (index++ == 0)
                prompt = prompt.Replace(path, "<image>");
            else
                prompt = prompt.Replace(path, "");
        }
        Console.WriteLine();


        // Initialize Images in executor
        //
        ex.ImagePaths = imagePaths.ToList();
    }

    Console.ForegroundColor = ConsoleColor.White;
    await foreach (var text in ex.InferAsync(prompt, inferenceParams))
    {
        Console.Write(text);
    }
    Console.Write(" ");
    Console.ForegroundColor = ConsoleColor.Green;
    prompt = Console.ReadLine();
    Console.WriteLine();

    // let the user finish with exit
    //
    if (prompt.Equals("/exit", StringComparison.OrdinalIgnoreCase))
        break;

}
while (true);

```

--------------------------------

### TensorSplitsCollection Length Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.abstractions.tensorsplitscollection.md

Gets the size of the tensor splits collection.

```csharp
public int Length { get; }
```

--------------------------------

### NativeLibraryWithVulkan Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibrarywithvulkan.md

Initializes a new instance of the NativeLibraryWithVulkan class. Requires Vulkan version, library name, AVX level, and a flag to skip checks.

```csharp
public NativeLibraryWithVulkan(string vulkanVersion, NativeLibraryName libraryName, AvxLevel avxLevel, bool skipCheck)
```

--------------------------------

### BatchedExecutor Create Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.batched.batchedexecutor.md

Starts a new conversation managed by the BatchedExecutor.

```csharp
public Conversation Create()
```

--------------------------------

### Get Buffer Type Name

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativeapi.md

Retrieves the name of a buffer type from its pointer.

```csharp
public static IntPtr ggml_backend_buft_name(IntPtr buft)
```

--------------------------------

### Initialize AntipromptProcessor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.antipromptprocessor.md

Use this constructor to create an AntipromptProcessor with an initial collection of antiprompts.

```csharp
public AntipromptProcessor(IEnumerable<string> antiprompts)
```

--------------------------------

### SafeLLamaContextHandle UBatchSize Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.safellamacontexthandle.md

Gets the physical maximum batch size for this context.

```csharp
public uint UBatchSize { get; }
```

--------------------------------

### Get Embeddings Count

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.llamabatchembeddings.md

Retrieves the current number of items in the embedding batch.

```csharp
public int EmbeddingsCount { get; private set; }
```

--------------------------------

### Implement Custom Native Library Interface

Source: https://github.com/scisharp/llamasharp/blob/master/docs/AdvancedTutorials/CustomizeNativeLibraryLoading.md

Abstract the native library to `INativeLibrary` for finer-grained control over loading. Implement the `Prepare` method to handle file preparation and return local paths.

```csharp
public interface INativeLibrary
{
    /// <summary>
    /// Metadata of this library.
    /// </summary>
    NativeLibraryMetadata? Metadata { get; }

    /// <summary>
    /// Prepare the native library file and returns the local path of it.
    /// If it's a relative path, LLamaSharp will search the path in the search directies you set.
    /// </summary>
    /// <param name="systemInfo">The system information of the current machine.</param>
    /// <param name="logCallback">The log callback.</param>
    /// <returns>
    /// The relative paths of the library. You could return multiple paths to try them one by one. If no file is available, please return an empty array.
    /// </returns>
    IEnumerable<string> Prepare(SystemInfo systemInfo, NativeLogConfig.LLamaLogCallback? logCallback = null);
}
```

--------------------------------

### Get Embedding Dimensions

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.llamabatchembeddings.md

Retrieves the size of an individual embedding within the batch.

```csharp
public int EmbeddingDimensions { get; }
```

--------------------------------

### UnknownNativeLibrary Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.unknownnativelibrary.md

Initializes a new instance of the UnknownNativeLibrary class. This constructor is public and has no parameters.

```csharp
public UnknownNativeLibrary()
```

--------------------------------

### PromptTemplateTransformer Constructor

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.transformers.prompttemplatetransformer.md

Initializes a new instance of the PromptTemplateTransformer class.

```APIDOC
## Constructor: PromptTemplateTransformer(LLamaWeights, Boolean)

A prompt formatter that will use llama.cpp's template formatter. If your model is not supported, you will need to define your own formatter according the cchat prompt specification for your model.

```csharp
public PromptTemplateTransformer(LLamaWeights model, bool withAssistant)
```

### Parameters

- **model** [LLamaWeights](./llama.llamaweights.md)<br>
- **withAssistant** [Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)<br>
```

--------------------------------

### InstructExecutor ClipModel Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.instructexecutor.md

Gets the LLavaWeights model used for clip operations.

```csharp
public LLavaWeights ClipModel { get; }
```

--------------------------------

### LLamaTemplate Count Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.llamatemplate.md

Gets the number of messages currently added to the template.

```csharp
public int Count { get; private set; }
```

--------------------------------

### NativeLibraryConfigContainer Methods

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibraryconfigcontainer.md

Methods for configuring the native library loading process.

```APIDOC
## WithSearchDirectory(String)

### Description
Add self-defined search directories. Note that the file structure of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.

### Method
`public NativeLibraryConfigContainer WithSearchDirectory(string directory)`

### Parameters
#### Path Parameters
- **directory** (String) - Required - The directory to search for native libraries.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

## WithSelectingPolicy(INativeLibrarySelectingPolicy)

### Description
Set the policy which decides how to select the desired native libraries and order them by priority. By default we use [DefaultNativeLibrarySelectingPolicy](./llama.native.defaultnativelibraryselectingpolicy.md).

### Method
`public NativeLibraryConfigContainer WithSelectingPolicy(INativeLibrarySelectingPolicy policy)`

### Parameters
#### Path Parameters
- **policy** (INativeLibrarySelectingPolicy) - Required - The policy to use for selecting native libraries.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

## WithLogCallback(LLamaLogCallback)

### Description
Set the log callback that will be used for all llama.cpp log messages.

### Method
`public NativeLibraryConfigContainer WithLogCallback(LLamaLogCallback callback)`

### Parameters
#### Path Parameters
- **callback** (LLamaLogCallback) - Required - The callback function for logging.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

### Exceptions
- **NotImplementedException**

## WithLogCallback(ILogger)

### Description
Set the log callback that will be used for all llama.cpp log messages.

### Method
`public NativeLibraryConfigContainer WithLogCallback(ILogger logger)`

### Parameters
#### Path Parameters
- **logger** (ILogger) - Required - The logger instance.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

### Exceptions
- **NotImplementedException**

## DryRun(INativeLibrary&, INativeLibrary&)

### Description
Try to load the native library with the current configurations, but do not actually set it to [NativeApi](./llama.native.nativeapi.md). You can still modify the configuration after this calling but only before any call from [NativeApi](./llama.native.nativeapi.md).

### Method
`public bool DryRun(INativeLibrary& loadedLLamaNativeLibrary, INativeLibrary& loadedLLavaNativeLibrary)`

### Parameters
#### Path Parameters
- **loadedLLamaNativeLibrary** (INativeLibrary&) - Required - Reference to load the LLama native library into.
- **loadedLLavaNativeLibrary** (INativeLibrary&) - Required - Reference to load the LLaVA native library into.

### Returns
[Boolean] - Whether the running is successful.
```

--------------------------------

### WithVulkan Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibraryconfigcontainer.md

Configures whether to use the Vulkan backend if possible.

```APIDOC
## Method WithVulkan(Boolean)

### Description
Configure whether to use vulkan backend if possible.

### Method
`WithVulkan`

### Parameters
#### Parameters
- **enable** (Boolean) - Required - Whether to enable Vulkan.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

### Exceptions
- [InvalidOperationException] - Thrown if `LibraryHasLoaded` is true.
```

--------------------------------

### BatchedExecutor BatchQueueCount Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.batched.batchedexecutor.md

Gets the number of batches waiting in the queue for inference.

```csharp
public int BatchQueueCount { get; }
```

--------------------------------

### StreamingTokenDecoder AvailableCharacters Property

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.streamingtokendecoder.md

Gets the number of decoded characters waiting to be read from the decoder.

```csharp
public int AvailableCharacters { get; }
```

--------------------------------

### WithLibrary Method

Source: https://github.com/scisharp/llamasharp/blob/master/docs/xmldocs/llama.native.nativelibraryconfigcontainer.md

Loads a specified native library as the backend for LLamaSharp. When this method is called, all other configurations are ignored.

```APIDOC
## Method WithLibrary(String, String)

### Description
Load a specified native library as backend for LLamaSharp. When this method is called, all the other configurations will be ignored.

### Method
`WithLibrary`

### Parameters
#### Parameters
- **llamaPath** (String) - Required - The full path to the llama library to load.
- **llavaPath** (String) - Required - The full path to the llava library to load.

### Returns
[NativeLibraryConfigContainer](./llama.native.nativelibraryconfigcontainer.md)

### Exceptions
- [InvalidOperationException] - Thrown if `LibraryHasLoaded` is true.
```

--------------------------------

### Start User Chat Session

Source: https://github.com/scisharp/llamasharp/blob/master/docs/Examples/KernelMemorySaveAndLoad.md

Continuously prompts the user for questions and displays answers using the provided Kernel Memory instance. The session ends when the user enters an empty line.

```csharp
private static async Task StartUserChatSession(IKernelMemory memory)
    {
        while (true)
        {
            Console.ForegroundColor = ConsoleColor.Green;
            Console.Write("Question: ");
            string question = Console.ReadLine()!;
            if (string.IsNullOrEmpty(question))
                return;

            await ShowAnswer(memory, question);
        }
    }
```