### Install and Initialize GPT4All Python SDK

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/index.md

This snippet demonstrates how to install the GPT4All Python package via pip and how to initialize a model instance. It includes a basic chat session to generate text responses from a local model file.

```bash
pip install gpt4all
```

```python
from gpt4all import GPT4All
model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf")
with model.chat_session():
    print(model.generate("How can I run LLMs efficiently on my laptop?", max_tokens=1024))
```

--------------------------------

### Install GPT4All SDK

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Instructions for installing the GPT4All Python package via pip, including optional support for NVIDIA CUDA acceleration.

```bash
pip install gpt4all

# For NVIDIA GPU support with CUDA
pip install gpt4all[cuda]
```

--------------------------------

### Interact with GPT4All via Command-Line Interface

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Provides instructions for installing CLI dependencies and running the interactive REPL to interact with local LLMs.

```bash
pip install gpt4all typer
curl -O https://raw.githubusercontent.com/nomic-ai/gpt4all/main/gpt4all-bindings/cli/app.py
python app.py repl
python app.py repl --model ~/.cache/gpt4all/Meta-Llama-3-8B-Instruct.Q4_0.gguf
```

--------------------------------

### Example LLM Chat Session Generation

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/home.md

Provides an example of initiating a chat session with a loaded LLM and generating a response to a specific prompt, demonstrating the conversational capabilities of the SDK.

```python
from gpt4all import GPT4All
model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf")
with model.chat_session():
    print(model.generate("quadratic formula"))
```

--------------------------------

### Install GPT4All Python Client

Source: https://github.com/nomic-ai/gpt4all/blob/main/README.md

This command installs the GPT4All Python package using pip. This package provides access to LLMs through Python, leveraging llama.cpp implementations for efficient inference.

```bash
pip install gpt4all
```

--------------------------------

### Install GPT4All Node.js Package

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/typescript/README.md

Instructions for installing the GPT4All Node.js package using popular package managers like yarn, npm, and pnpm.

```bash
yarn add gpt4all@latest
```

```bash
npm install gpt4all@latest
```

```bash
pnpm install gpt4all@latest
```

--------------------------------

### Node.js Chat Completion Example with GPT4All

Source: https://github.com/nomic-ai/gpt4all/wiki/Node.js-Bindings

This example demonstrates how to load a GPT4All model and perform chat completions using the Node.js bindings. It requires the gpt4all Node.js package and a compatible model file. The function takes a model and a prompt, returning a completion object.

```javascript
import { LLModel, createCompletion, DEFAULT_DIRECTORY, DEFAULT_LIBRARIES_DIRECTORY, loadModel } from '../src/gpt4all.js'

const model = await loadModel( 'mistral-7b-openorca.gguf2.Q4_0.gguf', { verbose: true, device: 'gpu' });

const completion1 = await createCompletion(model, 'What is 1 + 1?', { verbose: true, })
console.log(completion1.message)

const completion2 = await createCompletion(model, 'And if we add two?', {  verbose: true  })
console.log(completion2.message)

model.dispose()
```

--------------------------------

### Install OpenLIT for GPT4All Monitoring

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/monitoring.md

Installs the OpenLIT library, which is essential for enabling auto-instrumentation and real-time monitoring of GPT4All LLM applications. This step is a prerequisite for all subsequent monitoring configurations.

```shell
pip install openlit
```

--------------------------------

### Special Tokens Example (LLM)

Source: https://github.com/nomic-ai/gpt4all/wiki/Generative-AI-Terminology

Demonstrates the use of special tokens in Large Language Models (LLMs) to control model behavior, such as marking the start or end of a message. These tokens are model-specific and crucial for proper functioning.

```text
<|start|>
<|end|>
```

--------------------------------

### Install Project Targets

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt

This CMake code block defines installation rules for the 'chat' and 'llmodel' targets. It specifies the destination directories based on the operating system (bin for Windows/Linux, Frameworks for macOS) and assigns components for installation.

```cmake
# -- install --

if (APPLE)
    set(GPT4ALL_LIB_DEST bin/gpt4all.app/Contents/Frameworks)
else()
    set(GPT4ALL_LIB_DEST lib)
endif()

install(TARGETS chat DESTINATION bin COMPONENT ${COMPONENT_NAME_MAIN})

install(
    TARGETS llmodel
    LIBRARY DESTINATION ${GPT4ALL_LIB_DEST} COMPONENT ${COMPONENT_NAME_MAIN}  # .so/.dylib
    RUNTIME DESTINATION bin                 COMPONENT ${COMPONENT_NAME_MAIN}  # .dll
)

# We should probably iterate through the list of the cmake for backend, but these need to be installed

```

--------------------------------

### Get GPT4All CLI Help

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Displays the general help information for the GPT4All CLI, listing all available commands and top-level options.

```shell
python app.py --help
```

--------------------------------

### Generate Text with GPU Acceleration

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md

Example of initializing the GPT4All model with GPU device support for faster inference.

```python
from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu')
output = model.generate("The capital of France is ", max_tokens=3)
print(output)
```

--------------------------------

### Get GPT4All CLI REPL Help

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Shows detailed help information specifically for the REPL mode of the GPT4All CLI, including its options and commands.

```shell
python app.py repl --help
```

--------------------------------

### CMake Project Setup and Versioning

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-backend/CMakeLists.txt

Initializes the CMake build system, sets the project name to 'llmodel', and defines its version based on major, minor, and patch components. It also sets the C++ standard to C++23 and configures library output directories.

```cmake
cmake_minimum_required(VERSION 3.23)

include(../common/common.cmake)

set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

set(LLMODEL_VERSION_MAJOR 0)
set(LLMODEL_VERSION_MINOR 5)
set(LLMODEL_VERSION_PATCH 0)
set(LLMODEL_VERSION "${LLMODEL_VERSION_MAJOR}.${LLMODEL_VERSION_MINOR}.${LLMODEL_VERSION_PATCH}")
project(llmodel VERSION ${LLMODEL_VERSION} LANGUAGES CXX C)

set(CMAKE_CXX_STANDARD 23)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(BUILD_SHARED_LIBS ON)
```

--------------------------------

### Run GPT4All CLI REPL

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Starts the GPT4All command-line interface in REPL (Read-Eval-Print Loop) mode. This command automatically selects and downloads the Mistral Instruct model if it's not already cached.

```shell
python app.py repl
```

--------------------------------

### Command-Line Interface (CLI)

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Utilize the GPT4All Command-Line Interface (CLI) for interactive sessions and model management. This section covers installation and basic usage.

```APIDOC
## Command-Line Interface

### Description
Use GPT4All from the command line with the CLI application. This allows for interactive REPL sessions and model management.

### Method
N/A (CLI Usage)

### Endpoint
N/A (CLI Usage)

### Parameters
N/A (CLI Usage)

### Request Example
```bash
# Install CLI dependencies
pip install gpt4all typer

# Download the CLI script
curl -O https://raw.githubusercontent.com/nomic-ai/gpt4all/main/gpt4all-bindings/cli/app.py

# Start interactive REPL (downloads default model if needed)
python app.py repl

# Use a specific model
python app.py repl --model ~/.cache/gpt4all/Meta-Llama-3-8B-Instruct.Q4_0.gguf

# Example REPL session:
# > What is Python?
# Python is a high-level, interpreted programming language...
# > /quit
```

### Response
#### Success Response (200)
N/A (CLI Usage - output is printed to the console)

#### Response Example
```
> What is Python?
Python is a high-level, interpreted programming language...
> /quit
```
```

--------------------------------

### Use Local GPT4All API with OpenAI Python Client

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Shows how to configure the OpenAI Python client to communicate with the local GPT4All API server. Includes examples for both chat and text completion requests.

```python
import openai

# Configure to use local GPT4All server
openai.api_base = "http://localhost:4891/v1"
openai.api_key = "not-needed"  # API key not required for local server

# Chat completion
response = openai.ChatCompletion.create(
    model="Phi-3 Mini Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain machine learning in simple terms."}
    ],
    max_tokens=200,
    temperature=0.7
)
print(response.choices[0].message.content)

# Text completion
response = openai.Completion.create(
    model="Phi-3 Mini Instruct",
    prompt="Python is a programming language that",
    max_tokens=50,
    temperature=0.5
)
print(response.choices[0].text)
```

--------------------------------

### Interact with GPT4All Server using OpenAI Python Client

Source: https://github.com/nomic-ai/gpt4all/wiki/Local-API-Server

This Python code snippet demonstrates how to use the OpenAI Python client library (version ~0.28) to interact with the GPT4All local server. It configures the API base URL to the local server and makes a completion request. Ensure you have installed the library using 'pip install "openai ~= 0.28"'.

```python
import openai

#openai.api_base = "https://api.openai.com/v1"
opeopenai.api_base = "http://localhost:4891/v1"

opeopenai.api_key = "not needed for a local LLM"

# Set up the prompt and other parameters for the API request
prompt = "Who is Michael Jordan?"

#model = "gpt-3.5-turbo"
model = "Phi-3 Mini Instruct"

# Make the API request
response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=50,
    temperature=0.28,
    top_p=0.95,
    n=1,
    echo=True,
    stream=False
)

# Print the generated completion
print(response)
```

--------------------------------

### Initialize OpenLIT for GPT4All Monitoring

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/monitoring.md

Initializes the OpenLIT monitoring system within a Python script using the GPT4All library. This code snippet demonstrates how to start the monitoring process and optionally configure GPU statistics collection. It then proceeds to load a GPT4All model and initiate a chat session to generate responses.

```python
from gpt4all import GPT4All
import openlit

openlit.init()  # start
# openlit.init(collect_gpu_stats=True)  # Optional: To configure GPU monitoring

model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf')

# Start a chat session and send queries
with model.chat_session():
    response1 = model.generate(prompt='hello', temp=0)
    response2 = model.generate(prompt='write me a short poem', temp=0)
    response3 = model.generate(prompt='thank you', temp=0)

    print(model.current_chat_session)
```

--------------------------------

### Offline Generation with Custom Prompts in GPT4All

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Demonstrates how to instantiate GPT4All for offline use by setting `allow_download=False`. This requires manually specifying the system prompt and prompt template, ensuring a model is pre-downloaded and its path is correctly set. The example shows a chat session with a custom system prompt and template.

```python
from gpt4all import GPT4All
model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf', allow_download=False)

system_prompt = '### System:\nYou are Arthur Dent, a mostly harmless guy from planet Earth.\n\n'
prompt_template = '### User:\n{0}\n\n### Response:\n'

with model.chat_session(system_prompt=system_prompt, prompt_template=prompt_template):
    for token in model.generate("What is a good way to get around the galaxy?", streaming=True):
        print(token)
        ...
```

--------------------------------

### Run GPT4All Tests

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/typescript/README.md

Executes the test suite for the GPT4All Node.js bindings. This command helps verify the installation and functionality of the bindings after the build process.

```shell
yarn test
```

--------------------------------

### Configure CPack IFW Repositories by Platform

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt

This CMake snippet detects the current operating system and processor architecture to register the appropriate GPT4All installer repository. It utilizes the cpack_ifw_add_repository function to map specific URLs for Linux, Windows, and macOS builds.

```cmake
cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/linux/repository")
elseif (CMAKE_SYSTEM_NAME MATCHES Windows)
    if (CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$")
        cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/windows/repository")
    elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64|AARCH64|arm64|ARM64)$")
        cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/windows_arm/repository")
    endif()
elseif (CMAKE_SYSTEM_NAME MATCHES Darwin)
    cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/mac/repository")
endif()
```

--------------------------------

### Configure GPT4All Model Context and GPU Layers (Python)

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Demonstrates how to initialize a GPT4All model with custom context window size, GPU layer offloading, and CPU thread settings. It also shows how to perform a long context conversation and clean up the model resources.

```python
from gpt4all import GPT4All

model = GPT4All(
    "Meta-Llama-3-8B-Instruct.Q4_0.gguf",
    n_ctx=4096,    # Maximum context window (default: 2048)
    ngl=100,       # Number of layers to offload to GPU (default: 100)
    n_threads=8,   # CPU threads for inference (default: auto)
    verbose=True   # Print debug information
)

with model.chat_session():
    # Long context conversation
    long_prompt = "Here is a very long document... " * 500
    response = model.generate(long_prompt + "\nSummarize the above.", max_tokens=200)
    print(response)

model.close()
```

--------------------------------

### Get Chat Template using Python Transformers (Open Models)

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chat_templates.md

This Python code snippet utilizes the 'transformers' library to load a tokenizer and retrieve its associated chat template. It's suitable for open models available on HuggingFace. Ensure 'transformers' is installed and updated.

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('NousResearch/Hermes-2-Pro-Llama-3-8B')
print(tokenizer.get_chat_template())
```

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('NousResearch/Hermes-2-Pro-Llama-3-8B')
open('chat_template.txt', 'w').write(tokenizer.get_chat_template())
```

--------------------------------

### Install GPT4All Python Package (Linux)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Installs the gpt4all and typer Python packages using pip for user-level installation on Linux. It first ensures that the python3-pip package is installed using apt-get, then proceeds with the pip installation.

```shell
sudo apt-get install python3-pip
python3 -m pip install --user --upgrade gpt4all typer
```

--------------------------------

### Install GPT4All Python Package (macOS)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Installs the gpt4all and typer Python packages using pip for user-level installation on macOS. It assumes a Python 3 environment is available and pip is accessible.

```shell
python3 -m pip install --user --upgrade gpt4all typer
```

--------------------------------

### Load Model and Generate Text

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Demonstrates how to initialize a model instance and perform basic text generation. The model is automatically downloaded if not present locally.

```python
from gpt4all import GPT4All

# Load a model (downloads automatically if not present)
model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf")

# Simple text generation without chat session
output = model.generate("The capital of France is ", max_tokens=50)
print(output)

# Clean up resources when done
model.close()
```

--------------------------------

### Manage GPT4All Models in Python

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Illustrates how to list available models, download models to custom directories, and load models for offline use. It also shows how to access model configuration and check backend/device information.

```python
from gpt4all import GPT4All
from pathlib import Path

# List all available models from GPT4All repository
models = GPT4All.list_models()
for model in models[:5]:
    print(f"{model['name']}: {model['filename']} ({model['filesize']})")

# Download/load model to custom directory
custom_path = Path.home() / "my_models"
custom_path.mkdir(exist_ok=True)
model = GPT4All(
    "orca-mini-3b-gguf2-q4_0.gguf",
    model_path=custom_path,
    allow_download=True
)

# Use model without internet access (must already be downloaded)
offline_model = GPT4All(
    "orca-mini-3b-gguf2-q4_0.gguf",
    model_path=custom_path,
    allow_download=False
)

# Access model configuration
print(model.config)
# Output: {'name': 'Orca Mini', 'filename': 'orca-mini-3b-gguf2-q4_0.gguf', ...}

# Check backend and device information
print(f"Backend: {model.backend}")  # 'cpu', 'cuda', 'metal', or 'kompute'
print(f"Device: {model.device}")    # GPU device name or None

model.close()
```

--------------------------------

### Initialize GPT4All and Use Chat Session (Python)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Shows how to use the `chat_session` context manager for interactive conversations with GPT4All. This allows for follow-up questions within the same session, maintaining context.

```python
from gpt4all import GPT4All
model = GPT4All('Phi-3-mini-4k-instruct.Q4_0.gguf')
with model.chat_session():
    print(model.generate("How do you know she is a witch?"))
    ...
```

--------------------------------

### Install GPT4All Python Package (Windows)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI

Installs the gpt4all and typer Python packages using pip for user-level installation on Windows. It utilizes the 'py -3' command to ensure the correct Python 3 interpreter is used.

```shell
py -3 -m pip install --user --upgrade gpt4all typer
```

--------------------------------

### Build GPT4All Backend from Source

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md

Commands to clone the repository and build the backend using CMake for various platforms.

```bash
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backend
cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallel
```

--------------------------------

### Initiate Chat Session with Prompt Templates

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Demonstrates how to use a chat session context manager with a custom prompt template. It shows how to send multiple sequential prompts to the model while maintaining session state.

```python
prompt_template = 'USER: {0}\nASSISTANT: '
with model.chat_session(system_template, prompt_template):
    response1 = model.generate('why is the grass green?')
    print(response1)
    response2 = model.generate('why is the sky blue?')
    print(response2)
```

--------------------------------

### Initialize GPT4All and Generate Response (Python)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Demonstrates the basic usage of the GPT4All Python SDK to initialize a model and generate a single response. This is a one-off exchange without memory.

```python
from gpt4all import GPT4All
model = GPT4All('Phi-3-mini-4k-instruct.Q4_0.gguf')
print(model.generate("What is the airspeed velocity of an unladen swallow?"))
```

--------------------------------

### Example Embeddings Representation (Python)

Source: https://github.com/nomic-ai/gpt4all/wiki/Generative-AI-Terminology

Illustrates how text can be represented as numerical vectors (embeddings) to capture semantic meaning. This example shows a simple Python-like representation where 'Cat', 'Kitten', and 'Dog' are mapped to numerical lists.

```python
"Cat"    [0.2, 0.5, 0.1]
"Kitten" [0.3, 0.6, 0.2]
"Dog"    [-0.1, 0.4, 0.8]
```

--------------------------------

### GET /v1/models

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_api_server/home.md

Retrieves a list of all models currently available in the GPT4All application.

```APIDOC
## GET /v1/models

### Description
Returns a list of models that are loaded and available for inference.

### Method
GET

### Endpoint
http://localhost:4891/v1/models

### Response
#### Success Response (200)
- **data** (array) - A list of model objects containing model metadata.
```

--------------------------------

### Create Embeddings

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/typescript/README.md

Example of how to load a model specifically for creating text embeddings and then generating an embedding for a given piece of text.

```javascript
import { loadModel, createEmbedding } from '../src/gpt4all.js'

const embedder = await loadModel("nomic-embed-text-v1.5.f16.gguf", { verbose: true, type: 'embedding'})

console.log(createEmbedding(embedder, "Maybe Minecraft was the friends we made along the way"));
```

--------------------------------

### Retrieving Default System Prompt and Prompt Template in GPT4All

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Shows how to retrieve the default system prompt and prompt template configured for a GPT4All model. This is useful for understanding the default behavior or for later use in offline sessions. The `allow_download` parameter defaults to `True` in this context.

```python
from gpt4all import GPT4All
model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
print(repr(model.config['systemPrompt']))
print(repr(model.config['promptTemplate']))
```

--------------------------------

### Generate Text with GPT4All

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md

Basic usage example to load a model and generate text output using the GPT4All Python API.

```python
from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)
```

--------------------------------

### Launch GPT4All-J Training Script with Accelerate and DeepSpeed

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-training/README.md

This bash command initiates the GPT4All-J training process using the `accelerate` library for distributed training and DeepSpeed for optimization. It configures multi-process training, mixed precision (bf16), and specifies the DeepSpeed configuration file and training script arguments.

```bash
accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16  --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config_gptj.json train.py --config configs/train/finetune_gptj.yaml
```

--------------------------------

### Node.js SDK - Chat Completion

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

This section provides an example of using the Node.js SDK to perform chat completions with session support, allowing for multi-turn conversations.

```APIDOC
## Node.js SDK - Chat Completion

Use GPT4All with Node.js for chat completions with session support.

```javascript
import { loadModel, createCompletion } from 'gpt4all';

// Load model with GPU acceleration
const model = await loadModel('orca-mini-3b-gguf2-q4_0.gguf', {
    verbose: true,
    device: 'gpu',
    nCtx: 2048
});

// Create a chat session for multi-turn conversations
const chat = await model.createChatSession({
    temperature: 0.8,
    systemPrompt: '### System:\nYou are an advanced mathematician.\n\n'
});

// Chat completion with session context
const response1 = await createCompletion(chat, 'What is 1 + 1?');
console.log(response1.choices[0].message);
// Output: { role: 'assistant', content: 'The answer is 2.' }

// Follow-up maintains context
const response2 = await createCompletion(chat, 'What if we multiply that by 5?');
console.log(response2.choices[0].message);
// Output: { role: 'assistant', content: 'If we multiply 2 by 5, we get 10.' }

// Stateless completion (no session)
const stateless = await createCompletion(model, 'What is the speed of light?');
console.log(stateless.choices[0].message);

// Clean up
model.dispose();
```
```

--------------------------------

### Specifying Model Path in GPT4All (Linux)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Provides an example of setting the `model_path` for GPT4All on Linux, pointing to the chat application's default model folder.

```python
from pathlib import Path
from gpt4all import GPT4All

model_name = 'Phi-3-mini-4k-instruct.Q4_0.gguf'
model_path = Path.home() / '.local' / 'share' / 'nomic.ai' / 'GPT4All'
model = GPT4All(model_name, model_path)
```

--------------------------------

### Run OpenAI-Compatible Local API Server (Bash)

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Provides `curl` commands to interact with the GPT4All local API server. It shows how to list available models and perform both chat completion and text completion requests.

```bash
# List available models
curl http://localhost:4891/v1/models

# Chat completion request
curl -X POST http://localhost:4891/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Phi-3 Mini Instruct",
    "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"}]
    "max_tokens": 100,
    "temperature": 0.7
  }'

# Text completion request
curl -X POST http://localhost:4891/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Phi-3 Mini Instruct",
    "prompt": "The meaning of life is",
    "max_tokens": 50,
    "temperature": 0.8
  }'
```

--------------------------------

### Specifying Model Path in GPT4All (macOS)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Provides an example of setting the `model_path` for GPT4All on macOS, pointing to the chat application's default model folder.

```python
from pathlib import Path
from gpt4all import GPT4All

model_name = 'Phi-3-mini-4k-instruct.Q4_0.gguf'
model_path = Path.home() / 'Library' / 'Application Support' / 'nomic.ai' / 'GPT4All'
model = GPT4All(model_name, model_path)
```

--------------------------------

### Specifying Model Path in GPT4All (Windows)

Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings

Provides an example of setting the `model_path` for GPT4All on Windows, using the environment variable `LOCALAPPDATA` to locate the chat application's default model folder.

```python
from pathlib import Path
from gpt4all import GPT4All
import os
model_name = 'Phi-3-mini-4k-instruct.Q4_0.gguf'
model_path = Path(os.environ['LOCALAPPDATA']) / 'nomic.ai' / 'GPT4All'
model = GPT4All(model_name, model_path)
```

--------------------------------

### Monitoring with OpenLIT

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

Integrate GPT4All with OpenLIT for real-time monitoring of LLM performance, including latency, token usage, cost estimation, and GPU metrics.

```APIDOC
## Monitoring with OpenLIT

### Description
Integrate with OpenLIT for real-time monitoring of LLM performance and GPU metrics. This example shows how to initialize OpenLIT and use GPT4All within its tracing context.

### Method
N/A (SDK Usage)

### Endpoint
N/A (SDK Usage)

### Parameters
N/A (SDK Usage)

### Request Example
```python
from gpt4all import GPT4All
import openlit

# Initialize OpenLIT monitoring
openlit.init()
# openlit.init(collect_gpu_stats=True)  # Enable GPU monitoring

model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')

with model.chat_session():
    # All generations are automatically traced
    response1 = model.generate('Hello!', temp=0)
    response2 = model.generate('Write me a short poem', temp=0.7)
    response3 = model.generate('Thank you', temp=0)

    # View conversation history
    print(model.current_chat_session)
```

### Response
#### Success Response (200)
N/A (SDK Usage - metrics are reported to OpenLIT)

#### Response Example
Metrics are available in the OpenLIT UI, including:
- Latency per request
- Token usage (prompt + completion)
- Cost estimation
- GPU utilization, memory, temperature
```

--------------------------------

### Using with OpenAI Python Client

Source: https://context7.com/nomic-ai/gpt4all/llms.txt

This section demonstrates how to integrate the local GPT4All API server with the OpenAI Python client library for chat and text completions.

```APIDOC
## Using with OpenAI Python Client

Use the local API server with the OpenAI Python client library.

```python
import openai

# Configure to use local GPT4All server
openai.api_base = "http://localhost:4891/v1"
openai.api_key = "not-needed"  # API key not required for local server

# Chat completion
response = openai.ChatCompletion.create(
    model="Phi-3 Mini Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain machine learning in simple terms."}
    ],
    max_tokens=200,
    temperature=0.7
)
print(response.choices[0].message.content)

# Text completion
response = openai.Completion.create(
    model="Phi-3 Mini Instruct",
    prompt="Python is a programming language that",
    max_tokens=50,
    temperature=0.5
)
print(response.choices[0].text)
```
```

--------------------------------

### Perform Direct Model Generation with GPT4All

Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/home.md

Demonstrates how to initialize a GPT4All model and perform direct text generation without chat templates. This approach treats the model as a raw text completion engine rather than a conversational assistant.

```python
from gpt4all import GPT4All
model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf")
print(model.generate("quadratic formula"))
```