### Install dependencies and run GPT4All CLI Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/cli/README.md Installs the required packages `gpt4all` and `typer` via pip, then runs the CLI with `python app.py repl`. Omit `--user` if using a virtual environment. By default, it downloads the `Mistral Instruct` model to `.cache/gpt4all/` in your user directory if necessary. ```shell # pip-install the necessary packages; omit '--user' if using a virtual environment python -m pip install --user --upgrade gpt4all typer # run the CLI python app.py repl ``` -------------------------------- ### Nous Hermes 2 Mistral DPO: Run and Play Instructions Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Instructions to save the file, run npm start, and play the guessing game, as suggested by the model output. ```text Save the file and go back to your terminal or command line interface. Run npm start. This will start a local development server, and you can access the app in your browser at http://localhost:3000. Play the guessing game by entering a number between 1 and 10 into the input field and clicking "Submit". The app will tell you if your guess is correct or not. Remember that this is just a simple example, and you can expand upon it to make the game more interesting with additional features like high scores, multiple difficulty levels, etc. ``` -------------------------------- ### Example output from Llama 3 chat session Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/home.md Shows the expected output when running the chat session generation example with default sampling settings. The response explains the quadratic formula and includes a worked example. ```text The quadratic formula! The quadratic formula is a mathematical formula that provides the solutions to a quadratic equation of the form: ax^2 + bx + c = 0 where a, b, and c are constants. The formula is: x = (-b ± √(b^2 - 4ac)) / 2a Let's break it down: * x is the variable we're trying to solve for. * a, b, and c are the coefficients of the quadratic equation. * ± means "plus or minus". * √ denotes the square root. To use the formula, simply plug in the values of a, b, and c into the expression above. The resulting value(s) will be the solutions to the original quadratic equation! For example, let's say we have the quadratic equation: x^2 + 5x + 6 = 0 We can plug these values into the formula as follows: a = 1 b = 5 ``` -------------------------------- ### Install GPT4All Python package Source: https://github.com/nomic-ai/gpt4all/blob/main/README.md Install the gpt4all Python client via pip. This is the first step to access LLMs through the gpt4all library. ```bash pip install gpt4all ``` -------------------------------- ### Install Qt dependencies on Fedora 39 Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/build_and_run.md Install required Qt 6.5+ packages and build tools on Fedora 39 using dnf. Includes qt6-qthttpserver-devel and qt6-qt5compat for QPdf and Qt HTTP Server support. ```bash sudo dnf install cmake gcc-c++ ninja-build qt-creator qt5-qtgraphicaleffects qt6-qt5compat qt6-qtbase-private-devel qt6-qtdeclarative-devel qt6-qthttpserver-devel qt6-qtsvg-devel ``` -------------------------------- ### Install Qt dependencies on Ubuntu 23.04 Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/build_and_run.md Install required Qt 6.5+ packages and build tools on Ubuntu 23.04 using apt. Includes qt6-httpserver-dev and qt6-5compat for QPdf and Qt HTTP Server support. ```bash sudo apt install cmake g++ libgl-dev libqt6core5compat6 ninja-build qml6-module-qt5compat-graphicaleffects qt6-base-private-dev qt6-declarative-dev qt6-httpserver-dev qt6-svg-dev qtcreator ``` -------------------------------- ### Install Python package locally Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md Navigate to the Python bindings directory and install the package in editable mode. ```bash cd ../gpt4all-bindings/python pip install -e . ``` -------------------------------- ### User installation on Windows Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Installs or upgrades gpt4all and typer for the current user using the py launcher. Requires the official Python installer from python.org. ```shell py -3 -m pip install --user --upgrade gpt4all typer ``` -------------------------------- ### Streaming generation example output Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings Example output from the streaming generation snippet. Shows the token list and the final printed text. ```text [' The', ' capital', ' of', ' France', ' is', ' Paris', '.'] The highest mountain in South America is Aconcagua, which is also considered the highest peak outside of Asia. ``` -------------------------------- ### Install OpenLIT Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/monitoring.md Install the OpenLIT package to enable OpenTelemetry auto-instrumentation for monitoring GPT4All applications. ```shell pip install openlit ``` -------------------------------- ### Llama 3: Example Output for Sky Explanation Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Example output from Llama 3 when prompted with the sky explanation question, using default sampling settings. ```text What a great question! So, you know how we can see different colors like red, yellow, green, and orange? Well, when sunlight enters Earth's atmosphere, it starts to interact with tiny particles called molecules of gases like nitrogen (N2) and oxygen (02). These molecules are really small, but they're all around us! Now, here's the cool part: these molecules scatter, or bounce off, shorter wavelengths of light more than longer wavelengths. Think of it like a game of pool where the cue ball hits other balls with different sizes and speeds. The shortest wavelength of visible light is violet (about 400 nanometers), followed by blue (around 450-495 nanometers). These shorter wavelengths are scattered in all directions, kind of like how a basketball player might bounce the ball around on the court. This scattering effect makes the sky appear blue to our eyes! Here's why it doesn't look as bright or intense as a blue crayon: there are many more molecules in the atmosphere that scatter longer wavelengths (like red and orange) than shorter ones, so those colors get spread out and become less noticeable. To sum it up: 1. Sunlight enters Earth's atmosphere. 2. Tiny gas molecule particles interact with light, scattering shorter wavelengths like blue and violet. 3. This scattered blue light reaches our eyes from all directions, making the sky appear blue! Now, go outside and enjoy that beautiful blue sky! ``` -------------------------------- ### Direct Generation Output Example Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/home.md Shows the output of the direct generation example with default sampling settings. The response resembles a continuation of math-style text rather than a helpful answer, illustrating the effect of not using a chat session. ```text . The equation is in the form of a + bx = c, where a and b are constants. The solution to this problem involves using the quadratic formula which states that for any quadratic equation ax^2+bx+c=0, its solutions can be found by: x = (-b ± √(b^2-4ac)) / 2a In your case, since you have a + bx = c, we need to rewrite it in the form of ax^2+bx+c=0. To do this, subtract both sides from c, so that: c - (a + bx) = 0 Now, combine like terms on the left side and simplify: ax^2 + (-b)x + (c-a) = 0 So now we have a quadratic equation in standard form: ax^2+bx+c=0. We can use this to find its solutions using the quadratic formula: x = ((-b ± √((-b)^2 ``` -------------------------------- ### Create a virtual environment for GPT4All CLI Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/cli/README.md Creates a Python virtual environment named 'gpt4all-cli'. This is optional but recommended. Activation differs between Unix-like systems (`. gpt4all-cli/bin/activate`) and Windows (`gpt4all-cli\Scripts\activate`). ```shell # optional but recommended: create and use a virtual environment python -m venv gpt4all-cli ``` -------------------------------- ### Create and activate virtual environment on Windows Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Creates a virtual environment named 'gpt4all-cli' using the py launcher, activates it, and installs gpt4all and typer. Requires the official Python installer from python.org. ```shell py -3 -m venv gpt4all-cli gpt4all-cli\Scripts\activate py -m pip install gpt4all typer ``` -------------------------------- ### Install Qt dependencies on Arch Linux Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/build_and_run.md Install required Qt 6.5+ packages and build tools on Arch Linux using pacman. Includes qt6-httpserver and qt6-5compat for QPdf and Qt HTTP Server support. ```bash sudo pacman -S --needed cmake gcc ninja qt6-5compat qt6-base qt6-declarative qt6-httpserver qt6-svg qtcreator ``` -------------------------------- ### Launch GPT4All-J training with accelerate and deepspeed Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-training/README.md Command to start training GPT4All-J using accelerate with DeepSpeed, bf16 mixed precision, and the specified configuration files. ```bash accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16 --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config_gptj.json train.py --config configs/train/finetune_gptj.yaml ``` -------------------------------- ### Retrieving default system prompt and prompt template Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings Use an online instance (allow_download defaults to True) to get the model's default systemPrompt and promptTemplate. The output shows the exact strings to pass later to an offline instance. ```py from gpt4all import GPT4All model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf') print(repr(model.config['systemPrompt'])) print(repr(model.config['promptTemplate'])) ``` -------------------------------- ### Install Targets and Libraries Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Configures installation destinations for the chat executable and llmodel library, handling platform-specific paths for macOS frameworks and other platforms. Sets up model implementation targets based on LLMODEL_KOMPUTE, adding Metal on Apple. Installs the model targets to the appropriate library or runtime directories. ```cmake # -- install -- if (APPLE) set(GPT4ALL_LIB_DEST bin/gpt4all.app/Contents/Frameworks) else() set(GPT4ALL_LIB_DEST lib) endif() install(TARGETS chat DESTINATION bin COMPONENT ${COMPONENT_NAME_MAIN}) install( TARGETS llmodel LIBRARY DESTINATION ${GPT4ALL_LIB_DEST} COMPONENT ${COMPONENT_NAME_MAIN} # .so/.dylib RUNTIME DESTINATION bin COMPONENT ${COMPONENT_NAME_MAIN} # .dll ) # We should probably iterate through the list of the cmake for backend, but these need to be installed # to the this component's dir for the finicky qt installer to work if (LLMODEL_KOMPUTE) set(MODEL_IMPL_TARGETS llamamodel-mainline-kompute llamamodel-mainline-kompute-avxonly ) else() set(MODEL_IMPL_TARGETS llamamodel-mainline-cpu llamamodel-mainline-cpu-avxonly ) endif() if (APPLE) list(APPEND MODEL_IMPL_TARGETS llamamodel-mainline-metal) endif() install( TARGETS ${MODEL_IMPL_TARGETS} LIBRARY DESTINATION ${GPT4ALL_LIB_DEST} COMPONENT ${COMPONENT_NAME_MAIN} # .so/.dylib RUNTIME DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN} # .dll ) ``` -------------------------------- ### Install PDFium and Embedding Model Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Installs the PDFium library to the appropriate destination (bin on Windows, GPT4ALL_LIB_DEST otherwise) when GPT4ALL_USING_QTPDF is not set. Also installs the local embedding model file to resources on non-Apple platforms. ```cmake if (NOT GPT4ALL_USING_QTPDF) # Install PDFium if (WIN32) install(FILES ${PDFium_LIBRARY} DESTINATION bin COMPONENT ${COMPONENT_NAME_MAIN}) # .dll else() install(FILES ${PDFium_LIBRARY} DESTINATION ${GPT4ALL_LIB_DEST} COMPONENT ${COMPONENT_NAME_MAIN}) # .so/.dylib endif() endif() if (NOT APPLE) install(FILES "${LOCAL_EMBEDDING_MODEL_PATH}" DESTINATION resources COMPONENT ${COMPONENT_NAME_MAIN}) endif() ``` -------------------------------- ### GET /v1/models Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_api_server/home.md Lists all available models on the GPT4All API server. This endpoint is used to retrieve the names of models that can be used in other API calls. ```APIDOC ## GET /v1/models ### Description Lists available models on the GPT4All API server. ### Method GET ### Endpoint /v1/models ### Response #### Success Response (200) - **models** (array) - List of available model names. #### Response Example { "models": ["Phi-3 Mini Instruct"] } ``` -------------------------------- ### Add installer repository based on platform and build type Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Adds the appropriate Qt Installer Framework repository URL based on the build configuration. Uses a localhost repository when GPT4ALL_LOCALHOST is set, defines GPT4ALL_OFFLINE_INSTALLER for offline installers, and selects the correct remote repository for Linux, Windows (x86_64 or ARM64), and macOS otherwise. ```CMake if (GPT4ALL_LOCALHOST) cpack_ifw_add_repository("GPT4AllRepository" URL "http://localhost/repository") elseif (GPT4ALL_OFFLINE_INSTALLER) add_compile_definitions(GPT4ALL_OFFLINE_INSTALLER) else() if (CMAKE_SYSTEM_NAME MATCHES Linux) cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/linux/repository") elseif (CMAKE_SYSTEM_NAME MATCHES Windows) # To sign the target on windows have to create a batch script add use it as a custom target and then use CPACK_IFW_EXTRA_TARGETS to set this extra target if (CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$") cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/windows/repository") elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64|AARCH64|arm64|ARM64)$") cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/windows_arm/repository") endif() elseif (CMAKE_SYSTEM_NAME MATCHES Darwin) cpack_ifw_add_repository("GPT4AllRepository" URL "https://gpt4all.io/installer_repos/mac/repository") endif() endif() ``` -------------------------------- ### Initialize OpenLIT and Monitor GPT4All Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/monitoring.md Initialize OpenLIT to start monitoring traces and metrics. Optionally enable GPU stats collection with collect_gpu_stats=True. Then run a chat session with GPT4All to generate traces. ```python from gpt4all import GPT4All import openlit openlit.init() # start # openlit.init(collect_gpu_stats=True) # Optional: To configure GPU monitoring model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf') # Start a chat session and send queries with model.chat_session(): response1 = model.generate(prompt='hello', temp=0) response2 = model.generate(prompt='write me a short poem', temp=0) response3 = model.generate(prompt='thank you', temp=0) print(model.current_chat_session) ``` -------------------------------- ### Nous Hermes 2 Mistral DPO: Example Output for React Game Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Example output from Nous Hermes 2 Mistral DPO when prompted with the React game request, using default sampling settings. Includes instructions and code blocks. ```text I'll provide you with a simple React app that allows you to play a guessing game. This app will generate a random number between 1 and 10, and ask you to guess it. You can run this app locally using Create React App. Follow the steps below: Open your terminal or command line interface. Navigate to the directory where you want to create the project (e.g., cd Documents/Projects). Run the following commands: ``` -------------------------------- ### Configure maintenance tool component for macOS installer Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Configures the maintenance tool component for the Qt Installer Framework on macOS when signing is enabled. Hides the component and marks it as downloaded unless an offline installer is used, then sets it as essential with forced installation, assigns a version, and attaches a Qt Script for component behavior. ```CMake if (APPLE AND GPT4ALL_SIGN_INSTALL) if (GPT4ALL_OFFLINE_INSTALLER) cpack_add_component(maintenancetool HIDDEN) else() cpack_add_component(maintenancetool HIDDEN DOWNLOADED) endif() cpack_ifw_configure_component(maintenancetool ESSENTIAL FORCED_INSTALLATION) cpack_ifw_configure_component(maintenancetool VERSION ${APP_VERSION}) cpack_ifw_configure_component(maintenancetool SCRIPT "${CMAKE_CURRENT_SOURCE_DIR}/cmake/installer_maintenancetool_component.qs") endif() ``` -------------------------------- ### Chat Completion Example with LLModel and createCompletion Source: https://github.com/nomic-ai/gpt4all/wiki/Node.js-Bindings Loads a model using loadModel with verbose and device options, then creates two completions with createCompletion, logging each message. Disposes the model after use. ```javascript import { LLModel, createCompletion, DEFAULT_DIRECTORY, DEFAULT_LIBRARIES_DIRECTORY, loadModel } from '../src/gpt4all.js' const model = await loadModel( 'mistral-7b-openorca.gguf2.Q4_0.gguf', { verbose: true, device: 'gpu'}); const completion1 = await createCompletion(model, 'What is 1 + 1?', { verbose: true, }) console.log(completion1.message) const completion2 = await createCompletion(model, 'And if we add two?', { verbose: true }) console.log(completion2.message) model.dispose() ``` -------------------------------- ### Check pip availability on macOS Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Verifies that the Python installation provides the pip module. Should print help text; if not, consult the Python installation documentation or download a separate variant. ```shell python3 -m pip --help ``` -------------------------------- ### Nous Hermes 2 Mistral DPO: Create React App Commands Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Commands to create and navigate into a new React app project, as suggested by the model output. ```bash npx create-react-app guessing-game cd guessing-game ``` -------------------------------- ### Using OpenAI Python Client Library with Local API Server Source: https://github.com/nomic-ai/gpt4all/wiki/Local-API-Server Requires an old-style OpenAI library (e.g., pip install "openai ~= 0.28") in a virtual environment. Points the client to the local server at http://localhost:4891/v1 and uses the model "Phi-3 Mini Instruct". The API key is not needed for a local LLM. Sends a completion request and prints the response. ```python import openai #openai.api_base = "https://api.openai.com/v1" openai.api_base = "http://localhost:4891/v1" openai.api_key = "not needed for a local LLM" # Set up the prompt and other parameters for the API request prompt = "Who is Michael Jordan?" #model = "gpt-3.5-turbo" model = "Phi-3 Mini Instruct" # Make the API request response = openai.Completion.create( model=model, prompt=prompt, max_tokens=50, temperature=0.28, top_p=0.95, n=1, echo=True, stream=False ) # Print the generated completion print(response) ``` -------------------------------- ### Create and activate virtual environment on macOS Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Creates a virtual environment named 'gpt4all-cli', activates it, and installs the gpt4all and typer packages. Use this to avoid dependency conflicts. ```shell python3 -m venv gpt4all-cli . gpt4all-cli/bin/activate python3 -m pip install gpt4all typer ``` -------------------------------- ### GPT4All v1 template example Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chat_templates.md Use this template to control how LocalDocs sources and file attachments are inserted into the conversation. It begins with `{# gpt4all v1 #}` and must be used directly in the template for those features to work correctly. ```jinja {# gpt4all v1 #} {%- for message in messages %} {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' }} {%- if message['role'] == 'user' %} {%- for source in message['sources'] %} {%- if loop.first %} {{- '### Context:\n' }} {%- endif %} {{- 'Collection: ' + source['collection'] + '\n' + 'Path: ' + source['path'] + '\n' + 'Excerpt: ' + source['text'] + '\n\n' }} {%- endfor %} {%- endif %} {%- for attachment in message['prompt_attachments'] %} {{- attachment['processed_content'] + '\n\n' }} {%- endfor %} {{- message['content'] | trim }} {{- '<|eot_id|>' }} {%- endfor %} {%- if add_generation_prompt %} {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }} {%- endif %} ``` -------------------------------- ### CMakeLists.txt: Include FetchContent and Set Build Options Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/deps/CMakeLists.txt Configures CMake to use FetchContent and sets build options for dependencies. Disables shared libraries and fmt installation, sets the Qt application class, and disables DuckX installation. ```cmake include(FetchContent) set(BUILD_SHARED_LIBS OFF) set(FMT_INSTALL OFF) add_subdirectory(fmt) set(QAPPLICATION_CLASS QApplication) add_subdirectory(SingleApplication) set(DUCKX_INSTALL OFF) add_subdirectory(DuckX) set(QT_VERSION_MAJOR 6) add_subdirectory(QXlsx/QXlsx) ``` -------------------------------- ### Check Python venv and pip availability on macOS Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Run these commands to verify that the Python installation provides the venv and pip modules. Both should print help text; if not, consult the Python installation documentation or download a separate variant. ```shell python3 -m venv --help python3 -m pip --help ``` -------------------------------- ### User installation on macOS Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Installs or upgrades gpt4all and typer for the current user only, avoiding the need for a virtual environment. May share dependencies with other Python software. ```shell python3 -m pip install --user --upgrade gpt4all typer ``` -------------------------------- ### Custom session templates with chat_session() Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings Customizes the system prompt and prompt template when using `chat_session()`. These take precedence over defaults when `allow_download=True`. The example uses a simple system template and a prompt template with 'USER:' and 'ASSISTANT:' markers. ```python from gpt4all import GPT4All model = GPT4All('wizardlm-13b-v1.2.Q4_0.gguf') system_template = 'A chat between a curious user and an artificial intelligence assistant.\n' # many models use triple hash '###' for keywords, Vicunas are simpler: prompt_template = 'USER: {0}\nASSISTANT: ' with model.chat_session(system_template, prompt_template): response1 = model.generate('why is the grass green?') print(response1) print() response2 = model.generate('why is the sky blue?') print(response2) ``` -------------------------------- ### CPack and IFW Component Configuration Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Includes required system libraries and CPack modules, then configures the main component for the Qt Installer Framework. Sets the component as ESSENTIAL with FORCED_INSTALLATION, assigns version, MIT license, a Qt script, and marks it as replacing 'gpt4all-chat'. The component is marked DOWNLOADED when GPT4ALL_OFFLINE_INSTALLER is not set. ```cmake include(InstallRequiredSystemLibraries) include(CPack) include(CPackIFW) if(GPT4ALL_OFFLINE_INSTALLER) cpack_add_component(${COMPONENT_NAME_MAIN}) else() cpack_add_component(${COMPONENT_NAME_MAIN} DOWNLOADED) endif() cpack_ifw_configure_component(${COMPONENT_NAME_MAIN} ESSENTIAL FORCED_INSTALLATION) cpack_ifw_configure_component(${COMPONENT_NAME_MAIN} VERSION ${APP_VERSION}) cpack_ifw_configure_component(${COMPONENT_NAME_MAIN} LICENSES "MIT LICENSE" ${CPACK_RESOURCE_FILE_LICENSE}) cpack_ifw_configure_component(${COMPONENT_NAME_MAIN} SCRIPT "${CMAKE_CURRENT_SOURCE_DIR}/cmake/installer_gpt4all_component.qs") cpack_ifw_configure_component(${COMPONENT_NAME_MAIN} REPLACES "gpt4all-chat") #Was used in very earliest prototypes ``` -------------------------------- ### Python example of reversing a string with reversed() Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-training/TRAINING_LOG.md Example shown in the training log illustrating a model generation that exits early. The code defines a string, calls reversed(), and prints the result. The displayed output is 'world hello'. ```python string = 'hello world' reversed_string = reversed(string) print(reversed_string) ``` ```text world hello ``` -------------------------------- ### Install python3-pip on Debian/Ubuntu Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Ensures the pip package is present on Debian/Ubuntu and derived distros for user installations. Use sudo only for system packages. ```shell sudo apt-get install python3-pip ``` -------------------------------- ### Example output from custom stop callback Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings Shows the response generated when the custom stop callback stops after the first period. The output is a single sentence. ```text They can grow up to 100 feet long and weigh over 20 tons. ``` -------------------------------- ### Direct Generation Example: model.generate() Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_python/home.md Calls model.generate() directly without applying a chat session template. This can produce responses that mirror the tone of the prompt rather than helpful assistant-style answers. The example loads the Llama 3 model and prompts it with 'quadratic formula'. ```python from gpt4all import GPT4All model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf") print(model.generate("quadratic formula")) ``` -------------------------------- ### CUDA Model Installation with Runtime Dependencies Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Installs CUDA model targets with an INSTALL_RPATH of $ORIGIN and collects runtime dependencies into llama-cuda-deps. On Windows, installs the dependency set with regex filters to include CUDA and cuBLAS DLLs while excluding system DLLs like nvcuda and api-ms-*. ```cmake if (LLMODEL_CUDA) set_property(TARGET llamamodel-mainline-cuda llamamodel-mainline-cuda-avxonly APPEND PROPERTY INSTALL_RPATH "$ORIGIN") install( TARGETS llamamodel-mainline-cuda llamamodel-mainline-cuda-avxonly RUNTIME_DEPENDENCY_SET llama-cuda-deps LIBRARY DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN} # .so RUNTIME DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN} # .dll ) if (WIN32) install( RUNTIME_DEPENDENCY_SET llama-cuda-deps PRE_EXCLUDE_REGEXES "^(nvcuda|api-ms-.*)\\.dll$" POST_INCLUDE_REGEXES "(^|[/\\\\])(lib)?(cuda|cublas)" POST_EXCLUDE_REGEXES . DIRECTORIES "${CUDAToolkit_BIN_DIR}" DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN} ) endif() endif() ``` -------------------------------- ### Install GPT4All with yarn, npm, or pnpm Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/typescript/README.md Install the gpt4all package using your preferred package manager. Choose one command based on your environment. ```sh yarn add gpt4all@latest ``` ```sh npm install gpt4all@latest ``` ```sh pnpm install gpt4all@latest ``` -------------------------------- ### Search query example Source: https://github.com/nomic-ai/gpt4all/wiki/Web-Search-Beta-Release Type this query in the chat view after loading the Llama 3.1 8B Instruct model to test web search. ```text Search the web and tell me who the 44th president of the United States was ``` -------------------------------- ### Get Qt6Core properties and locate qmake Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Retrieves Qt6::Core include directories and release library location, then finds the qmake executable and computes the Qt 6 root directory. Prints the discovered paths for diagnostics. ```cmake # Get the Qt6Core target properties get_target_property(Qt6Core_INCLUDE_DIRS Qt6::Core INTERFACE_INCLUDE_DIRECTORIES) get_target_property(Qt6Core_LIBRARY_RELEASE Qt6::Core LOCATION_RELEASE) # Find the qmake binary find_program(QMAKE_EXECUTABLE NAMES qmake qmake6 PATHS ${Qt6Core_INCLUDE_DIRS}/../.. NO_DEFAULT_PATH) # Get the Qt 6 root directory get_filename_component(Qt6_ROOT_DIR "${Qt6Core_LIBRARY_RELEASE}" DIRECTORY) get_filename_component(Qt6_ROOT_DIR "${Qt6_ROOT_DIR}/.." ABSOLUTE) message(STATUS "qmake binary: ${QMAKE_EXECUTABLE}") message(STATUS "Qt 6 root directory: ${Qt6_ROOT_DIR}") ``` -------------------------------- ### Define llmodel library with sources and public headers Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-backend/CMakeLists.txt Creates the llmodel library target with source files, adds warning options, sets public headers, defines LIB_FILE_EXT, and includes directories. Sets version and SOVERSION properties, and defines install prefix. ```cmake add_library(llmodel src/dlhandle.cpp src/llmodel.cpp src/llmodel_c.cpp src/llmodel_shared.cpp ) gpt4all_add_warning_options(llmodel) target_sources(llmodel PUBLIC FILE_SET public_headers TYPE HEADERS BASE_DIRS include FILES include/gpt4all-backend/llmodel.h include/gpt4all-backend/llmodel_c.h include/gpt4all-backend/sysinfo.h ) target_compile_definitions(llmodel PRIVATE LIB_FILE_EXT="${CMAKE_SHARED_LIBRARY_SUFFIX}") target_include_directories(llmodel PRIVATE src include/gpt4all-backend) set_target_properties(llmodel PROPERTIES VERSION ${PROJECT_VERSION} SOVERSION ${PROJECT_VERSION_MAJOR}) set(COMPONENT_NAME_MAIN ${PROJECT_NAME}) set(CMAKE_INSTALL_PREFIX ${CMAKE_BINARY_DIR}/install) ``` -------------------------------- ### CMakeLists.txt for gpt4all-chat Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Top-level CMake configuration for the gpt4all-chat application. Sets project version 3.10.1-dev0, requires CMake 3.25, and includes common.cmake. Defines build options for tests, localhost installer, offline installer, signing, CPack config generation, QtPDF usage, and D3D12 backend. On macOS, optionally builds a universal binary. Uses C++23 and enables accurate __cplusplus on MSVC. ```cmake cmake_minimum_required(VERSION 3.25) # for try_compile SOURCE_FROM_VAR include(../common/common.cmake) set(APP_VERSION_MAJOR 3) set(APP_VERSION_MINOR 10) set(APP_VERSION_PATCH 1) set(APP_VERSION_BASE "${APP_VERSION_MAJOR}.${APP_VERSION_MINOR}.${APP_VERSION_PATCH}") set(APP_VERSION "${APP_VERSION_BASE}-dev0") project(gpt4all VERSION ${APP_VERSION_BASE} LANGUAGES CXX C) if (CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) set(CMAKE_INSTALL_PREFIX ${CMAKE_BINARY_DIR}/install CACHE PATH "..." FORCE) endif() if(APPLE) option(BUILD_UNIVERSAL "Build a Universal binary on macOS" OFF) if(BUILD_UNIVERSAL) # Build a Universal binary on macOS # This requires that the found Qt library is compiled as Universal binaries. set(CMAKE_OSX_ARCHITECTURES "arm64;x86_64" CACHE STRING "" FORCE) else() # Build for the host architecture on macOS set(CMAKE_OSX_ARCHITECTURES "${CMAKE_HOST_SYSTEM_PROCESSOR}" CACHE STRING "" FORCE) endif() endif() find_package(Python3 3.12 QUIET COMPONENTS Interpreter) option(GPT4ALL_TEST "Build the tests" ${Python3_FOUND}) option(GPT4ALL_LOCALHOST "Build installer for localhost repo" OFF) option(GPT4ALL_OFFLINE_INSTALLER "Build an offline installer" OFF) option(GPT4ALL_SIGN_INSTALL "Sign installed binaries and installers (requires signing identities)" OFF) option(GPT4ALL_GEN_CPACK_CONFIG "Generate the CPack config.xml in the package step and nothing else." OFF) set(GPT4ALL_USE_QTPDF "AUTO" CACHE STRING "Whether to Use QtPDF for LocalDocs. If OFF or not available on this platform, PDFium is used.") set_property(CACHE GPT4ALL_USE_QTPDF PROPERTY STRINGS AUTO ON OFF) set(GPT4ALL_FORCE_D3D12 "AUTO" CACHE STRING "Whether to use Direct3D 12 as the Qt scene graph backend. Defaults to ON on Windows ARM.") set_property(CACHE GPT4ALL_FORCE_D3D12 PROPERTY STRINGS AUTO ON OFF) include(cmake/cpack_config.cmake) if (GPT4ALL_GEN_CPACK_CONFIG) configure_file("${CMAKE_CURRENT_SOURCE_DIR}/cmake/cpack-steal-config.cmake.in" "${CMAKE_BINARY_DIR}/cmake/cpack-steal-config.cmake" @ONLY) set(CPACK_POST_BUILD_SCRIPTS ${CMAKE_BINARY_DIR}/cmake/cpack-steal-config.cmake) include(CPack) include(CPackIFW) return() endif() set(CMAKE_EXPORT_COMPILE_COMMANDS ON) set(CMAKE_CXX_STANDARD 23) set(CMAKE_CXX_STANDARD_REQUIRED ON) if (MSVC) # Enable accurate __cplusplus macro add_compile_options($<$:/Zc:__cplusplus>) endif() ``` -------------------------------- ### POST /v1/completions Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_api_server/home.md Generates text completions based on the provided prompt. This endpoint is used for non-chat completion tasks. ```APIDOC ## POST /v1/completions ### Description Generate text completions. ### Method POST ### Endpoint /v1/completions ### Parameters #### Request Body - **model** (string) - Required - The model to use for completion. - **prompt** (string) - Required - The input prompt for completion. - **max_tokens** (integer) - Optional - Maximum number of tokens to generate. - **temperature** (number) - Optional - Sampling temperature. ### Request Example { "model": "Phi-3 Mini Instruct", "prompt": "Once upon a time", "max_tokens": 50, "temperature": 0.28 } ### Response #### Success Response (200) - **choices** (array) - List of generated completions. #### Response Example { "choices": [ { "text": "..." } ] } ``` -------------------------------- ### Start the CLI with a custom model path Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Uses the `-m`/`--model` parameter to specify a model. A bare filename is looked up in `.cache/gpt4all/`; use a full path for models outside that cache folder. ```shell python app.py repl --model /home/user/my-gpt4all-models/mistral-7b-instruct-v0.1.Q4_0.gguf ``` -------------------------------- ### Install python3-venv and python3-pip on Debian/Ubuntu Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Ensures the required Python packages for virtual environments are present on Debian/Ubuntu and derived distros. Use sudo only for system packages. ```shell sudo apt-get install python3-venv python3-pip ``` -------------------------------- ### POST /v1/chat/completions Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_api_server/home.md Generates chat completions based on a conversation history. This endpoint is used for chat-based interactions and supports OpenAI-compatible clients. ```APIDOC ## POST /v1/chat/completions ### Description Generate chat completions. ### Method POST ### Endpoint /v1/chat/completions ### Parameters #### Request Body - **model** (string) - Required - The model to use for chat completion. - **messages** (array) - Required - List of message objects with role and content. - **max_tokens** (integer) - Optional - Maximum number of tokens to generate. - **temperature** (number) - Optional - Sampling temperature. ### Request Example { "model": "Phi-3 Mini Instruct", "messages": [ { "role": "user", "content": "Who is Lionel Messi?" } ], "max_tokens": 50, "temperature": 0.28 } ### Response #### Success Response (200) - **choices** (array) - List of generated chat completions. - **references** (array) - Optional, present when LocalDocs is enabled. Contains reference snippets. #### Response Example { "choices": [ { "message": { "role": "assistant", "content": "..." }, "references": [ { "text": "...", "author": "...", "date": "...", "page": 1, "title": "..." } ] } ] } ``` -------------------------------- ### Load GPT4All-J v1.2-jazzy model and dataset Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-training/README.md Specify the revision argument to load a specific version of the GPT4All-J model and dataset. This example loads the v1.2-jazzy version. ```python from datasets import load_dataset from transformers import AutoModelForCausalLM dataset = load_dataset("nomic-ai/gpt4all-j-prompt-generations", revision="v1.2-jazzy") model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-j", revision="v1.2-jazzy") ``` -------------------------------- ### GET /v1/models/ Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_api_server/home.md Retrieves details of a specific model by its name. This endpoint is used to get information about a particular model available on the server. ```APIDOC ## GET /v1/models/ ### Description Get details of a specific model. ### Method GET ### Endpoint /v1/models/ ### Parameters #### Path Parameters - **name** (string) - Required - The name of the model to retrieve details for. ### Response #### Success Response (200) - **model** (object) - Details of the requested model. #### Response Example { "model": { "name": "Phi-3 Mini Instruct" } } ``` -------------------------------- ### Build backend on Windows with Visual Studio Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md Configure and build the backend using CMake with RelWithDebInfo configuration on Windows. ```bash cmake -B build cmake --build build --parallel --config RelWithDebInfo ``` -------------------------------- ### Llama 3: Prompt for Sky Explanation Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Prompt to use with the Llama 3 model to generate a child-friendly explanation of why the sky is blue. With default sampling settings, the model produces a detailed response. ```text explain why the sky is blue in a way that is correct and makes sense to a child ``` -------------------------------- ### Show REPL help Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Displays a description of the REPL options. While inside the running REPL, use `/help` instead. ```shell python app.py repl --help ``` -------------------------------- ### Simple System Prompt Source: https://github.com/nomic-ai/gpt4all/wiki/Configuring-Custom-Models A clean system prompt without any jinja2 templating, as recommended for GPT4All model configuration. ```text You are a helpful AI assistant. ``` -------------------------------- ### Install dependencies for GPT4All CLI Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Install the required Python packages for the CLI: gpt4all and typer. Requires Python 3.8 or later. ```shell pip install gpt4all typer ``` -------------------------------- ### Show CLI help Source: https://github.com/nomic-ai/gpt4all/wiki/Python-CLI Displays help for all available commands and options on the command-line. ```shell python app.py --help ``` -------------------------------- ### Build backend on other platforms Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md Configure and build the backend using CMake with RelWithDebInfo configuration on macOS and Linux. ```bash cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo cmake --build build --parallel ``` -------------------------------- ### Deployment Tool Configuration per Platform Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Configures CPACK_PRE_BUILD_SCRIPTS by finding the appropriate deployment tool (linuxdeployqt, windeployqt, or macdeployqt) and generating a platform-specific deploy script from a template. The script is configured with @ONLY substitution. ```cmake if (CMAKE_SYSTEM_NAME MATCHES Linux) find_program(LINUXDEPLOYQT linuxdeployqt HINTS "$ENV{HOME}/dev/linuxdeployqt/build/tools/linuxdeployqt" "$ENV{HOME}/project/linuxdeployqt/bin") configure_file("${CMAKE_CURRENT_SOURCE_DIR}/cmake/deploy-qt-linux.cmake.in" "${CMAKE_BINARY_DIR}/cmake/deploy-qt-linux.cmake" @ONLY) set(CPACK_PRE_BUILD_SCRIPTS ${CMAKE_BINARY_DIR}/cmake/deploy-qt-linux.cmake) elseif (CMAKE_SYSTEM_NAME MATCHES Windows) find_program(WINDEPLOYQT windeployqt) configure_file("${CMAKE_CURRENT_SOURCE_DIR}/cmake/deploy-qt-windows.cmake.in" "${CMAKE_BINARY_DIR}/cmake/deploy-qt-windows.cmake" @ONLY) set(CPACK_PRE_BUILD_SCRIPTS ${CMAKE_BINARY_DIR}/cmake/deploy-qt-windows.cmake) elseif (CMAKE_SYSTEM_NAME MATCHES Darwin) find_program(MACDEPLOYQT macdeployqt) configure_file("${CMAKE_CURRENT_SOURCE_DIR}/cmake/deploy-qt-mac.cmake.in" "${CMAKE_BINARY_DIR}/cmake/deploy-qt-mac.cmake" @ONLY) set(CPACK_PRE_BUILD_SCRIPTS ${CMAKE_BINARY_DIR}/cmake/deploy-qt-mac.cmake) endif() ``` -------------------------------- ### Link libraries for chat target Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/CMakeLists.txt Links Qt6 libraries, optional Qt6::Pdf or pdfium, llmodel, SingleApplication, fmt, duckx, QXlsx, and Cocoa on Apple platforms. ```cmake target_link_libraries(chat PRIVATE Qt6::Core Qt6::HttpServer Qt6::Quick Qt6::Sql Qt6::Svg) ``` ```cmake if (GPT4ALL_USING_QTPDF) target_compile_definitions(chat PRIVATE GPT4ALL_USE_QTPDF) target_link_libraries(chat PRIVATE Qt6::Pdf) else() # Link PDFium target_link_libraries(chat PRIVATE pdfium) endif() ``` ```cmake target_link_libraries(chat PRIVATE llmodel SingleApplication fmt::fmt duckx::duckx QXlsx) ``` ```cmake if (APPLE) target_link_libraries(chat PRIVATE ${COCOA_LIBRARY}) endif() ``` -------------------------------- ### Output of retrieving system prompt and prompt template Source: https://github.com/nomic-ai/gpt4all/wiki/Python-Bindings Shows the repr output of the systemPrompt and promptTemplate for the orca-mini-3b model. ```py '### System:\nYou are an AI assistant that follows instruction extremely well. Help as much as you can.\n\n' '### User:\n{0}\n\n### Response:\n' ``` -------------------------------- ### Initialize llama.cpp git submodule Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/typescript/README.md Updates and initializes the llama.cpp git submodule for GPT4All. Run this in the llama.cpp parent directory if the submodule is absent. ```sh git submodule update --init --recursive ``` -------------------------------- ### Clone GPT4All repository and change directory Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md Clone the repository with submodules and navigate to the backend directory. ```bash git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git cd gpt4all/gpt4all-backend ``` -------------------------------- ### Extract chat template with jq CLI Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chat_templates.md Use jq to extract the chat_template field from tokenizer_config.json. Requires jq installed. Output is human-readable; if output is "null", the model has no chat template. ```bash jq -r ".chat_template" tokenizer_config.json ``` ```bash jq -r ".chat_template" tokenizer_config.json >chat_template.txt ``` -------------------------------- ### Nous Hermes 2 Mistral DPO: React App Code Source: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/docs/gpt4all_desktop/chats.md React component code for a guessing game, as suggested by the model output. Replace the content of src/App.js with this code. ```javascript import React, { useState } from 'react'; function App() { const [guess, setGuess] = useState(''); const [message, setMessage] = useState(''); const [numberToGuess, setNumberToGuess] = useState(Math.floor(Math.random() * 10) + 1); function handleChange(e) { setGuess(e.target.value); } function handleSubmit(e) { e.preventDefault(); if (parseInt(guess) === numberToGuess) { setMessage('You guessed it right!'); } else { setMessage(`Wrong guess, the correct number was ${numberToGuess}. Try again next time.`); } } return (

Guess a Number

{message}

); } export default App; ```