### Install and Verify MAX Environment Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Commands to navigate to the quickstart directory, and verify the installation of MAX and Python versions. Also includes a command to clean cached data. ```bash cd src/quickstart ``` ```bash pixi run max --version ``` ```bash pixi run python --version ``` ```bash pixi clean ``` -------------------------------- ### Navigate to Mojo Example Directory Source: https://docs.modular.com/mojo/manual/install This command navigates the user into a specific subdirectory within the cloned Mojo repository, typically containing an example project like Conway's Game of Life. This prepares the user to install dependencies and run the example. ```bash cd modular/examples/mojo/life ``` -------------------------------- ### Install and Initialize Conda for Project Setup Source: https://builds.modular.com/models/Llama-3.2-Vision-Instruct/11B Installs conda if not present, initializes it for shell interaction, and creates a new conda environment named 'quickstart'. This sets up the foundational environment for the project. ```bash brew install miniconda conda init conda init zsh conda create -n quickstart conda activate quickstart ``` -------------------------------- ### Run Mojo Example with Pixi Source: https://docs.modular.com/mojo/manual/install This command uses the `pixi` package manager to install project dependencies, build, and run a Mojo example. It's specifically shown for the 'lifev1' example from the Game of Life tutorial. Ensure `pixi` is installed and configured. ```bash pixi run lifev1 ``` -------------------------------- ### Get Started with Pixi and Node.js Source: https://pixi.sh/latest This guide shows how to set up a Node.js project using Pixi. It includes initializing a workspace, adding Node.js as a dependency, creating a simple JavaScript file, defining a task to run the script, and executing the task. ```shell pixi init pixi-node cd pixi-node ``` ```shell pixi add nodejs ``` ```javascript console.log("Hello Pixi fans!"); ``` ```shell pixi task add start "node hello.js" ``` ```shell pixi run start ``` -------------------------------- ### Setup Project and Install Modular (uv) Source: https://builds.modular.com/models/InternVL3/14B This snippet guides users through setting up a project and virtual environment using 'uv', a fast Python package installer. It then proceeds to install the Modular Python package. This is an alternative to the pip-based setup. ```shell curl -LsSf https://astral.sh/uv/install.sh | sh # Restart terminal after installation uv init quickstart && cd quickstart uv venv && source .venv/bin/activate ``` ```shell uv pip install modular \ --index-url https://dl.modular.com/public/nightly/python/simple/ \ --prerelease allow ``` ```shell uv pip install modular \ --extra-index-url https://modular.gateway.scarf.sh/simple/ ``` -------------------------------- ### Mojo Setup and Configuration for Vector Addition Source: https://docs.modular.com/mojo/manual/gpu/basics Sets up constants and imports necessary for the vector addition example. This includes defining data types, vector size, layout, block size, and calculating the number of blocks required for GPU execution. ```mojo from math import ceildiv from sys import has_accelerator from gpu.host import DeviceContext from gpu import block_dim, block_idx, thread_idx from layout import Layout, LayoutTensor # Vector data type and size comptime float_dtype = DType.float32 comptime vector_size = 1000 comptime layout = Layout.row_major(vector_size) # Calculate the number of thread blocks needed by dividing the vector size # by the block size and rounding up. comptime block_size = 256 comptime num_blocks = ceildiv(vector_size, block_size) ``` -------------------------------- ### Start Gemma 3 Endpoint with MAX CLI Source: https://docs.modular.com/max/get-started Starts a MAX endpoint for the Google Gemma 3 model. Requires Hugging Face token and compatible GPU. Ensure you agree to the model license before starting. ```bash export HF_TOKEN="hf_..." max serve --model google/gemma-3-27b-it ``` -------------------------------- ### Get Started with Pixi and Python Source: https://pixi.sh/latest A step-by-step guide to setting up a Python project with Pixi. It covers initializing a workspace, adding Python and PyPI dependencies, creating a script, defining a task, running the task, and entering the environment shell. ```shell pixi init hello-world cd hello-world ``` ```shell pixi add cowpy python ``` ```python from cowpy.cow import Cowacter message = Cowacter().milk("Hello Pixi fans!") print(message) ``` ```shell pixi task add start python hello.py ``` ```shell pixi run start ``` ```shell pixi shell ``` -------------------------------- ### Run Mojo Example with Mojo CLI Source: https://docs.modular.com/mojo/manual/install This command executes a Mojo example file directly using the `mojo` CLI. It's an alternative to using `pixi` for running examples, especially when `pixi` is not preferred or available. This requires the `mojo` executable to be in your system's PATH. ```bash mojo lifev1.mojo ``` -------------------------------- ### Execute Basic Layouts Example with Pixi Source: https://github.com/modular/modular/tree/main/mojo/examples/layouts This command executes the basic layouts example using the Pixi package manager. Ensure Pixi is installed and configured to run Mojo examples. ```shell pixi run mojo basic_layouts ``` -------------------------------- ### Execute Tiled Layouts Example with Pixi Source: https://github.com/modular/modular/tree/main/mojo/examples/layouts This command executes the tiled layouts example using the Pixi package manager. Ensure Pixi is installed and configured to run Mojo examples. ```shell pixi run mojo tiled_layouts ``` -------------------------------- ### Verify Mojo Installation Source: https://docs.modular.com/mojo/manual/get-started Verifies the installed Mojo version within the project's virtual environment using 'pixi run'. This ensures the project is set up correctly. ```bash pixi run mojo --version ``` -------------------------------- ### Install OpenAI Package via Pixi Source: https://docs.modular.com/max/get-started Installs the OpenAI Python package using the Pixi package manager. This is a prerequisite for making inference requests. ```bash pixi add openai ``` -------------------------------- ### Install Mojo with Modular CLI Source: https://github.com/modular/modular/issues/538 This snippet demonstrates the command used to install the 'mojo' package using the 'modular' command-line interface. It shows the output, including package version, download progress, and post-installation tests confirming successful setup. It requires the 'modular' CLI to be installed and configured. ```bash ▶ modular -v modular 0.1.4 (6b54d308) ▶ modular clean ▶ modular install mojo # Found release for https://packages.modular.com/mojo @ 0.2.1, installing to /home/megamorf/.modular/pkg/packages.modular.com_mojo # Downloads complete, setting configs... # Configs complete, running post-install hooks... [...output omitted...] [notice] A new release of pip is available: 23.0.1 -> 23.2.1 [notice] To update, run: python3.11 -m pip install --upgrade pip Testing `MODULAR_HOME=/home/megamorf/.modular` * `/home/megamorf/.modular/pkg/packages.modular.com_mojo/bin/mojo`... TEST: `mojo --help`... OK TEST: `mojo run --help`... OK TEST: `mojo build test_mandelbrot.mojo`... OK TEST: `mojo build test_python.mojo`... OK TEST: `mojo demangle`... OK TEST: `mojo format`... OK TEST: `mojo package`... OK TEST: `mojo test_mandelbrot.mojo`... OK TEST: `mojo test_python.mojo`... OK ``` -------------------------------- ### Install Pixi and Initialize Project Source: https://builds.modular.com/models/Llama-3.2-Vision-Instruct/11B Installs the pixi package manager and initializes a new project named 'quickstart'. It specifies the conda-forge and modular-nightly channels for package resolution. ```bash curl -fsSL https://pixi.sh/install.sh | sh pixi init quickstart \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd quickstart ``` -------------------------------- ### Install OpenAI Package via UV Source: https://docs.modular.com/max/get-started Installs the OpenAI Python package using the UV package manager. This is a prerequisite for making inference requests. ```bash uv add openai ``` -------------------------------- ### Verify Setup and Run First Puzzle (uv) Source: https://puzzles.modular.com/howto.html Verifies the setup and runs the first puzzle using the uv package manager. This involves installing GPU-specific dependencies, checking GPU specifications, and executing the initial puzzle. ```bash # Install GPU-specific dependencies uv pip install -e ".[nvidia]" # For NVIDIA GPUs # OR uv pip install -e ".[amd]" # For AMD GPUs # Check your GPU specifications uv run poe gpu-specs # Run your first puzzle # This fails waiting for your implementation! follow the content uv run poe p01 ``` -------------------------------- ### Installing Modular APIs and Tools with pip (Shell) Source: https://docs.modular.com/max/changelog Instructions for installing Modular APIs and tools, including the 'max' CLI, Python library, and Mojo CLI, using pip. This simplifies the setup process for developers. ```shell pip install modular \ --index-url https://download.pytorch.org/whl/cpu ``` -------------------------------- ### Install and Verify Mojo with Pixi Source: https://docs.modular.com/mojo/manual/gpu/basics Installs Mojo using the 'pixi' package manager, adds necessary conda channels, and verifies the installation by checking the Mojo version. This is the initial setup for a Mojo project intended for GPU programming. ```bash curl -fsSL https://pixi.sh/install.sh | sh pixi init gpu-intro \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd gpu-intro pixi add mojo pixi run mojo --version pixi shell ``` -------------------------------- ### Quick Demo: Initialize Project and Run Python Source: https://pixi.sh/latest This sequence demonstrates initializing a new Pixi project named 'hello-world', adding Python to its environment, and running a simple Python command. ```bash pixi init hello-world cd hello-world pixi add python pixi run python -c 'print("Hello World!")' ``` -------------------------------- ### Print 'Hello World' in Mojo Source: https://docs.modular.com/mojo/manual/install This snippet demonstrates how to create a basic Mojo file and execute it using the `mojo` CLI. It defines a `main` function that prints the string 'Hello, World!' to the console. Ensure you have the `mojo` environment set up before running. ```mojo def main(): print("Hello, World!") ``` -------------------------------- ### Install Safetensors via Pip Source: https://github.com/huggingface/safetensors Installs the safetensors library using the pip package manager. This is the recommended method for most users to quickly get started. ```bash pip install safetensors ``` -------------------------------- ### Running Benchmark with Command Line Arguments (Example) Source: https://docs.modular.com/max/cli/benchmark This example shows how to run the benchmark using individual command-line arguments. It covers model selection, backend, endpoint, host, port, number of prompts, dataset name, and dataset-specific input/output lengths. This method is an alternative to using a configuration file. ```bash max benchmark \ --model google/gemma-3-27b-it \ --backend modular \ --endpoint /v1/chat/completions \ --host localhost \ --port 8000 \ --num-prompts 50 \ --dataset-name arxiv-summarization \ --arxiv-summarization-input-len 12000 \ --max-output-len 1200 ``` -------------------------------- ### Check MAX version with pip Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Displays the installed version of the 'modular' Python package using pip. This helps verify the installation and version compatibility. ```bash pip show modular ``` -------------------------------- ### Get Started with Pixi and Rust Source: https://pixi.sh/latest A guide to using Pixi with Rust projects. It demonstrates initializing a workspace, adding the Rust toolchain, creating a Rust project using `cargo init`, defining a build task, and running it. Note: This is an example and not the recommended way to build Rust projects. ```shell pixi init pixi-rust cd pixi-rust ``` ```shell pixi add rust ``` ```shell pixi run cargo init ``` ```shell pixi task add start cargo run ``` ```shell pixi run start ``` -------------------------------- ### Start Local Endpoint and Send Request (Conda) Source: https://builds.modular.com/models/all-mpnet-base-v2/5B This section guides you through installing necessary tools with Conda, starting a local endpoint for the 'all-mpnet-base-v2' model, and sending a sample request. ```APIDOC ## Start Local Endpoint and Send Request (Conda) ### Description This covers the steps to set up a local environment using Conda, serve the `all-mpnet-base-v2` model, and make an embedding request. ### Setup Steps 1. **Install Conda**: If you don't have it, install Miniconda. ```bash brew install miniconda ``` 2. **Initialize Conda**: Prepare Conda for shell interaction. ```bash conda init # For Mac, use: conda init zsh ``` Restart your terminal for changes to take effect. 3. **Create Project Environment**: Create a new Conda environment. ```bash conda create -n quickstart ``` 4. **Activate Environment**: Start the virtual environment. ```bash conda activate quickstart ``` 5. **Install Modular Package**: Install the `modular` package (Nightly or Stable). ```bash # Nightly build conda install -c conda-forge -c https://conda.modular.com/max-nightly/ modular # Stable build conda install -c conda-forge -c https://conda.modular.com/max/ modular ``` 6. **Start Local Endpoint**: Serve the `all-mpnet-base-v2` model locally. ```bash max serve --model sentence-transformers/all-mpnet-base-v2 ``` The server is ready when you see `🚀 Server ready`. ### Request Embedding Send a request to the local endpoint using `curl`: ```bash curl -N http://0.0.0.0:8000/v1/embeddings \ -H "Content-Type: application/json" \ -d '{ "input": "Turn this sentence into embeddings", "model": "sentence-transformers/all-mpnet-base-v2", "encoding_format": "float" }' ``` ### Response Example Upon a successful request, you will receive a JSON response containing the embeddings. ```json { "object": "embedding", "data": [ { "object": "embedding", "embedding": [ 0.00123, -0.00456, ... ] } ], "model": "sentence-transformers/all-mpnet-base-v2", "usage": { "prompt_tokens": 1, "total_tokens": 1 } } ``` ``` -------------------------------- ### Create Project Folder for Llama 3 Tutorial Source: https://docs.modular.com/max/tutorials/max-serve-local-to-cloud Creates a new directory named 'llama3-tutorial' to serve as the root for the project and then changes the current directory into it. ```shell mkdir llama3-tutorial && cd llama3-tutorial ``` -------------------------------- ### Initialize uv Project for Llama 3 Tutorial Source: https://docs.modular.com/max/tutorials/max-serve-local-to-cloud Initializes a new uv project named 'llama3-tutorial' and changes the directory into it. uv will manage the project's dependencies and virtual environment. ```shell uv init llama3-tutorial && cd llama3-tutorial ``` -------------------------------- ### Install Mojo using pixi CLI Source: https://docs.modular.com/mojo/manual/gpu/intro-tutorial Installs the pixi package manager and initializes a new Mojo project. It adds the Modular conda package channel and installs the 'mojo' package within the project's virtual environment. Finally, it activates the environment. ```shell curl -fsSL https://pixi.sh/install.sh | sh pixi init gpu-intro \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd gpu-intro pixi add mojo pixi run mojo --version pixi shell ``` -------------------------------- ### Activate conda environment Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Activates the 'example-project' conda environment. This makes the packages installed within this environment available in the current shell session. ```bash conda activate example-project ``` -------------------------------- ### Create Project Folder Source: https://builds.modular.com/models/gpt-oss-20b-BF16/20B Creates a new directory named 'quickstart' and changes the current directory to it. This is the first step in setting up a new project. ```shell mkdir quickstart && cd quickstart ``` -------------------------------- ### Create and activate Python virtual environment Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Creates a Python virtual environment using the 'venv' module and activates it. This isolates project dependencies from the system Python installation. ```bash python3 -m venv .venv/example-project && source .venv/example-project/bin/activate ``` -------------------------------- ### Go 'Hello, World!' Example Source: https://go.dev/ A fundamental 'Hello, World!' program in Go. This snippet demonstrates the basic structure of a Go program, including package declaration, import statements, and the main function. It's a common starting point for learning the language. ```go package main import "fmt" func main() { fmt.Println("Hello, 世界") } ``` -------------------------------- ### Terminal Output: Addition Result Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python This is the expected output when running the Python example. It shows the final result of the tensor addition, confirming that [1.0] + [1.0] correctly yields [2.0]. ```text result: [2.] ``` -------------------------------- ### Add modular package with pixi Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Adds the 'modular' conda package to the pixi project. Two options are provided: installing the nightly build or a specific stable version (v25.7). ```bash pixi add modular ``` ```bash pixi add "modular==25.7" ``` -------------------------------- ### Create project folder Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Creates a new directory named 'example-project' and changes the current directory into it. This serves as the root for the project. ```bash mkdir example-project && cd example-project ``` -------------------------------- ### Create and Activate Virtual Environment with venv Source: https://docs.modular.com/max/get-started Creates a Python virtual environment named '.venv/quickstart' using the 'venv' module and activates it for isolated package management. ```shell python3 -m venv .venv/quickstart \ && source .venv/quickstart/bin/activate ``` -------------------------------- ### Check MAX version with uv Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Displays the installed version of the 'modular' Python package using uv pip. This command is used within a uv-managed project to check package details. ```bash uv pip show modular ``` -------------------------------- ### Initialize uv project Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python Initializes a new project named 'example-project' using uv and changes the directory into the project folder. uv will automatically create a virtual environment. ```bash uv init example-project && cd example-project ``` -------------------------------- ### Run ShareGPTv3 Benchmark Client Source: https://www.modular.com/blog/max-gpu-state-of-the-art-throughput-on-a-new-genai-platform This snippet shows how to initiate the benchmark client for the ShareGPTv3 workload. It specifies the backend to be used ('vllm' in this example) and the model to benchmark. This command assumes the ShareGPTv3 dataset has already been downloaded. ```bash python benchmark_serving.py \ --backend vllm \ --model meta-llama/M ``` -------------------------------- ### Initialize Mojo Project with Pixi Source: https://docs.modular.com/mojo/manual/get-started Initializes a new Mojo project named 'life' using pixi, adds the Modular conda package channel, and navigates into the project directory. It then installs the 'mojo' package. ```bash curl -fsSL https://pixi.sh/install.sh | sh pixi init life \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd life pixi add mojo ``` -------------------------------- ### Python: Execute Addition Graph Example Source: https://docs.modular.com/max/develop/get-started-with-max-graph-in-python This Python code snippet defines input tensors and calls the add_tensors function to execute a pre-defined MAX graph for addition. It requires NumPy for array creation. The output shows the result of the tensor addition. ```python if __name__ == "__main__": input0 = np.array([1.0], dtype=np.float32) input1 = np.array([1.0], dtype=np.float32) result = add_tensors(input0, input1) print("result:", result) ``` -------------------------------- ### Initialize a new Pixi project Source: https://docs.modular.com/pixi This command initializes a new Pixi project named 'example-project'. It automatically configures the project environment, including setting up the virtual environment and `pixi.toml` file, using the default channels specified in the user configuration. ```bash pixi init example-project ``` -------------------------------- ### Create Project Directory Source: https://docs.modular.com/max/inference/image-to-text Creates a new directory named 'vision-quickstart' for the project. This is a standard step before initializing project-specific configurations. ```bash mkdir vision-quickstart && cd vision-quickstart ``` -------------------------------- ### Initialize Project with uv Source: https://docs.modular.com/max/develop/serve-custom-model-architectures Initializes a new project named 'qwen2' using uv and navigates the user into the project directory. uv is a fast Python package installer and virtual environment manager. ```bash uv init qwen2 && cd qwen2 ``` -------------------------------- ### Vector Addition Main Function and GPU Setup in Mojo Source: https://docs.modular.com/mojo/manual/gpu/gpu-basics Sets up the GPU context, initializes host buffers, creates device buffers, copies data, enqueues the vector addition kernel, and retrieves results. Includes error handling for missing accelerators. ```mojo def main(): @parameter if not has_accelerator(): print("No compatible GPU found") else: # Get the context for the attached GPU ctx = DeviceContext() # Create HostBuffers for input vectors lhs_host_buffer = ctx.enqueue_create_host_buffer[float_dtype]( vector_size ) rhs_host_buffer = ctx.enqueue_create_host_buffer[float_dtype]( vector_size ) ctx.synchronize() # Initialize the input vectors for i in range(vector_size): lhs_host_buffer[i] = Float32(i) rhs_host_buffer[i] = Float32(i * 0.5) print("LHS buffer: ", lhs_host_buffer) print("RHS buffer: ", rhs_host_buffer) # Create DeviceBuffers for the input vectors lhs_device_buffer = ctx.enqueue_create_buffer[float_dtype](vector_size) rhs_device_buffer = ctx.enqueue_create_buffer[float_dtype](vector_size) # Copy the input vectors from the HostBuffers to the DeviceBuffers ctx.enqueue_copy(dst_buf=lhs_device_buffer, src_buf=lhs_host_buffer) ctx.enqueue_copy(dst_buf=rhs_device_buffer, src_buf=rhs_host_buffer) # Create a DeviceBuffer for the result vector result_device_buffer = ctx.enqueue_create_buffer[float_dtype]( vector_size ) # Wrap the DeviceBuffers in LayoutTensors lhs_tensor = LayoutTensor[float_dtype, layout](lhs_device_buffer) rhs_tensor = LayoutTensor[float_dtype, layout](rhs_device_buffer) result_tensor = LayoutTensor[float_dtype, layout](result_device_buffer) # Compile and enqueue the kernel ctx.enqueue_function_checked[vector_addition, vector_addition]( lhs_tensor, rhs_tensor, result_tensor, grid_dim=num_blocks, block_dim=block_size, ) # Create a HostBuffer for the result vector result_host_buffer = ctx.enqueue_create_host_buffer[float_dtype]( vector_size ) # Copy the result vector from the DeviceBuffer to the HostBuffer ctx.enqueue_copy( dst_buf=result_host_buffer, src_buf=result_device_buffer, ) # Finally, synchronize the DeviceContext to run all enqueued operations ctx.synchronize() print("Result vector:", result_host_buffer) ``` -------------------------------- ### Python Virtual Environment Setup Source: https://docs.modular.com/max/container These commands create and activate a Python virtual environment named '.venv/quickstart' in the current directory. This isolates project dependencies. ```bash mkdir quickstart && cd quickstart ``` ```bash python3 -m venv .venv/quickstart \ && source .venv/quickstart/bin/activate ``` -------------------------------- ### Install a Package using Homebrew Source: https://brew.sh/ This command installs a specified package using Homebrew. For example, `brew install wget` will install the `wget` utility. Homebrew manages package installations within its own directory structure. ```bash $ brew install wget ``` -------------------------------- ### MAX Server and Installation Commands Source: https://github.com/modular/modular/blob/main/CLAUDE Commands for installing and running the MAX server, including Python pip installation, global Pixi installation, starting the OpenAI-compatible server, and running with Docker. These commands facilitate model serving and inference. ```bash # Install the MAX nightly within a Python virtual environment using pip pip install modular --index-url https://dl.modular.com/public/nightly/python/simple/ # Install MAX globally using Pixi, an alternative to the above pixi global install -c conda-forge -c https://conda.modular.com/max-nightly # Start OpenAI-compatible server max serve --model modularai/Llama-3.1-8B-Instruct-GGUF # Run with Docker docker run --gpus=1 -p 8000:8000 docker.modular.com/modular/max-nvidia-full:latest --model modularai/Llama-3.1-8B-Instruct-GGUF ``` -------------------------------- ### Create Project Directory Source: https://docs.modular.com/max/develop/serve-custom-model-architectures Creates a new directory named 'qwen2' and then changes the current working directory into it. This is a standard way to start a new project. ```bash mkdir qwen2 && cd qwen2 ``` -------------------------------- ### Install CrewAI with Pip Source: https://github.com/crewAIInc/crewAI Installs the base CrewAI package using pip. This command is essential for starting any CrewAI project. Ensure Python version 3.10 to 3.14 is installed prior to execution. ```shell pip install crewai ``` -------------------------------- ### Pip Editable Installs Fix (Commit Message) Source: https://pixi.sh/latest/CHANGELOG A commit message reporting a fix for pip editable installs getting uninstalled. This ensures that packages installed in editable mode are not inadvertently removed. ```text Pip editable installs getting uninstalled by @renan-r-santos in #902 ```