### Run Modal Examples Source: https://modal.com/docs/examples/text_to_image After cloning the repository, navigate into the directory and run the provided example commands. This step is crucial for testing your setup and exploring Modal's features. ```bash cd modal-examples python -m modal run your_example.py ``` -------------------------------- ### Initialize Modal Environment Source: https://modal.com/docs/examples/flux Standard commands to install the Modal Python package and clone the examples repository to begin development. ```bash pip install modal git clone https://github.com/modal-labs/modal-examples ``` -------------------------------- ### modal setup CLI Command Source: https://modal.com/docs/reference/cli/setup This section details the usage and options for the 'modal setup' command-line interface tool. ```APIDOC ## modal setup ### Description Bootstrap Modal’s configuration. ### Method CLI Command ### Endpoint N/A (CLI command) ### Parameters #### Options - **--profile** (TEXT) - Optional - Specify a Modal profile to use. - **--help** (BOOLEAN) - Optional - Show this message and exit. ### Request Example ```bash modal setup --profile my-profile ``` ### Response #### Success Response (0) - **Output**: Confirmation message indicating successful setup or configuration. #### Response Example ``` Modal configuration bootstrapped successfully. ``` ``` -------------------------------- ### Initialize SvelteKit Application Source: https://modal.com/docs/guide/queues This script initializes the SvelteKit application by dynamically importing necessary modules and starting the app with provided configuration. It identifies the parent element for mounting and passes configuration options like node IDs and data to the SvelteKit start function. Dependencies include the SvelteKit framework itself. ```javascript const __sveltekit_1vg83v6 = { base: "" }; const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 7, 10, 102], data: [{type:"data",data:{},uses:{}}, null, null, null], form: null, error: null }); }); ``` -------------------------------- ### Initialize Modal Application Source: https://modal.com/blog/cache-dict-launch This snippet demonstrates the basic setup for initializing a Modal application using JavaScript. It imports necessary modules and starts the application, attaching it to a specific DOM element. ```javascript { "__sveltekit_1vg83v6": { "base": "" } }; const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 4, 5, 53], data: [ { type: "data", data: {}, uses: {} }, { type: "data", data: { flags: {}, announcementBanner: null }, uses: {} }, null, { type: "data", data: { page: null }, uses: { params: ["slug"] } } ], form: null, error: null }); }); ``` -------------------------------- ### Finetune an LLM with Modal Source: https://modal.com/blog/jono-containers-talk This example shows how to finetune a Large Language Model (LLM) using Modal. It covers the necessary setup for training, including data loading and model configuration, enabling you to customize LLMs for specific tasks or domains. ```python from modal import Image, Stub, method, build, enter, exit, gpu MODEL_DIR = "/model" DATA_DIR = "/data" OUTPUT_DIR = "/output" image = ( Image.debian_slim() .pip_install("torch", "transformers", "datasets", "peft", "accelerate", "bitsandbytes") .env({"HF_TOKEN": "YOUR_HUGGINGFACE_TOKEN"}) ) stub = Stub(name="llm-finetuner", image=image) @stub.function(gpu=gpu.A100(count=1), volumes={MODEL_DIR: "hf-llm-models", DATA_DIR: "my-finetuning-data", OUTPUT_DIR: "my-finetuning-output"}) def finetune_llm(): from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer from peft import LoraConfig, get_peft_model from datasets import load_dataset # Load dataset dataset = load_dataset(f"json", data_files=f"{DATA_DIR}/my_dataset.jsonl") # Load base model and tokenizer model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf", load_in_8bit=True, device_map="auto") tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf") tokenizer.pad_token = tokenizer.eos_token # Configure LoRA peft_config = LoraConfig( r=16, lora_alpha=32, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", ) model = get_peft_model(model, peft_config) # Training arguments training_args = TrainingArguments( output_dir=OUTPUT_DIR, per_device_train_batch_size=4, gradient_accumulation_steps=4, learning_rate=2e-4, num_train_epochs=1, logging_steps=10, save_steps=50, fp16=True, ) # Trainer trainer = Trainer( model=model, args=training_args, train_dataset=dataset["train"], tokenizer=tokenizer, ) # Finetune trainer.train() # Save adapter trainer.save_model(f"{OUTPUT_DIR}/final_adapter") @stub.local_entrypoint() def main(): finetune_llm.remote() ``` -------------------------------- ### Clone Modal Examples Repository Source: https://modal.com/docs/examples/text_to_image This command clones the official Modal examples repository from GitHub. This repository contains various examples demonstrating Modal's capabilities. ```bash git clone https://github.com/modal-labs/modal-examples ``` -------------------------------- ### Package Installation Source: https://modal.com/docs/reference/modal.Image Installs necessary packages into your Modal image using different package managers. ```APIDOC ## Package Installation ### `apt_install(*packages)` Installs a list of Debian packages using `apt`. ### `pip_install(*packages)` Installs Python packages using `pip`. ### `pip_install_from_requirements(requirements_file)` Installs Python packages from a `requirements.txt` file. ### `pip_install_from_pyproject(pyproject_file)` Installs Python packages from a `pyproject.toml` file using `pip`. ### `uv_pip_install(*packages)` Installs Python packages using `uv`. ### `uv_sync(lock_file)` Synchronizes Python packages using `uv` based on a lock file. ### `poetry_install_from_file(poetry_file)` Installs Python packages from a `poetry.lock` file. ### `micromamba_install(*packages)` Installs packages using `micromamba`. ``` -------------------------------- ### Initialize SvelteKit App with Dynamic Imports Source: https://modal.com/settings/image-config This snippet demonstrates how to initialize a SvelteKit application. It dynamically imports the 'start' and 'app' modules from the immutable entry points and then calls the 'start' function with the application instance, the target element, and configuration data. This is typically found in the main HTML file to bootstrap the client-side application. ```javascript const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 6, 81], data: [{type:"data",data:{},uses:{}}, null, null], form: null, error: null }); }); ``` -------------------------------- ### Modal CLI Commands Source: https://modal.com/docs/guide/notebooks Shows examples of using the Modal command-line interface (CLI) for deploying applications and managing resources. ```bash modal deploy my_app.py modal run my_app.py::my_function %modal ``` -------------------------------- ### Create a Modal Sandbox with Images and Volumes Source: https://modal.com/docs/guide/sandboxes Demonstrates how to initialize a sandbox with a specific container image and mount persistent volumes. This setup allows for isolated execution environments with access to external data. ```python sb = modal.Sandbox.create(image=modal.Image.debian_slim().pip_install("pandas"), volumes={"/data": modal.Volume.from_name("my-volume")}, workdir="/repo", app=my_app, ) sb.detach() ``` ```javascript const image = modal.images.fromRegistry("python:3.13-slim"); const volume = modal.volumes.fromName("my-volume"); const sb = await modal.sandboxes.create(app, image, { volumes: { "/data": volume }, workdir: "/repo", }); sb.detach(); ``` ```go image := mc.Images.FromRegistry("python:3.13-slim", nil) volume := mc.Volumes.FromName("my-volume", nil) sb, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Volumes: map[string]*modal.Volume{"/data": volume}, Workdir: "/repo", }) defer sb.Detach() ``` -------------------------------- ### Create and Execute Commands in a Modal Sandbox Source: https://modal.com/docs/guide/sandboxes Demonstrates how to initialize a Sandbox, execute shell commands, and handle standard output. This pattern is consistent across Python, JavaScript, and Go SDKs. ```python import modal app = modal.App.lookup("my-app", create_if_missing=True) sb = modal.Sandbox.create(app=app) p = sb.exec("python", "-c", "print('hello')", timeout=3) print(p.stdout.read()) p = sb.exec("bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done", timeout=5) for line in p.stdout: print(line, end="") sb.terminate() sb.detach() ``` ```javascript import { ModalClient } from "modal"; const modal = new ModalClient(); const app = await modal.apps.fromName("my-app", { createIfMissing: true }); const image = modal.images.fromRegistry("python:3.13-slim"); const sb = await modal.sandboxes.create(app, image); const p = await sb.exec(["python", "-c", "print('hello')"], { timeout: 3 * 1000 }); console.log(await p.stdout.readText()); const p2 = await sb.exec(["bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done"], { timeout: 5 * 1000 }); for await (const line of p2.stdout) { process.stdout.write(line); } await sb.terminate(); ``` ```go package main import ( "context" "fmt" "io" "os" "time" "github.com/modal-labs/modal-client/go" ) func main() { ctx := context.Background() mc, _ := modal.NewClient() app, _ := mc.Apps.FromName(ctx, "my-app", &modal.AppFromNameParams{CreateIfMissing: true}) image := mc.Images.FromRegistry("python:3.13-slim", nil) sb, _ := mc.Sandboxes.Create(ctx, app, image, nil) defer sb.Terminate(ctx, nil) p, _ := sb.Exec(ctx, []string{"python", "-c", "print('hello')"}, &modal.SandboxExecParams{Timeout: 3 * time.Second}) stdout, _ := io.ReadAll(p.Stdout) fmt.Println(string(stdout)) p2, _ := sb.Exec(ctx, []string{"bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done"}, &modal.SandboxExecParams{Timeout: 5 * time.Second}) io.Copy(os.Stdout, p2.Stdout) } ``` -------------------------------- ### Identifying Host Overhead with PyTorch Profiler Source: https://modal.com/blog/host-overhead-inference-efficiency This example demonstrates how to use the PyTorch Profiler to trace inference operations and identify 'gaps' in CUDA streams, which indicate host overhead. It requires PyTorch to be installed. ```python import torch # Assume model and input_data are defined # model = YourModel() # input_data = ... with torch.profiler.profile( activities=[ torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA, ], schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2), on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/inference'), record_shapes=True, profile_memory=True, with_stack=True ) as pbar: for step in range(10): # Perform inference # output = model(input_data) pbar.step() print("Profiling complete. Check TensorBoard logs at ./log/inference") ``` -------------------------------- ### Initialize SvelteKit App with Modal Source: https://modal.com/notebooks This JavaScript snippet initializes the SvelteKit application within the Modal environment. It dynamically imports necessary modules and starts the application, passing configuration details like node IDs and data. This is crucial for the initial rendering and setup of the web interface. ```javascript const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 6, 81], data: [{type:"data",data:{},uses:{}},null,null], form: null, error: null }); }); ``` -------------------------------- ### Create and Tag Modal Sandboxes (Python, JavaScript, Go) Source: https://modal.com/docs/guide/sandboxes Demonstrates creating multiple sandboxes with different commands and applying key-value tags for filtering. It also shows how to list sandboxes based on these tags. ```python notest sandbox_v1_1 = modal.Sandbox.create("sleep", "10", app=my_app) sandbox_v1_2 = modal.Sandbox.create("sleep", "20", app=my_app) sandbox_v1_1.set_tags({"major_version": "1", "minor_version": "1"}) sandbox_v1_2.set_tags({"major_version": "1", "minor_version": "2"}) for sandbox in modal.Sandbox.list(app_id=my_app.app_id): # All sandboxes. print(sandbox.object_id) for sandbox in modal.Sandbox.list( app_id=my_app.app_id, tags={"major_version": "1"}, ): # Also all sandboxes. print(sandbox.object_id) for sandbox in modal.Sandbox.list( app_id=app.app_id, tags={"major_version": "1", "minor_version": "2"}, ): # Just the latest sandbox. print(sandbox.object_id) sandbox_v1_1.detach() sandbox_v1_2.detach() ``` ```javascript notest const sandboxV1_1 = await modal.sandboxes.create(app, image, { command: ["sleep", "10"], }); const sandboxV1_2 = await modal.sandboxes.create(app, image, { command: ["sleep", "20"], }); await sandboxV1_1.setTags({ major_version: "1", minor_version: "1" }); await sandboxV1_2.setTags({ major_version: "1", minor_version: "2" }); // All sandboxes. for await (const sandbox of modal.sandboxes.list({ appId: app.appId })) { console.log(sandbox.sandboxId); } // Also all sandboxes. for await (const sandbox of modal.sandboxes.list({ appId: app.appId, tags: { major_version: "1" }, })) { console.log(sandbox.sandboxId); } // Just the latest sandbox. for await (const sandbox of modal.sandboxes.list({ appId: app.appId, tags: { major_version: "1", minor_version: "2" }, })) { console.log(sandbox.sandboxId); } sandboxV1_1.detach(); sandboxV1_2.detach(); ``` ```go notest sandboxV1_1, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Command: []string{"sleep", "10"}, }) sandboxV1_2, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Command: []string{"sleep", "20"}, }) defersb1.Detach() defersb2.Detach() sandboxV1_1.SetTags(ctx, map[string]string{"major_version": "1", "minor_version": "1"}) sandboxV1_2.SetTags(ctx, map[string]string{"major_version": "1", "minor_version": "2"}) // All sandboxes. it, _ := mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } // Also all sandboxes. it, _ = mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, Tags: map[string]string{"major_version": "1"}, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } // Just the latest sandbox. it, _ = mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, Tags: map[string]string{"major_version": "1", "minor_version": "2"}, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } ``` -------------------------------- ### Inference with vLLM and LoRA Adapters Source: https://modal.com/docs/examples/llm-finetuning This Python code snippet demonstrates how to perform inference using vLLM, a high-throughput LLM serving engine. It specifically highlights the support for dynamically swapping LoRA adapters, allowing for efficient use of fine-tuned models without merging them. The example shows a basic setup for a Modal function that can handle inference requests. ```python from modal import Stub, Image, Mount, Secret from vllm import LLM, SamplingParams stub = Stub("inference-service") # Define the base model and LoRA adapter paths BASE_MODEL = "meta-llama/Meta-Llama-3-8B" LORA_ADAPTER = "./lora-adapter" @stub.function( image=Image.debian_slim().pip_install("vllm", "torch", "transformers", "datasets", "peft"), mounts=[Mount("./models", remote_path="/models")], secret=Secret.from_name("hf-token"), gpu=True, ) def generate_response(prompt: str, lora_path: str = None) -> str: """Generates a response from the LLM, optionally using a LoRA adapter.""" # Initialize vLLM llm = LLM(model=BASE_MODEL) # Load LoRA adapter if provided if lora_path: llm.load_lora_weights(lora_path) # Define sampling parameters sampling_params = SamplingParams(temperature=0.7, top_p=0.95, max_tokens=100) # Generate output outputs = llm.generate(prompt, sampling_params) # Return the generated text return outputs[0].outputs[0].text if __name__ == "__main__": # Example usage: # response = generate_response("What is the capital of France?", lora_path=LORA_ADAPTER) # print(response) stub.run(generate_response) ``` -------------------------------- ### Install Modal Python Package Source: https://modal.com/docs/examples/text_to_image This command installs the Modal Python package, which is necessary for interacting with the Modal service. Ensure you have Python and pip installed. ```bash pip install modal ``` -------------------------------- ### Create and Tag Modal Sandboxes (Python, JavaScript, Go) Source: https://modal.com/docs/guide/sandbox Demonstrates how to create sandboxes with specific commands and apply tags for filtering. It also shows how to list sandboxes based on these tags. Sandboxes are detached at the end. ```python sandbox_v1_1 = modal.Sandbox.create("sleep", "10", app=my_app) sandbox_v1_2 = modal.Sandbox.create("sleep", "20", app=my_app) sandbox_v1_1.set_tags({"major_version": "1", "minor_version": "1"}) sandbox_v1_2.set_tags({"major_version": "1", "minor_version": "2"}) for sandbox in modal.Sandbox.list(app_id=my_app.app_id): # All sandboxes. print(sandbox.object_id) for sandbox in modal.Sandbox.list( app_id=my_app.app_id, tags={"major_version": "1"}, ): # Also all sandboxes. print(sandbox.object_id) for sandbox in modal.Sandbox.list( app_id=app.app_id, tags={"major_version": "1", "minor_version": "2"}, ): # Just the latest sandbox. print(sandbox.object_id) sandbox_v1_1.detach() sandbox_v1_2.detach() ``` ```javascript const sandboxV1_1 = await modal.sandboxes.create(app, image, { command: ["sleep", "10"], }); const sandboxV1_2 = await modal.sandboxes.create(app, image, { command: ["sleep", "20"], }); await sandboxV1_1.setTags({ major_version: "1", minor_version: "1" }); await sandboxV1_2.setTags({ major_version: "1", minor_version: "2" }); // All sandboxes. for await (const sandbox of modal.sandboxes.list({ appId: app.appId })) { console.log(sandbox.sandboxId); } // Also all sandboxes. for await (const sandbox of modal.sandboxes.list({ appId: app.appId, tags: { major_version: "1" }, })) { console.log(sandbox.sandboxId); } // Just the latest sandbox. for await (const sandbox of modal.sandboxes.list({ appId: app.appId, tags: { major_version: "1", minor_version: "2" }, })) { console.log(sandbox.sandboxId); } sandboxV1_1.detach(); sandboxV1_2.detach(); ``` ```go sandboxV1_1, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Command: []string{"sleep", "10"}, }) sandboxV1_2, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Command: []string{"sleep", "20"}, }) defer sandboxV1_1.Detach() defer sandboxV1_2.Detach() sandboxV1_1.SetTags(ctx, map[string]string{"major_version": "1", "minor_version": "1"}) sandboxV1_2.SetTags(ctx, map[string]string{"major_version": "1", "minor_version": "2"}) // All sandboxes. it, _ := mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } // Also all sandboxes. it, _ = mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, Tags: map[string]string{"major_version": "1"}, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } // Just the latest sandbox. it, _ = mc.Sandboxes.List(ctx, &modal.SandboxListParams{ AppID: app.AppID, Tags: map[string]string{"major_version": "1", "minor_version": "2"}, }) for sandbox := range it { fmt.Println(sandbox.SandboxID) } ``` -------------------------------- ### Finetune an LLM with Modal Source: https://modal.com/blog/gpu-utilization-guide This example demonstrates how to finetune a Large Language Model (LLM) using Modal. It covers setting up the training environment, loading a dataset, and executing the finetuning process on Modal's infrastructure. ```python from modal import Image, Stub, Volume, gpu from modal.functions import FunctionCall import torch # Define a base image with necessary libraries for finetuning # This is a placeholder, replace with your actual image and dependencies image = ( Image.debian_slim() .pip_install("transformers", "datasets", "accelerate", "peft", "torch") .apt_install("git") .env({"HF_TOKEN": "YOUR_HUGGINGFACE_TOKEN"}) # Ensure you have a Hugging Face token set ) stub = Stub(name="llm-finetuner", image=image) # Define a volume for storing datasets and model checkpoints # Replace with your desired volume name and mount path model_volume = Volume.persisted("modal-finetune-data") @stub.function( volumes={ "/data": model_volume }, gpu=gpu.A100(count=1), # Request a GPU for training timeout=3600 # Set a timeout for the training job (in seconds) ) def finetune_llm(dataset_path: str, model_id: str, output_dir: str): """Finetunes an LLM on a given dataset.""" from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments from peft import LoraConfig, get_peft_model from datasets import load_dataset print(f"Loading dataset from {dataset_path}") # Load your dataset (e.g., from a CSV or JSON file) # Replace with your actual dataset loading logic try: dataset = load_dataset(dataset_path, split="train") except Exception as e: print(f"Error loading dataset: {e}") return print(f"Loading base model: {model_id}") model = AutoModelForCausalLM.from_pretrained(model_id) tokenizer = AutoTokenizer.from_pretrained(model_id) # Configure LoRA (Low-Rank Adaptation) lora_config = LoraConfig( r=16, # Rank lora_alpha=32, # Alpha scaling target_modules=["q_proj", "v_proj"], # Modules to apply LoRA to lora_dropout=0.05, bias="none", task_type="CAUSAL_LM" ) model = get_peft_model(model, lora_config) model.print_trainable_parameters() # Define training arguments training_args = TrainingArguments( output_dir=output_dir, # Directory to save model checkpoints per_device_train_batch_size=4, # Batch size per GPU gradient_accumulation_steps=4, # Accumulate gradients over steps learning_rate=2e-4, num_train_epochs=1, # Number of training epochs logging_steps=10, save_steps=50, fp16=True, # Use mixed precision training if supported # Add other relevant training arguments ) # Initialize Trainer trainer = Trainer( model=model, args=training_args, train_dataset=dataset, tokenizer=tokenizer, # data_collator can be added here if needed ) print("Starting finetuning...") trainer.train() print(f"Saving finetuned model to {output_dir}") trainer.save_model(output_dir) print("Finetuning complete.") @stub.local_entrypoint() def main(): # Define parameters for finetuning # Ensure the dataset path is accessible (e.g., uploaded to a volume or accessible via URL) # Replace with your actual dataset path and desired model ID dataset_path = "/data/my_finetune_dataset.jsonl" # Example path on the volume model_id = "meta-llama/Llama-2-7b-hf" # Example base model output_directory = "/data/finetuned_model" # Example output directory on the volume # Ensure the output directory exists on the volume model_volume.mkdir(output_directory) # Call the finetune function remotely # This will upload the script and run it on Modal infrastructure finetune_llm.remote(dataset_path=dataset_path, model_id=model_id, output_dir=output_directory) print(f"Finetuning job submitted. Model will be saved to {output_directory}") ``` -------------------------------- ### POST /sandboxes/create Source: https://modal.com/docs/guide/sandboxes Creates a new sandbox instance with specific configurations like images, volumes, and working directories. ```APIDOC ## POST /sandboxes/create ### Description Creates a new isolated sandbox environment. This endpoint allows for the attachment of volumes, custom images, and specific working directories. ### Method POST ### Endpoint /sandboxes/create ### Parameters #### Request Body - **image** (Object) - Required - The container image to use for the sandbox. - **volumes** (Map) - Optional - A mapping of mount paths to Volume objects. - **workdir** (String) - Optional - The working directory inside the container. - **app** (Object) - Required - The parent application context. ### Request Example { "image": "debian_slim", "volumes": {"/data": "my-volume"}, "workdir": "/repo" } ### Response #### Success Response (200) - **id** (String) - The unique identifier of the created sandbox. #### Response Example { "id": "sb-12345" } ``` -------------------------------- ### Create and Execute Code in a Modal Sandbox Source: https://modal.com/docs/guide/sandbox Demonstrates how to create a Modal Sandbox, execute commands within it, and read the output. Includes examples for running Python and Bash commands. Sandboxes require an App object, which can be looked up or created if missing. ```python notest import modal app = modal.App.lookup("my-app", create_if_missing=True) sb = modal.Sandbox.create(app=app) p = sb.exec("python", "-c", "print('hello')", timeout=3) print(p.stdout.read()) p = sb.exec("bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done", timeout=5) for line in p.stdout: # Avoid double newlines by using end="". print(line, end="") sb.terminate() sb.detach() ``` ```javascript notest import { ModalClient } from "modal"; const modal = new ModalClient(); const app = await modal.apps.fromName("my-app", { createIfMissing: true, }); const image = modal.images.fromRegistry("python:3.13-slim"); const sb = await modal.sandboxes.create(app, image); const p = await sb.exec(["python", "-c", "print('hello')"], { timeout: 3 * 1000, }); console.log(await p.stdout.readText()); const p2 = await sb.exec( ["bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done"], { timeout: 5 * 1000 }, ); for await (const line of p2.stdout) { process.stdout.write(line); } await sb.terminate(); ``` ```go notest package main import ( "context" "fmt" "io" "os" "time" "github.com/modal-labs/modal-client/go" ) func main() { ctx := context.Background() mc, _ := modal.NewClient() app, _ := mc.Apps.FromName(ctx, "my-app", &modal.AppFromNameParams{ CreateIfMissing: true, }) image := mc.Images.FromRegistry("python:3.13-slim", nil) sb, _ := mc.Sandboxes.Create(ctx, app, image, nil) defer sb.Terminate(ctx, nil) p, _ := sb.Exec(ctx, []string{"python", "-c", "print('hello')"}, &modal.SandboxExecParams{ Timeout: 3 * time.Second, }) stdout, _ := io.ReadAll(p.Stdout) fmt.Println(string(stdout)) p2, _ := sb.Exec(ctx, []string{"bash", "-c", "for i in {1..10}; do date +%T; sleep 0.5; done"}, &modal.SandboxExecParams{ Timeout: 5 * time.Second, }) io.Copy(os.Stdout, p2.Stdout) } ``` -------------------------------- ### Add Python Packages using uv_pip_install Source: https://modal.com/docs/guide/images Installs Python packages using uv, the fast, reliable Python package installer. ```APIDOC ## Image.uv_pip_install ### Description Installs Python packages using uv. ### Method `Image.uv_pip_install` ### Parameters #### Arguments - **packages** (list[str]) - Required - A list of package names to install, optionally with version specifiers. ### Request Example ```python from modal import Image my_image = Image.debian_slim().uv_pip_install("pandas", "torch==2.8.0") ``` ### Response #### Success Response (200) Returns the modified Image object. #### Response Example ```json { "message": "Image updated with packages", "packages_installed": ["pandas", "torch==2.8.0"] } ``` ``` -------------------------------- ### Initialize Modal SDK Client Source: https://modal.com/gpu-glossary/device-hardware/cuda-core This snippet demonstrates the initialization of the Modal SDK client using SvelteKit's dynamic import pattern. It loads the necessary application modules and starts the client lifecycle. ```javascript { __sveltekit_1vg83v6 = { base: "" }; const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 13, 110], data: [{type:"data",data:{},uses:{}},null,null], form: null, error: null }); }); } ``` -------------------------------- ### Run Sandboxes with Custom Entrypoints Source: https://modal.com/docs/guide/sandboxes Demonstrates how to override the default container command by providing a specific entrypoint and timeout, useful for long-running services. ```python sb = modal.Sandbox.create("python", "-m", "http.server", "8080", app=my_app, timeout=10) for line in sb.stdout: print(line, end="") sb.detach() ``` ```javascript const sb = await modal.sandboxes.create(app, image, { entrypoint: ["python", "-m", "http.server", "8080"], timeout: 10 * 1000, }); sb.detach(); ``` ```go sb, err := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{ Entrypoint: []string{"python", "-m", "http.server", "8080"}, Timeout: 10 * time.Second, }) sb.Detach() ``` -------------------------------- ### Initialize SvelteKit Application Source: https://modal.com/settings/tokens/member-tokens This JavaScript code snippet initializes the SvelteKit application. It finds the parent element of the current script, dynamically imports the SvelteKit start and app modules, and then starts the application with provided configuration including node IDs, data, form, and error states. ```javascript const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 6, 81], data: [{type: "data", data: {}, uses: {}}, null, null], form: null, error: null }); }); ``` -------------------------------- ### Initialize SvelteKit Application Source: https://modal.com/secrets This script initializes the SvelteKit application by dynamically importing the start and app modules. It then triggers the kit.start method with the required node IDs and initial data state. ```javascript { __sveltekit_1vg83v6 = { base: "" }; const element = document.currentScript.parentElement; Promise.all([ import("/_app/immutable/entry/start.BHsQNYu4.js"), import("/_app/immutable/entry/app.KdrcnPtc.js") ]).then(([kit, app]) => { kit.start(app, element, { node_ids: [0, 6, 81], data: [{type:"data",data:{},uses:{}}, null, null], form: null, error: null }); }); } ``` -------------------------------- ### GET /ls Source: https://modal.com/docs/reference/modal.Sandbox Lists the contents of a specified directory within the Sandbox. ```APIDOC ## GET /ls ### Description List the contents of a directory in the Sandbox. ### Method GET ### Endpoint /ls ### Parameters #### Query Parameters - **path** (string) - Required - The directory path to list. ### Response #### Success Response (200) - **files** (array) - A list of file and directory names within the specified path. ```