### Start Studio with CloudProvider and Machine Source: https://lightning.ai/docs/platform/developers/sdk/studio Basic example of starting a Studio instance with a specific cloud provider and machine type. ```python from lightning_sdk import Machine, CloudProvider studio = Studio(name="my-new-studio", create_ok=True, cloud_provider=CloudProvider.AWS) studio.start(Machine.L4) ``` -------------------------------- ### on_start.sh example Source: https://lightning.ai/docs/platform/build/ai-studio/on-start-actions Example on_start.sh script that sets an environment variable and starts a server. Environment variables set here are not available in subsequent studio terminals. ```bash #!/bin/bash # This script runs every time your Studio starts, from your home directory. export SOME_IMPORTANT_PATH="/teamspace/studios/this_studio/path/to/some/important/folder" # start a server python server.py ``` -------------------------------- ### Start VM with CloudProvider Source: https://lightning.ai/docs/platform/developers/sdk/vms Basic example of creating a VM instance and starting it with a specific machine type on AWS. ```python from lightning_sdk import Machine, CloudProvider vm = VM(name="my-new-vm", create_ok=True, cloud_provider=CloudProvider.AWS) vm.start(Machine.L4) ``` -------------------------------- ### CLIPAPI setup and inference implementation Source: https://lightning.ai/docs/litserve/api-reference/litapi This example demonstrates using the setup method to initialize a CLIP model and pre-encode text captions on a specified device. It includes the full LitAPI implementation for image classification. ```python # pip install torch clip import torch, clip, multiprocessing from PIL import Image from io import BytesIO import litserve as ls class CLIPAPI(ls.LitAPI): def setup(self, device): # load the model on the device automatically (CPU, GPU etc...) self.model, self.preprocess = clip.load("ViT-B/32", device=device) # Load candidate captions with open('captions.txt', 'r') as file: self.candidate_captions = [line.strip() for line in file.readlines()] # Pre-tokenize and pre-encode text captions with torch.no_grad(): self.text_tokens = clip.tokenize(self.candidate_captions).to(self.device) self.text_embeddings = self.model.encode_text(self.text_tokens) def decode_request(self, request): image_bytes = bytes.fromhex(request["image_bytes"]) image = Image.open(BytesIO(image_bytes)) return self.preprocess(image).unsqueeze(0).to(self.device) def predict(self, image_tensor): with torch.no_grad(): image_embedding = self.model.encode_image(image_tensor) similarities = (100.0 * image_embedding @ self.text_embeddings.T).softmax(dim=-1) max_indices = similarities.argmax(dim=-1) return [self.candidate_captions[idx] for idx in max_indices.cpu().numpy()] def encode_response(self, output): return {"description": output[0]} if __name__ == "__main__": api = CLIPAPI() server = ls.LitServer(api, accelerator="auto") server.run(port=8000) ``` -------------------------------- ### Implementing prepare_data and setup for NLP tasks Source: https://lightning.ai/docs/pytorch/stable/reference/data/datamodule Example of using prepare_data to tokenize and save data to disk, and setup to load the processed data back. ```python class LitDataModule(L.LightningDataModule): def prepare_data(self): dataset = load_Dataset(...) train_dataset = ... val_dataset = ... # tokenize # save it to disk def setup(self, stage): # load it back here dataset = load_dataset_from_disk(...) ``` -------------------------------- ### setup(trainer) Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.accelerators.Accelerator Called by the Trainer to set up the accelerator before the model starts running on the device. ```APIDOC ## setup(trainer) ### Description Called by the Trainer to set up the accelerator before the model starts running on the device. ### Method `setup` ### Parameters - **trainer** (`Trainer`) - Required - the trainer instance ### Return Type `None` ``` -------------------------------- ### Full FSDP Transformer example Source: https://lightning.ai/docs/pytorch/stable/reference/advanced/model_parallel/fsdp Complete example of training a 1B-parameter Transformer language model on WikiText2 with FSDP. Includes a LightningModule, DataLoader, Trainer setup with FSDPStrategy, and memory summary printing. Reused throughout the guide for optimizing speed and memory usage and comparing to DDP. ```python import torch import torch.nn as nn import torch.nn.functional as F from torch.utils.data import DataLoader import lightning as L from lightning.pytorch.strategies import FSDPStrategy from lightning.pytorch.demos import Transformer, WikiText2 class LanguageModel(L.LightningModule): def __init__(self, vocab_size): super().__init__() self.model = Transformer( # 1B parameters vocab_size=vocab_size, nlayers=32, nhid=4096, ninp=1024, nhead=64, ) def training_step(self, batch): input, target = batch output = self.model(input, target) loss = F.nll_loss(output, target.view(-1)) self.log("train_loss", loss, prog_bar=True) return loss def configure_optimizers(self): return torch.optim.Adam(self.parameters(), lr=0.1) L.seed_everything(42) # Data dataset = WikiText2() train_dataloader = DataLoader(dataset) # Model model = LanguageModel(vocab_size=dataset.vocab_size) # Trainer trainer = L.Trainer(accelerator="cuda", devices=2, strategy=FSDPStrategy()) trainer.fit(model, train_dataloader) trainer.print(torch.cuda.memory_summary()) ``` -------------------------------- ### Submit a Multi-Machine Job with Lightning SDK Source: https://lightning.ai/docs/platform/train/multi-node-training/sdk-commands Full example of submitting a multi-machine job. Includes installation and authentication setup, initializing a Studio, running a job with MMT.run, monitoring status, and stopping or deleting the job. ```python # Install the Lightning SDK # pip install lightning-sdk # Export authentication variables # export LIGHTNING_USER_ID=your-user-id # export LIGHTNING_API_KEY=your-api-key from lightning_sdk import Studio, Machine, MMT # Step 1: Initialize a Studio studio = Studio(name="batch-processing", teamspace="my-teamspace", user="my-user") studio.start() # Step 3: Run a Multi-machine job job = MMT.run(command="python process_data.py", name="data-job", machine=Machine.CPU, studio=studio, num_machines=2) # Step 4: Monitor Job Status print(job.status) # Running, Completed, Failed # Step 5: Stop or Delete a Job # job.stop() # Gracefully stop a running job # job.delete() # Cancel and remove the job # Step 6: Shut Down the Studio studio.stop() ``` -------------------------------- ### Submit a job - full example Source: https://lightning.ai/docs/platform/developers/sdk/batch Complete example showing how to initialize a Studio, run a batch job, monitor its status, stop or delete it, and shut down the Studio. ```python # Install the Lightning SDK # pip install lightning-sdk # Export authentication variables # export LIGHTNING_USER_ID=your-user-id # export LIGHTNING_API_KEY=your-api-key from lightning_sdk import Studio, Machine, Job # Step 1: Initialize a Studio studio = Studio(name="batch-processing", teamspace="my-teamspace", user="my-user") studio.start() # Step 3: Run a Batch Job job = Job.run(command="python process_data.py", name="data-job", machine=Machine.CPU, studio=studio) # Step 4: Monitor Job Status print(job.status) # Running, Completed, Failed # Step 5: Stop or Delete a Job # job.stop() # Gracefully stop a running job # job.delete() # Cancel and remove the job # Step 6: Shut Down the Studio studio.stop() ``` -------------------------------- ### LambdaCallback usage example Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.callbacks.LambdaCallback Creates a Trainer with a LambdaCallback that prints 'setup' when the setup hook is called. The example shows the required imports and the callback instantiation. ```python >>> from lightning.pytorch import Trainer >>> from lightning.pytorch.callbacks import LambdaCallback >>> trainer = Trainer(callbacks=[LambdaCallback(setup=lambda *args: print('setup'))]) ``` -------------------------------- ### setup Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.strategies.ModelParallelStrategy Sets up the accelerator, plugins and initializes the optimizers (if needed). ```APIDOC ## setup ### Description Sets up the accelerator, plugins and initializes the optimizers (if needed). ### Method `setup(trainer)` ### Parameters - **trainer** (`Trainer`) - Required - The trainer instance. ### Return Type `None` ``` -------------------------------- ### Install torchvision nightly Source: https://lightning.ai/docs/thunder/stable/basic/mlp_mnist Command to install the required nightly version of torchvision for the MNIST example. ```bash pip install --pre torchvision --index-url https://download.pytorch.org/whl/nightly/cu121 ``` -------------------------------- ### setup(_trainer_) Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.strategies.FSDPStrategy Sets up the accelerator, plugins and initializes the optimizers (if needed). ```APIDOC ## setup(_trainer_) ### Description Sets up the accelerator, plugins and initializes the optimizers (if needed). ### Method setup(_trainer_) ### Parameters - **trainer** (`Trainer`) - The trainer instance. ### Return Type `None` ``` -------------------------------- ### Clone and install the example Source: https://lightning.ai/docs/platform/build/sandbox/examples/code-agent-langchain Clone the SDK repo and install the local 'brain' dependencies. Note that torch and transformers are installed inside the sandbox at create time, not here. ```bash git clone https://github.com/Lightning-AI/sdk.git cd sdk/python/examples # Install the local "brain" dependencies (torch + transformers are installed # *in the sandbox* at create time, not here). pip install -r langchain/requirements.txt ``` -------------------------------- ### Set up model and optimizers with setup_module Source: https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.fabric.Fabric Demonstrates setting up a model independently of optimizers, which is useful for strategies like FSDP. ```python # Set up model first (useful for FSDP) model = fabric.setup_module(model) # Then create and set up optimizer optimizer = torch.optim.Adam(model.parameters()) optimizer = fabric.setup_optimizers(optimizer) ``` -------------------------------- ### setup(module, *optimizers, scheduler=None, move_to_device=True, _reapply_compile=True) Source: https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.fabric.Fabric Set up a model and its optimizers for accelerated training. Returns the wrapped module and optimizers, and optionally the scheduler. ```APIDOC ## setup(module, *optimizers, scheduler=None, move_to_device=True, _reapply_compile=True) ### Description Set up a model and its optimizers for accelerated training. ### Method setup ### Signature setup(module, *optimizers, scheduler=None, move_to_device=True, _reapply_compile=True) ### Parameters - **module** (Module) - Required - A torch.nn.Module to set up. - ***optimizers** (Optimizer) - Optional - The optimizer(s) to set up. Can be zero or more optimizers. - **scheduler** (Optional[_LRScheduler]) - Optional - An optional learning rate scheduler to set up. Must be provided after optimizers if used. - **move_to_device** (bool) - Optional - If set True (default), moves the model to the correct device. Set this to False and alternatively use to_device() manually. - **_reapply_compile** (bool) - Optional - If True (default), and the model was torch.compiled before, the corresponding torch._dynamo.OptimizedModule wrapper will be removed and reapplied with the same settings after the model was set up by the strategy (e.g., after the model was wrapped by DDP, FSDP etc.). Set it to False if compiling DDP/FSDP is causing issues. ### Return Type Any ### Returns If no optimizers are passed, returns the wrapped module. If optimizers are passed, returns a tuple containing the wrapped module and optimizers, and optionally the scheduler if provided, in the same order they were passed in. ### Note For certain strategies like FSDP, you may need to set up the model first using setup_module(), then create the optimizer, and finally set up the optimizer using setup_optimizers(). ### Example ```python # Basic usage model, optimizer = fabric.setup(model, optimizer) # With multiple optimizers and scheduler model, opt1, opt2, scheduler = fabric.setup(model, opt1, opt2, scheduler=scheduler) # Model only model = fabric.setup(model) ``` ``` -------------------------------- ### Example: Compute STOI with short_time_objective_intelligibility Source: https://lightning.ai/docs/torchmetrics/stable/audio/short_time_objective_intelligibility Shows how to compute the STOI metric using the functional interface. Requires pystoi to be installed (e.g., via pip install torchmetrics[audio] or pip install pystoi). The example uses random tensors of shape (8000,) and a sampling frequency of 8000 Hz, returning a tensor value. ```python >>> from torch import randn >>> from torchmetrics.functional.audio.stoi import short_time_objective_intelligibility >>> preds = randn(8000) >>> target = randn(8000) >>> short_time_objective_intelligibility(preds, target, 8000).float() tensor(-0.084...) ``` -------------------------------- ### Set up optimizers with setup_optimizers Source: https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.fabric.Fabric Demonstrates setting up single or multiple optimizers for accelerated training. ```python # Single optimizer optimizer = fabric.setup_optimizers(optimizer) # Multiple optimizers opt1, opt2 = fabric.setup_optimizers(opt1, opt2) ``` -------------------------------- ### setup Source: https://lightning.ai/docs/litserve/api-reference/litapi The setup method is called once at the start of the server's lifecycle. It is used to initialize models, databases, caches, and other necessary systems. The method accepts a device string to specify the hardware (CPU, GPU, MPS) or a list of specific GPUs. ```APIDOC ## setup ### Description Called once at the start of the server's lifecycle to initialize models, databases, caches, and other necessary systems. Centralizes all setup tasks, simplifying complex configurations like multiple models or external resources. ### Method Signature `def setup(self, device: str)` ### Parameters - **device** (str) - Required - String representing the device (CPU, GPU, MPS). Can also specify specific GPUs by passing a list [1, 2, 3]. ### Example ```python # pip install torch clip import torch, clip, multiprocessing from PIL import Image from io import BytesIO import litserve as ls class CLIPAPI(ls.LitAPI): def setup(self, device): # load the model on the device automatically (CPU, GPU etc...) self.model, self.preprocess = clip.load("ViT-B/32", device=device) # Load candidate captions with open('captions.txt', 'r') as file: self.candidate_captions = [line.strip() for line in file.readlines()] # Pre-tokenize and pre-encode text captions with torch.no_grad(): self.text_tokens = clip.tokenize(self.candidate_captions).to(self.device) self.text_embeddings = self.model.encode_text(self.text_tokens) def decode_request(self, request): image_bytes = bytes.fromhex(request["image_bytes"]) image = Image.open(BytesIO(image_bytes)) return self.preprocess(image).unsqueeze(0).to(self.device) def predict(self, image_tensor): with torch.no_grad(): image_embedding = self.model.encode_image(image_tensor) similarities = (100.0 * image_embedding @ self.text_embeddings.T).softmax(dim=-1) max_indices = similarities.argmax(dim=-1) return [self.candidate_captions[idx] for idx in max_indices.cpu().numpy()] def encode_response(self, output): return {"description": output[0]} if __name__ == "__main__": api = CLIPAPI() server = ls.LitServer(api, accelerator="auto") server.run(port=8000) ``` ``` -------------------------------- ### Install the lightning-sdk Source: https://lightning.ai/docs/platform/build/ai-studio Install the Lightning SDK with pip. This is required to automate Studio operations as shown in the following pipeline example. ```bash pip install lightning-sdk ``` -------------------------------- ### Initialize and manage a Studio Source: https://lightning.ai/docs/platform/developers/sdk/studio Demonstrates basic Studio initialization, starting, stopping, and duplication. ```python from lightning_sdk import Studio studio = Studio(name='my-studio-name', teamspace='my-teamspace', org='my-org') studio.start() studio.stop() duplicated_studio = studio.duplicate() ``` -------------------------------- ### Quickstart Sandbox Execution Source: https://lightning.ai/docs/platform/developers/sdk/sandbox/js-sandbox-sdk Demonstrates creating a sandbox, running a simple shell command, and cleaning up the environment. Credentials are automatically read from environment variables or can be configured via Sandbox.configure. ```ts import { Sandbox } from "@lightningai/sdk"; const sandbox = await Sandbox.create({ name: "quickstart", instanceType: "cpu-1" }); const result = await sandbox.runCommand("echo", ["Hello, World!"]); console.log(result.output.trimEnd()); // Hello, World! console.log(result.exitCode); // 0 await sandbox.delete(); ``` -------------------------------- ### Install toolchains at runtime Source: https://lightning.ai/docs/platform/developers/sdk/sandbox/python-sandbox-sdk Example of installing additional software like GCC inside a sandbox using run_command, which requires network egress. ```python sandbox = Sandbox.create(name="build", instance_type="cpu-1", runtime="python313") # Install a toolchain (needs egress — see Network policy) then use it: sandbox.run_command("sh", ["-c", "apt-get update && apt-get install -y gcc && gcc --version"]) ``` -------------------------------- ### Setup models and optimizers with fabric.setup Source: https://lightning.ai/docs/fabric/stable/full-api-reference/api/fabric_methods The setup method prepares models, optimizers, and optional schedulers for accelerated training. It automatically moves objects to the correct device and handles precision casting. ```python model = nn.Linear(32, 64) optimizer = torch.optim.SGD(model.parameters(), lr=0.001) scheduler = torch.optim.lr_scheduler.LinearLR(optimizer, start_factor=1.0, end_factor=0.3, total_iters=10) # Set up model and optimizer for accelerated training model, optimizer = fabric.setup(model, optimizer) # If you don't want Fabric to set the device model, optimizer = fabric.setup(model, optimizer, move_to_device=False) # If you want to additionally register a learning rate scheduler with compatible strategies such as DeepSpeed model, optimizer, scheduler = fabric.setup(model, optimizer, scheduler) ``` -------------------------------- ### Setup multi-language sandbox with run_and_wait Source: https://lightning.ai/docs/platform/build/sandbox/examples/safe-code-execution Creates a python313 sandbox and installs Node.js, Ruby, and PHP using the run_and_wait helper to handle the installation process. ```python def setup_multi_language_sandbox() -> Sandbox: """Create a sandbox and install Node.js, Ruby and PHP alongside Python.""" print("🏖️ Creating sandbox (python313 base)...") sandbox = Sandbox.create( name="example-safe-code-execution", instance_type="cpu-1", runtime="python313", timeout=30 * MINUTES_MS, ) print(f"Sandbox ID: {sandbox.sandbox_id}") print("🏖️ Installing nodejs, ruby and php (this can take a minute)...") install_script = ( "set -eux; " "export DEBIAN_FRONTEND=noninteractive; " "apt-get update; " "apt-get install -y nodejs ruby php-cli" ) exit_code, output = run_and_wait(sandbox, install_script, timeout=600) if exit_code != 0: raise RuntimeError( f"toolchain install failed (exit {exit_code}):\n{output}" ) return sandbox ``` -------------------------------- ### DDPStrategy.setup Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.strategies.DDPStrategy Sets up the accelerator, plugins and initializes the optimizers (if needed). ```APIDOC ## DDPStrategy.setup ### Description Sets up the accelerator, plugins and initializes the optimizers (if needed). ### Method setup ### Parameters - **trainer** (`Trainer`) - The trainer instance. ### Return Type `None` ``` -------------------------------- ### LitServer with accelerator='cpu' Source: https://lightning.ai/docs/litserve/api-reference/litserver Example of starting a LitServer with accelerator set to 'cpu'. ```python import litserve as ls api = ls.examples.SimpleLitAPI() server = ls.LitServer(api, accelerator='cpu') server.run(port=8000) ``` -------------------------------- ### setup_environment() Source: https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.strategies.DDPStrategy Setup any processes or distributed connections. ```APIDOC ## setup_environment() ### Description Setup any processes or distributed connections. This must be called by the framework at the beginning of every process, before any distributed communication takes place. ### Method `setup_environment()` ### Return Type `None` ``` -------------------------------- ### LitServer with devices=4 Source: https://lightning.ai/docs/litserve/api-reference/litserver Example of starting a LitServer with 4 GPUs (accelerator='cuda', devices=4). ```python import litserve as ls api = ls.examples.SimpleLitAPI() server = ls.LitServer(api, accelerator='cuda', devices=4) server.run(port=8000) ``` -------------------------------- ### Compute VMAF score and features with video_multi_method_assessment_fusion Source: https://lightning.ai/docs/torchmetrics/stable/video/vmaf Example usage of the functional VMAF metric. Requires vmaf-torch to be installed (e.g., via pip install torchmetrics[video]). The example creates random video tensors, computes the VMAF score, and then with features=True retrieves a dictionary of elementary features like 'vmaf' and 'integer_adm2'. ```python >>> import torch >>> from torchmetrics.functional.video import video_multi_method_assessment_fusion >>> # 2 videos, 3 channels, 10 frames, 32x32 resolution >>> preds = torch.rand(2, 3, 10, 32, 32, generator=torch.manual_seed(42)) >>> target = torch.rand(2, 3, 10, 32, 32, generator=torch.manual_seed(43)) >>> vmaf_score = video_multi_method_assessment_fusion(preds, target) >>> torch.round(vmaf_score, decimals=2) tensor([[ 9.9900, 15.9000, 14.2600, 16.6100, 15.9100, 14.3000, 13.5800, 13.4900, 15.4700, 20.2800], [ 6.2500, 11.3000, 17.3000, 11.4600, 19.0600, 14.9300, 14.0500, 14.4100, 12.4700, 14.8200]]) >>> vmaf_dict = video_multi_method_assessment_fusion(preds, target, features=True) >>> # show a couple of features, more features are available >>> vmaf_dict['vmaf'].round(decimals=2) tensor([[ 9.9900, 15.9000, 14.2600, 16.6100, 15.9100, 14.3000, 13.5800, 13.4900, 15.4700, 20.2800], [ 6.2500, 11.3000, 17.3000, 11.4600, 19.0600, 14.9300, 14.0500, 14.4100, 12.4700, 14.8200]]) >>> vmaf_dict['integer_adm2'].round(decimals=2) tensor([[0.4500, 0.4500, 0.3600, 0.4700, 0.4300, 0.3600, 0.3900, 0.4100, 0.3700, 0.4700], [0.4200, 0.3900, 0.4400, 0.3700, 0.4500, 0.3900, 0.3800, 0.4800, 0.3900, 0.3900]]) ``` -------------------------------- ### Full example: submit a multi-machine job Source: https://lightning.ai/docs/platform/developers/sdk/multi-machine Complete example showing how to initialize a Studio, run a multi-machine job, monitor its status, and stop or delete it. ```python # Install the Lightning SDK # pip install lightning-sdk # Export authentication variables # export LIGHTNING_USER_ID=your-user-id # export LIGHTNING_API_KEY=your-api-key from lightning_sdk import Studio, Machine, MMT # Step 1: Initialize a Studio studio = Studio(name="batch-processing", teamspace="my-teamspace", user="my-user") studio.start() # Step 3: Run a Multi-machine job job = MMT.run(command="python process_data.py", name="data-job", machine=Machine.CPU, studio=studio, num_machines=2) # Step 4: Monitor Job Status print(job.status) # Running, Completed, Failed # Step 5: Stop or Delete a Job # job.stop() # Gracefully stop a running job # job.delete() # Cancel and remove the job # Step 6: Shut Down the Studio studio.stop() ``` -------------------------------- ### BinaryFBetaScore example with int preds Source: https://lightning.ai/docs/torchmetrics/stable/classification/fbeta_score Shows how to use BinaryFBetaScore with beta=2.0. The example uses integer tensors for preds and target, creates the metric, and calls it to get the F-beta score. ```python >>> from torch import tensor >>> from torchmetrics.classification import BinaryFBetaScore >>> target = tensor([0, 1, 0, 1, 0, 1]) >>> preds = tensor([0, 0, 1, 1, 0, 1]) >>> metric = BinaryFBetaScore(beta=2.0) >>> metric(preds, target) tensor(0.6667) ``` -------------------------------- ### Implementing setup in MNISTDataModule Source: https://lightning.ai/docs/pytorch/stable/reference/data/datamodule Shows how to use the setup method to perform data splits and dataset creation. This method is called from every process. ```python import lightning as L class MNISTDataModule(L.LightningDataModule): def setup(self, stage: str): # Assign Train/val split(s) for use in Dataloaders if stage == "fit": mnist_full = MNIST(self.data_dir, train=True, download=True, transform=self.transform) self.mnist_train, self.mnist_val = random_split( mnist_full, [55000, 5000], generator=torch.Generator().manual_seed(42) ) # Assign Test split(s) for use in Dataloaders if stage == "test": self.mnist_test = MNIST(self.data_dir, train=False, download=True, transform=self.transform) ``` -------------------------------- ### LightningCLI.setup_parser Source: https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.cli.LightningCLI Initialize and setup the parser, subcommands, and arguments. Returns None. ```APIDOC ## LightningCLI.setup_parser ### Description Initialize and setup the parser, subcommands, and arguments. ### Method setup_parser ### Signature setup_parser(add_subcommands, main_kwargs, subparser_kwargs) ### Parameters - **add_subcommands** - Required - Whether to add subcommands. - **main_kwargs** - Required - Main parser keyword arguments. - **subparser_kwargs** - Required - Subparser keyword arguments. ### Returns - **None** - No return value. ``` -------------------------------- ### Push from Linux Source: https://lightning.ai/docs/platform/inference/deploy-containers/lightning-container-registry Install and start Docker on Linux, then upload a container to Lightning registry. ```bash # [Install Docker] Setup Docker # install curl -fsSL https://get.docker.com | bash # [Start Docker] Make sure Docker is running # start sudo systemctl restart docker # [Upload] Upload the container (same vLLM container as above) to Lightning registry # upload lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest ``` -------------------------------- ### Start a Studio Source: https://lightning.ai/docs/platform/build/ai-studio/cli Starts an existing Studio. --name and --teamspace are optional and will be prompted if not provided as arguments or in the config file. ```bash lightning studio start --name --teamspace / --machine T4 ``` -------------------------------- ### Run LitServe server with basic, production, and development configurations Source: https://lightning.ai/docs/litserve/api-reference/litserver Shows how to start a LitServe server with the default configuration, a production setup with multiple API servers and warning-level logging, and a development setup with debug logging, pretty logs, and client file generation. ```python import litserve as ls api = ls.examples.SimpleLitAPI() server = ls.LitServer(api) # Basic server.run() # Production server.run( port=8080, num_api_servers=4, log_level="warning" ) # Development server.run( log_level="debug", pretty_logs=True, generate_client_file=True ) ``` -------------------------------- ### Push from Mac Source: https://lightning.ai/docs/platform/inference/deploy-containers/lightning-container-registry Install and start Docker on macOS, then upload a container to Lightning registry. ```bash # [Install] Install Docker if it's not already installed # install brew install docker [Run Docker] Make sure Docker is running # start open -a Docker # [Upload] Upload the vLLM container built earlier to the organization or username and teamspace # upload lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest ``` -------------------------------- ### Configuring fixed optimizer and scheduler Source: https://lightning.ai/docs/pytorch/stable/reference/cli/lightning_cli_advanced_3 Example configuration structure for a fixed optimizer and scheduler setup. ```yaml optimizer: lr: 0.01 lr_scheduler: gamma: 0.2 model: ... trainer: ... ```