TinyTorch

https://mlsysbook.ai/tinytorch
Admin
TinyTorch is an educational ML framework that teaches you how PyTorch, TensorFlow, and JAX work by...
Tokens:136,101
Snippets:1,897
Trust Score:9.1
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install and Setup TinyTorch

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Installs TinyTorch, sets up the virtual environment, and performs initial profile configuration. Ensure you are in the correct directory after installation.

```bash
curl -sSL mlsysbook.ai/tinytorch/install.sh | bash
cd tinytorch
source .venv/bin/activate
tito setup
tito system health
```

--------------------------------

### Setup TinyTorch Environment

Source: https://mlsysbook.ai/tinytorch/community/about.html

Use this command for initial setup. It installs dependencies, creates your profile, and initializes your workspace. You can choose to log in to the TinyTorch community during this process.

```bash
git clone https://github.com/harvard-edge/cs249r_book.git
cd TinyTorch
tito setup
```

--------------------------------

### Initial Setup

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Run this command for the first-time setup. It is safe to re-run if needed.

```bash
tito setup
```

--------------------------------

### Typical Session Flow: Start, Work, Export, Test, Track

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

A step-by-step guide for a typical TinyTorch session, from starting the environment to tracking progress after completing work.

```bash
cd tinytorch
source .venv/bin/activate
tito system health         # Verify environment

tito module start 03       # Or: tito module resume 03
# Edit in Jupyter Lab...

tito module complete 03

tito milestone list        # Check if ready
tito milestone run 03      # Run with YOUR code

tito module status         # See module progress
tito milestone status      # See milestone achievements
```

--------------------------------

### Install TinyTorch on macOS/Linux

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Installs TinyTorch, activates its virtual environment, and verifies the setup. Run this from your project folder.

```bash
# Install TinyTorch (run from a project folder like ~/projects)
curl -sSL mlsysbook.ai/tinytorch/install.sh | bash

# Activate and verify
cd tinytorch
source .venv/bin/activate
tito setup
```

--------------------------------

### Install TinyTorch on Windows using Git Bash

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Installs TinyTorch using Git Bash on Windows, activates its virtual environment, and verifies the setup. Run this from your project folder.

```bash
# In Git Bash (run from a project folder like ~/projects)
curl -sSL mlsysbook.ai/tinytorch/install.sh | bash

# Activate and verify
cd tinytorch
source .venv/Scripts/activate
tito setup
```

--------------------------------

### Start Architecture Tier Modules

Source: https://mlsysbook.ai/tinytorch/tiers/architecture.html

Use the tito CLI to start building the Architecture tier, either with the vision track (convolutions) or the language track (tokenization).

```bash
# Start the Architecture tier with vision
tito module start 09_convolutions

# Or jump to language models
tito module start 10_tokenization
```

--------------------------------

### Milestone Output Example

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Example output from running a milestone, showing the process and achievement.

```text
Checking prerequisites for Milestone 01...
All required modules completed!

Testing YOUR implementations...
  * Tensor import successful
  * Activations import successful
  * Layers import successful
YOUR TinyTorch is ready!

+----------------------- Milestone 01 (1958) -----------------------+
|  Milestone 01: Perceptron (1958)                                  |
|  Frank Rosenblatt's First Neural Network                          |
|                                                                   |
|  Running: milestones/01_1958_perceptron/01_rosenblatt_forward.py  |
|  All code uses YOUR TinyTorch implementations!                    |
+-------------------------------------------------------------------+

Starting Milestone 01...

Assembling perceptron with YOUR TinyTorch modules...
   * Linear layer: 2 -> 1 (YOUR Module 03!)
   * Activation: Sigmoid (YOUR Module 02!)

+-------------------- Achievement Unlocked --------------------+
|  MILESTONE ACHIEVED!                                         |
|                                                              |
|  You completed Milestone 01: Perceptron (1958)               |
|  Frank Rosenblatt's First Neural Network                     |
|                                                              |
|  What makes this special:
|  - Every tensor operation: YOUR Tensor class                 |
|  - Every layer: YOUR Linear implementation                   |
|  - Every activation: YOUR Sigmoid function                   |
+--------------------------------------------------------------+
```

--------------------------------

### Start a TinyTorch Module

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Starts a specific module, opening its notebook and tracking progress.

```bash
tito module start 01
```

--------------------------------

### Training Loop with Optimizer Example

Source: https://mlsysbook.ai/tinytorch/modules/07_optimizers.html

An example demonstrating the typical training loop pattern using an optimizer.

```APIDOC
```python
# Training loop with optimizer
optimizer = Adam(model.parameters(), lr=0.001)
loss.backward()  # Compute gradients (Module 06)
optimizer.step()  # Update parameters using gradients
optimizer.zero_grad()  # Clear gradients for next iteration
```
```

--------------------------------

### Benchmark Commands

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Commands to validate setup and measure performance using baseline and capstone benchmarks.

```bash
tito benchmark baseline
tito benchmark capstone
```

--------------------------------

### Troubleshooting: Re-activating Environment and Setup

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Steps to re-activate the TinyTorch environment and re-run setup if `tito system health` shows errors.

```bash
# Re-activate environment
cd tinytorch
source .venv/bin/activate

# Re-run setup if needed
tito setup

# Verify
tito system health
```

--------------------------------

### Validate Setup and Run Benchmarks with TinyTorch CLI

Source: https://mlsysbook.ai/tinytorch/community.html

Validate your setup with a baseline benchmark and run full capstone benchmarks after Module 20. Results are saved locally and can be optionally submitted to the community.

```bash
# Quick setup validation (after initial setup)
tito benchmark baseline

# Full capstone benchmarks (after Module 20)
tito benchmark capstone

# Submit results to community (optional)
# Prompts automatically after benchmarks complete
```

--------------------------------

### Typical Development Session Setup

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Commands to set up your development environment for a typical TinyTorch module development session.

```bash
cd tinytorch
source .venv/bin/activate
tito system health # Verify environment

```

--------------------------------

### Install Tito in Editable Mode

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

Resolve 'Permission denied' errors during 'tito setup' on macOS by installing the project in editable mode using 'pip install -e .'. Then, re-run 'tito setup'.

```bash
pip install -e .
tito setup
```

--------------------------------

### Install TinyTorch Dependencies

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

Ensure all necessary dependencies like NumPy and Jupyter are installed in your virtual environment. Activate the environment first, then use pip to install the required packages.

```bash
# Activate environment
source .venv/bin/activate

# Install dependencies
pip install numpy jupyter jupyterlab jupytext rich

# Verify
python -c "import numpy; print(numpy.__version__)"
```

--------------------------------

### Start Module 01: Tensor

Source: https://mlsysbook.ai/tinytorch/tiers/foundation.html

Use this command to begin working on the Tensor module. Follow the daily workflow for implementation, testing, and exporting.

```bash
# Start with Module 01: Tensor
tito module start 01_tensor

# Follow the daily workflow
# 1. Read the ABOUT guide
# 2. Implement in *_dev.py
# 3. Test with tito module test
# 4. Export to *_sol.py
```

--------------------------------

### Start Optimization Tier with Tito

Source: https://mlsysbook.ai/tinytorch/tiers/optimization.html

Use the 'tito' command-line tool to begin the profiling module, initiating the optimization learning cycle.

```bash
# Start the Optimization tier
tito module start 14_profiling

# Follow the measure → optimize → validate cycle
```

--------------------------------

### Start Working on a Module

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Use this command to begin working on a specific module. Replace N with the module number.

```bash
tito module start <N>
```

--------------------------------

### Install Rich Library

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

Install the 'rich' library to enable colorful output in your terminal. This is necessary for TITO's rich formatting.

```bash
pip install rich
```

--------------------------------

### Student Workflow: Start and Manage Modules

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Commands for students to begin new modules, continue existing ones, mark them as complete, and check their progress.

```bash
# Start or continue a module
tito module start 01      # First time
tito module resume 01     # Continue later

# Export when complete
tito module complete 01

# Check progress
tito module status
```

--------------------------------

### Install Jupyter and Dependencies

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

Ensure Jupyter, JupyterLab, and Jupytext are installed. This is a prerequisite for running notebooks and managing them with TinyTorch.

```bash
pip install jupyter jupyterlab jupytext
```

--------------------------------

### Training Computational Complexity Example

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Provides a concrete numerical example for training cost calculation, showing operations per sample, per batch, per epoch, and total operations.

```text
d² × L              = 512² × 2          = 524,288 ops per sample
Batch operations    = 524,288 × 32      = 16.8M ops per batch
Iterations / epoch  = 10,000 / 32       = 313
Total iterations    = 313 × 100         = 31,300
Total operations    = 31,300 × 16.8M    ≈ 525 billion ops
```

--------------------------------

### Start a TinyTorch Module

Source: https://mlsysbook.ai/tinytorch/community/about.html

Begin working on a specific module. This command initializes the workspace for the chosen module.

```bash
tito module start 01_tensor
```

--------------------------------

### Verify Environment Health

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Run this command first to ensure your development environment is correctly set up and all dependencies are installed.

```bash
tito system health
```

--------------------------------

### TinyTorch CPU Acceleration Example

Source: https://mlsysbook.ai/tinytorch/modules/17_acceleration.html

Demonstrates CPU-based acceleration using TinyTorch's `vectorized_matmul` and `fused_gelu` for matrix multiplication and activation.

```python
from tinytorch.perf.acceleration import vectorized_matmul, fused_gelu

# CPU-based acceleration
x = Tensor(np.random.randn(128, 512))
w = Tensor(np.random.randn(512, 256))

# Vectorized matrix multiplication
h = vectorized_matmul(x, w)

# Fused activation
output = fused_gelu(h)
```

--------------------------------

### System Information and Updates

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Commands for checking system resources, starting Jupyter, and updating TinyTorch.

```bash
tito system info
```

```bash
tito system jupyter
```

```bash
tito system update
```

```bash
tito system reset
```

--------------------------------

### Manage Module Completion for Milestones

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

Track module status with 'tito module status', start missing modules with 'tito module start <module_number>', and complete them with 'tito module complete <module_number>' to meet milestone requirements.

```bash
tito module status # See what's completed
tito module start 08 # Complete missing modules
# ... implement and export
tito module complete 08
```

--------------------------------

### Trainer Usage Example

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Demonstrates how to use the Trainer class with a model, optimizer, loss function, scheduler, and gradient clipping.

```APIDOC
## Trainer Usage Example

### Description
This example shows the typical usage pattern for the `Trainer` class, integrating various components for a complete training pipeline.

### Code Example
```python
# Complete training pipeline (modules 01-07 working together)
trainer = Trainer(model, optimizer, loss_fn, scheduler, grad_clip_norm=1.0)
for epoch in range(100):
    train_loss = trainer.train_epoch(train_data)
    eval_loss, accuracy = trainer.evaluate(val_data)
    trainer.save_checkpoint(f"checkpoint_{epoch}.pkl")
```
```

--------------------------------

### Start a New Module

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Initiates a new module development session, opening Jupyter Lab for the specified module. Creates a backup of existing work.

```bash
tito module start 01
```

```bash
tito module start 05 # Start Module 05 (DataLoader)
```

--------------------------------

### Example DataLoader Usage

Source: https://mlsysbook.ai/tinytorch/modules/05_dataloader.html

Demonstrates how to use the DataLoader to transform individual samples into training-ready batches. This pattern is reusable across different models and datasets.

```python
# Transform individual samples into training-ready batches
dataset = TensorDataset(features, labels)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

for batch_features, batch_labels in loader:
    # batch_features: (32, feature_dim) - ready for model.forward()
    predictions = model(batch_features)
```

--------------------------------

### PyTorch GPU Acceleration Example

Source: https://mlsysbook.ai/tinytorch/modules/17_acceleration.html

Shows GPU acceleration in PyTorch using similar concepts to TinyTorch, leveraging CUDA for computations.

```python
import torch

# GPU acceleration with same concepts
x = torch.randn(128, 512, device='cuda')
w = torch.randn(512, 256, device='cuda')

# Vectorized (cuBLAS on GPU)
h = x @ w
```

--------------------------------

### Multi-Layer Parameter Aggregation Example

Source: https://mlsysbook.ai/tinytorch/modules/03_layers.html

Demonstrates how to aggregate parameters from multiple Linear layers for use with an optimizer. This pattern is extended by containers like Sequential.

```python
layer1 = Linear(784, 256)
layer2 = Linear(256, 10)

all_params = layer1.parameters() + layer2.parameters()
# Pass all_params to optimizer.step() during training
```

--------------------------------

### Schema Validation Example (With Schema)

Source: https://mlsysbook.ai/tinytorch/modules/20_capstone.html

Illustrates how a strict schema ensures consistent data formats, enabling automated validation, aggregation, and comparison of submissions.

```text
# All submissions use same format (aggregation works)
{"accuracy": 0.92}  # Always float in [0.0, 1.0]
```

--------------------------------

### Student/Learner Workflow

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Commands for students learning ML systems by building from scratch.

```bash
tito module start 01
```

```bash
tito module complete 01
```

```bash
tito milestone run 03
```

```bash
tito module status
```

--------------------------------

### Install and Run TinyTorch on Windows using Git Bash

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

TinyTorch commands should be run within Git Bash on Windows. This script installs TinyTorch, activates its virtual environment, and then runs 'tito setup'.

```bash
# Open Git Bash from Start Menu
# Navigate to your project folder
cd ~/

# Install TinyTorch
curl -sSL mlsysbook.ai/tinytorch/install.sh | bash

# Activate (note the Scripts path on Windows)
cd tinytorch
source .venv/Scripts/activate

# Run TinyTorch
tito setup
```

--------------------------------

### Handle Missing Prerequisites Error

Source: https://mlsysbook.ai/tinytorch/tito/milestones.html

Example of an error message displayed when required modules for a milestone are missing. It lists the missing modules and provides commands to start them.

```bash
 Missing Required Modules

Milestone 03 requires modules: 01, 02, 03, 04, 05, 06, 07, 08
Missing: 06, 07, 08

Complete the missing modules first:
 tito module start 06
 tito module start 07
 tito module start 08
```

--------------------------------

### Complete Training Pipeline Example

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Demonstrates how to use the `Trainer` class with a model, optimizer, loss function, scheduler, and gradient clipping to run a full training pipeline.

```python
# Complete training pipeline (modules 01-07 working together)
trainer = Trainer(model, optimizer, loss_fn, scheduler, grad_clip_norm=1.0)
for epoch in range(100):
    train_loss = trainer.train_epoch(train_data)
    eval_loss, accuracy = trainer.evaluate(val_data)
    trainer.save_checkpoint(f"checkpoint_{epoch}.pkl")
```

--------------------------------

### Example Sequential Model Instantiation

Source: https://mlsysbook.ai/tinytorch/modules/03_layers.html

Demonstrates how to instantiate a multi-layer neural network using the Sequential container, chaining Linear layers, ReLU activations, and Dropout layers.

```python
model = Sequential(
    Linear(784, 256), ReLU(), Dropout(0.5),
    Linear(256, 128), ReLU(), Dropout(0.3),
    Linear(128, 10)
)
```

--------------------------------

### PyTorch Lightning Trainer Setup and Training

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Sets up a PyTorch Lightning trainer with a model, optimizer, and learning rate scheduler. Uses the `trainer.fit()` method to abstract the training loop, checkpointing, and callbacks.

```python
import torch
from torch.optim.lr_scheduler import CosineAnnealingLR
from pytorch_lightning import Trainer

# Setup (nearly identical!)
model = MyModel()
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
scheduler = CosineAnnealingLR(optimizer, T_max=100, eta_min=0.01)
trainer = Trainer(max_epochs=100, gradient_clip_val=1.0)

# Training (abstracted by Lightning)
trainer.fit(model, train_dataloader, val_dataloader)
# Lightning handles the loop, checkpointing, and callbacks automatically
```

--------------------------------

### Encode Method Example

Source: https://mlsysbook.ai/tinytorch/modules/10_tokenization.html

Example implementation of the `encode` method for a character-level tokenizer, demonstrating how text is converted to character IDs.

```APIDOC
## encode

### Description
Encode text to list of character IDs.

### Method Signature
`encode(self, text: str) -> List[int]`

### Example Implementation
```python
def encode(self, text: str) -> List[int]:
    """Encode text to list of character IDs."""
    tokens = []
    for char in text:
        tokens.append(self.char_to_id.get(char, self.unk_id))
    return tokens
```
```

--------------------------------

### Professional ML Workflow Example

Source: https://mlsysbook.ai/tinytorch/modules/20_capstone.html

This snippet illustrates a typical professional machine learning workflow using the TinyTorch framework. It shows how to initialize a benchmark report, run model benchmarks, generate a submission artifact with optimization details, and save the results.

```python
# Professional ML workflow
report = BenchmarkReport(model_name="my_model")
report.benchmark_model(model, X_test, y_test, num_runs=100)

submission = generate_submission(
    baseline_report=baseline_report,
    optimized_report=optimized_report,
    techniques_applied=["quantization", "pruning"]
)
save_submission(submission, "results.json")
```

--------------------------------

### Systematic Optimization Workflow Example

Source: https://mlsysbook.ai/tinytorch/milestones/06_mlperf.html

Illustrates the 'measure, optimize, validate, repeat' loop for ML model optimization, contrasting it with a 'heroic' approach. This highlights how systematic profiling and targeted optimizations lead to significant gains.

```text
1. MEASURE:   Profile shows 70% of time is in attention,
              80% of memory is Linear layers
2. OPTIMIZE:  Add KV-cache (targets the 70%),
              quantize Linear layers (targets the 80%)
3. VALIDATE:  Accuracy drops 1.5% (acceptable),
              8x faster (huge win)
4. REPEAT:    Profile again, find next bottleneck
```

--------------------------------

### Troubleshoot Failed Update

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

If an update fails or is interrupted, check your environment, try a manual install, and verify the installation. Re-export modules if necessary.

```bash
# Step 1: Check environment
source .venv/bin/activate
tito system health
```

```bash
# Step 2: Try manual install
curl -fsSL mlsysbook.ai/tinytorch/install.sh | bash
```

```bash
# Step 3: Verify installation
tito --version
tito system health
```

```bash
# Step 4: Re-export modules if needed
# If imports fail, re-export your completed modules
tito module complete 01
tito module complete 02
```

--------------------------------

### Restart Jupyter Lab

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

If Jupyter Lab fails to start, you can try killing the existing process using 'pkill -f jupyter' and then restarting it with 'tito module start XX'.

```bash
pkill -f jupyter && tito module start XX
```

--------------------------------

### TinyTorch Trainer Setup and Training Loop

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Sets up a custom trainer with model, optimizer, loss function, scheduler, and gradient clipping. Manually iterates through epochs, performing training and evaluation, and saving checkpoints.

```python
model = MyModel()
optimizer = SGD(model.parameters(), lr=0.1, momentum=0.9)
scheduler = CosineSchedule(max_lr=0.1, min_lr=0.01, total_epochs=100)
trainer = Trainer(model, optimizer, MSELoss(), scheduler, grad_clip_norm=1.0)

# Training loop
for epoch in range(100):
    train_loss = trainer.train_epoch(train_data)
    eval_loss, acc = trainer.evaluate(val_data)

    if epoch % 10 == 0:
        trainer.save_checkpoint(f"ckpt_{epoch}.pkl")
```

--------------------------------

### Example Inline Test for Tensor Creation

Source: https://mlsysbook.ai/tinytorch/tito/testing.html

An example of an inline test embedded within a module's source file, using nbgrader format. These tests validate module implementations and are typically student-facing.

```python
# %% nbgrader={"grade": true, "grade_id": "tensor_creation", "points": 5}
# Test tensor creation
t = Tensor([1, 2, 3])
assert t.shape == (3,), "Shape should be (3,)"
assert t.data[0] == 1, "First element should be 1"
```

--------------------------------

### Reset and Start a Specific Module in TinyTorch

Source: https://mlsysbook.ai/tinytorch/tito/data.html

Use these commands to reset a specific module to its initial state and then start working on it again. This operation only affects the progress tracking in the .tito directory and does not touch your source code.

```bash
# Reset module 03 to start fresh
tito module reset 03

# Start working on it again
tito module start 03
```

--------------------------------

### Instructor Workflow

Source: https://mlsysbook.ai/tinytorch/tito/overview.html

Commands for instructors teaching ML systems engineering, focusing on assignment management.

```bash
tito nbgrader init
```

```bash
tito nbgrader generate 01
```

```bash
tito nbgrader release 01
```

```bash
tito nbgrader collect 01
```

```bash
tito nbgrader autograde 01
```

```bash
tito nbgrader feedback 01
```

```bash
tito nbgrader status
```

```bash
tito nbgrader analytics 01
```

```bash
tito nbgrader report
```

--------------------------------

### TinyTorch Latency Benchmark Setup and Run

Source: https://mlsysbook.ai/tinytorch/modules/19_benchmarking.html

Set up and run a latency benchmark using the TinyTorch Benchmark class. This includes configuring warmup and measurement runs, and specifying input shapes.

```python
from tinytorch.perf.benchmarking import Benchmark

# Setup models and benchmark
benchmark = Benchmark(
    models=[baseline_model, optimized_model],
    warmup_runs=5,
    measurement_runs=10
)

# Run latency benchmark
results = benchmark.run_latency_benchmark(input_shape=(1, 28, 28))
```

--------------------------------

### Check TinyTorch Version

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Displays the currently installed version of TinyTorch.

```bash
tito --version
```

--------------------------------

### Display Help

Source: https://mlsysbook.ai/tinytorch/getting-started.html

Access the full command reference for TinyTorch.

```bash
tito --help
```

--------------------------------

### Implement a Module Class

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Example of implementing a module class within a Jupyter notebook during development.

```python
class Linear:
 def __init__(self, in_features, out_features):
 # YOUR implementation here
 ...

```

--------------------------------

### TinyTorch Training Pipeline Initialization

Source: https://mlsysbook.ai/tinytorch/modules/08_training.html

Imports necessary components from TinyTorch to set up a training pipeline, including the Trainer, a learning rate schedule, an optimizer, and a loss function.

```python
from tinytorch import Trainer, CosineSchedule, SGD, MSELoss
```

--------------------------------

### Profiler Constructor

Source: https://mlsysbook.ai/tinytorch/modules/14_profiling.html

Initializes the profiler with necessary measurement tracking structures. No specific setup is required before calling.

```python
Profiler()
```

--------------------------------

### Environment Validation Passed Output

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Example output indicating that the environment validation checks have passed successfully.

```text
 Environment validation passed
 • Virtual environment: Active
 • Dependencies: NumPy, Jupyter, Rich installed
 • TinyTorch: Development mode

```

--------------------------------

### TinyTorch Autograd Example

Source: https://mlsysbook.ai/tinytorch/modules/06_autograd.html

Demonstrates tensor creation with gradient tracking, forward pass for computation graph construction, and backward pass for gradient computation in TinyTorch.

```python
from tinytorch import Tensor

# Create tensors with gradient tracking
x = Tensor([[1.0, 2.0]], requires_grad=True)
W = Tensor([[3.0], [4.0]], requires_grad=True)

# Forward pass builds computation graph
y = x.matmul(W)  # y = x @ W
loss = (y * y).sum()  # loss = sum(y²)

# Backward pass computes gradients
loss.backward()

# Access gradients
print(f"x.grad: {x.grad}")  # ∂loss/∂x
print(f"W.grad: {W.grad}")  # ∂loss/∂W
```

--------------------------------

### Manual TinyTorch Update

Source: https://mlsysbook.ai/tinytorch/tito/troubleshooting.html

If automatic updates fail, you can manually update TinyTorch by downloading and executing the install script.

```bash
curl -fsSL mlsysbook.ai/tinytorch/install.sh | bash
```

--------------------------------

### Module Export Process Steps

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

Illustrates the steps involved when running `tito module complete XX`, including validation, export, and tracking.

```text
Checking NBGrader metadata
Validating Python syntax
Running inline tests
```

```text
Converting src/XX_name/XX_name.py
 → modules/XX_name/XX_name.ipynb (notebook)
 → tinytorch/path/name.py (package)
Adding "DO NOT EDIT" warning
Making file read-only
```

```text
Recording completion in .tito/progress.json
Updating module status
```

```text
Module XX complete!
Your code is now part of TinyTorch!

Import with: from tinytorch import YourClass
```

--------------------------------

### Launch Jupyter Lab

Source: https://mlsysbook.ai/tinytorch/tito/modules.html

A convenience command to start Jupyter Lab from the correct directory within your TinyTorch project.

```bash
tito system jupyter
```

--------------------------------

### Run Quick Tests in CI/CD

Source: https://mlsysbook.ai/tinytorch/tito/testing.html

This command executes unit and CLI tests, providing quick feedback on every push to the repository.

```bash
tito dev test --unit --cli
```

--------------------------------

### Optional Optimization Module Imports

Source: https://mlsysbook.ai/tinytorch/modules/20_capstone.html

These imports are for optional optimization modules and are wrapped in a try-except block for graceful degradation if not installed.

```python
# These imports use try/except blocks for graceful degradation
try:
    from tinytorch.perf.profiling import Profiler, quick_profile
    from tinytorch.perf.compression import magnitude_prune, structured_prune
    from tinytorch.perf.benchmarking import Benchmark, BenchmarkResult
except ImportError:
    # Core benchmarking still works without optimization modules
    pass
```

--------------------------------

### Join and Manage TinyTorch Community via CLI

Source: https://mlsysbook.ai/tinytorch/community.html

Use the TinyTorch CLI for initial setup, logging in, viewing your profile, checking status, and accessing the community map.

```bash
# Join the community (first-time setup)
tito setup

# Or, if already set up, log in
tito community login

# View your profile
tito community profile

# Check your community status
tito community status

# Open the community map
tito community map
```
Tokens:
Raw