### Complete Ranking Example

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/ranking-engine.md

This example shows the full workflow of detecting hardware, loading models, fetching benchmarks, grouping models, and ranking them based on performance and hardware compatibility. It includes caching mechanisms for models.

```python
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models, models_to_dicts
from whichllm.models.grouper import group_models
from whichllm.models.benchmark import fetch_benchmark_scores
from whichllm.models.cache import load_cache, save_cache
from whichllm.engine.ranker import rank_models
import asyncio

async def complete_ranking():
    # Detect hardware
    hardware = detect_hardware()
    print(f"Hardware: {len(hardware.gpus)} GPUs, {hardware.cpu_cores} CPU cores")
    
    # Load models
    cached = load_cache()
    if cached:
        from whichllm.models.fetcher import dicts_to_models
        models = dicts_to_models(cached)
    else:
        models = await fetch_models()
        save_cache(models_to_dicts(models))
    
    print(f"Models: {len(models)}")
    
    # Load benchmarks
    bench_scores = await fetch_benchmark_scores()
    print(f"Benchmarks: {len(bench_scores)} models")
    
    # Group and flatten
    families = group_models(models)
    all_models = []
    for family in families:
        all_models.append(family.base_model)
        all_models.extend(family.variants)
    
    # Rank
    results = rank_models(
        all_models,
        hardware,
        context_length=4096,
        top_n=10,
        benchmark_scores=bench_scores,
        task_profile="general",
    )
    
    # Display
    print(f"\nTop recommendations:")
    for rank, result in enumerate(results, 1):
        check = "✓" if result.can_run else "✗"
        print(
            f"{check} #{rank} {result.model.id:40s} "
            f"score={result.quality_score:5.1f} "
            f"speed={result.estimated_tok_per_sec or 0:6.0f} tok/s "
            f"vram={result.vram_required_bytes / 1e9:5.1f}GB"
        )

asyncio.run(complete_ranking())

```

--------------------------------

### Clone and Set Up Development Environment

Source: https://github.com/andyyyy64/whichllm/blob/main/CONTRIBUTING.md

Clone the repository and install development dependencies using uv.

```bash
git clone https://github.com/Andyyyy64/whichllm.git
cd whichllm
uv sync --dev
```

--------------------------------

### Complete Output Pipeline Example

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/output-display.md

This example shows a complete pipeline for detecting hardware, fetching and grouping models, retrieving benchmark scores, ranking models, and then displaying the results in JSON, Markdown, or a human-readable format based on command-line arguments.

```python
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models
from whichllm.models.grouper import group_models
from whichllm.models.benchmark import fetch_benchmark_scores
from whichllm.engine.ranker import rank_models
from whichllm.output.display import (
    display_hardware,
    display_ranking,
    display_json,
    display_markdown,
)
import asyncio
import json
import sys

async def complete():
    # Setup
    hardware = detect_hardware()
    models = await fetch_models()
    families = group_models(models)
    scores = await fetch_benchmark_scores()
    
    all_models = []
    for family in families:
        all_models.append(family.base_model)
        all_models.extend(family.variants)
    
    # Rank
    results = rank_models(
        all_models,
        hardware,
        top_n=10,
        benchmark_scores=scores,
    )
    
    # Display options
    if "--json" in sys.argv:
        display_json(results, hardware)
    elif "--markdown" in sys.argv:
        display_markdown(results, hardware)
    else:
        # Human-readable (default)
        display_hardware(hardware)
        display_ranking(results, has_gpu=bool(hardware.gpus))

asyncio.run(complete())
```

--------------------------------

### Manual Execution Command Example

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/run-snippet.md

Provides an example of the `uv run` command used to execute a generated Python script, including specifying dependencies.

```bash
uv run --no-project --with llama-cpp-python --with huggingface-hub script.py
```

--------------------------------

### Install uv dependency

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/troubleshooting.md

Install the 'uv' package manager using the provided script before running whichllm commands.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Recommend Models for Current Hardware (Python)

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/INDEX.md

Use this function to detect your hardware and get a ranked list of the top 10 recommended LLM models based on benchmark scores. Ensure you have the necessary libraries installed.

```python
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models
from whichllm.models.grouper import group_models
from whichllm.models.benchmark import fetch_benchmark_scores
from whichllm.engine.ranker import rank_models
import asyncio

async def recommend():
    hardware = detect_hardware()
    models = await fetch_models()
    families = group_models(models)
    scores = await fetch_benchmark_scores()
    
    all_models = []
    for family in families:
        all_models.append(family.base_model)
        all_models.extend(family.variants)
    
    results = rank_models(all_models, hardware, top_n=10)
    
    for rank, result in enumerate(results, 1):
        print(f"#{rank} {result.model.id} (score={result.quality_score:.1f})")

asyncio.run(recommend())
```

--------------------------------

### Multi-GPU Setup

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Find the top 10 LLMs that can run on a multi-GPU setup, such as '2x RTX 4090'.

```bash
whichllm --gpu "2x RTX 4090" --top 10
```

--------------------------------

### Examples for 'whichllm run'

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/run-snippet.md

Demonstrates various ways to invoke the 'whichllm run' command with different model names and options.

```bash
whichllm run
```

```bash
whichllm run "qwen 2.5 1.5b gguf"
```

```bash
whichllm run "phi 3 mini gguf" --cpu-only
```

```bash
whichllm run "llama 3 8b gguf" --quant Q5_K_M
```

--------------------------------

### Download and Run a Model Instantly

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use the 'run' subcommand to download and immediately start chatting with a specified LLM, or let the tool auto-pick the best model for your hardware.

```bash
whichllm run "qwen 2.5 1.5b gguf"
whichllm run                       # auto-pick best for your hardware
```

--------------------------------

### Examples for 'whichllm snippet'

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/run-snippet.md

Demonstrates how to use the 'whichllm snippet' command to generate scripts for specific models and quantizations.

```bash
whichllm snippet "qwen 7b"
```

```bash
whichllm snippet "llama 3 8b gguf" --quant Q5_K_M
```

--------------------------------

### Fetch and Group Models Example

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/model-fetching.md

Demonstrates fetching all available models and then grouping them into families. This is useful for organizing models for better comparison and selection.

```python
from whichllm.models.fetcher import fetch_models
from whichllm.models.grouper import group_models
import asyncio

async def group():
    models = await fetch_models()
    families = group_models(models)
    return families

families = asyncio.run(group())

for family in families[:5]:
    print(f"{family.display_name}: {len(family.variants)} variants")
    print(f
```

--------------------------------

### Run a Model

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Execute the 'run' command to download a model, set up an environment, and start an interactive chat. An optional model name can be provided; otherwise, the best model is auto-selected.

```bash
whichllm run [model-name] [OPTIONS]
```

--------------------------------

### Multi-GPU simulation

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Simulates a multi-GPU setup with two RTX 4090s and shows the top 3 models.

```bash
whichllm --gpu "2x RTX 4090" --top 3
```

--------------------------------

### Install whichllm via Homebrew

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Install whichllm using the Homebrew package manager.

```bash
brew install andyyyy64/whichllm/whichllm

```

--------------------------------

### Install whichllm with uv tool

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Install or upgrade whichllm using the uv tool for frequent use.

```bash
uv tool install whichllm

```

```bash
uv tool upgrade whichllm  # update an existing install

```

--------------------------------

### Get JSON Output for Scripting

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/README.md

Output recommendations in JSON format for use in scripting. This example pipes the output to 'jq' to extract the model_id of the top recommendation.

```bash
whichllm --top 1 --json | jq '.models[0].model_id'
```

--------------------------------

### Install whichllm with pip

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Install whichllm using pip, the Python package installer.

```bash
pip install whichllm

```

--------------------------------

### WhichLLM Output Example

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Example of WhichLLM's output when run with specific GPU constraints, showing model ranking, size, quantization, score, and speed.

```text
$ whichllm --gpu "RTX 4090"

#1  Qwen/Qwen3.6-27B     27.8B  Q5_K_M   score 92.8    27 t/s
#2  Qwen/Qwen3-32B       32.0B  Q4_K_M   score 83.0    31 t/s
#3  Qwen/Qwen3-30B-A3B   30.0B  Q5_K_M   score 82.7   102 t/s
```

--------------------------------

### Get General WhichLLM Help

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Use the --help flag to display general usage information and a list of available commands for the WhichLLM CLI.

```bash
whichllm --help
```

--------------------------------

### Run WhichLLM for Current Hardware

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/README.md

Execute the WhichLLM CLI to get model recommendations based on your detected hardware.

```bash
whichllm
```

--------------------------------

### Get Help for WhichLLM Plan Command

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Use the --help flag with the 'plan' subcommand to view detailed usage instructions and options specific to planning operations.

```bash
whichllm plan --help
```

--------------------------------

### Get Help for WhichLLM Run Command

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Use the --help flag with the 'run' subcommand to view detailed usage instructions and options specific to running operations.

```bash
whichllm run --help
```

--------------------------------

### Compare Current Machine Against Simulated GPUs

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/cli.md

Compares your current machine's CPU, RAM, disk, and OS against simulated GPUs to assess upgrade potential. Use this to evaluate how different GPUs would perform with your existing setup.

```bash
whichllm upgrade TARGET_GPUS... [OPTIONS]
```

```bash
whichllm upgrade "RTX 4090" "RTX 5090" "H100"
```

```bash
whichllm upgrade "Apple M4 Max" --top 5
```

```bash
whichllm upgrade "RX 7900 XTX" --profile coding
```

```bash
whichllm upgrade "RTX 4090" --context-length 128k
```

--------------------------------

### Safe Recommendations (Conservative)

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Get conservative LLM recommendations, filtering for GPU-only execution, usable speed, and a VRAM headroom of 1.5GB, showing the top 10 models.

```bash
whichllm --gpu-only --speed usable --vram-headroom 1.5GB --top 10
```

--------------------------------

### Run Interactive Chat

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/run-snippet.md

Starts an interactive chat session with a selected model. If MODEL_NAME is omitted, it ranks models for the current hardware and uses the top result.

```bash
whichllm run [MODEL_NAME]
```

--------------------------------

### Select GPU by Index for Multi-GPU Systems

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/hardware.md

When multiple GPUs are detected, use the `--gpu-index` flag to specify which GPU should be displayed or used by commands like `whichllm hardware`. This allows targeting specific cards in a multi-GPU setup.

```bash
whichllm --gpu-index 1 --vram 8 --ram-bandwidth 68
```

--------------------------------

### Detect CPU Name

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/hardware-detection.md

Get the CPU model name. For example, 'Intel Core i7-13700K'.

```python
from whichllm.hardware.cpu import detect_cpu_name

name = detect_cpu_name()
print(f"CPU: {name}")  # e.g., "Intel Core i7-13700K"
```

--------------------------------

### Simulate Hardware and Show Top Models

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Simulate a specific GPU (e.g., RTX 4090) and display the top 3 recommended models for that hardware.

```bash
whichllm --gpu "RTX 4090" --top 3
```

--------------------------------

### Main Command (Default)

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

The default command to detect hardware, fetch models, rank by quality, and display recommendations.

```bash
whichllm [OPTIONS]
```

--------------------------------

### List Top 5 Models

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Use this command to list the top 5 recommended models for your current hardware configuration.

```bash
whichllm --top 5
```

--------------------------------

### Clone and Run WhichLLM

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Steps to clone the repository, set up development environment, and run the project and tests.

```bash
git clone https://github.com/Andyyyy64/whichllm.git
cd whichllm
uv sync --dev
uv run whichllm
uv run pytest
```

--------------------------------

### Detect CPU Cores

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/hardware-detection.md

Get the logical CPU core count, including hyperthreads.

```python
from whichllm.hardware.cpu import detect_cpu_cores

cores = detect_cpu_cores()
print(f"Cores: {cores}")  # includes hyperthreads
```

--------------------------------

### Initialize Typer CLI Application

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Sets up the main entry point for the whichllm CLI application using Typer. This is the default command that handles hardware detection, model ranking, and displaying recommendations.

```python
from whichllm.cli import app

app = typer.Typer(name="llm-checker", help="Find the best LLM that runs on your hardware.")
```

--------------------------------

### Detect Hardware, Fetch Models, and Rank

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/INDEX.md

This pattern demonstrates the core workflow of detecting available hardware, fetching a list of models, and then ranking these models based on the detected hardware. It's a fundamental sequence for using the library.

```python
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models
from whichllm.engine.ranker import rank_models

hardware = detect_hardware()
models = await fetch_models()
results = rank_models(models, hardware)
```

--------------------------------

### Run whichllm with uvx

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Execute the latest version of whichllm directly using uvx without prior installation.

```bash
uvx whichllm@latest

```

--------------------------------

### Check WhichLLM Version

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Use the --version flag to display the currently installed version of the WhichLLM CLI.

```bash
whichllm --version
```

--------------------------------

### Determine GPU Requirements for a Model with WhichLLM

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use the 'plan' command to find out the specific GPU requirements for running a particular model. This helps in planning hardware purchases.

```bash
whichllm plan "llama 3 70b"
```

--------------------------------

### Simulate GPU Hardware for Planning

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use the --gpu flag to simulate specific GPU models, useful for planning hardware purchases or testing compatibility without the actual hardware.

```bash
whichllm --gpu "RTX 4090"
whichllm --gpu "RTX 5090"
# Specify variant
whichllm --gpu "RTX 5060 16"
```

--------------------------------

### Detect Free Disk Space

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/hardware-detection.md

Gets the free disk space in bytes within the whichllm cache directory.

```python
from whichllm.hardware.memory import detect_disk_free_bytes

disk_free = detect_disk_free_bytes()
print(f"Free disk: {disk_free / 1e9:.1f} GB")
```

--------------------------------

### Run Chat with Best Model

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/README.md

Launch an interactive chat session using the best model identified for your current hardware.

```bash
whichllm run
```

--------------------------------

### Safer LLM recommendation with specific parameters

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Run whichllm with parameters to prioritize models that fit entirely in GPU VRAM, filter slow estimates, and reserve VRAM headroom.

```bash
uvx whichllm@latest --gpu-only --speed usable --vram-headroom 1GB

```

--------------------------------

### Python Library Imports

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/INDEX.md

Import necessary components from the WhichLLM Python library to utilize its functionalities in your scripts. Ensure these modules are installed.

```python
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models
from whichllm.models.grouper import group_models
from whichllm.models.benchmark import fetch_benchmark_scores
from whichllm.engine.ranker import rank_models
from whichllm.output.display import display_ranking
```

--------------------------------

### Manage RAM Budget and Headroom

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Configure VRAM headroom and RAM budget to avoid edge cases and unexpected memory usage.

```bash
# Avoid edge fits and background-RAM surprises
whichllm --vram-headroom 1.5GB
whichllm --ram-budget available
whichllm --ram-budget 8GB
```

--------------------------------

### Simulate Specific GPU and Top Models

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/README.md

Use the --gpu and --top flags to simulate a specific GPU and limit the number of recommendations.

```bash
whichllm --gpu "RTX 4090" --top 10
```

--------------------------------

### Simulate LLM for Hardware

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use this command to simulate which LLM would perform best on your specific hardware before purchasing. It helps in understanding potential performance.

```bash
whichllm --gpu "<your card>"
```

--------------------------------

### Simulate Multi-GPU Workstation with WhichLLM

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Configure WhichLLM to simulate a multi-GPU setup by specifying multiple GPUs with the --gpu flag. Useful for testing multi-GPU performance.

```bash
whichllm --gpu "2x RTX 4090"
```

--------------------------------

### Plan Hardware Purchase

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Determine the top 10 LLMs that can run on a specific GPU, like an RTX 5090.

```bash
whichllm --gpu "RTX 5090" --top 10
```

--------------------------------

### CLI Error Message Formatting

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Errors, warnings, and notes are printed to stderr with color coding for better readability. This example shows the format for different message types.

```text
[red]Error:[/] --cpu-only and --gpu are mutually exclusive.
[yellow]Warning:[/] Benchmark data unavailable: ...
[dim]Note:[/] Found N matches, using: ...
```

--------------------------------

### Plan GPU Requirements for a Model

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Use the 'plan' command to find out the VRAM requirements and compatible GPU recommendations for a specific model. You can specify quantization and context length.

```bash
whichllm plan "llama 3.1 70b"
```

```bash
whichllm plan "qwen 27b" --quant Q8_0 --context-length 32k
```

```bash
whichllm plan "mistral 7b" --json | jq .
```

--------------------------------

### Compare GPU Upgrade Path

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Compare LLM performance across different GPU upgrades, showing the top 5 models for each.

```bash
whichllm upgrade "RTX 4090" "RTX 5090" "H100" --top 5
```

--------------------------------

### Detect All Hardware

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/hardware-detection.md

Call this function to get a complete hardware inventory. It detects GPUs, CPU, RAM, disk, and OS. Each detector is fail-safe and returns empty results on error.

```python
from whichllm.hardware.detector import detect_hardware

hw = detect_hardware()

print(f"GPUs detected: {len(hw.gpus)}")
for gpu in hw.gpus:
    print(f"  {gpu.name}: {gpu.vram_bytes / 1e9:.1f} GB {gpu.vendor}")

print(f"CPU: {hw.cpu_name} ({hw.cpu_cores} cores)")
print(f"RAM: {hw.ram_bytes / 1e9:.1f} GB")
print(f"OS: {hw.os}")
```

--------------------------------

### Show top 5 with RTX 4090 simulation

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Displays the top 5 models, simulating an RTX 4090 GPU.

```bash
whichllm --gpu "RTX 4090" --top 5
```

--------------------------------

### CLI Entry Point

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/INDEX.md

The command-line interface for interacting with WhichLLM. Use this to run commands and options directly from your terminal.

```bash
whichllm [COMMAND] [OPTIONS]
```

--------------------------------

### Get Top Ollama Model ID with JSON

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use this command to retrieve the HuggingFace model ID of the top-ranked model, formatted as JSON, for scripting purposes. Requires `jq` for JSON parsing.

```bash
# Pick the top HuggingFace model ID
whichllm --top 1 --json | jq -r '.models[0].model_id'
```

```bash
# Find the best coding model ID
whichllm --profile coding --top 1 --json | jq -r '.models[0].model_id'
```

--------------------------------

### Get Top Model as JSON with WhichLLM

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Retrieve the top-ranked model information in JSON format using the --top 1 --json flags. Ideal for scripting and automated processing.

```bash
whichllm --top 1 --json
```

--------------------------------

### Plan Model Compatibility

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Use the 'plan' command to determine which GPUs and configurations can run a specific model. Specify the model name and optional context length or quantization.

```bash
whichllm plan "model-name" [OPTIONS]
```

--------------------------------

### Compare GPU Upgrades

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/README.md

Utilize the 'upgrade' command to compare the performance of different GPUs for LLM inference.

```bash
whichllm upgrade "RTX 4090" "RTX 5090" "H100"
```

--------------------------------

### Get LLM Rankings as JSON

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

This command retrieves LLM rankings in JSON format, suitable for use in automated pipelines or further processing with tools like jq. It allows for programmatic access to the LLM selection data.

```bash
whichllm --json | jq
```

--------------------------------

### Plan GPU Requirements for a Specific Model

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Use the 'plan' subcommand to determine the necessary hardware, particularly GPU, for running a specified LLM model.

```bash
whichllm plan "llama 3 70b"
whichllm plan "Qwen2.5-72B" --quant Q8_0
whichllm plan "mistral 7b" --context-length 32768
```

--------------------------------

### Fetch and Inspect Models

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/api-reference/model-fetching.md

Fetches a list of available models and inspects a specific model's details. Requires importing `fetch_models` and `asyncio`. The example demonstrates accessing various attributes of the `ModelInfo` object, including GGUF variants and base model information.

```python
from whichllm.models.fetcher import fetch_models
import asyncio

async def inspect():
    models = await fetch_models()
    model = next(m for m in models if "qwen3-27b" in m.id.lower())
    
    print(f"Model: {model.id}")
    print(f"  Parameters: {model.parameter_count / 1e9:.1f}B")
    print(f"  Architecture: {model.architecture}")
    print(f"  Published: {model.published_at}")
    print(f"  Downloads: {model.downloads:,}")
    
    if model.gguf_variants:
        print(f"  GGUF variants:")
        for variant in model.gguf_variants[:3]:
            print(f"    • {variant.quant_type}: {variant.file_size_bytes / 1e9:.1f} GB")
    
    if model.base_model:
        print(f"  Base model: {model.base_model}")

asyncio.run(inspect())
```

--------------------------------

### Safe recommendations: full-GPU only, usable speed, 1.5GB headroom

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/cli-reference.md

Provides safe recommendations by enforcing full-GPU usage, requiring usable speed, and setting a 1.5GB VRAM headroom.

```bash
whichllm --gpu-only --speed usable --vram-headroom 1.5GB
```

--------------------------------

### Inspect Model Memory Requirements

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/troubleshooting.md

Use the `plan` command to estimate the memory requirements for a specific model and quantization. Adjust context length as needed.

```bash
whichllm plan "Qwen2.5-72B" --quant Q4_K_M
```

```bash
whichllm plan "Qwen2.5-72B" --quant Q8_0 --context-length 32768
```

--------------------------------

### Multi-GPU Simulation Syntax

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/hardware.md

Simulate multiple GPUs using various syntaxes: repeating the `--gpu` flag, providing comma-separated values, or using count shorthand. Note that `--vram` is only supported for single GPU simulations; for multi-GPU setups, rely on known GPU names for VRAM resolution.

```bash
whichllm --gpu "2x RTX 4090"
```

```bash
whichllm --gpu "RTX 4090" --gpu "RTX 3090"
```

```bash
whichllm --gpu "RTX 4090, RTX 3090"
```

--------------------------------

### Accessing Nested Types in Model Ranking

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/types.md

This Python script demonstrates how to use the whichllm library to detect hardware, fetch available models, rank them based on hardware compatibility, and then access detailed information from the ranking results, including model specifics, performance metrics, and GGUF variant details. Ensure the necessary libraries are installed and an asyncio event loop is available.

```python
from whichllm.engine.ranker import rank_models
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models
import asyncio

async def access_types():
    hardware = detect_hardware()
    models = await fetch_models()
    
    results = rank_models(models, hardware, top_n=1)
    
    if results:
        result = results[0]  # CompatibilityResult
        model = result.model  # ModelInfo
        
        # Access hardware
        gpu = hardware.gpus[0] if hardware.gpus else None
        
        # Access model info
        print(f"Model: {model.id} ({model.parameter_count / 1e9:.1f}B)")
        print(f"Architecture: {model.architecture}")
        print(f"License: {model.license}")
        
        # Access ranking result
        print(f"Score: {result.quality_score:.1f}")
        print(f"Speed: {result.estimated_tok_per_sec} tok/s ({result.speed_confidence})")
        print(f"Fit: {result.fit_type}")
        print(f"Benchmark source: {result.benchmark_source}")
        
        # Access GGUF variant
        if result.gguf_variant:
            print(f"GGUF: {result.gguf_variant.quant_type} ({result.gguf_variant.file_size_bytes / 1e9:.1f} GB)")

asyncio.run(access_types())
```

--------------------------------

### Troubleshoot No Compatible Models Found

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/troubleshooting.md

If no compatible models are found, try running without filters, with `--refresh`, or check common causes like restrictive quantizations or speed filters.

```bash
whichllm
```

```bash
whichllm --cpu-only
```

```bash
whichllm --refresh
```

```bash
whichllm --top 20
```

--------------------------------

### Set RAM budget

Source: https://github.com/andyyyy64/whichllm/blob/main/docs/cli.md

Caps the RAM available for partial offloading. 'available' uses the current free RAM.

```bash
whichllm --ram-budget available
```

--------------------------------

### Output Rendering Functions (Python)

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

Provides functions for displaying ranking results, hardware details, JSON, Markdown tables, plans, and upgrade comparisons. Import these functions to customize output presentation.

```python
from whichllm.output.display import (
    display_ranking,       # Rich table + hardware info
    display_hardware,      # Hardware details only
    display_json,          # JSON output
    display_markdown,      # GitHub-Flavored Markdown table
    display_plan,          # Plan command output
    display_upgrade,       # Upgrade comparison table
)

# Example: display ranking results
display_ranking(results, has_gpu=True, show_status=True)

# Example: JSON output
display_json(results, hardware)

# Example: Markdown for GitHub issues
display_markdown(results, hardware, show_status=True)
```

--------------------------------

### Inspect Hardware

Source: https://github.com/andyyyy64/whichllm/blob/main/_autodocs/REFERENCE.md

The 'hardware' command displays detected hardware information. GPU simulation flags can be used to override detected hardware details.

```bash
whichllm hardware [OPTIONS]
```

--------------------------------

### Run a Model Directly with WhichLLM

Source: https://github.com/andyyyy64/whichllm/blob/main/README.md

Launch an interactive chat session with a specified model using the 'run' command. Requires the model to be compatible and available.

```bash
whichllm run "qwen 2.5 1.5b gguf"
```