### Generate Trace and Start MCP Server for Analysis

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/README.md

This example demonstrates a multi-step process for analyzing Perfetto traces. First, generate a trace with specific options. Then, start the MCP server in daemon mode. Finally, interact with the MCP server using a client to load the trace and perform analysis.

```bash
# 1. Generate trace
sudo scxtop trace -d 5000 -o trace.proto -s

# 2. Start MCP server
sudo scxtop mcp --daemon

# 3. Via MCP client (e.g., Claude):
#    - load_perfetto_trace(file_path="trace.proto")
#    - analyze_trace_scheduling(analysis_type="cpu_utilization")
#    - find_scheduling_bottlenecks(limit=10)
```

--------------------------------

### Profiling Workflow Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example demonstrating a profiling workflow. It involves starting profiling, optionally waiting, stopping profiling, and retrieving results.

```text
"Profile the system and show me the hottest kernel functions"
→ Claude calls: start_perf_profiling with event="hw:cpu-clock", freq=99
→ (waits or sets duration)
→ Claude calls: stop_perf_profiling
→ Claude calls: get_perf_results with limit=20, include_stacks=true
→ Claude analyzes and presents results
```

--------------------------------

### Perfetto Trace Analysis Workflow Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/PERFETTO_ANALYZER_README.md

Demonstrates a typical workflow for analyzing a Perfetto trace, including loading the trace, getting a summary, running all analyzers, detecting outliers, finding bottlenecks, and performing custom queries.

```json
// 1. Load trace
{ "tool": "load_perfetto_trace", "arguments": { "file_path": "/path/to/trace.proto" } }
```

```json
// 2. Get summary
{ "tool": "get_trace_summary", "arguments": { "trace_id": "trace" } }
```

```json
// 3. Run all applicable analyzers
{ "tool": "run_all_analyzers", "arguments": { "trace_id": "trace" } }
```

```json
// 4. Detect outliers
{ "tool": "detect_outliers", "arguments": { "trace_id": "trace", "method": "IQR", "category": "all" } }
```

```json
// 5. Find bottlenecks
{ "tool": "find_scheduling_bottlenecks", "arguments": { "trace_id": "trace" } }
```

```json
// 6. Custom query
{
  "tool": "query_trace",
  "arguments": {
    "trace_id": "trace",
    "event_type": "sched_switch",
    "aggregation": { "function": "count_by", "field": "comm" }
  }
}
```

--------------------------------

### scxtop Configuration Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/README.md

An example configuration file for scxtop, demonstrating customization of theme, tick rates, and keymaps.

```toml
theme = "IAmBlue"
tick_rate_ms = 250
debug = false
exclude_bpf = false
worker_threads = 4

[keymap]
d = "AppStateDefault"
"?" = "AppStateHelp"
"[" = "DecBpfSampleRate"
q = "Quit"
"+" = "IncTickRate"
u = "ToggleUncoreFreq"
"Page Down" = "PageDown"
S = "SaveConfig"
Up = "Up"
P = "RecordTrace"
"-" = "DecTickRate"
L = "ToggleLocalization"
t = "ChangeTheme"
"]" = "IncBpfSampleRate"
Down = "Down"
l = "AppStateLlc"
k = "NextEvent"
a = "RecordTrace"
j = "PrevEvent"
v = "NextViewState"
h = "AppStateHelp"
n = "AppStateNode"
s = "AppStateScheduler"
e = "AppStateEvent"
w = "RecordTrace"
f = "ToggleCpuFreq"
Enter = "Enter"
"Page Up" = "PageUp"
x = "ClearEvent"

```

--------------------------------

### Example Trace File Paths

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PROTOBUF_LOADING_VERIFIED.md

These are example paths for perfetto trace files that can be loaded by scxtop.

```bash
/home/hodgesd/scx/scxtop_trace_0.proto  # 40MB, 722K events
my_scheduler_trace.proto                 # Any perfetto trace
workload_analysis.proto                  # Generated by scxtop
```

--------------------------------

### Tool Usage Example: Get Topology

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example of using a tool to retrieve hardware topology. Claude calls the 'get_topology' tool with a 'full' detail level.

```text
"Show me the hardware topology"
→ Claude calls: get_topology with detail_level="full"
```

--------------------------------

### Server Example Execution

Source: https://github.com/sched-ext/scx/blob/main/rust/scx_stats/README.md

This example shows how to run the server. It listens on a UNIX domain socket and requires the client to connect to the specified path.

```bash
> cargo run --example server -- ~/tmp/socket
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/examples/server /home/htejun/tmp/socket`
Server listening. Run `client "/home/htejun/tmp/socket"`.
Use `socat - UNIX-CONNECT:"/home/htejun/tmp/socket"` for raw connection.
Press any key to exit.
```

--------------------------------

### Install scx Schedulers from Source on Ubuntu

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs the built scx schedulers by iterating through the scheduler directories and using Cargo install.

```bash
$ ls -d scheds/rust/scx_* | xargs -I{} cargo install --path {}
```

--------------------------------

### Build Specific Schedulers (Examples)

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Examples of building individual schedulers with the release profile. Each entry in the table demonstrates building a specific scheduler.

```bash
cargo build --release -p scx_beerland
```

```bash
cargo build --release -p scx_bpfland
```

```bash
cargo build --release -p scx_cake
```

```bash
cargo build --release -p scx_chaos
```

```bash
cargo build --release -p scx_cosmos
```

```bash
cargo build --release -p scx_flash
```

```bash
cargo build --release -p scx_flow
```

```bash
cargo build --release -p scx_lavd
```

```bash
cargo build --release -p scx_layered
```

```bash
cargo build --release -p scx_mitosis
```

```bash
cargo build --release -p scx_p2dq
```

```bash
cargo build --release -p scx_pandemonium
```

```bash
cargo build --release -p scx_rlfifo
```

```bash
cargo build --release -p scx_rustland
```

```bash
cargo build --release -p scx_rusty
```

```bash
cargo build --release -p scx_tickless
```

--------------------------------

### Enable and Start SCX Service

Source: https://github.com/sched-ext/scx/blob/main/services/README.md

Commands to enable the SCX service to start at boot and start it immediately. Requires root privileges.

```bash
systemctl enable scx.service
```

```bash
systemctl start scx.service
```

--------------------------------

### Install scxtop from crates.io

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Installs the scxtop crate directly from crates.io. Ensure your PATH includes ~/.cargo/bin to access the installed binary.

```bash
cargo install scxtop
```

--------------------------------

### Build and Install Manager Commands

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Use the pandemonium.py script for managing the build and installation process. Options include cleaning, rebuilding, installing, and checking status.

```bash
# Build manager (recommended)
./pandemonium.py rebuild        # Force clean rebuild
./pandemonium.py install        # Build + install to /usr/local/bin + systemd service file
./pandemonium.py status         # Show build/install status
./pandemonium.py clean          # Wipe build artifacts
```

--------------------------------

### Client Example Execution with Tracing

Source: https://github.com/sched-ext/scx/blob/main/rust/scx_stats/README.md

Execute the client example with `RUST_LOG=trace` to observe the communication protocol details, including sent and received JSON messages.

```bash
$ RUST_LOG=trace cargo run --example client -- ~/tmp/socket
...
===== Requesting "stats" but receiving with serde_json::Value:
2024-08-15T22:13:23.769Z TRACE [scx_stats::client] Sending: {"req":"stats","args":{"target":"top"}}
2024-08-15T22:13:23.769Z TRACE [scx_stats::client] Received: {"errno":0,"args":{"resp":{"at":12345,"bitmap":[3735928559,3203391149],"doms_dict":{"0":{"events":1234,"name":"domain 0","pressure":1.234},"3":{"events":5678,"name":"domain 3","pressure":5.678}},"name":"test cluster"}}}
Ok(
    Object {
        "at": Number(12345),
        "bitmap": Array [
            Number(3735928559),
            Number(3203391149),
        ],
        "doms_dict": Object {
            "0": Object {
                "events": Number(1234),
                "name": String("domain 0"),
                "pressure": Number(1.234),
            },
            "3": Object {
                "events": Number(5678),
                "name": String("domain 3"),
                "pressure": Number(5.678),
            },
        },
        "name": String("test cluster"),
    },

```

--------------------------------

### Enable and Start SCX Service (Shortened)

Source: https://github.com/sched-ext/scx/blob/main/services/README.md

A combined command to enable the SCX service for boot and start it immediately. Requires root privileges.

```bash
systemctl enable --now scx.service
```

--------------------------------

### Basic Resource Read Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example of a basic query to read scheduler information. Claude reads the 'scheduler://current' resource.

```text
"What scheduler is currently running?"
→ Claude reads: scheduler://current
```

--------------------------------

### Build and Run Minimal Scheduler

Source: https://github.com/sched-ext/scx/wiki/Home

These commands are used to build the scheduler binary and then start it with root privileges. Ensure the scheduler is built before starting.

```bash
# build the scheduler binary
./build.sh

# start the scheduler
sudo ./start.sh

# do something ...

# stop the scheduler
sudo ./stop.sh
```

--------------------------------

### Install scx and Run Rusty Scheduler on openSUSE

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs the scx package and runs the scx_rusty scheduler on openSUSE Tumbleweed.

```sh
$ sudo zypper install scx
$ sudo scx_rusty
```

--------------------------------

### Install Ubuntu Development Dependencies

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs essential build tools and development libraries required for building scx on Ubuntu.

```bash
$ sudo apt install -y build-essential cmake cargo rustc clang llvm pkg-config libelf-dev protobuf-compiler libseccomp-dev libbpf-dev pahole
```

--------------------------------

### Clone Minimal Scheduler Repository

Source: https://github.com/sched-ext/scx/wiki/Home

Clone the minimal-scheduler repository to get started with the tutorial code. This is the first step to access the scheduler files.

```bash
git clone https://github.com/parttimenerd/minimal-scheduler
cd minimal-scheduler
```

--------------------------------

### Install scx and pahole on Gentoo Linux

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs the scx scheduler and pahole utility on Gentoo Linux using emerge.

```bash
emerge sys-kernel/scx dev-util/pahole
```

--------------------------------

### Install scx_flash from crates.io

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Installs the scx_flash crate directly from crates.io. The binary will be placed in ~/.cargo/bin, which should be added to your system's PATH.

```bash
cargo install scx_flash
```

--------------------------------

### Install Rust Scheduler from Crates.io

Source: https://github.com/sched-ext/scx/blob/main/README.md

Rust schedulers can be installed directly using 'cargo install' if they are published on crates.io.

```shell
$ cargo install scx_rusty
```

--------------------------------

### Event Monitoring Example (Daemon Mode)

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example of setting up event monitoring in daemon mode. Claude subscribes to events, filters for high latency, and reports anomalies.

```text
"Monitor scheduling events and alert me if you see high latency"
→ Claude subscribes to: events://stream
→ Claude filters sched_switch events for dsq_lat_us > 1000
→ Claude reports anomalies in real-time
```

--------------------------------

### Claude Code CLI Usage Examples for SCXTop

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/README.md

Demonstrates direct command-line usage of the Claude Code CLI with SCXTop. Includes examples for quick queries, interactive sessions, using specific prompts, generating reports, and profiling.

```bash
# Quick query
claude --mcp scxtop "What scheduler is running and how's performance?"
```

```bash
# Interactive session
claude --mcp scxtop
> Show me CPU utilization across NUMA nodes
> Which processes have high scheduling latency?
> What perf events are available for profiling?
> Profile the system at 99 Hz for 5 seconds and show the top 20 functions
> Start profiling and collect 10000 samples, then show me kernel stack traces
```

```bash
# Use a specific workflow prompt
claude --mcp scxtop --prompt analyze_scheduler_performance --arg focus_area=latency
```

```bash
# Generate a performance report
claude --mcp scxtop "Create a scheduler performance report" > report.md
```

```bash
# Profile and analyze
claude --mcp scxtop "Profile the system and identify performance bottlenecks"
```

--------------------------------

### Install Cross-Compilation Target

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Installs a specific target triple for cross-compilation. This is a prerequisite for building for that target.

```bash
rustup target add x86_64-unknown-linux-musl
```

--------------------------------

### Per-Second Telemetry Output Example

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Example of the detailed per-second telemetry data provided by scx_pandemonium, showing various performance metrics.

```text
d/s: 251000 idle: 5% shared: 230000 preempt: 12 keep: 0 kick: H=8000 S=22000 enq: W=8000 R=22000 wake: 4us p99: 10us [B:8 I:9 L:7] lat_idle: 3us lat_kick: 6us procdb: 42/5 sleep: io=87% slice: 1000us batch: 20000us reenq: 4 sjrn: 3ms/5ms rescue: 0 l2: B=67% I=72% L=85% chaos: lam=2.10 H=0.40 det=0.95 x=0 frozen: 0 (n=12) retune_iv: 2 [MIXED]
```

--------------------------------

### Copy installed binary to system directory

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Copies an installed binary (e.g., scxtop) from ~/.cargo/bin to a system-wide location like /usr/local/bin. This requires superuser privileges.

```bash
sudo cp ~/.cargo/bin/scxtop /usr/local/bin/
```

--------------------------------

### Resource Handler Registration Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example of registering a resource handler in Rust. It demonstrates capturing state like topology data to provide resource information.

```rust
self.resources.register_handler("topology://info".to_string(), move || {
    Ok(serde_json::json!({
        "nr_cpus": topo.all_cpus.len(),
        // ... topology data
    }))
});
```

--------------------------------

### Prompt Usage Example: Analyze Scheduler Performance

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Example of using a prompt to analyze scheduler performance issues. Claude invokes 'analyze_scheduler_performance' with a 'latency' focus.

```text
"Analyze scheduler latency issues"
→ Claude invokes: analyze_scheduler_performance prompt with focus_area="latency"
```

--------------------------------

### Bottleneck Detection Output Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PERFETTO_TRACE_ANALYSIS.md

Example JSON output for the find_scheduling_bottlenecks function, providing a description of the bottleneck, its severity, type, and the affected time range.

```json
{
  "description": "High migration rate: 30547 migrations/sec",
  "severity": 10.0,
  "bottleneck_type": "ExcessiveMigration",
  "time_range": [174606978728139, 174608228685859]
}
```

--------------------------------

### scxtop Project Structure

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/README.md

Overview of the scxtop project directory layout, highlighting key documentation and example files.

```text
tools/scxtop/
├── README.md                           # Main README
├── docs/                               # User documentation
│   ├── README.md                       # This index
│   ├── PERFETTO_TRACE_ANALYSIS.md     # Perfetto analysis guide
│   ├── TASK_THREAD_DEBUGGING_GUIDE.md # Debugging workflows
│   └── PROTOBUF_LOADING_VERIFIED.md   # Protobuf verification
├── examples/
│   └── perfetto_trace_analysis_examples.json
└── CLAUDE_INTEGRATION.md              # Claude setup

```

--------------------------------

### Error Handling: File Not Found

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PROTOBUF_LOADING_VERIFIED.md

Example error message when the specified trace file cannot be found.

```text
Error: Failed to read file: No such file or directory
```

--------------------------------

### Trace Caching Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PROTOBUF_LOADING_VERIFIED.md

Demonstrates loading a trace once and then performing multiple analyses using the cached trace ID for speed.

```javascript
// Load once
load_perfetto_trace({file_path: "/path/to/trace.proto", trace_id: "my_trace"})

// Analyze many times (uses cache, fast!)
analyze_trace_scheduling({trace_id: "my_trace", analysis_type: "task_states"})
analyze_trace_scheduling({trace_id: "my_trace", analysis_type: "preemptions"})
analyze_trace_scheduling({trace_id: "my_trace", analysis_type: "wakeup_chains"})
// No re-parsing needed!
```

--------------------------------

### Example Process Timeline Events for CPU Affinity

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/TASK_THREAD_DEBUGGING_GUIDE.md

This JSON structure represents example events from a process timeline, including 'Scheduled' and 'Migrated' events. Counting unique CPUs used from these events helps determine affinity.

```json
{
  "events": [
    {"Scheduled": {"cpu": 0, "timestamp": ...}},
    {"Migrated": {"from_cpu": 0, "to_cpu": 5, "timestamp": ...}},
    {"Scheduled": {"cpu": 5, "timestamp": ...}},
    {"Migrated": {"from_cpu": 5, "to_cpu": 12, "timestamp": ...}}
  ]
}
```

--------------------------------

### Basic Trace Analysis Workflow

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/PERFETTO_ANALYZER_GUIDE.md

A three-step process to load a Perfetto trace, get a summary, and run parallel CPU utilization analysis.

```json
// 1. Load trace
{
  "tool": "load_perfetto_trace",
  "arguments": {
    "file_path": "/traces/my_app.proto",
    "trace_id": "app_trace"
  }
}

// 2. Get summary
{
  "tool": "get_trace_summary",
  "arguments": {
    "trace_id": "app_trace"
  }
}

// 3. Run CPU utilization analysis
{
  "tool": "analyze_trace_scheduling",
  "arguments": {
    "trace_id": "app_trace",
    "analysis_type": "cpu_utilization",
    "use_parallel": true
  }
}
```

--------------------------------

### start_perf_profiling

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Start perf profiling with stack trace collection and symbolization. Configurable by event, frequency, CPU, PID, sample count, and duration.

```APIDOC
## start_perf_profiling

### Description
Start perf profiling with stack trace collection and symbolization.

### Parameters
#### Query Parameters
- **event** (string) - Optional - Default: `hw:cpu-clock`. Event to profile. Examples: `hw:cpu-clock`, `sw:task-clock`, `tracepoint:subsystem:event`.
- **freq** (integer) - Optional - Default: `99`. Sampling frequency in Hz.
- **cpu** (integer) - Optional - Default: `-1`. CPU to profile (-1 for all CPUs, specific CPU ID otherwise).
- **pid** (integer) - Optional - Default: `-1`. Process ID to profile (-1 for system-wide).
- **max_samples** (integer) - Optional - Default: `10000`. Maximum samples to collect (0 for unlimited).
- **duration_secs** (integer) - Optional - Default: `0`. Duration in seconds (0 for manual stop).

### Request Example
```json
{
  "event": "hw:cpu-clock",
  "freq": 99,
  "duration_secs": 10,
  "max_samples": 0
}
```

### Returns
Confirmation with profiling configuration.
```

--------------------------------

### Systemd Service Management

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Control the pandemonium service using systemctl after installation. Start the service immediately or enable it to start on boot.

```bash
sudo systemctl start pandemonium          # Start now
sudo systemctl enable pandemonium         # Start on boot
```

--------------------------------

### Debug high scheduling latency workflow

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Start a guided workflow to debug high scheduling latency issues. Optionally specify a process ID for targeted analysis.

```string
debug_high_latency
```

--------------------------------

### Running and Monitoring a sched_ext Scheduler

Source: https://github.com/sched-ext/scx/blob/main/README.md

Demonstrates how to activate a sched_ext scheduler by running its binary, monitor its state, suspend it, and observe its output. It also shows how to return to the default scheduler.

```bash
root@test ~# cat /sys/kernel/sched_ext/state /sys/kernel/sched_ext/*/ops 2>/dev/null
disabled
root@test ~# scx_simple
local=1 global=0
local=74 global=15
local=78 global=32
local=82 global=42
local=86 global=54
^Zfish: Job 1, 'scx_simple' has stopped
root@test ~# cat /sys/kernel/sched_ext/state /sys/kernel/sched_ext/*/ops 2>/dev/null
enabled
simple
root@test ~# fg
Send job 1 (scx_simple) to foreground
local=635 global=179
local=696 global=192
^CEXIT: BPF scheduler unregistered
```

--------------------------------

### Summarize system overview workflow

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Initiate a workflow to gather a comprehensive system summary, including hardware topology, statistics, and resource distribution. This workflow requires no arguments.

```string
summarize_system
```

--------------------------------

### Analyze Task States for Specific Threads

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/TASK_THREAD_DEBUGGING_GUIDE.md

To analyze a multi-threaded process, first get all thread PIDs (TIDs) in the thread group (TGID), then run analysis for each thread PID. This example shows how to analyze task states for individual threads or fetch all and filter results.

```javascript
// Process with threads: 1000 (main), 1001, 1002, 1003

// Analyze each thread
analyze_trace_scheduling({analysis_type: "task_states", pid: 1000})
analyze_trace_scheduling({analysis_type: "task_states", pid: 1001})
analyze_trace_scheduling({analysis_type: "task_states", pid: 1002})
analyze_trace_scheduling({analysis_type: "task_states", pid: 1003})

// Or get all and filter
analyze_trace_scheduling({analysis_type: "task_states", limit: 1000})
// Filter JSON results to your thread PIDs
```

--------------------------------

### Build and Run scx_cake

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_cake/README.md

Instructions for cloning the repository, building the scx_cake binary, and running it with root privileges. Running with the -v flag enables live statistics.

```bash
# Prerequisites: Linux Kernel 6.12+ with sched_ext, Rust toolchain

# Clone and build
git clone https://github.com/sched-ext/scx.git
cd scx && cargo build --release -p scx_cake

# Run (requires root)
sudo ./target/release/scx_cake

# Run with live stats TUI
sudo scx_cake -v
```

--------------------------------

### Run scx_cake with Custom Quantum and Verbose Output

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_cake/README.md

Launches scx_cake with a custom quantum of 1500 microseconds and enables live TUI statistics display.

```bash
sudo scx_cake --quantum 1500 -v
```

--------------------------------

### Install scx Schedulers on Arch Linux

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs the scx schedulers package directly using pacman.

```bash
sudo pacman -S scx-scheds
```

--------------------------------

### Enable scx-scheds COPR and Install on Fedora

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Enables the CachyOS community COPR repository and installs the scx-scheds package on Fedora.

```sh
$ sudo dnf copr enable bieszczaders/kernel-cachyos-addons
$ sudo dnf install scx-scheds
```

--------------------------------

### Install Arch Linux Development Dependencies

Source: https://github.com/sched-ext/scx/blob/main/INSTALL.md

Installs additional development tools required for building scx on Arch Linux.

```bash
$ sudo pacman -Sy cargo bpf pahole
```

--------------------------------

### Analyze CPU imbalance workflow

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Begin a workflow to analyze CPU load imbalance and migration patterns. This workflow requires no specific arguments.

```string
analyze_cpu_imbalance
```

--------------------------------

### Start MCP Server in Daemon Mode

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PROTOBUF_LOADING_VERIFIED.md

Command to start the scxtop MCP server in daemon mode, which is required for trace loading and caching.

```bash
# Correct (daemon mode)
sudo scxtop mcp --daemon

# Wrong (one-shot mode) - cache not persistent
sudo scxtop mcp
```

--------------------------------

### Run Full Benchmarking Suite

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Execute the complete suite of benchmarks, including throughput, latency, burst, longrun, mixed, deadline, IPC, and launch tests. Results are archived locally.

```bash
./pandemonium.py bench-scale
```

--------------------------------

### Start scxtop MCP Server for AI Integration

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/README.md

Launch the Model Context Protocol (MCP) server in daemon mode for AI-assisted scheduler analysis.

```bash
sudo scxtop mcp --daemon
```

--------------------------------

### Build with Tiny Profile

Source: https://github.com/sched-ext/scx/blob/main/CARGO_BUILD.md

Use the `--profile` option to select a specific build profile, such as `release-tiny` for optimized small binary size.

```bash
cargo build --profile=release-tiny
```

--------------------------------

### Start performance profiling session

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/MCP_INTEGRATIONS.md

Initiate a performance profiling session. Configure the event, sampling frequency, target CPU, process ID, maximum samples, and duration.

```json
{
  "event": "hw:cpu-clock",
  "freq": 99,
  "duration_secs": 10,
  "max_samples": 0
}
```

--------------------------------

### Run scxtop MCP Server (One-shot)

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/README.md

Execute the scxtop MCP server for a single query and then exit. Requires root privileges.

```bash
sudo scxtop mcp
```

--------------------------------

### DHQ vs. DSQ: System Configuration Example

Source: https://github.com/sched-ext/scx/blob/main/DHQ_README.md

Illustrates the difference in task management between a traditional Dispatch Queue (DSQ) and a Dual Heap Queue (DHQ) in a system with two Last-Level Caches (LLCs). DHQ aims to maintain cache affinity and controlled migration.

```text
System: 2 LLCs in same NUMA node
Traditional DSQ approach:
  - mig_dsq: Tasks from both LLCs mixed together
  - No way to prioritize cache-warm tasks
  - High migration rate destroys cache affinity

DHQ approach:
  - Strand A: Tasks from LLC 0 (cache-warm to LLC 0)
  - Strand B: Tasks from LLC 1 (cache-warm to LLC 1)
  - Priority mode: Migrate task with highest urgency (lowest vtime)
  - Strand constraint: Prevents excessive migration from one LLC
```

--------------------------------

### Wakeup-Schedule Correlation Output Example

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PERFETTO_TRACE_ANALYSIS.md

Example JSON output for the correlate_wakeup_to_schedule function, detailing waker and wakee PIDs, timestamps, latency, and the CPU where scheduling occurred.

```json
{
  "pid": 2952187,
  "waker_pid": 2952260,
  "wakeup_timestamp": 174607123456789,
  "schedule_timestamp": 174607234567890,
  "wakeup_latency_ns": 111111101,
  "cpu": 5
}
```

--------------------------------

### Install Build Dependencies on Arch Linux

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Installs the necessary packages for building and running the SCX scheduler on Arch Linux, including clang, libbpf, and the Rust toolchain.

```bash
# Arch Linux
pacman -S clang libbpf bpf rust
```

--------------------------------

### DHQ vs. ATQ: Performance Scenario Example

Source: https://github.com/sched-ext/scx/blob/main/DHQ_README.md

Compares the behavior of Arena Task Queue (ATQ) and Dual Heap Queue (DHQ) in a high-frequency task migration scenario between LLCs. DHQ's priority mode with imbalance limits helps maintain cache hit rates and controlled migration.

```text
Scenario: High-frequency task migration between LLCs

ATQ behavior:
  - All tasks in single pool
  - No preference for cache affinity
  - LLC 0 may consume all LLC 1's tasks
  - Result: Thrashing, poor cache utilization

DHQ behavior (Priority mode, max_imbalance=3):
  - Tasks primarily stay on their strand (LLC)
  - Cross-strand steal only for high-priority tasks
  - Imbalance limit prevents migration storms
  - Result: Good cache hit rates, controlled migration
```

--------------------------------

### Claude System Overview Query

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/CLAUDE_INTEGRATION.md

Example of a user query to Claude for a system scheduler summary. Claude uses specific resource paths to gather information.

```text
System Summary:
- Hardware: 16 CPUs (8 cores), 2 LLCs, 1 NUMA node, SMT enabled
- Active Scheduler: scx_rusty (sched_ext)
- CPU Utilization: Average 45%, range 32-67%
- Top Process: chrome (PID 1234) using 18% CPU time
- Context Switch Rate: 12,450/sec
- Load: Well balanced across CPUs (variance: 8%)
```

--------------------------------

### Example sched_ext Scheduler for Core Scheduling

Source: https://github.com/sched-ext/scx/blob/main/OVERVIEW.md

This C code snippet illustrates a sched_ext scheduler designed to co-schedule pairs of tasks from the same cgroup, providing resilience against L1TF vulnerabilities. It serves as an example for core scheduling semantics but is not production-ready.

```c
/*
 * scx_pair.bpf.c
 * 
 * This is a simple example of a sched_ext scheduler that co-schedules
 * pairs of tasks from the same cgroup. It is intended to illustrate
 * core scheduling semantics and provide resilience against L1TF
 * vulnerabilities.
 * 
 * NOTE: This scheduler is not suitable for production in its current form.
 * A more performant and featureful scheduler would be required for
 * production use cases.
 */

#include "vmlinux.h"
#include "bpf/bpf_helpers.h"

// Define a structure to hold task information, potentially including cgroup ID
struct task_info {
    u64 cgroup_id;
    // Add other relevant task information if needed
};

// Map to store task information, keyed by task PID
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 10240);
    __uint(key_size, sizeof(pid_t));
    __uint(value_size, sizeof(struct task_info));
} task_map SEC(".maps");

// Function to get the cgroup ID for a given task
// This is a placeholder and would need to be implemented based on kernel specifics
static inline u64 get_cgroup_id(struct task_struct *task) {
    // Placeholder implementation: In a real scenario, you would traverse
    // the cgroup hierarchy or use BPF helpers to get the cgroup ID.
    // For demonstration, returning a dummy value.
    return 0;
}

SEC("sched_ext")
int sched_ext_main(struct bpf_raw_tracepoint_args *ctx) {
    struct task_struct *current_task;
    u64 cgroup_id;
    struct task_info *task_info;

    // Get the current task
    current_task = (struct task_struct *)bpf_get_current_task();
    if (!current_task) {
        return 0;
    }

    // Get the cgroup ID for the current task
    cgroup_id = get_cgroup_id(current_task);

    // Store or retrieve task information from the map
    task_info = bpf_map_lookup_elem(&task_map, &current_task->pid);
    if (!task_info) {
        // If task is not in map, add it
        struct task_info new_task_info = {
            .cgroup_id = cgroup_id,
        };
        bpf_map_update_elem(&task_map, &current_task->pid, &new_task_info, BPF_ANY);
    } else {
        // Update cgroup ID if it changed (e.g., task moved to a different cgroup)
        task_info->cgroup_id = cgroup_id;
    }

    // Core scheduling logic would go here:
    // - Find other tasks in the same cgroup.
    // - Decide which tasks to run together (e.g., pairs).
    // - Potentially use bpf_sched_setattr or similar BPF helpers
    //   to influence scheduling decisions (if available and applicable).
    // 
    // For this example, we'll just log that the scheduler was invoked.
    bpf_printk("sched_ext: Scheduler invoked for PID %d in cgroup %llu\n", current_task->pid, cgroup_id);

    return 0;
}

char _license[] SEC("license") = "GPL";

```

--------------------------------

### Run Live System Telemetry Benchmark

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_pandemonium/README.md

Capture live system telemetry data for analysis during benchmarking.

```bash
./pandemonium.py bench-sys
```

--------------------------------

### get_trace_summary

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/PERFETTO_ANALYZER_GUIDE.md

Gets a comprehensive trace summary including capabilities and applicable analyzers.

```APIDOC
## get_trace_summary

### Description
Gets comprehensive trace summary including capabilities and applicable analyzers.

### Parameters
- `trace_id` (required): Trace ID

### Returns
- Trace duration, CPU/process/event counts
- Trace capabilities (available events, process tree, sched_ext)
- Applicable analyzers grouped by category
```

--------------------------------

### Basic scx_mitosis Usage

Source: https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_mitosis/README.md

Starts scx_mitosis with a specified parent cgroup for workload isolation.

```bash
scx_mitosis --cell-parent-cgroup /workloads
```

--------------------------------

### vmlinux_docify Usage and Options

Source: https://github.com/sched-ext/scx/blob/main/tools/vmlinux_docify/README.md

This snippet shows the command-line usage and available options for the vmlinux_docify tool. It outlines how to specify kernel source and vmlinux.h file paths, as well as output file and help options.

```bash
Usage: vmlinux_docify [OPTIONS] --kernel-dir <KERNEL_DIR> --vmlinux-h <VMLINUX_H>

Options:
  -k, --kernel-dir <KERNEL_DIR>
          Path to the kernel source directory

  -v, --vmlinux-h <VMLINUX_H>
          Path to the vmlinux.h file to annotate

  -o, --output <OUTPUT>
          Path to the output file (default: vmlinux_annotated.h)
          
          [default: vmlinux_annotated.h]

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

--------------------------------

### Query

Source: https://github.com/sched-ext/scx/blob/main/DHQ_README.md

Provides functions to peek at tasks without removing them and to get the current queue sizes.

```APIDOC
## Query

**⚠️ IMPORTANT**: For vtime comparisons in dispatch, always use `scx_dhq_peek_strand()` to peek at the specific strand associated with the current CPU.

### PREFERRED: Peek at specific strand (for vtime comparison)
```c
// This is the correct way to peek from DHQ in scheduler context
u64 taskc_ptr = scx_dhq_peek_strand(scx_dhq_t *dhq, u64 strand);
```

### Generic peek without removing (respects mode)
```c
// Use only for debugging or when strand doesn't matter
u64 taskc_ptr = scx_dhq_peek(scx_dhq_t *dhq);
```

### Get queue sizes
```c
// Get total number of queued tasks
int total = scx_dhq_nr_queued(scx_dhq_t *dhq);

// Get number of queued tasks for a specific strand
int strand_size = scx_dhq_nr_queued_strand(scx_dhq_t *dhq, u64 strand);
```
```

--------------------------------

### Generate and Load scxtop Trace

Source: https://github.com/sched-ext/scx/blob/main/tools/scxtop/docs/PROTOBUF_LOADING_VERIFIED.md

Command to generate a perfetto protobuf file using scxtop and a note on how to load it with the MCP tool.

```bash
sudo scxtop trace -d 5000 -o trace.proto
# Generates perfetto protobuf file
# MCP can load it: load_perfetto_trace({file_path: "trace.proto"})
```