### Rust: Initialize and Run VCpu in Main

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

This snippet shows the initialization of the GuestPageTable and the VCpu, followed by calling the vcpu.run() method to start the guest execution. It's a minimal setup for launching the guest.

```rust
fn main() -> ! {
    /* ... */
    table.map(guest_entry, kernel_memory as u64, PTE_R | PTE_W | PTE_X);

    let mut vcpu = VCpu::new(&table, guest_entry);
    vcpu.run();
}

```

--------------------------------

### Install QEMU on Ubuntu

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/01-getting-started.md

Install the QEMU system emulator for RISC-V 64-bit on Ubuntu using apt.

```shell
sudo apt install qemu-system-riscv64
```

--------------------------------

### Install QEMU on macOS

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/01-getting-started.md

Use Homebrew to install the QEMU emulator on macOS systems.

```shell
brew install qemu
```

--------------------------------

### Hypervisor Boot Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

Example output from the hypervisor's boot script, showing the loading of the kernel and initial SBI calls made during the boot process.

```bash
$ ./run.sh
Booting hypervisor...
loaded kernel: size=5658KB
SBI call: eid=0x10, fid=0x0, a0=0x0 ('')
SBI call: eid=0x1, fid=0x0, a0=0x5b ('[')
SBI call: eid=0x1, fid=0x0, a0=0x20 (' ')
SBI call: eid=0x1, fid=0x0, a0=0x20 (' ')
SBI call: eid=0x1, fid=0x0, a0=0x20 (' ')
SBI call: eid=0x1, fid=0x0, a0=0x20 (' ')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x2e ('.')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x30 ('0')
SBI call: eid=0x1, fid=0x0, a0=0x5d (']')
SBI call: eid=0x1, fid=0x0, a0=0x20 (' ')
SBI call: eid=0x1, fid=0x0, a0=0x4c ('L')
SBI call: eid=0x1, fid=0x0, a0=0x69 ('i')
SBI call: eid=0x1, fid=0x0, a0=0x6e ('n')
SBI call: eid=0x1, fid=0x0, a0=0x75 ('u')
SBI call: eid=0x1, fid=0x0, a0=0x78 ('x')
...
```

--------------------------------

### Configure and Run QEMU with OpenSBI

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

This script configures and starts a QEMU virtual machine using the 'virt' machine type and the default OpenSBI firmware. It directs serial output to stdio and disables automatic reboot on crash.

```bash
#!/bin/bash
set -xue

# QEMU file path
QEMU=qemu-system-riscv64

# Start QEMU
$QEMU -machine virt -bios default -nographic -serial mon:stdio --no-reboot
```

--------------------------------

### Initialize and Run Virtual CPU

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

Sets up the virtual CPU with the loaded kernel and device tree, then starts its execution. The `a0` and `a1` registers are set for hart ID and device tree address, respectively.

```rust
use crate::linux_loader::GUEST_DTB_ADDR;

fn main() -> ! {
    /* ... */
    linux_loader::load_linux_kernel(&mut table, kernel_image);
    let mut vcpu = VCpu::new(&table, GUEST_BASE_ADDR);
    vcpu.a0 = 0; // hart ID
    vcpu.a1 = GUEST_DTB_ADDR; // device tree address
    vcpu.run();
 }

```

--------------------------------

### QEMU Register Information

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Example output from QEMU showing CPU register information after running the hypervisor. The 'pc' register indicates the current program counter.

```text
QEMU 10.0.0 monitor - type 'help' for more information
(qemu) info registers

CPU#0
 V      =   0
 pc       0000000080200010
...
```

--------------------------------

### Guest Page Fault Panic Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Example output showing a panic due to a load guest-page fault within the guest system.

```text
$ ./run.sh
... 
panic: panicked at src/trap.rs:193:14:
trap handler: load guest-page fault at 0xffffffff801d7d78 (stval=0xff20000000002080)

```

--------------------------------

### Main Function: Kernel Loading and VCPU Initialization

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

The `main` function initializes the guest page table, loads the Linux kernel image using `linux_loader::load_linux_kernel`, and sets up a virtual CPU (VCPU). It configures the VCPU's initial arguments, including the hart ID and a placeholder for the device tree address, before starting its execution.

```Rust
fn main() -> ! {
    /* ... */

    let mut table = GuestPageTable::new();
    let kernel_image = include_bytes!("../linux/Image");
    linux_loader::load_linux_kernel(&mut table, kernel_image);
    let mut vcpu = VCpu::new(&table, GUEST_BASE_ADDR);
    vcpu.a0 = 0; // hart ID
    vcpu.a1 = 0xdeadbeef_00000000; // device tree address (TODO)
    vcpu.run();
}
```

--------------------------------

### Guest Kernel Panic Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Example output showing a kernel panic in the guest system due to a missing interrupt controller.

```text
$ ./run.sh
... 
[guest] [    0.000000] Kernel panic - not syncing: No interrupt controller found.
[guest] [    0.000000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.12.34 #1
[guest] [    0.000000] Hardware name: riscv-virtio (DT)
[guest] [    0.000000] Call Trace:
[guest] [    0.000000] [<ffffffff80004650>] dump_backtrace+0x1c/0x24
[guest] [    0.000000] [<ffffffff8036b2ba>] show_stack+0x2c/0x38
[guest] [    0.000000] [<ffffffff8037234c>] dump_stack_lvl+0x52/0x74
[guest] [    0.000000] [<ffffffff80372382>] dump_stack+0x14/0x1c
[guest] [    0.000000] [<ffffffff8036b44e>] panic+0xf8/0x2e0
[guest] [    0.000000] [<ffffffff8037ce04>] init_IRQ+0x88/0xba
[guest] [    0.000000] [<ffffffff8037b334>] start_kernel+0x360/0x428

```

--------------------------------

### Enter Guest Mode in Rust

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/05-guest-mode.md

This Rust code snippet demonstrates the minimal setup to enter guest mode. It configures the `hstatus` register for the guest's supervisor execution mode and sets the `sepc` to the entry point, then uses `sret` to transition.

```rust
fn main() -> ! {
    /* ... */

    let mut hstatus: u64 = 0;
    hstatus |= 2 << 32; // VSXL: XLEN for VS-mode (64-bit)
    hstatus |= 1 << 7; // SPV: Supervisor Previous Virtualization mode

    let sepc: u64 = 0x1234abcd;
    unsafe {
        asm!(
            "csrw hstatus, {hstatus}",
            "csrw sepc, {sepc}",
            "sret",
            hstatus = in(reg) hstatus,
            sepc = in(reg) sepc,
        );
    }

    unreachable!();
}

```

--------------------------------

### Initialize BSS and Allocator in `main`

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/04-memory-allocation.md

Initializes the BSS section and the global bump allocator by providing the heap start and end pointers obtained from linker symbols.

```rust
unsafe extern "C" {
    static mut __bss: u8;
    static mut __bss_end: u8;
    static mut __heap: u8;
    static mut __heap_end: u8;
}

fn main() -> ! {
    init_bss();
    println!("\nBooting hypervisor...");

    allocator::GLOBAL_ALLOCATOR.init(&raw mut __heap, &raw mut __heap_end);

```

--------------------------------

### Initialize Trap Handler in Main

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/03-hello-world.md

Initializes the trap handler by writing its address to the `stvec` CSR. This setup occurs after clearing the BSS section.

```rust
fn main() -> ! {
    unsafe {
        let bss_start = &raw mut __bss;
        let bss_size = (&raw mut __bss_end as usize) - (&raw mut __bss as usize);
        core::ptr::write_bytes(bss_start, 0, bss_size);

        asm!("csrw stvec, {}", in(reg) trap::trap_handler as usize);
    }

```

--------------------------------

### Guest Trap Handler Panic Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Example output showing a panic within the guest trap handler, indicating a virtual instruction or a load guest-page fault.

```text
$ ./run.sh
... 
panic: panicked at src/trap.rs:188:14: 
trap handler: virtual instruction at 0xffffffff8022674e (stval=0xc0102573)

```

--------------------------------

### Unknown SBI Call Panic Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Example output showing a panic when an unknown SBI call (eid=0x0, fid=0x0) is encountered.

```text
$ ./run.sh
... 
panic: panicked at src/trap.rs:127:13:
unknown SBI call: eid=0x0, fid=0x0

```

--------------------------------

### Implement SBI Get Machine IDs

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Handles SBI calls for retrieving machine vendor, architecture, and implementation IDs. This snippet is part of the trap handling logic.

```rust
    let result: Result<i64, i64> = match (eid, fid) {
        // Get SBI specification version
        (0x10, 0x0) => Ok(0),
        // Probe SBI extension
        (0x10, 0x3) => Err(-1),
        // Get machine vendor/arch/implementation ID
        (0x10, 0x4 | 0x5 | 0x6) => Ok(0),
        // Console Putchar.
        (0x1, 0x0) => {

```

--------------------------------

### Linux Kernel Assembly for MMU Setup

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

This RISC-V assembly code from the Linux kernel (arch/riscv/kernel/head.S) demonstrates the process of loading the trampoline page directory and updating the SATP register. It includes a fence instruction to ensure memory translations are in effect.

```assembly
	/*
	 * Load trampoline page directory, which will cause us to trap to
	 * stvec if VA != PA, or simply fall through if VA == PA.  We need a
	 * full fence here because setup_vm() just wrote these PTEs and we need
	 * to ensure the new translations are in use.
	 */
	la a0, trampoline_pg_dir
	XIP_FIXUP_OFFSET a0
	srl a0, a0, PAGE_SHIFT
	or a0, a0, a1
	sfence.vma
	csrw CSR_SATP, a0

```

--------------------------------

### Build and Run Hypervisor

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Command to build and run the hypervisor. Observe the QEMU output to verify execution.

```bash
$ ./run.sh
```

--------------------------------

### Get Cycles Function

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

A C function that reads the current cycle count from the CSR_TIME register.

```c
static inline cycles_t get_cycles(void)
{
	return csr_read(CSR_TIME);
}

```

--------------------------------

### Define heap area in linker script

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/04-memory-allocation.md

Allocates memory for the heap in the linker script, defining the start (`__heap`) and end (`__heap_end`) symbols for the allocator.

```linker
. += 1024 * 1024; /* 1MB */
    __stack_top = .;

    __heap = .;
    . += 100 * 1024 * 1024; /* 100MB */
    __heap_end = .;

    /DISCARD/ : {
        *(.eh_frame);
    }

```

--------------------------------

### Create and Make run.sh Executable

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Creates a shell script named run.sh and makes it executable. This script is used to launch the QEMU virtual machine.

```bash
$ touch run.sh
$ chmod +x run.sh
```

--------------------------------

### Enable Performance Counters

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Configures the `hcounteren` CSR to enable hardware performance counters, specifically 'cycle' and 'time'. This is part of the trap handler setup.

```rust
                hgatp = in(reg) self.hgatp,
                hedeleg = in(reg) self.hedeleg,
                hcounteren = in(reg) 0b11, /* cycle and time */
                sepc = in(reg) self.sepc,

```

--------------------------------

### Initialize New Rust Project

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Initializes a new binary Rust project named 'hypervisor' using Cargo. This command creates the basic project structure and necessary configuration files.

```bash
$ cargo init --bin hypervisor  
    Creating binary (application) package
note: see more `Cargo.toml` keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
```

--------------------------------

### Dockerfile for Linux Kernel Build Environment

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/08-build-linux-kernel.md

Sets up a Docker image with necessary tools like GCC, binutils, ncurses, flex, and bison for building the Linux kernel. It downloads and extracts the kernel source and copies a default configuration file.

```Dockerfile
FROM ubuntu:24.04

RUN apt-get update && apt-get install -y \
    curl \
    tar \
    build-essential \
    gcc-riscv64-linux-gnu \
    binutils-riscv64-linux-gnu \
    libncurses-dev \
    flex \
    bison \
    bc

RUN curl -fSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.12.34.tar.xz
RUN tar xf linux-6.12.34.tar.xz && \
    mv linux-6.12.34 kernel

WORKDIR /kernel

ENV CROSS_COMPILE=riscv64-linux-gnu-
ENV ARCH=riscv

COPY linux.config .config
```

--------------------------------

### Exit QEMU Monitor

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Demonstrates how to exit the QEMU monitor console. This is done by typing the 'q' command within the monitor interface.

```text
QEMU 8.0.2 monitor - type 'help' for more information
(qemu) q
```

--------------------------------

### Compile and Prepare Guest Kernel

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/06-guest-page-table.md

Shell script to compile the guest assembly code using Clang/LLVM and convert it to a raw binary. It also builds the hypervisor using Cargo.

```shell
#!/bin/sh
set -ev

# macOS users need to install "llvm" in Homebrew:
export PATH="$(brew --prefix llvm)/bin:$PATH"

clang \
    -Wall -Wextra --target=riscv64-unknown-elf -ffreestanding -nostdlib \
    -Wl,-eguest_boot -Wl,-Ttext=0x100000 -Wl,-Map=guest.map \
    guest.S -o guest.elf

llvm-objcopy -O binary guest.elf guest.bin

RUSTFLAGS="-C link-arg=-Thypervisor.ld -C linker=rust-lld" \
  cargo build --bin hypervisor --target riscv64gc-unknown-none-elf

```

--------------------------------

### Use println! for boot message

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/03-hello-world.md

Utilizes the custom `println!` macro to display a boot message to the debug console.

```rust
fn main() -> ! {
    init_bss();
    println!("\nBooting hypervisor...");
    loop {}
}
```

--------------------------------

### Minimal Rust Boot Code

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

This Rust code defines the entry point for the hypervisor, loads the stack pointer, and jumps to the main function. It also initializes the BSS section and enters an infinite loop.

```rust
#![no_std]
#![no_main]

use core::arch::asm;

#[unsafe(no_mangle)]
#[unsafe(link_section = ".text.boot")]
pub extern "C" fn boot() -> ! {
    unsafe {
        asm!(
            "la sp, __stack_top",  // Load __stack_top address into sp
            "j {main}",            // Jump to main
            main = sym main,       // Defines {main} in the assembly code
            options(noreturn)      // No return from this function
        );
    }
}


unsafe extern "C" {
    static mut __bss: u8;
    static mut __bss_end: u8;
}

fn main() -> ! {
    // Fill the BSS section with zeros.
    unsafe {
        let bss_start = &raw mut __bss;
        let bss_size = (&raw mut __bss_end as usize) - (&raw mut __bss as usize);
        core::ptr::write_bytes(bss_start, 0, bss_size);
    }

    // Infinite loop.
    loop {}
}

```

--------------------------------

### RISC-V Assembly for MMU Setup

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

This snippet shows the RISC-V assembly code responsible for updating the SATP (Supervisor Address Translation and Protection) register, which configures the MMU. The instruction immediately following this is where Linux intentionally causes a page fault.

```assembly
ffffffff80000040 <relocate_enable_mmu>:
...
ffffffff80000084: 18051073      csrw    satp, a0
ffffffff80000088: 00000517      auipc   a0, 0x0      <-- page fault here!

```

--------------------------------

### QEMU Control-A Commands

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

Lists common control commands for the QEMU emulator, accessible via the Control-A key combination. These commands allow for various interactions with the emulator, such as exiting, saving state, and toggling timestamps.

```text
C-a h    print this help
C-a x    exit emulator
C-a s    save disk data back to file (if -snapshot)
C-a t    toggle console timestamps
C-a b    send break (magic sysrq)
C-a c    switch between console and monitor
C-a C-a  sends C-a
```

--------------------------------

### Verify build output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/08-build-linux-kernel.md

Lists the files in the linux directory after the build process, showing the generated kernel Image and vmlinux file.

```sh
$ ls -alh linux 
total 99M
drwxr-xr-x  7 seiya staff  224 Jul 24 15:37 .
drwxr-xr-x 24 seiya staff  768 Jul 24 15:37 ..
-rwxr-xr-x  1 seiya staff  304 Jul 24 15:37 build.sh
-rw-r--r--  1 seiya staff  461 Jul 24 15:37 Dockerfile
-rwxr-xr-x  1 seiya staff  5.2M Jul 23 20:12 Image
-rw-r--r--  1 seiya staff  43K Jul 24 15:37 linux.config
-rwxr-xr-x  1 seiya staff  93M Jul 23 20:12 vmlinux
```

--------------------------------

### Linker Script for Hypervisor Memory Layout

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

This linker script defines the memory layout for the hypervisor, including the entry point, section placement, and stack top address. It ensures the boot section is at the beginning and discards exception handling frames.

```linker script
ENTRY(boot)

SECTIONS {
    . = 0x80200000;

    .text :{
        KEEP(*(.text.boot)); // Keep `boot` function at the beginning
        *(.text .text.*);
    }

    .rodata : ALIGN(8) {
        *(.rodata .rodata.*);
    }

    .data : ALIGN(8) {
        *(.data .data.*);
    }

    .bss : ALIGN(8) {
        __bss = .;
        *(.bss .bss.* .sbss .sbss.*);
        __bss_end = .;
    }

    . = ALIGN(16);
    . += 1024 * 1024; /* 1MB */
    __stack_top = .;

    /DISCARD/ {
        *(.eh_frame);
    }
}

```

--------------------------------

### Guest SBI Calls

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

These assembly instructions demonstrate how a guest program can make SBI calls to the hypervisor. Each `ecall` instruction triggers a hypercall with a character ('A', 'B', or 'C') passed as an argument in register a0.

```asm
    li a0, 'A'      # Parameter: 'A'
    ecall           # Call SBI (hypervisor)
    li a0, 'B'      # Parameter: 'B'
    ecall           # Call SBI (hypervisor)
    li a0, 'C'      # Parameter: 'C'
    ecall           # Call SBI (hypervisor)
```

--------------------------------

### Build and Run Script for Hypervisor

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/02-boot.md

This shell script compiles the hypervisor using Cargo with specific Rust flags for linking and targets the RISC-V architecture. It then copies the executable and runs it using QEMU.

```shell
#!/bin/sh
set -ev

RUSTFLAGS="-C link-arg=-Thypervisor.ld -C linker=rust-lld" \
  cargo build --bin hypervisor --target riscv64gc-unknown-none-elf

cp target/riscv64gc-unknown-none-elf/debug/hypervisor hypervisor.elf

qemu-system-riscv64 \
    -machine virt \
    -cpu rv64 \
    -bios default \
    -smp 1 \
    -m 128M \
    -nographic \
    -d cpu_reset,unimp,guest_errors,int -D qemu.log \
    -serial mon:stdio \
    --no-reboot \
    -kernel hypervisor.elf

```

--------------------------------

### Enable Hypervisor Extension in QEMU

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/05-guest-mode.md

To enable the hypervisor extension for RISC-V, add `h=true` to the `-cpu` option when launching QEMU.

```shell
    -cpu rv64,h=true \

```

--------------------------------

### Minimal Guest Kernel Assembly

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/06-guest-page-table.md

A basic assembly program for the guest that creates an infinite loop. This serves as the entry point for the guest kernel.

```assembly
.section .text
.global guest_boot
guest_boot:
    j guest_boot

```

--------------------------------

### Guest Assembly: Call SBI Console Putchar

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

This assembly code demonstrates calling the SBI's Console Putchar extension from the guest to print the character 'A'. It sets up the necessary registers for the hypercall and then executes the ecall instruction.

```asm
.section .text
.global guest_boot
guest_boot:
    li a7, 1        # Extension ID: 1 (legacy console)
    li a6, 0        # Function ID: 0 (putchar)
    li a0, 'A'      # Parameter: 'A'
    ecall           # Call SBI (hypervisor)

halt:
    j halt          # Infinite loop

```

--------------------------------

### Load and Map Guest Kernel in Rust

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/06-guest-page-table.md

Rust code to load the compiled guest kernel binary into memory and map it into the guest page table. This prepares the guest environment for execution.

```rust
use crate::{
    allocator::alloc_pages,
    guest_page_table::{GuestPageTable, PTE_R, PTE_W, PTE_X},
};

fn main() -> ! {
    /* ... */

    let kernel_image = include_bytes!("../guest.bin");
    let guest_entry = 0x100000;

    // Copy guest kernel to a guest memory buffer.
    let kernel_memory = alloc_pages(kernel_image.len());
    unsafe {
        let dst = kernel_memory as *mut u8;
        let src = kernel_image.as_ptr();
        core::ptr::copy_nonoverlapping(src, dst, kernel_image.len());
    }

    // Map the guest memory into the guest page table.
    let mut table = GuestPageTable::new();
    table.map(guest_entry, kernel_memory as u64, PTE_R | PTE_W | PTE_X);

    let mut hstatus = 0;
    hstatus |= 2u64 << 32; // VSXL: XLEN for VS-mode (64-bit)
    hstatus |= 1u64 << 7; // SPV: Supervisor Previous Virtualization mode (HS-mode)

    unsafe {
        asm!(
            "csrw hstatus, {hstatus}",
            "csrw hgatp, {hgatp}",
            "csrw sepc, {sepc}",
            "sret",
            hstatus = in(reg) hstatus,
            hgatp = in(reg) table.hgatp(),
            sepc = in(reg) guest_entry,
        );
    }
}

```

--------------------------------

### Add vm-fdt Dependency for Device Tree

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

This snippet shows the Cargo.toml configuration to include the `vm-fdt` crate, which is necessary for building the device tree.

```toml
[dependencies]
vm-fdt = { version = "0.3.0", features = ["alloc"], default-features = false }
```

--------------------------------

### Load Linux Kernel and Device Tree

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

Loads the Linux kernel image and the generated device tree into guest memory. It asserts the kernel image header and checks the size of the device tree.

```rust
pub const GUEST_DTB_ADDR: u64 = 0x7000_0000;

pub fn load_linux_kernel(table: &mut GuestPageTable, image: &[u8]) {
    assert!(image.len() >= size_of::<RiscvImageHeader>());
    let header = unsafe { &*(image.as_ptr() as *const RiscvImageHeader) };
    assert_eq!(u32::from_le(header.magic2), 0x05435352, "invalid magic");

    let kernel_size = u64::from_le(header.image_size);
    assert!(image.len() <= MEMORY_SIZE);
    copy_and_map(table, image, GUEST_BASE_ADDR, MEMORY_SIZE, PTE_R | PTE_W | PTE_X);

    let dtb = build_device_tree().unwrap();
    assert!(dtb.len() <= 0x10000, "DTB is too large");
    copy_and_map(table, &dtb, GUEST_DTB_ADDR, dtb.len(), PTE_R);

    println!("loaded kernel: size={}KB", kernel_size / 1024);
}

```

--------------------------------

### Print "Hi!" using sbi_putchar

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/03-hello-world.md

Demonstrates using the `sbi_putchar` function to print the string "Hi!" to the debug console.

```rust
fn main() -> ! {
    /* ... */
    print::sbi_putchar(b'H');
    print::sbi_putchar(b'i');
    print::sbi_putchar(b'!');
    print::sbi_putchar(b'\n');
    loop {}
}
```

--------------------------------

### Use `Vec` with the bump allocator

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/04-memory-allocation.md

Demonstrates using `alloc::vec::Vec` after initializing the bump allocator. The `Vec` can be populated and printed, showcasing dynamic memory allocation.

```rust
fn main() -> ! {
    /* ... */

    allocator::GLOBAL_ALLOCATOR.init(&raw mut __heap, &raw mut __heap_end);

    let mut v = alloc::vec::Vec::new();
    v.push('a');
    v.push('b');
    v.push('c');
    println!("v = {:?}", v);

    loop {}
}

```

--------------------------------

### Implement SBI Set Timer

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Handles the SBI Set Timer call. Currently, it prints a warning and ignores the call, returning Ok(0).

```rust
    let result: Result<i64, i64> = match (eid, fid) {
        // Set Timer
        (0x00, 0x0) => {
            println!("[sbi] WARN: set_timer is not implemented, ignoring");
            Ok(0)
        }

```

--------------------------------

### Enable `alloc` crate in `main.rs`

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/04-memory-allocation.md

Enables the `alloc` crate by adding `extern crate alloc;` to `main.rs`. This makes standard library features requiring dynamic allocation available.

```rust
#![no_std]
#![no_main]

extern crate alloc;

```

--------------------------------

### Guest Page Table Implementation in Rust

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/06-guest-page-table.md

Implements a guest page table structure for managing guest memory. It includes definitions for page table entries and tables, along with methods for allocation, address translation, and mapping guest physical addresses to host physical addresses.

```Rust
use core::mem::size_of;

pub const PTE_R: u64 = 1 << 1; /* Readable */
pub const PTE_W: u64 = 1 << 2; /* Writable */
pub const PTE_X: u64 = 1 << 3; /* Executable */
const PTE_V: u64 = 1 << 0; /* Valid */
const PTE_U: u64 = 1 << 4; /* User */
const PPN_SHIFT: usize = 12;
const PTE_PPN_SHIFT: usize = 10;

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(transparent)]
struct Entry(u64);

impl Entry {
    pub fn new(paddr: u64, flags: u64) -> Self {
        let ppn = (paddr as u64) >> PPN_SHIFT;
        Self(ppn << PTE_PPN_SHIFT | flags)
    }

    pub fn is_valid(&self) -> bool {
        self.0 & PTE_V != 0
    }

    pub fn paddr(&self) -> u64 {
        (self.0 >> PTE_PPN_SHIFT) << PPN_SHIFT
    }
}

#[repr(transparent)]
struct Table([Entry; 512]);

impl Table {
    pub fn alloc() -> *mut Table {
        crate::allocator::alloc_pages(size_of::<Table>()) as *mut Table
    }

    pub fn entry_by_addr(&mut self, guest_paddr: u64, level: usize) -> &mut Entry {
        let index = (guest_paddr >> (12 + 9 * level)) & 0x1ff; // extract 9-bits index
        &mut self.0[index as usize]
    }
}

pub struct GuestPageTable {
    table: *mut Table,
}

impl GuestPageTable {
    pub fn new() -> Self {
        Self {
            table: Table::alloc(),
        }
    }

    pub fn hgatp(&self) -> u64 {
        (9u64 << 60/* Sv48x4 */) | (self.table as u64 >> PPN_SHIFT)
    }

    pub fn map(&mut self, guest_paddr: u64, host_paddr: u64, flags: u64) {
        let mut table = unsafe { &mut *self.table };
        for level in (1..=3).rev() { // level = 3, 2, 1
            let entry = table.entry_by_addr(guest_paddr, level);
            if !entry.is_valid() {
                let new_table_ptr = Table::alloc();
                *entry = Entry::new(new_table_ptr as u64, PTE_V);
            }

            table = unsafe { &mut *(entry.paddr() as *mut Table) };
        }

        let entry = table.entry_by_addr(guest_paddr, 0);
        println!("map: {:08x} -> {:08x}", guest_paddr, host_paddr);
        assert!(!entry.is_valid(), "already mapped");
        *entry = Entry::new(host_paddr, flags | PTE_V | PTE_U);
    }
}
```

--------------------------------

### Execute the build script

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/08-build-linux-kernel.md

Command to run the shell script that builds the Linux kernel within a Docker container.

```sh
./linux/build.sh
```

--------------------------------

### Handle SBI Call and Resume Guest

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

This Rust code snippet shows how to handle an SBI (Supervisor Binary Interface) call within the trap handler. It prints the call details, advances the guest's program counter, and then calls `vcpu.run()` to resume guest execution.

```rust
    if scause == 10 {
        println!("SBI call: eid={:#x}, fid={:#x}, a0={:#x} ('{}')", vcpu.a7, vcpu.a6, vcpu.a0, vcpu.a0 as u8 as char);
        vcpu.sepc = sepc + 4; // Resume the guest after ECALL instruction.
    } else {
        panic!("trap handler: {} at {:#x} (stval={:#x})", scause_str, sepc, stval);
    }

    vcpu.run();
```

--------------------------------

### Enable println! macro

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/03-hello-world.md

Enable the custom `println!` macro by declaring the `print` module with the `#[macro_use]` attribute in `main.rs`.

```rust
#[macro_use]
mod print;
```

--------------------------------

### Shell script to build Linux kernel using Docker

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/08-build-linux-kernel.md

Automates the Docker build process for the Linux kernel and runs the build inside the container. It mounts the current directory to copy the resulting kernel image and vmlinux file.

```bash
#!/bin/bash
cd "$(dirname "$0")"

docker build -t guest-linux-builder -f Dockerfile .

# Build Linux kernel, and copy the Image to this directory.
docker run -v $PWD:/linux -it guest-linux-builder \
    bash -c 'make -j$(nproc) Image && cp arch/riscv/boot/Image /linux/Image && cp vmlinux /linux/vmlinux'
```

--------------------------------

### Implement sbi_probe_extension

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Adds the `sbi_probe_extension` call (eid=0x10, fid=0x3) to the SBI handler, returning an error code (-1) to indicate the extension is not supported.

```rust
    let result: Result<i64, i64> = match (eid, fid) {
        // Get SBI specification version
        (0x10, 0x0) => Ok(0),
        // Probe SBI extension
        (0x10, 0x3) => Err(-1),
        // Console Putchar.
        (0x1, 0x0) => {

```

--------------------------------

### Guest SBI Calls for State Restoration

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

These SBI calls are used by the guest to request specific state restoration actions from the hypervisor. 'B' typically signifies a begin operation, and 'C' a commit or continue operation.

```assembly
SBI call: eid=0x1, fid=0x0, a0=0x42 ('B')
SBI call: eid=0x1, fid=0x0, a0=0x43 ('C')
```

--------------------------------

### Rust: VCpu Struct for Guest State Management

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

Defines the VCpu struct to manage guest CPU state, including hypervisor and supervisor control and status registers. It provides a constructor to initialize these registers and a run method to enter guest mode.

```rust
use core::arch::asm;

use crate::{allocator::alloc_pages, guest_page_table::GuestPageTable};

#[derive(Debug, Default)]
pub struct VCpu {
    pub hstatus: u64,
    pub hgatp: u64,
    pub sstatus: u64,
    pub sepc: u64,
}

impl VCpu {
    pub fn new(table: &GuestPageTable, guest_entry: u64) -> Self {
        let mut hstatus: u64 = 0;
        hstatus |= 2 << 32; // VSXL: XLEN for VS-mode (64-bit)
        hstatus |= 1 << 7; // SPV: Supervisor Previous Virtualization mode

        let sstatus: u64 = 1 << 8; // SPP: Supervisor Previous Privilege mode (VS-mode)

        Self {
            hstatus,
            hgatp: table.hgatp(),
            sstatus,
            sepc: guest_entry,
            ..Default::default()
        }
    }

    pub fn run(&mut self) -> ! {
        unsafe {
            asm!(
                "csrw hstatus, {hstatus}",
                "csrw sstatus, {sstatus}",
                "csrw sscratch, {sscratch}",
                "csrw hgatp, {hgatp}",
                "csrw sepc, {sepc}",
                "sret",
                hstatus = in(reg) self.hstatus,
                sstatus = in(reg) self.sstatus,
                hgatp = in(reg) self.hgatp,
                sepc = in(reg) self.sepc,
                sscratch = in(reg) (self as *mut VCpu as usize),
            );
        }

        unreachable!();
    }
}

```

--------------------------------

### Address to Line Information

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Uses `llvm-addr2line` to convert a virtual address from a kernel panic trace to a source code location.

```text
$ llvm-addr2line -e linux/vmlinux 0xffffffff8022674e
/kernel/./arch/riscv/include/asm/timex.h:53 (discriminator 1)
```

--------------------------------

### Linux Kernel Image Header and Loading Logic

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/09-boot-linux.md

Defines the RiscvImageHeader structure and the `load_linux_kernel` function. This function parses the kernel image header, copies the kernel data to allocated memory, and maps it into the guest's address space with read, write, and execute permissions. It asserts the validity of the kernel image magic number and checks for memory size constraints.

```Rust
use crate::allocator::alloc_pages;
use crate::guest_page_table::{GuestPageTable, PTE_R, PTE_W, PTE_X};
use core::mem::size_of;

#[repr(C)]
struct RiscvImageHeader {
    code0: u32,
    code1: u32,
    text_offset: u64,
    image_size: u64,
    flags: u64,
    version: u32,
    reserved1: u32,
    reserved2: u64,
    magic: u64,
    magic2: u32,
    reserved3: u32,
}

pub const GUEST_BASE_ADDR: u64 = 0x8000_0000;
pub const MEMORY_SIZE: usize = 64 * 1024 * 1024;

fn copy_and_map(
    table: &mut GuestPageTable,
    data: &[u8],
    guest_addr: u64,
    len: usize,
    flags: u64,
) {
    // Allocate a memory region, and copy the data to it.
    assert!(data.len() <= len, "data is beyond the region");
    let raw_ptr = alloc_pages(len);
    unsafe {
        core::ptr::copy_nonoverlapping(data.as_ptr(), raw_ptr, data.len());
    }

    // Map the memory region to the guest's address space.
    let host_addr = raw_ptr as u64;
    for off in (0..len).step_by(4096) {
        table.map(guest_addr + off as u64, host_addr + off as u64, flags);
    }
}

pub fn load_linux_kernel(table: &mut GuestPageTable, image: &[u8]) {
    assert!(image.len() >= size_of::<RiscvImageHeader>());
    let header = unsafe { &*(image.as_ptr() as *const RiscvImageHeader) };
    assert_eq!(u32::from_le(header.magic2), 0x05435352, "invalid magic");

    let kernel_size = u64::from_le(header.image_size);
    assert!(image.len() <= MEMORY_SIZE);

    copy_and_map(
        table,
        image,
        GUEST_BASE_ADDR,
        MEMORY_SIZE,
        PTE_R | PTE_W | PTE_X,
    );

    println!("loaded kernel: size={}KB", kernel_size / 1024);
}
```

--------------------------------

### Save Guest State in trap_handler (Assembly)

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/07-hello-from-guest.md

Assembly code to save the guest's CPU state, including general-purpose registers and the return address, onto the hypervisor's stack. It also switches the stack pointer to the hypervisor's stack before calling the trap handler.

```assembly
        // Swap a0 and sscratch.
        "csrrw a0, sscratch, a0",

        // a0 is now a pointer to a VCpu. Save registers except a0.
        "sd ra, {ra_offset}(a0)",
        "sd sp, {sp_offset}(a0)",
        "sd gp, {gp_offset}(a0)",
        "sd tp, {tp_offset}(a0)",
        "sd t0, {t0_offset}(a0)",
        "sd t1, {t1_offset}(a0)",
        "sd t2, {t2_offset}(a0)",
        "sd s0, {s0_offset}(a0)",
        "sd s1, {s1_offset}(a0)",
        "sd a1, {a1_offset}(a0)",
        "sd a2, {a2_offset}(a0)",
        "sd a3, {a3_offset}(a0)",
        "sd a4, {a4_offset}(a0)",
        "sd a5, {a5_offset}(a0)",
        "sd a6, {a6_offset}(a0)",
        "sd a7, {a7_offset}(a0)",
        "sd s2, {s2_offset}(a0)",
        "sd s3, {s3_offset}(a0)",
        "sd s4, {s4_offset}(a0)",
        "sd s5, {s5_offset}(a0)",
        "sd s6, {s6_offset}(a0)",
        "sd s7, {s7_offset}(a0)",
        "sd s8, {s8_offset}(a0)",
        "sd s9, {s9_offset}(a0)",
        "sd s10, {s10_offset}(a0)",
        "sd s11, {s11_offset}(a0)",
        "sd t3, {t3_offset}(a0)",
        "sd t4, {t4_offset}(a0)",
        "sd t5, {t5_offset}(a0)",
        "sd t6, {t6_offset}(a0)",

        // Restore a0 from sscratch, and save in to VCpu.
        "csrr t0, sscratch",
        "sd t0, {a0_offset}(a0)",

        // Switch to the hypervisor's stack.
        "ld sp, {host_sp_offset}(a0)",

        // a0 (first argument) is still the vcpu pointer here.
        "call {handle_trap}",
        handle_trap = sym handle_trap,
        host_sp_offset = const offset_of!(VCpu, host_sp),
        ra_offset = const offset_of!(VCpu, ra),
        sp_offset = const offset_of!(VCpu, sp),
        gp_offset = const offset_of!(VCpu, gp),
        tp_offset = const offset_of!(VCpu, tp),
        t0_offset = const offset_of!(VCpu, t0),
        t1_offset = const offset_of!(VCpu, t1),
        t2_offset = const offset_of!(VCpu, t2),
        s0_offset = const offset_of!(VCpu, s0),
        s1_offset = const offset_of!(VCpu, s1),
        a0_offset = const offset_of!(VCpu, a0),
        a1_offset = const offset_of!(VCpu, a1),
        a2_offset = const offset_of!(VCpu, a2),
        a3_offset = const offset_of!(VCpu, a3),
        a4_offset = const offset_of!(VCpu, a4),
        a5_offset = const offset_of!(VCpu, a5),
        a6_offset = const offset_of!(VCpu, a6),
        a7_offset = const offset_of!(VCpu, a7),
        s2_offset = const offset_of!(VCpu, s2),
        s3_offset = const offset_of!(VCpu, s3),
        s4_offset = const offset_of!(VCpu, s4),
        s5_offset = const offset_of!(VCpu, s5),
        s6_offset = const offset_of!(VCpu, s6),
        s7_offset = const offset_of!(VCpu, s7),
        s8_offset = const offset_of!(VCpu, s8),
        s9_offset = const offset_of!(VCpu, s9),

```

--------------------------------

### Implement Buffered Console Output

Source: https://github.com/nuta/hypervisor-in-1000-lines/blob/main/website/en/10-supervisor-binary-interface.md

Implements buffered console output for `sbi_console_putchar`. It collects characters and prints them as a line when a newline is encountered, clearing the buffer.

```rust
use alloc::vec::Vec;
use spin::Mutex;

static CONSOLE_BUFFER: Mutex<Vec<u8>> = Mutex::new(Vec::new());

fn handle_sbi_call(vcpu: &mut VCpu) {
    let eid = vcpu.a7;
    let fid = vcpu.a6;
    let result: Result<i64, i64> = match (eid, fid) {
        // Get SBI specification version
        (0x10, 0x0) => Ok(0),
        // Console Putchar.
        (0x1, 0x0) => {
            let ch = vcpu.a0 as u8;
            let mut buffer = CONSOLE_BUFFER.lock();
            if ch == b'\n' {
                let output = core::str::from_utf8(&buffer).unwrap_or("(not utf-8)");
                println!("[guest] {}", output);
                buffer.clear();
            } else {
                buffer.push(ch);
            }
            Ok(0)
        }
        _ => {
            panic!("unknown SBI call: eid={:#x}, fid={:#x}", eid, fid);
        }
    };
}
```