### Example Pointer Printouts Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-alignment.md Sample pointer printouts demonstrating memory alignment for different data types on x86_64, showing typical allocator behavior. ```text p=0x7fb78b421028 u8 p=0x7fb78b421030 u16 p=0x7fb78b421038 u32 p=0x7fb78b421050 u64 p=0x7fb78b420060 "spam" p=0x7fb78b4220f0 Hoge { y: 2, z: "ほげ", x: 1 } ``` -------------------------------- ### Interpreter Mutator Implementation Example in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md An example implementation of the Mutator trait for an Interpreter struct, demonstrating heap allocation and scoped pointer usage. ```rust struct Stack {} impl Stack { fn say_hello(&self) { println!("I'm the stack!"); } } struct Roots { stack: CellPtr } impl Roots { fn new(stack: ScopedPtr<'_, Stack>) -> Roots { Roots { stack: CellPtr::new_with(stack) } } } struct Interpreter {} impl Mutator for Interpreter { type Input: (); type Output: Roots; fn run(&self, mem: &MutatorView, input: Self::Input) -> Result { let stack = mem.alloc(Stack {})?; stack.say_hello(); let roots = Roots::new(stack); let stack_ptr = roots.stack.get(mem); stack_ptr.say_hello(); Ok(roots) } } fn main() { ... let interp = Interpreter {}; let result = memory.mutate(&interp, ()); let roots = result.unwrap(); // no way to do this - compile error let stack = roots.stack.get(); ... } ``` -------------------------------- ### Define a named function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-design.md Example of a function definition with a name and one argument. ```Lisp-like (def is_true (x) (is? x true)) ``` -------------------------------- ### Access RawArray Capacity and Pointer in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-arrays.md Provides methods to get the capacity and the raw pointer of a `RawArray`. `as_ptr()` returns an `Option>` representing the start of the array's data. ```rust impl RawArray { pub fn capacity(&self) -> usize { self.capacity } pub fn as_ptr(&self) -> Option> { self.ptr } } ``` -------------------------------- ### Define an anonymous function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-design.md Example of a lambda function definition without a name. ```Lisp-like (lambda (x) (is? x true)) ``` -------------------------------- ### Get Next Opcode Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Retrieves the next opcode from the current instruction stream. ```rust {{#include ../interpreter/src/bytecode.rs:DefInstructionStreamGetNextOpcode}} ``` -------------------------------- ### Access Block Pointer Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-blocks.md Provides a raw pointer to the start of the allocated memory block. ```rust {{#include ../blockalloc/src/lib.rs:BlockAsPtr}} ``` -------------------------------- ### Iterate Over Lines Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-simple-bump.md Begins the loop to identify holes by iterating backwards from the starting line. ```rust for index in (0..starting_line).rev() { let marked = unsafe { *self.lines.add(index) }; ``` -------------------------------- ### Implement TaggedCellPtr get Method Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Retrieves a TaggedScopedPtr from a TaggedCellPtr. ```rust impl TaggedCellPtr { {{#include ../interpreter/src/safeptr.rs:DefTaggedCellPtrGet}} } ``` -------------------------------- ### Define a closure Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-design.md Example of a function that captures a nonlocal variable from an outer scope. ```Lisp-like (def make_adder (n) (lambda (x) (+ x n)) ) ``` -------------------------------- ### Symbol As String Method Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-symbols-and-pairs.md Provides a safe method to get the string representation of a Symbol when provided with a MutatorScope guard, ensuring safe access to the underlying string data. ```rust pub fn as_str(&self, _scope: &MutatorScope) -> &'static str { self.name } ``` -------------------------------- ### Initialize and Run REPL Source: https://context7.com/rust-hosted-langs/book/llms.txt Demonstrates how to initialize the memory environment and execute an interactive read-eval-print loop. ```rust use crate::memory::Memory; use crate::repl::RepMaker; fn main() { let mem = Memory::new(); let rep_maker = RepMaker {}; // Initialize REPL environment let rep = mem.mutate(&rep_maker, ())?; // Evaluate expressions interactively loop { let line = read_line("> "); mem.mutate(&rep, line)?; } } ``` -------------------------------- ### Running the REPL Source: https://context7.com/rust-hosted-langs/book/llms.txt Information on how to run the Read-Eval-Print-Loop (REPL) for interactive evaluation of expressions within the interpreter. ```APIDOC ## Running the REPL The interpreter includes a Read-Eval-Print-Loop for interactive evaluation of expressions. ### Initialization and Evaluation 1. Initialize the memory and REPL maker. 2. Create the REPL environment using `mem.mutate(&rep_maker, ())?`. 3. Enter an infinite loop to read user input. 4. Evaluate each line of input using `mem.mutate(&rep, line)?`. ### Example REPL Session ``` > (def fact (n) (cond (is? n 0) 1 true (* n (fact (- n 1))))) > (fact 5) 120 > (def map (f l) (cond (nil? l) nil true (cons (f (car l)) (map f (cdr l))))) > (map (lambda (x) (* x 2)) '(1 2 3 4 5)) (2 4 6 8 10) ``` ``` -------------------------------- ### Define VM Opcodes in Rust Source: https://context7.com/rust-hosted-langs/book/llms.txt Lists the available bytecode instructions for the register-based VM, including control flow, memory operations, and closure management. ```rust use crate::bytecode::{Opcode, Register, LiteralId, JumpOffset}; // Key opcodes and their semantics: // Control flow Opcode::Return { reg: Register } // Return value from register Opcode::Jump { offset: JumpOffset } // Unconditional jump Opcode::JumpIfTrue { test: Register, offset } // Jump if test == 'true Opcode::JumpIfNotTrue { test: Register, offset } // Jump if test != 'true // Load operations Opcode::LoadNil { dest: Register } // dest = nil Opcode::LoadInteger { dest, integer: i16 } // dest = inline integer Opcode::LoadLiteral { dest, literal_id: LiteralId } // dest = literals[id] Opcode::LoadGlobal { dest, name: Register } // dest = globals[name] Opcode::StoreGlobal { src, name: Register } // globals[name] = src // Pair operations (Lisp car/cdr/cons) Opcode::FirstOfPair { dest, reg } // dest = car(reg) Opcode::SecondOfPair { dest, reg } // dest = cdr(reg) Opcode::MakePair { dest, reg1, reg2 } // dest = cons(reg1, reg2) // Type tests Opcode::IsNil { dest, test } // dest = (test == nil) ? 'true : nil Opcode::IsAtom { dest, test } // dest = atom?(test) ? 'true : nil Opcode::IsIdentical { dest, test1, test2 } // dest = (test1 == test2) ? 'true : nil // Function calls Opcode::Call { function, dest, arg_count } // call function with args Opcode::MakeClosure { dest, function } // create closure from function // Closure upvalue operations Opcode::GetUpvalue { dest, src: UpvalueId } // dest = upvalues[src] Opcode::SetUpvalue { dest: UpvalueId, src } // upvalues[dest] = src Opcode::CloseUpvalues { reg1, reg2, reg3 } // close upvalues for registers // Register operations Opcode::CopyRegister { dest, src } // dest = src ``` -------------------------------- ### Implement Block Creation Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-blocks.md Constructor for creating a new memory block, requiring a power-of-two size for alignment. ```rust {{#include ../blockalloc/src/lib.rs:BlockNew}} ``` -------------------------------- ### Thread Execution Loop in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-vm-impl.md Illustrates the primary control structure of the VM's execution loop, which iterates through instructions and uses a match expression to decode and execute them. ```rust impl Thread { pub fn eval_next_instr(&mut self) { let mut frame = self.frames.borrow_mut(); let ip = frame.ip; let bytecode = &frame.function.prototype.code; let opcode = bytecode.get(ip); frame.ip += 1; match opcode { // ... cases for each opcode ... } } } ``` -------------------------------- ### Implement TaggedPtr Instantiation Methods Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Helper functions for instantiating TaggedPtr variants including Nil, Number, Symbol, and Pair. ```rust impl TaggedPtr { {{#include ../interpreter/src/taggedptr.rs:DefTaggedPtrNil}} {{#include ../interpreter/src/taggedptr.rs:DefTaggedPtrNumber}} {{#include ../interpreter/src/taggedptr.rs:DefTaggedPtrSymbol}} {{#include ../interpreter/src/taggedptr.rs:DefTaggedPtrPair}} } ``` -------------------------------- ### Implement CellPtr Get Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md Obtains a ScopedPtr from a CellPtr to allow safe dereferencing. ```rust impl CellPtr { {{#include ../interpreter/src/safeptr.rs:DefCellPtrGet}} } ``` -------------------------------- ### Implement From FatPtr for TaggedPtr Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Conversion implementation from FatPtr to TaggedPtr. ```rust {{#include ../interpreter/src/taggedptr.rs:DefFromFatPtrForTaggedPtr}} ``` -------------------------------- ### Implement switch-style instruction dispatch Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-vm-design.md Uses a loop and a match expression to handle bytecode dispatching, which is the recommended cross-platform approach in Rust. ```rust loop { let opcode = get_next_opcode(); match opcode { Opcode::Add(a, x, y) => { ... }, Opcode::Call(f, r, p) => { ... }, } } ``` -------------------------------- ### Define Header Constructor Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-allocation-api.md Method signature for initializing an object header. ```rust pub trait AllocHeader: Sized { ... fn new>( size: u32, size_class: SizeClass, mark: Mark ) -> Self; ... } ``` -------------------------------- ### Handle Unmarked Lines Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-simple-bump.md Increments the counter for unmarked lines and checks for a valid hole at the start of the block. ```rust if marked == 0 { count += 1; if index == 0 && count >= lines_required { let limit = index * constants::LINE_SIZE; let cursor = end * constants::LINE_SIZE; return Some((cursor, limit)); } } else { ``` -------------------------------- ### Implement Sticky Immix heap allocation Source: https://context7.com/rust-hosted-langs/book/llms.txt Configures a custom heap using the Sticky Immix algorithm by defining type IDs, object headers, and allocation logic. ```rust use stickyimmix::{ AllocError, AllocHeader, AllocObject, AllocRaw, AllocTypeId, ArraySize, Mark, RawPtr, SizeClass, StickyImmixHeap }; // Define your type system #[derive(PartialEq, Copy, Clone)] enum MyTypeId { String, Number, Array, } impl AllocTypeId for MyTypeId {} // Implement AllocObject for your types impl AllocObject for String { const TYPE_ID: MyTypeId = MyTypeId::String; } impl AllocObject for usize { const TYPE_ID: MyTypeId = MyTypeId::Number; } // Define a custom object header struct MyHeader { size_class: SizeClass, mark: Mark, type_id: MyTypeId, size_bytes: u32, } impl AllocHeader for MyHeader { type TypeId = MyTypeId; fn new>(size: u32, size_class: SizeClass, mark: Mark) -> Self { MyHeader { size_class, mark, type_id: O::TYPE_ID, size_bytes: size } } fn new_array(size: ArraySize, size_class: SizeClass, mark: Mark) -> Self { MyHeader { size_class, mark, type_id: MyTypeId::Array, size_bytes: size } } fn mark(&mut self) { self.mark = Mark::Marked; } fn is_marked(&self) -> bool { self.mark == Mark::Marked } fn size_class(&self) -> SizeClass { self.size_class } fn size(&self) -> u32 { self.size_bytes } fn type_id(&self) -> MyTypeId { self.type_id } } // Create heap and allocate objects let heap = StickyImmixHeap::::new(); // Allocate a String object let string_ptr: RawPtr = heap.alloc(String::from("hello"))?; let value = unsafe { string_ptr.as_ref() }; assert_eq!(*value, "hello"); // Allocate a number let num_ptr: RawPtr = heap.alloc(42usize)?; // Allocate a byte array (zero-initialized) let array_ptr: RawPtr = heap.alloc_array(2048)?; // Access object header from object pointer let header_ptr = StickyImmixHeap::::get_header(string_ptr.as_untyped()); let header = unsafe { &*header_ptr.as_ptr() }; assert_eq!(header.type_id(), MyTypeId::String); ``` -------------------------------- ### Get Array Element Offset Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-arrays.md Retrieves a pointer to a memory location within an array based on an index, performing bounds checks. ```rust impl Array { } ``` -------------------------------- ### Compile AST to Bytecode in Rust Source: https://context7.com/rust-hosted-langs/book/llms.txt Compiles various AST structures into bytecode for the virtual machine. Supports expressions, function definitions, conditionals, closures, and let bindings. Ensure necessary imports are present. ```rust use crate::compiler::compile; use crate::parser::parse; use crate::bytecode::{ByteCode, Opcode, Register}; use crate::function::Function; fn compile_example(mem: &MutatorView) -> Result<(), RuntimeError> { // Compile a simple expression let ast = parse(mem, "(nil? nil)")?; let function: ScopedPtr = compile(mem, ast)?; // Compile a function definition let ast = parse(mem, "(def square (x) (* x x))")?; let main_fn = compile(mem, ast)?; // Compile conditional expressions let ast = parse(mem, r#" (cond (nil? x) 'is-nil (atom? x) 'is-atom true 'is-list) "#)?; let cond_fn = compile(mem, ast)?; // Compile lambda with closure let ast = parse(mem, r#" (def make-counter (start) (let ((count start)) (lambda () (set 'result count) result))) "#)?; let closure_fn = compile(mem, ast)?; // Compile let binding let ast = parse(mem, r#" (let ((a 1) (b 2)) (cons a b)) "#)?; let let_fn = compile(mem, ast)?; // Access function properties println!("Arity: {}", function.arity()); println!("Is closure: {}", function.is_closure()); Ok(()) } ``` -------------------------------- ### Set Initial Boundary Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-simple-bump.md Sets the initial end boundary for the hole search. ```rust let mut end = starting_line; ``` -------------------------------- ### Implement AllocRaw::alloc() for StickyImmixHeap Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-allocation-impl.md Implements the alloc() function which allocates space for an object and its header, instantiates the header, copies the object, and returns a pointer to the object's location. Requires space calculation, allocation, and header instantiation. ```rust impl AllocRaw for StickyImmixHeap { type Header = H; fn alloc(&mut self, obj: T) -> NonNull { let space = self.find_space(size_of::()); let header_ptr = space as *mut Self::Header; let obj_ptr = unsafe { (space + size_of::()) as *mut T }; unsafe { write(header_ptr, Self::Header::new()); write(obj_ptr, obj); } NonNull::new(obj_ptr).unwrap() } ... } ``` -------------------------------- ### InstructionStream Switch Frame Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Implementation for switching between bytecode instances. ```rust impl InstructionStream { {{#include ../interpreter/src/bytecode.rs:DefInstructionStreamSwitchFrame}} } ``` -------------------------------- ### Create RawArray with Capacity in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-arrays.md Implements `RawArray::with_capacity()`. This method allocates the backing store for the array using a `MutatorView` and initializes the `RawArray` struct. It requires `MutatorView` for allocation. ```rust impl RawArray { pub fn with_capacity(mutator: &MutatorView, capacity: usize) -> Result { let len_bytes = capacity.checked_mul(std::mem::size_of::()) .ok_or(AllocError::CapacityOverflow)?; let ptr = if capacity == 0 { None } else { mutator.alloc_array(len_bytes)? .cast::().into() // Cast to T }; Ok(RawArray { ptr, capacity }) } } ``` -------------------------------- ### Compile nil? Function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-impl.md This snippet shows the implementation of the nil? function compilation logic. ```rust ... {{#include ../interpreter/src/compiler.rs:DefCompileApplyIsNil}} ... ``` -------------------------------- ### Bytecode Execution Support Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Functions and structures necessary for executing bytecode, including managing instruction streams and switching execution contexts. ```APIDOC ## Bytecode Execution Support This section covers the components required for executing the compiled bytecode. ### `InstructionStream` A struct representing the execution view of bytecode, including a pointer to the current instruction sequence and an instruction pointer. It is designed to allow switching between different `ByteCode` instances, facilitating function calls. ```rust,ignore {{#include ../interpreter/src/bytecode.rs:DefInstructionStream}} ``` ### `InstructionStream::switch_frame` Methods to support switching execution frames within the `InstructionStream`. ```rust,ignore impl InstructionStream { {{#include ../interpreter/src/bytecode.rs:DefInstructionStreamSwitchFrame}} } ``` ### `InstructionStream::get_next_opcode` Retrieves the next opcode from the instruction stream. The current implementation involves multiple dereferences to access the opcode. ```rust,ignore {{#include ../interpreter/src/bytecode.rs:DefInstructionStreamGetNextOpcode}} ``` - **Description**: This method advances the instruction pointer and returns the next opcode to be executed. The current implementation is less efficient, requiring dereferencing the `ByteCode` instance, then the `ArrayOpcode` instance, and finally indexing into the `ArrayOpcode`. ``` -------------------------------- ### Initialize Hole Finder Variables Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-simple-bump.md Initializes the counter for consecutive available lines. ```rust let mut count = 0; ``` -------------------------------- ### VM Opcodes Reference Source: https://context7.com/rust-hosted-langs/book/llms.txt Reference for the 32-bit opcodes used in the virtual machine, which features a register-based architecture supporting 256 registers per call frame. ```APIDOC ## VM Opcodes Reference The bytecode VM uses 32-bit opcodes with a register-based architecture supporting 256 registers per call frame. ### Control Flow - `Opcode::Return { reg: Register }`: Return value from register. - `Opcode::Jump { offset: JumpOffset }`: Unconditional jump. - `Opcode::JumpIfTrue { test: Register, offset }`: Jump if test == 'true'. - `Opcode::JumpIfNotTrue { test: Register, offset }`: Jump if test != 'true'. ### Load Operations - `Opcode::LoadNil { dest: Register }`: `dest = nil`. - `Opcode::LoadInteger { dest, integer: i16 }`: `dest = inline integer`. - `Opcode::LoadLiteral { dest, literal_id: LiteralId }`: `dest = literals[id]`. - `Opcode::LoadGlobal { dest, name: Register }`: `dest = globals[name]`. - `Opcode::StoreGlobal { src, name: Register }`: `globals[name] = src`. ### Pair Operations (Lisp car/cdr/cons) - `Opcode::FirstOfPair { dest, reg }`: `dest = car(reg)`. - `Opcode::SecondOfPair { dest, reg }`: `dest = cdr(reg)`. - `Opcode::MakePair { dest, reg1, reg2 }`: `dest = cons(reg1, reg2)`. ### Type Tests - `Opcode::IsNil { dest, test }`: `dest = (test == nil) ? 'true : nil`. - `Opcode::IsAtom { dest, test }`: `dest = atom?(test) ? 'true : nil`. - `Opcode::IsIdentical { dest, test1, test2 }`: `dest = (test1 == test2) ? 'true : nil`. ### Function Calls - `Opcode::Call { function, dest, arg_count }`: Call function with arguments. - `Opcode::MakeClosure { dest, function }`: Create closure from function. ### Closure Upvalue Operations - `Opcode::GetUpvalue { dest, src: UpvalueId }`: `dest = upvalues[src]`. - `Opcode::SetUpvalue { dest: UpvalueId, src }`: `upvalues[dest] = src`. - `Opcode::CloseUpvalues { reg1, reg2, reg3 }`: Close upvalues for registers. ### Register Operations - `Opcode::CopyRegister { dest, src }`: `dest = src`. ``` -------------------------------- ### Manage Tagged Pointers and Value Types Source: https://context7.com/rust-hosted-langs/book/llms.txt Demonstrates encoding type information into pointer bits and converting them into safe, lifetime-bounded Value types. ```rust use crate::taggedptr::{TaggedPtr, FatPtr, Value}; use crate::safeptr::{MutatorScope, TaggedScopedPtr}; // TaggedPtr uses low 2 bits for type tags // TAG_NUMBER = 0b01 - inline small integers // TAG_SYMBOL = 0b10 - symbol pointers // TAG_PAIR = 0b11 - pair/cons cell pointers // TAG_OBJECT = 0b00 - other heap objects (type from header) // Create tagged pointers for different value types let nil_ptr = TaggedPtr::nil(); let num_ptr = TaggedPtr::number(42); let sym_ptr = TaggedPtr::symbol(symbol_raw_ptr); let literal = TaggedPtr::literal_integer(-100i16); // Check if nil assert!(nil_ptr.is_nil()); assert!(!num_ptr.is_nil()); // Convert to fat pointer for pattern matching let fat_ptr: FatPtr = num_ptr.into_fat_ptr(); match fat_ptr { FatPtr::Number(n) => println!("Got number: {}", n), FatPtr::Symbol(ptr) => println!("Got symbol"), FatPtr::Pair(ptr) => println!("Got pair"), FatPtr::Nil => println!("Got nil"), _ => println!("Got other object type"), } // Convert to Value for safe, lifetime-bounded access fn process_value<'guard>(guard: &'guard dyn MutatorScope, ptr: TaggedPtr) { let value: Value<'guard> = FatPtr::from(ptr).as_value(guard); match value { Value::Number(n) => println!("Number: {}", n), Value::Symbol(s) => println!("Symbol: {}", s.as_str(&value)), Value::Pair(p) => println!("Pair: ({}, {})", p.first.get(guard), p.second.get(guard)), Value::List(l) => println!("List length: {}", l.length()), Value::Nil => println!("Nil"), _ => println!("Other type"), } } ``` -------------------------------- ### Bytecode Compiler Support Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Functions provided to support the compilation process, allowing opcodes and literals to be added to the ByteCode structure. ```APIDOC ## Bytecode Compiler Support This section details the methods available for compiling code into the `ByteCode` structure. ### `push` Adds a new opcode to the `ArrayOpcode` instance. - **Method**: `fn push<'guard>(&self, mem: &'MutatorView, op: Opcode) -> Result<(), RuntimeError>` ### `update_jump_offset` Updates the jump offset for a given jump instruction within the `ArrayOpcode`. - **Method**: `fn update_jump_offset<'guard>(&self, mem: &'guard MutatorView, instruction: ArraySize, offset: JumpOffset) -> Result<(), RuntimeError>` - **Description**: Necessary for forward jumps, as their offsets can only be determined after subsequent instructions are compiled. ### `push_lit` Pushes a literal onto the `Literals` list and returns its `LiteralId`. - **Method**: `fn push_lit<'guard>(&self, mem: &'guard MutatorView, literal: TaggedScopedPtr) -> Result` ### `push_loadlit` Pushes a `LiteralId` referencing instruction onto the `ArrayOpcode` list. - **Method**: `fn push_loadlit<'guard>(&self, mem: &'guard MutatorView, dest: Register, literal_id: LiteralId) -> Result<(), RuntimeError>` - **Description**: This is typically called after a literal has been pushed using `push_lit`. ``` -------------------------------- ### Sticky Immix Heap Allocator Source: https://context7.com/rust-hosted-langs/book/llms.txt Implements the Sticky Immix garbage collection algorithm, providing bump-pointer allocation with line-based marking. Supports various object size classes and uses a header system for runtime type identification. ```APIDOC ## Sticky Immix Heap Allocator ### Description Implements the Sticky Immix garbage collection algorithm, providing bump-pointer allocation with line-based marking. It supports small, medium, and large object size classes and uses a header system for runtime type identification. ### Method N/A (Library API) ### Endpoint N/A (Library API) ### Parameters N/A (Library API) ### Request Example ```rust use stickyimmix::{ AllocError, AllocHeader, AllocObject, AllocRaw, AllocTypeId, ArraySize, Mark, RawPtr, SizeClass, StickyImmixHeap }; // Define your type system #[derive(PartialEq, Copy, Clone)] enum MyTypeId { String, Number, Array, } impl AllocTypeId for MyTypeId {} // Implement AllocObject for your types impl AllocObject for String { const TYPE_ID: MyTypeId = MyTypeId::String; } impl AllocObject for usize { const TYPE_ID: MyTypeId = MyTypeId::Number; } // Define a custom object header struct MyHeader { size_class: SizeClass, mark: Mark, type_id: MyTypeId, size_bytes: u32, } impl AllocHeader for MyHeader { type TypeId = MyTypeId; fn new>(size: u32, size_class: SizeClass, mark: Mark) -> Self { MyHeader { size_class, mark, type_id: O::TYPE_ID, size_bytes: size } } fn new_array(size: ArraySize, size_class: SizeClass, mark: Mark) -> Self { MyHeader { size_class, mark, type_id: MyTypeId::Array, size_bytes: size } } fn mark(&mut self) { self.mark = Mark::Marked; } fn is_marked(&self) -> bool { self.mark == Mark::Marked } fn size_class(&self) -> SizeClass { self.size_class } fn size(&self) -> u32 { self.size_bytes } fn type_id(&self) -> MyTypeId { self.type_id } } // Create heap and allocate objects let heap = StickyImmixHeap::::new(); // Allocate a String object let string_ptr: RawPtr = heap.alloc(String::from("hello"))?; let value = unsafe { string_ptr.as_ref() }; assert_eq!(*value, "hello"); // Allocate a number let num_ptr: RawPtr = heap.alloc(42usize)?; // Allocate a byte array (zero-initialized) let array_ptr: RawPtr = heap.alloc_array(2048)?; // Access object header from object pointer let header_ptr = StickyImmixHeap::::get_header(string_ptr.as_untyped()); let header = unsafe { &*header_ptr.as_ptr() }; assert_eq!(header.type_id(), MyTypeId::String); ``` ### Response #### Success Response (200) N/A (Library API) #### Response Example N/A (Library API) ``` -------------------------------- ### Define InstructionStream Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Represents the execution stream of bytecode instructions. ```rust {{#include ../interpreter/src/bytecode.rs:DefInstructionStream}} ``` -------------------------------- ### Define ByteCode Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Defines the composition of ArrayOpcode and Literals. ```rust {{#include ../interpreter/src/bytecode.rs:DefByteCode}} ``` -------------------------------- ### Implement MakeClosure VM Instruction in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-vm-impl.md Decodes and executes the MakeClosure VM instruction, which creates a Partial object from a Function and its environment, writing the result to a destination register. ```rust Opcode::MakeClosure => { let (dest, function) = operands.read_two_registers(); let function = vm.registers[function].as_function().unwrap(); let closure = vm.make_closure(function)?; vm.registers[dest] = Value::Partial(closure); } ``` -------------------------------- ### Binding a function in a let expression Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-design.md Demonstrates binding the result of a function call to a symbol and invoking it within a let scope. ```Lisp (let ((add_3 (make_adder 3))) (add_3 4)) ``` -------------------------------- ### Execute Bytecode in Rust VM Source: https://context7.com/rust-hosted-langs/book/llms.txt Executes compiled bytecode using a register-based virtual machine. Supports function calls, closures, and global bindings. Ensure the Thread is allocated and necessary compilation functions are available. ```rust use crate::vm::{Thread, EvalStatus, FIRST_ARG_REG}; use crate::compiler::compile; use crate::parser::parse; use crate::function::Function; fn vm_example(mem: &MutatorView) -> Result<(), RuntimeError> { // Create execution thread let thread = Thread::alloc(mem)?; // Compile and evaluate a simple expression let code = "(cons 'a 'b)"; let function = compile(mem, parse(mem, code)?)?; let result = thread.quick_vm_eval(mem, function)?; println!("Result: {}", result); // (a . b) // Define and call functions let define_fn = "(def is-y (x) (is? x 'y))"; compile_and_eval(mem, thread, define_fn)?; let call_fn = "(is-y 'y)"; let result = compile_and_eval(mem, thread, call_fn)?; // result = Symbol("true") // Recursive function example let map_def = r###" (def map (f lst) (cond (nil? lst) nil true (cons (f (car lst)) (map f (cdr lst))))) "###; compile_and_eval(mem, thread, map_def)?; let map_call = "(map is-y '(x y z y))"; let result = compile_and_eval(mem, thread, map_call)?; // result = (nil true nil true) // Partial application (currying) let partial_def = "(def add (a b) (+ a b))"; compile_and_eval(mem, thread, partial_def)?; let partial_call = "((add 5) 3)"; // Returns 8 let result = compile_and_eval(mem, thread, partial_call)?; // Closures capturing variables let closure_def = r###" (def make-adder (n) (lambda (x) (+ n x))) "###; compile_and_eval(mem, thread, closure_def)?; let use_closure = r###" (let ((add5 (make-adder 5))) (add5 10)) "###; let result = compile_and_eval(mem, thread, use_closure)?; // result = 15 Ok(()) } fn compile_and_eval<'guard>( mem: &'guard MutatorView, thread: ScopedPtr<'guard, Thread>, code: &str ) -> Result, RuntimeError> { let function = compile(mem, parse(mem, code)?)?; thread.quick_vm_eval(mem, function) } ``` -------------------------------- ### Compile Function Implementation Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-impl.md The implementation of compile_function, which handles scope instantiation, body evaluation, and closure creation. ```rust {{#include ../interpreter/src/compiler.rs:DefCompilerCompileFunction}} ``` -------------------------------- ### Implement TaggedPtr to FatPtr Conversion Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Logic for identifying object types from tags and converting back to FatPtr. ```rust {{#include ../interpreter/src/taggedptr.rs:FromTaggedPtrForFatPtr}} impl TaggedPtr { {{#include ../interpreter/src/taggedptr.rs:DefTaggedPtrIntoFatPtr}} } ``` -------------------------------- ### Implement Association for Dict Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-dicts.md Implementation of the assoc method for the Dict container using find_entry. ```rust impl HashIndexedAnyContainer for Dict { {{#include ../interpreter/src/dict.rs:DefHashIndexedAnyContainerForDictAssoc}} } ``` -------------------------------- ### Compiler push_op2 Implementation Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-impl.md This function handles the compilation of simple function calls that take one argument and return one result. ```rust {{#include ../interpreter/src/compiler.rs:DefCompilerPushOp2}} ``` -------------------------------- ### Block Allocator API Source: https://context7.com/rust-hosted-langs/book/llms.txt API for allocating size-aligned memory blocks using Rust's standard allocator. Blocks must be power-of-two sizes and are aligned to their own size. ```APIDOC ## Block Allocator API ### Description Provides size-aligned memory block allocation using Rust's standard allocator. Blocks must be power-of-two sizes and are aligned to their own size, which is essential for implementing efficient memory management schemes like Immix. ### Method N/A (Library API) ### Endpoint N/A (Library API) ### Parameters N/A (Library API) ### Request Example ```rust use blockalloc::{Block, BlockError, BlockPtr, BlockSize}; // Allocate a 32KB memory block (must be power of 2) let size: BlockSize = 32768; let block = Block::new(size)?; // Get a raw pointer to the block let ptr: *const u8 = block.as_ptr(); // Verify alignment: block address AND alignment bits should be exclusive let mask = size - 1; assert!((ptr as usize & mask) ^ mask == mask); // Consume block and get the pointer let block_ptr: BlockPtr = block.into_mut_ptr(); // Reconstruct block from raw parts (unsafe) let block = unsafe { Block::from_raw_parts(block_ptr, size) }; // Block is automatically deallocated when dropped drop(block); ``` ### Response #### Success Response (200) N/A (Library API) #### Response Example N/A (Library API) ``` -------------------------------- ### Convert FatPtr to Value Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Provides a method to convert a `FatPtr` into a `Value`. This is necessary because the standard `From for Value` implementation cannot define the required `'guard` lifetime. ```rust pub fn as_value<'guard>(&self) -> Value<'guard> where Self: Copy, { Value::from(*self) } ``` -------------------------------- ### Bytecode Structures Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Defines the core data structures for representing bytecode, including opcodes, literals, and the main bytecode container. ```APIDOC ## Bytecode Structures This section describes the fundamental data structures used for representing bytecode in the interpreter. ### Opcode Definition An `Opcode` enum is used to define the various operations that can be performed. A specific definition for `ArrayOpcode` is provided, which is a generic array type. ```rust,ignore {{#include ../interpreter/src/bytecode.rs:DefArrayOpcode}} ``` ### Literals and Literal IDs To handle various literal types beyond 16-bit signed integers, a `Literals` type is introduced. This type is a collection of literals that can be referenced by bytecode instructions. A `LiteralId` (a `u16`) is used as an index into this `Literals` list. ```rust,ignore {{#include ../interpreter/src/bytecode.rs:DefLiterals}} ``` ```rust pub type LiteralId = u16; ``` ### ByteCode Type The `ByteCode` type is a composition of `ArrayOpcode` and `Literals`, serving as the primary container for compiled code. ```rust,ignore {{#include ../interpreter/src/bytecode.rs:DefByteCode}} ``` ``` -------------------------------- ### Define HashKey function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-dicts.md Retrieves a 64-bit hash value from a TaggedScopedPtr. ```rust {{#include ../interpreter/src/dict.rs:DefHashKey}} ``` -------------------------------- ### Implement Heap Allocation Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md Proxies the Sticky Immix allocation function within the Heap struct. ```rust impl Heap { {{#include ../interpreter/src/memory.rs:DefHeapAlloc}} } ``` -------------------------------- ### Implement AllocRaw::alloc_array() for StickyImmixHeap Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-allocation-impl.md Implements the alloc_array() function for allocating arrays. The type is fixed to a u8 pointer, and the array is initialized to zero bytes. The interpreter is responsible for writing into the array itself. ```rust impl AllocRaw for StickyImmixHeap { type Header = H; fn alloc_array(&mut self, size: usize) -> NonNull { let space = self.find_space(size); let header_ptr = space as *mut Self::Header; let obj_ptr = space as *mut u8; unsafe { write(header_ptr, Self::Header::new()); // Initialize array to zero bytes std::ptr::write_bytes(obj_ptr, 0, size + size_of::()); } NonNull::new(obj_ptr).unwrap() } ... } ``` -------------------------------- ### Sticky Immix Allocator Interface Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md This code defines the raw allocator interface from the Sticky Immix crate. It is used as the basis for building safe Rust abstractions. ```rust pub trait DefAllocRaw { type Allocator: AllocatorRaw; fn allocator(&self) -> &Self::Allocator; } pub trait AllocatorRaw { type Ptr: PtrRaw; fn alloc(&self, value: T) -> Self::Ptr; fn dealloc(&self, ptr: Self::Ptr); } pub trait PtrRaw { fn as_usize(&self) -> usize; fn from_usize(ptr: usize) -> Self; } ``` -------------------------------- ### Define compile_function Signature Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-impl.md The main entrypoint for compiling functions, setting up variable scopes and triggering expression evaluation. ```rust {{#include ../interpreter/src/compiler.rs:DefCompilerCompileFunctionSig}} ... } ``` -------------------------------- ### Implement inner_alloc Function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-simple-bump.md Allocates memory by moving a cursor downwards and aligning to word boundaries. ```rust impl BumpBlock { pub fn inner_alloc(&mut self, alloc_size: usize) -> Option<*const u8> { let block_start_ptr = self.block.as_ptr() as usize; let cursor_ptr = self.cursor as usize; // align to word boundary let align_mask: usize = !(size_of::() - 1); let next_ptr = cursor_ptr.checked_sub(alloc_size)? & align_mask; if next_ptr < block_start_ptr { // allocation would start lower than block beginning, which means // there isn't space in the block for this allocation None } else { self.cursor = next_ptr as *const u8; Some(next_ptr as *const u8) } } } ``` -------------------------------- ### Implement Pair::new() Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-symbols-and-pairs.md Creates a new Pair instance with 'first' and 'second' members set to nil. This function does not require heap allocation. ```rust pub fn new<'guard>(guard: &'guard MutatorView<'guard>) -> Pair<'guard> { Pair { first: guard.nil(), second: guard.nil(), source: None, } } ``` -------------------------------- ### Push Load Literal Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-bytecode.md Pushes a load instruction into the ArrayOpcode list. ```rust fn push_loadlit<'guard>( &self, mem: &'guard MutatorView, dest: Register, literal_id: LiteralId, ) -> Result<(), RuntimeError> ``` -------------------------------- ### Allocate memory blocks with blockalloc Source: https://context7.com/rust-hosted-langs/book/llms.txt Allocates power-of-two sized memory blocks aligned to their own size. The block is automatically deallocated when dropped. ```rust use blockalloc::{Block, BlockError, BlockPtr, BlockSize}; // Allocate a 32KB memory block (must be power of 2) let size: BlockSize = 32768; let block = Block::new(size)?; // Get a raw pointer to the block let ptr: *const u8 = block.as_ptr(); // Verify alignment: block address AND alignment bits should be exclusive let mask = size - 1; assert!((ptr as usize & mask) ^ mask == mask); // Consume block and get the pointer let block_ptr: BlockPtr = block.into_mut_ptr(); // Reconstruct block from raw parts (unsafe) let block = unsafe { Block::from_raw_parts(block_ptr, size) }; // Block is automatically deallocated when dropped drop(block); ``` -------------------------------- ### Array::push for StackAnyContainer Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-arrays.md Implements the `push` method for `Array` by leveraging the existing `StackContainer::push` implementation. ```rust impl StackAnyContainer for Array { } ``` -------------------------------- ### Compile Anonymous Function Implementation Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-compiler-impl.md The full implementation of the anonymous function compilation logic. ```rust {{#include ../interpreter/src/compiler.rs:DefCompilerCompileAnonymousFunction}} ``` -------------------------------- ### Implement Pair::cons() Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-symbols-and-pairs.md Allocates a new Pair on the heap with specified head and rest values. Requires a MutatorView for heap access and lifetime management. ```rust pub fn cons<'guard>( guard: &'guard MutatorView<'guard>, head: TaggedScopedPtr<'guard>, rest: TaggedScopedPtr<'guard>, ) -> TaggedCellPtr<'guard> { let mut pair = Pair::new(guard); pair.first.set(head); pair.second.set(rest); guard.alloc_tagged(pair) } ``` -------------------------------- ### Implement Dissociation for Dict Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-dicts.md Implementation of the dissoc method for the Dict container using find_entry. ```rust impl HashIndexedAnyContainer for Dict { {{#include ../interpreter/src/dict.rs:DefHashIndexedAnyContainerForDictDissoc}} } ``` -------------------------------- ### Tokenize input string Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-parsing.md Converts a string slice into a vector of tokens. This is the first step in parsing s-expressions. ```rust fn tokenize(input: &str) -> Result, RuntimeError>; ``` -------------------------------- ### Implement ObjectHeader get_object_fatptr Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Method to retrieve a FatPtr from an object header, marked unsafe due to potential for type mismatch errors. ```rust impl ObjectHeader { {{#include ../interpreter/src/headers.rs:DefObjectHeaderGetObjectFatPtr}} } ``` -------------------------------- ### Mutator Execution Function Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md This function outlines the typical steps involved in running the mutator in an interpreted language environment, including parsing, compiling, and executing bytecode. ```rust fn run_mutator() { parse_source_code(); compile(); execute_bytecode(); } ``` -------------------------------- ### Implement MutatorView alloc_tagged Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-tagged-ptrs.md Allocates a new object and returns a TaggedScopedPtr. ```rust impl MutatorView { {{#include ../interpreter/src/memory.rs:DefMutatorViewAllocTagged}} } ``` -------------------------------- ### Implement Hashable for Symbol Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-dicts.md Wraps the standard library Hash implementation for Symbol types. ```rust {{#include ../interpreter/src/symbol.rs:DefImplHashableForSymbol}} ``` -------------------------------- ### Implement find_space for StickyImmixHeap Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-managing-blocks.md Defines the structure for the find_space function used to locate available memory within the heap blocks. ```rust impl StickyImmixHeap { fn find_space( &self, alloc_size: usize, size_class: SizeClass, ) -> Result<*const u8, AllocError> { let blocks = unsafe { &mut *self.blocks.get() }; ... } } ``` -------------------------------- ### Implement MyHeader Struct and AllocHeader in Rust Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-allocation-api.md Defines a possible object header struct and implements the `AllocHeader::new()` function. The `type_id` is derived from the `AllocObject`'s `TYPE_ID`. ```rust struct MyHeader { size: u32, size_class: SizeClass, mark: Mark, type_id: MyTypeId, } impl AllocHeader for MyHeader { type TypeId = MyTypeId; fn new>( size: u32, size_class: SizeClass, mark: Mark ) -> Self { MyHeader { size, size_class, mark, type_id: O::TYPE_ID, } } ... } ``` -------------------------------- ### Define Heap Struct Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-interp-alloc.md Defines the main heap structure containing interpreter-global storage. ```rust {{#include ../interpreter/src/memory.rs:DefHeap}} ``` -------------------------------- ### Allocate into a new overflow block Source: https://github.com/rust-hosted-langs/book/blob/master/booksrc/chapter-managing-blocks.md Handles the case where no overflow block exists yet. A new `BumpBlock` is created, and the object is allocated within it. This allocation is expected to succeed as the object size is less than the block size. ```rust match self.overflow { Some ..., None => { let mut overflow = BumpBlock::new()?; // object size < block size means we can't fail this expect let space = overflow .inner_alloc(alloc_size) .expect("We expected this object to fit!"); self.overflow = Some(overflow); space } } ```