# Ultimate Commodore 64 Reference Guide

## Introduction

The Ultimate Commodore 64 Reference Guide is a comprehensive project that collects, formats, and presents Commodore 64 reference material in machine-readable form. This repository aggregates historical documentation about the C64 including ROM disassemblies, memory maps, KERNAL API documentation, 6502 CPU instruction references, character sets, PETSCII codes, and keyboard layouts. The project was initially created by Michael Steil to preserve and make accessible critical technical documentation about the iconic Commodore 64 computer system and related machines in the Commodore family (PET, VIC-20, C128, C65, TED, CBM2).

The project generates a multi-page interactive HTML website from structured text files and Python scripts. Each reference category (6502 CPU, KERNAL API, ROM Disassembly, Memory Map, I/O Map, Character Sets, and Colors) has its own generation scripts that parse text-based source files and produce richly formatted HTML with cross-references, side-by-side commentary views, and interactive features. The build system includes validation, git integration for revision tracking, and deployment capabilities to publish the documentation online.

## Build System

### Main Build Script

The primary build script that orchestrates the entire generation and deployment process.

```python
#!/usr/bin/env python3
# Generate the entire Ultimate C64 Reference website

# Build locally with all stable categories
./generate.py local

# Build with work-in-progress categories (colors, c64io)
./generate.py local --wip

# Build only specific categories by path name
./generate.py local --only 6502 kernal c64mem

# Deploy to production server (requires clean git state on main branch)
./generate.py upload

# Example: Typical development workflow
# 1. Make changes to source files
./generate.py local --only c64mem  # Build just the changed category
# 2. Test locally at http://localhost:6464/c64ref/c64mem/
python3 -m http.server 6464
```

### Build Configuration

Configure build parameters programmatically using the BuildConfig class.

```python
from dataclasses import dataclass

@dataclass
class BuildConfig():
    source_dir: str = "src"           # Source files location
    build_dir: str = "out"            # Output directory
    base_dir: str = "c64ref"          # Web base path
    server_path: str = "user@server:/var/www/html/"
    deploy: bool = False              # Upload to server
    build_wips: bool = False          # Include WIP categories
    enabled_paths: list = None        # Selective builds

# Usage in custom build script
config = BuildConfig()
config.build_dir = "custom_out"
config.enabled_paths = ["6502", "kernal"]
# ... proceed with build using config
```

### Adding New Reference Categories

Define new reference categories by extending the CATEGORIES list.

```python
from dataclasses import dataclass

@dataclass
class RefCategory():
    path: str              # Folder name in src/
    long_title: str        # Full title for HTML
    short_title: str       # Menu title
    authors: list          # Author names/links
    is_wip: bool = False   # Work in progress flag

# Add a new category
CATEGORIES.append(
    RefCategory(
        'sid',
        'C64 SID Sound Interface Device Reference',
        'SID Audio',
        ['<a href="http://example.com">Your Name</a>'],
        is_wip=True
    )
)

# The build system will automatically:
# 1. Look for src/sid/generate.py or out.sh
# 2. Generate navigation with "SID Audio" menu item
# 3. Create index at /c64ref/sid/
# 4. Skip on production if is_wip=True (unless --wip flag used)
```

## Memory Map Generator

### Memory Map HTML Generation

Generates interactive HTML tables comparing multiple C64 memory map references side-by-side.

```python
#!/usr/bin/env python3
# src/c64mem/generate.py - Generate memory map comparison

# Input file format (c64mem_*.txt):
# $0000-$0001 D6510   I/O port data direction and I/O port
#
#                     This is the detailed description that can span
#                     multiple lines and include markdown formatting.

# Generate the memory map
import sys
sys.path.insert(0, './src/c64mem')
from generate import *

# Example: Cross-reference memory addresses
text = "See address $D000 and routine $E000"
result = cross_reference(text)
# Output: 'See address <a href="#D000">$D000</a> and
#          routine <a href="../c64disasm/#E000">$E000</a>'

# Memory addresses < $0400 link to memory map
# Addresses $A000-$BFFF and $E000-$FFFF link to disassembly

# Generate full HTML output
import subprocess
subprocess.run(['python3', 'src/c64mem/generate.py'],
               stdout=open('out/c64ref/c64mem/index.html', 'w'))
```

### Memory Map Data Format

Structure memory map data files for multiple sources.

```python
# Example memory map entry format in c64mem_*.txt

# Address range, symbol, and short description
$0000-$0001 D6510   I/O port DDR and data
# First blank line separates heading from details

                    Detailed description with **markdown** support.
                    Can include tables and cross-references to $FFD2.

                    | Bit | Function |
                    |-----|----------|
                    | 0-2 | Memory configuration |

# Comment lines (ignored)
#----------------------------

# Single address entry
$0002       UNUSED  Unused memory location

# Parse and process programmatically
filenames = ['c64mem_mapc64.txt', 'c64mem_sta.txt']
data = []
for filename in filenames:
    lines = [line.rstrip() for line in open(filename)]
    data.append(lines)

# The generator merges entries by address range
# Longest range wins for defining address spans
# Multiple commentaries displayed in columns
```

## KERNAL API Generator

### KERNAL API Documentation

Generate comprehensive KERNAL API reference with multiple commentary sources.

```python
#!/usr/bin/env python3
# src/kernal/generate.py - Generate KERNAL API reference

# Input file format (kernal_*.txt):
# $FFD2  CHROUT  Output a character
#
#                Detailed description of the routine,
#                including register usage and examples.

# Define categories for color-coding
categories = {
    0xFFD2: 'IO',      # Input/Output
    0xFFE4: 'KBD',     # Keyboard
    0xFFDE: 'TIME',    # Time/Jiffy clock
    0xFF9C: 'MEM',     # Memory management
    0xFF93: 'IEEE',    # IEEE-488 bus
    0xFFED: 'EDITOR',  # Screen editor
}

# Example: Generate API entry with category
address = 0xFFD2
symbol = 'CHROUT'
category = categories[address]
summary = 'Output a character'

html = f'''
<tr>
  <th>${address:04X}</th>
  <th>{symbol}</th>
  <th style="background-color: var(--cat-color-io);">{category}</th>
  <td><b>{summary}</b>
  <p>Outputs a character to the current output device.</p>
  <p>Input: .A = character to output</p>
  <p>Output: None</p>
  </td>
</tr>
'''
```

### KERNAL API Cross-Referencing

Implement automatic cross-referencing between KERNAL routines.

```python
import re

# Automatic cross-referencing of symbols and addresses
def cross_reference(text):
    # Link hex addresses to appropriate sections
    hex_numbers = re.findall(r'\$[0-9A-F]{4}', text)
    for hex_num in hex_numbers:
        dec = int(hex_num[1:], 16)
        if dec < 0x0400:
            # Link to memory map
            text = text.replace(
                hex_num,
                f'<a href="../c64mem/#{dec:04X}">${dec:04X}</a>'
            )
        elif (0xA000 <= dec <= 0xBFFF) or (0xE000 <= dec <= 0xFFFF):
            # Link to disassembly
            text = text.replace(
                hex_num,
                f'<a href="../c64disasm/#{dec:04X}">{hex_num}</a>'
            )
    return text

# Example usage
description = """
CHROUT calls $E716 internally and uses zero page
locations $D3 and $D6. See also CHRIN at $FFCF.
"""
html_description = cross_reference(description)
# Links are automatically created for $E716, $D3, $D6, $FFCF
```

### KERNAL API Smart Text Joining

Handle hyphenated line breaks intelligently when processing multi-line text.

```python
def smart_join(lines):
    """Join lines with intelligent hyphen handling"""
    s = ''
    while len(lines) > 0:
        if s == '':
            s = lines[0]
        elif s[-1] == '-':
            # Check if next word continues (lowercase) or is new (uppercase)
            if len(lines[0]) > 0 and lines[0][0].islower():
                s = s[:-1] + lines[0]  # Remove hyphen, join words
            else:
                s += lines[0]  # Keep hyphen
        else:
            s += '\n' + lines[0]
        lines = lines[1:]
    return s

# Example
input_lines = [
    'This is a demon-',
    'stration of the smart',
    'join function. It han-',
    'dles word breaks.'
]
result = smart_join(input_lines)
# Output: 'This is a demonstration of the smart\njoin function. It handles word breaks.'
```

## Character Set and PETSCII

### PETSCII to Screen Code Conversion

Convert between PETSCII character codes and screen codes.

```python
# Generate scrcode_from_petscii mapping
scrcode_from_petscii = []
for petscii in range(0, 256):
    # Special handling for $FF (pi symbol)
    if petscii == 0xff:
        petscii = 0xde

    # Conversion logic
    if petscii < 0x20:
        scrcode = petscii + 0x80  # Inverted control chars
    elif petscii < 0x40:
        scrcode = petscii
    elif petscii < 0x60:
        scrcode = petscii - 0x40
    elif petscii < 0x80:
        scrcode = petscii - 0x20
    elif petscii < 0xa0:
        scrcode = petscii + 0x40  # Inverted
    elif petscii < 0xc0:
        scrcode = petscii - 0x40
    else:
        scrcode = petscii - 0x80

    scrcode_from_petscii.append(scrcode)

# Usage example
petscii_a = 0x41  # Letter 'A'
scrcode = scrcode_from_petscii[petscii_a]
print(f"PETSCII ${petscii_a:02X} = Screen Code ${scrcode:02X}")
# Output: PETSCII $41 = Screen Code $01
```

### Keyboard Layout Generation

Generate SVG keyboard layouts from ASCII art descriptions.

```python
def keyboard_layout_html(machine, layout_lines):
    """Generate interactive SVG keyboard from text layout"""

    # Example layout format (keyboard_c64.txt):
    # layout    +---+---+---+
    # layout    | 3E| 3F| 40|    (Scancode in hex)
    # layout    +---+---+---+

    SCALE = 8
    VSCALE = 2
    total_width = max(len(line) for line in layout_lines) * SCALE
    total_height = len(layout_lines) * VSCALE * SCALE

    svg = f'<svg width="{total_width}" height="{total_height}">'

    # Flood fill to find key regions
    cells_from_scancode = {}
    for y in range(len(layout_lines)):
        for x in range(len(layout_lines[y])):
            c = layout_lines[y][x]
            if c not in ['+', '|', '-', 'X']:
                # Found a key - extract scancode
                scancode_text = ''
                for x2 in range(x, len(layout_lines[y])):
                    if layout_lines[y][x2] in ['+', '|', '-']:
                        break
                    scancode_text += layout_lines[y][x2]
                scancode = int(scancode_text.strip(), 16)

                # Use flood fill to find all cells for this key
                # ... (flood fill implementation)

                # Draw rectangle and label
                svg += f'<rect x="{x*SCALE}" y="{y*VSCALE*SCALE}" '
                svg += f'width="{width}" height="{height}" '
                svg += f'class="keyrect_{machine}_{scancode}" />'

    svg += '</svg>'
    return svg

# Load keyboard layouts for all machines
machines = ['C64', 'C128', 'VIC-20', 'PET-N', 'PET-B']
for machine in machines:
    layout = []
    for line in open(f'keyboard_{machine.lower()}.txt'):
        if line.startswith('layout'):
            layout.append(line[8:])  # Skip 'layout  ' prefix
    html = keyboard_layout_html(machine, layout)
```

### PETSCII Control Codes

Map and display control code functions across different Commodore machines.

```python
# Control code descriptions
description_from_control_code_symbol = {
    'CRSR_DOWN':   ('Crsr ↓', 'Cursor Down'),
    'CRSR_HOME':   ('HOME', 'Cursor Home'),
    'CLEAR':       ('CLR', 'Clear Screen'),
    'DEL':         ('DEL', 'Delete'),
    'INST':        ('INST', 'Insert'),
    'COL_WHITE':   ('Wht', 'Set text color to White'),
    'COL_RED':     ('Red', 'Set text color to Red'),
    'RVS_ON':      ('RVS On', 'Reverse Video On'),
    'RVS_OFF':     ('RVS Off', 'Reverse Video Off'),
}

# Load control codes for each machine
machines = ['C64', 'C128', 'VIC-20', 'PET-N', 'PET-B', 'CBM2', 'C65', 'TED']
description_from_control_code = {}

for machine in machines:
    description_from_control_code[machine] = {}
    for line in open(f'control_codes_{machine.lower()}.txt'):
        line = line.split('#')[0].rstrip()
        if len(line) == 0:
            continue
        petscii = int(line[0:2], 16)
        symbol = line[3:].split()[0]

        if symbol in description_from_control_code_symbol:
            description_from_control_code[machine][petscii] = \
                description_from_control_code_symbol[symbol]

# Example: Display control code for PETSCII $13 (HOME)
petscii = 0x13
for machine in machines:
    if petscii in description_from_control_code[machine]:
        short, long = description_from_control_code[machine][petscii]
        print(f"{machine}: {short} - {long}")
```

### Unicode Mapping

Map PETSCII characters to Unicode using the Symbols for Legacy Computing standard.

```python
# Load PETSCII to Unicode mappings
unicode_from_petscii = {'upper': {}, 'lower': {}}
description_from_unicode = {}

# C64IPRI.TXT - Primary (uppercase/graphics) character set
for line in open('C64IPRI.TXT'):
    if line.startswith('#') or len(line.strip()) == 0:
        continue
    # Format: 0x41 -> U+1FB00 LATIN CAPITAL LETTER A
    petscii = int(line[2:4], 16)
    unicode_val = int(line[7:12], 16)
    unicode_from_petscii['upper'][petscii] = unicode_val
    description_from_unicode[unicode_val] = line[14:].strip()

# C64IALT.TXT - Alternate (lowercase) character set
for line in open('C64IALT.TXT'):
    if line.startswith('#') or len(line.strip()) == 0:
        continue
    petscii = int(line[2:4], 16)
    unicode_val = int(line[7:12], 16)
    unicode_from_petscii['lower'][petscii] = unicode_val

# Usage example: Convert PETSCII to Unicode
def petscii_to_unicode(petscii, charset='upper'):
    """Convert PETSCII code to Unicode character"""
    if petscii in unicode_from_petscii[charset]:
        unicode_val = unicode_from_petscii[charset][petscii]
        char = chr(unicode_val)
        desc = description_from_unicode.get(unicode_val, 'Unknown')
        return char, unicode_val, desc
    return None, None, None

# Example
char, code, desc = petscii_to_unicode(0x41, 'upper')
print(f"PETSCII $41 = U+{code:04X} ({desc}): {char}")
# Output: PETSCII $41 = U+1FB01 (LATIN CAPITAL LETTER A): 🬁
```

## Header and Navigation Generation

### HTML Header with Navigation

Generate consistent headers with navigation across all reference pages.

```python
def get_header_str(current_category, categories, source_path,
                   base_path, git_has_changes):
    """Generate HTML header with GitHub corner and navigation"""

    # GitHub corner (octocat icon)
    octocat_svg = """
    <a href="https://github.com/mist64/c64ref" class="github-corner">
      <svg width="80" height="80" viewBox="0 0 250 250">
        <!-- SVG paths for GitHub octocat -->
      </svg>
    </a>
    """

    # Navigation menu
    nav = f'<div>{GLOBAL_TITLE}</div>\n'
    for cat in categories:
        if cat == current_category:
            nav += f'<a class="active" href="#">{cat.short_title}</a>\n'
        else:
            nav += f'<a href="/{base_path}/{cat.path}/">{cat.short_title}</a>\n'

    # Git revision info
    revision = os.popen(f'git log -1 --pretty=format:%h {source_path}').read()
    if git_has_changes:
        revision += "+"
    date = os.popen(f'git log -1 --date=short --pretty=format:%cd {source_path}').read()

    authors = ', '.join(current_category.authors)
    byline = f'''
    by <em>{authors}.</em>
    [<small><a href="https://github.com/mist64/c64ref">
    github.com/mist64/c64ref</a>, rev {revision}, {date}</small>]
    '''

    return f"""
    <header>
      <div id="cat">{octocat_svg}</div>
      <nav class="topnav">{nav}</nav>
      <h1>{current_category.long_title}</h1>
      <p id="byline">{byline}</p>
    </header>
    """

# Usage in build script
header_html = get_header_str(
    current_category=CATEGORIES[0],  # 6502 CPU
    categories=CATEGORIES,
    source_path='src/6502',
    base_path='c64ref',
    git_has_changes=False
)
```

### Path Utilities

Safely create directories for build output.

```python
import os

def ensured_path(path, *paths, is_dir):
    """Join paths and ensure parent directories exist"""
    result = os.path.join(path, *paths)

    dir_name = result if is_dir else os.path.dirname(result)
    if not os.path.exists(dir_name):
        os.makedirs(dir_name)

    return result

# Usage examples
# Ensure directory exists
output_dir = ensured_path('out', 'c64ref', 'kernal', is_dir=True)

# Ensure parent directory exists for file
output_file = ensured_path('out', 'c64ref', 'kernal', 'index.html', is_dir=False)

# Complex nested path
css_file = ensured_path('out', 'c64ref', 'assets', 'css', 'style.css', is_dir=False)
# Creates: out/c64ref/assets/css/ if it doesn't exist
```

## HTML Validation

### Shell Script Validation

Validate generated HTML files before deployment.

```bash
#!/bin/bash
# validate_html.sh - Validate all generated HTML files

# Find all HTML files in output directory
find out/c64ref -name "*.html" -type f | while read file; do
    echo "Validating: $file"

    # Use tidy for HTML validation
    tidy -q -e -utf8 "$file" 2>&1 | grep -v "Warning:"

    if [ $? -eq 0 ]; then
        echo "✓ Valid: $file"
    else
        echo "✗ Errors in: $file"
        exit 1
    fi
done

echo "All HTML files validated successfully"

# Usage in build pipeline
# ./generate.py local
# ./validate_html.sh
# if [ $? -eq 0 ]; then ./generate.py upload; fi
```

## Summary

The Ultimate Commodore 64 Reference Guide provides a complete, maintainable documentation system for Commodore 64 technical reference material. The project's strength lies in its structured approach to aggregating multiple historical sources, presenting them in comparable side-by-side views, and automatically generating rich cross-references between related documentation sections. Developers can extend the system by adding new reference categories, contributing additional source commentaries, or improving the generation scripts.

The build system is designed for both local development and production deployment, with selective building capabilities for rapid iteration and comprehensive validation before publishing. The generated HTML is self-contained with embedded CSS and JavaScript, creating an accessible reference that works offline. Integration patterns include extending the category system, adding new data sources in the standardized text formats, implementing custom generators for new reference types, and deploying to web servers via rsync. The modular architecture allows contributors to focus on individual reference categories without affecting the broader system.