### Clone and Install oMLX for Development

Source: https://github.com/jundot/omlx/blob/main/README.md

Clone the oMLX repository and install it with development dependencies. This is the initial setup step for local development.

```bash
git clone https://github.com/jundot/omlx.git
cd omlx
pip install -e " .[dev]"
pytest -m "not slow"
```

--------------------------------

### Install oMLX from Source

Source: https://github.com/jundot/omlx/blob/main/README.md

Install oMLX directly from its source repository. Supports installing core functionality or with additional MCP support.

```bash
git clone https://github.com/jundot/omlx.git
cd omlx
pip install -e .          # Core only
pip install -e " .[mcp]"   # With MCP (Model Context Protocol) support
```

--------------------------------

### Install oMLX with Homebrew

Source: https://github.com/jundot/omlx/blob/main/README.md

Install oMLX using Homebrew, including instructions for upgrading and running it as a background service. Optional MCP support can also be installed.

```bash
brew tap jundot/omlx https://github.com/jundot/omlx
brew install omlx

# Upgrade to the latest version
brew update && brew upgrade omlx

# Run as a background service (auto-restarts on crash)
omlx start

# Optional: MCP (Model Context Protocol) support
/opt/homebrew/opt/omlx/libexec/bin/pip install mcp
```

--------------------------------

### Start oMLX Server

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX managed background server. This command is typically used on macOS or systems with a Homebrew installation.

```bash
omlx start
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/jundot/omlx/blob/main/docs/CONTRIBUTING.md

Install the project's development dependencies using pip. This includes necessary packages for testing and development.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Serve Models with MCP Tools Configuration

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server with a specified MCP configuration file. Ensure your models are in the specified directory.

```bash
omlx serve --model-dir ~/models --mcp-config mcp.json
```

--------------------------------

### Internationalization (i18n) Setup

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/base.html

Sets up global locale strings for internationalization. The `t` function retrieves translated strings based on the provided key.

```javascript
window._t = {{ locale_json | safe }};
window.t = function(key) {
  return window._t[key] !== undefined ? window._t[key] : key;
};
```

--------------------------------

### Start Update Check Timer

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Starts an interval timer to periodically check for updates.

```javascript
startUpdateCheckTimer() {
 this._updateCheckTimer = setInterval(() => this.checkForUpdate(), 3600000);
 }
```

--------------------------------

### Serve Models with HuggingFace Mirror Endpoint

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server and configures it to use a HuggingFace mirror endpoint, useful for restricted regions. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --hf-endpoint https://hf-mirror.com
```

--------------------------------

### Serve Models with API Key Authentication

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server with API key authentication enabled. Replace 'your-secret-key' with your actual secret key. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --api-key your-secret-key
```

--------------------------------

### Serve Models with Default Settings

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server with default memory guard settings and enables management via the admin UI. Ensure your models are located in the specified directory.

```bash
omlx serve --model-dir ~/models
```

--------------------------------

### GET /v1/models

Source: https://github.com/jundot/omlx/blob/main/README.md

Lists the available models supported by the API.

```APIDOC
## GET /v1/models

### Description
List available models

### Method
GET

### Endpoint
/v1/models

### Parameters
(Parameters not provided in source)

### Request Body
(Schema not provided in source)

### Response
#### Success Response (200)
(Schema not provided in source)

#### Response Example
(Example not provided in source)
```

--------------------------------

### Enable SSD Cache for KV Blocks

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server and enables an SSD cache for KV blocks, specifying the cache directory. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --paged-ssd-cache-dir ~/.omlx/cache
```

--------------------------------

### Serve Models with Custom Memory Guard Ceiling

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server with a custom memory guard ceiling set to 48 GB. Ensure your models are in the specified directory.

```bash
omlx serve --model-dir ~/models --memory-guard-gb 48
```

--------------------------------

### Serve Models with Safe Memory Guard Tier

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server and selects the 'safe' memory guard tier for stricter memory management. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --memory-guard safe
```

--------------------------------

### Start New Chat

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Initializes a new chat session. It saves the previous chat's messages, creates a new chat ID, sets default system prompts and model settings, and resets the UI elements for a new chat.

```javascript
async startNewChat() {
      this.cancelPersistModelSettingsTimer();
      const prevChatId = this.currentChatId;
      if (prevChatId) {
        const prevSession = this.getChatSession(prevChatId, false);
        if (prevSession) {
          prevSession.messages = this.messages;
          this.saveCurrentChat(prevChatId);
        }
      }
      const chatId = 'chat_' + Date.now();
      const sysDefault = this.promptProfiles.find(p => p.name === 'System Default');
      const systemPrompt = sysDefault ? sysDefault.content : '';
      this.currentChatId = chatId;
      this.uploadImages = [];
      this.uploadDocuments = [];
      this.inputMessage = '';
      this.systemPrompt = systemPrompt;
      this.activePromptProfile = sysDefault ? sysDefault.name : null;
      this.editingIndex = null;
      this.editContent = '';
      this.editImages = [];
      this.renamingChatId = null;
      this.isMobile = window.innerWidth < 768;
      this.sidebarOpen = window.innerWidth >= 768;
      const session = {
        messages: [],
        model: this.currentModel,
        systemPrompt,
        activeProfile: this.activePromptProfile,
        modelSettings: null,
        speculativeEngine: null,
        speculativeDraftModel: null,
        modelSettingsByModel: {},
      };
      this.chatSessions[chatId] = session;
      this.messages = session.messages;
      await this.ensureSessionModelSettings(session, this.currentModel);
      if (this.currentModel) {
        this.saveModelSettingsForModel(session, this.currentModel);
      }
      this.resetStreamSession(this.getStreamSession(chatId, true));
      this.recentStats = this.emptyStats();
      this.stopStatsPollingIfIdle();
      this.saveCurrentChat(chatId);
      this.$nextTick(() => {
        const input = this.$refs.messageInput;
        if (input) {
          this.resetTextareaHeight(input);
          input.focus();
        }
        if (window.innerWidth < 768) {
          this.sidebarOpen = false;
        }
        const container = document.getElementById('messagesContainer');
        if (container) {
          container.scrollTop = 0;
        }
        this.computeTimelineDots();
        this.computeTimelineActive();
      });
    }
```

--------------------------------

### Set In-Memory Hot Cache Size

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server and configures the in-memory hot cache size to 20% of available memory. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --hot-cache-max-size 20%
```

--------------------------------

### DFlash Generation Completion Log

Source: https://github.com/jundot/omlx/blob/main/docs/experimental/dflash_mlx_integration.md

Example log message indicating successful DFlash generation, showing key performance metrics like token count, tokens per second, acceptance rate, and number of cycles.

```log
DFlash generation complete: 502 tokens, 45.3 tok/s, acceptance=87.2%, cycles=38
```

--------------------------------

### Manage oMLX Homebrew Service

Source: https://github.com/jundot/omlx/blob/main/README.md

Start, stop, or restart the oMLX service using Homebrew commands. These commands delegate to `brew services` for managing the background service.

```bash
omlx start                    # Start via brew services
omlx stop                     # Stop
omlx restart                  # Restart

brew services start omlx    # Start (auto-restarts on crash)
brew services stop omlx     # Stop
brew services restart omlx  # Restart
brew services info omlx     # Check status
```

--------------------------------

### JavaScript for Login and API Key Management

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/login.html

This JavaScript code handles the login form submission, API key setup, and random key generation. It includes client-side validation and asynchronous API calls.

```javascript
function loginForm(apiKeyConfigured) {
  return {
    apiKeyConfigured: apiKeyConfigured,
    apiKey: '',
    apiKeyConfirm: '',
    remember: false,
    error: '',
    loading: false,
    showSetupKey: false,
    generateRandomKey() {
      const chars = 'abcdefghijklmnopqrstuvwxyz0123456789';
      const rand = Array.from(crypto.getRandomValues(new Uint8Array(16)))
        .map(b => chars[b % chars.length]).join('');
      const key = 'omlx-' + rand;
      this.apiKey = key;
      this.apiKeyConfirm = key;
      this.showSetupKey = true;
    },
    async login() {
      this.error = '';
      this.loading = true;
      try {
        const response = await fetch('/admin/api/login', {
          method: 'POST',
          headers: {
            'Content-Type': 'application/json',
          },
          body: JSON.stringify({ api_key: this.apiKey, remember: this.remember }),
        });
        const data = await response.json();
        if (response.ok && data.success) {
          window.location.href = '/admin/dashboard';
        } else {
          this.error = data.detail || data.message || window.t('login.error.invalid_api_key');
        }
      } catch (err) {
        this.error = window.t('login.error.connect_failed');
        console.error('Login error:', err);
      } finally {
        this.loading = false;
      }
    },
    async setup() {
      this.error = '';
      // Client-side validation
      if (this.apiKey.length < 4) {
        this.error = window.t('login.error.min_length');
        return;
      }
      if (/\s/.test(this.apiKey)) {
        this.error = window.t('login.error.no_whitespace');
        return;
      }
      if (this.apiKey !== this.apiKeyConfirm) {
        this.error = window.t('login.error.mismatch');
        return;
      }
      this.loading = true;
      try {
        const response = await fetch('/admin/api/setup-api-key', {
          method: 'POST',
          headers: {
            'Content-Type': 'application/json',
          },
          body: JSON.stringify({
            api_key: this.apiKey,
            api_key_confirm: this.apiKeyConfirm,
          }),
        });
        const data = await response.json();
        if (response.ok && data.success) {
          window.location.href = '/admin/dashboard';
        } else {
          this.error = data.detail || data.message || window.t('login.error.setup_failed');
        }
      } catch (err) {
        this.error = window.t('login.error.connect_failed');
        console.error('Setup error:', err);
      } finally {
        this.loading = false;
      }
    }
  }
}
```

--------------------------------

### DFlash Context Fallback Log

Source: https://github.com/jundot/omlx/blob/main/docs/experimental/dflash_mlx_integration.md

Example log message indicating that DFlash fell back to a standard engine due to the prompt exceeding the maximum context length.

```log
DFlash context fallback: 5120 >= 4096, using vlm engine
```

--------------------------------

### Start Renaming Chat

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Initiates the renaming process for a given chat. It sets the renaming chat ID and prepares the draft title, then focuses the input field for immediate editing.

```javascript
startRenamingChat(chat) {
      this.renamingChatId = chat.id;
      this.renameDraft = chat.title || this.untitledChatTitle();
      this.$nextTick(() => {
        const ref = this.$refs.renameInput;
        const input = Array.isArray(ref) ? ref[0] : ref;
        if (!input) return;
        input.focus();
        input.select();
      });
    }
```

--------------------------------

### Get Variant Chain From Specific Message

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Constructs a variant chain starting from a specific message ID. If the message is not found, it falls back to getting the active variant chain.

```javascript
getVariantChainFrom(turnMessages, variantId) {
  if (!turnMessages.length || !variantId) return [];
  const start = turnMessages.findIndex(m => m.id === variantId);
  if (start < 0) return this.getActiveVariantChain(turnMessages, variantId);
  const chain = [];
  for (let i = start; i < turnMessages.length; i++) {
    const m = turnMessages[i];
    if (i > start && m.role === 'assistant' && m._ui !== false) break;
    chain.push(m);
  }
  return chain;
}
```

--------------------------------

### DFlashEngine Fallback Mechanism Overview

Source: https://github.com/jundot/omlx/blob/main/docs/experimental/dflash_mlx_integration.md

Illustrates the startup process of DFlashEngine and the request routing logic based on prompt length. It also shows how the engine pool handles DFlash startup failures by directly creating fallback engines.

```text
DFlashEngine.start()
  ├── load target model (dflash-mlx)
  ├── load draft model (dflash-mlx)
  └── start fallback engine
        ├── VLMBatchedEngine (if model detected as VLM)
        └── BatchedEngine (otherwise)

Request arrives:
  ├── len(prompt_tokens) < DFLASH_MAX_CTX → DFlash path
  └── len(prompt_tokens) >= DFLASH_MAX_CTX → fallback engine path

DFlashEngine.start() fails:
  └── engine_pool catches exception → creates VLMBatchedEngine or BatchedEngine directly
```

--------------------------------

### oQ+ Pipeline Steps

Source: https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md

The oQ+ pipeline involves loading the model, measuring sensitivity, building a budget plan, optimizing weights with GPTQ, quantizing, and saving. This process is used for enhanced quantization.

```text
1. Load model (full)
2. Measure per-layer sensitivity (relative MSE)
3. Build budget plan (sensitivity-driven bit allocation)
4. GPTQ weight optimization (all quantizable weights)
5. Quantize with mixed-precision predicate
6. Save
```

--------------------------------

### Load and Initialize Prompt Profiles

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Loads prompt profiles from local storage, migrates from older formats, and ensures a 'System Default' profile exists. Applies the default prompt if none is set.

```javascript
loadPromptProfiles() {
  try {
    const raw = localStorage.getItem('omlx_chat_prompt_profiles');
    if (raw) {
      this.promptProfiles = JSON.parse(raw);
    } else {
      // Migrate from old single-default format
      const oldDefault = localStorage.getItem('omlx_chat_system_prompt');
      if (oldDefault) {
        this.promptProfiles = [{ name: 'System Default', content: oldDefault }];
        this._persistPromptProfiles();
        localStorage.removeItem('omlx_chat_system_prompt');
      }
    }

    // Ensure System Default profile always exists
    if (!this.promptProfiles.some(p => p.name === 'System Default')) {
      this.promptProfiles.unshift({ name: 'System Default', content: '' });
      this._persistPromptProfiles();
    }

    // Apply System Default profile's prompt if none set
    if (!this.systemPrompt) {
      const sysDefault = this.promptProfiles.find(p => p.name === 'System Default');
      if (sysDefault) {
        this.systemPrompt = sysDefault.content;
        this.activePromptProfile = 'System Default';
      }
    }
  } catch (e) {
    this.promptProfiles = [];
  }
}
```

--------------------------------

### Get Turn Variants For User

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Convenience function to get turn variants for a user based on the current messages array.

```javascript
getTurnVariantsForUser(userIndex) {
  return this.getTurnVariantsAt(this.messages, userIndex);
}
```

--------------------------------

### Apply Theme Immediately

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/base.html

Applies the theme from local storage immediately to prevent a flash of the wrong theme. It defaults to 'light' if no theme is found.

```javascript
(function() { var theme = localStorage.getItem('omlx-chat-theme') || 'light'; document.documentElement.setAttribute('data-theme', theme); })();
```

--------------------------------

### Start and Stop Chat Statistics Polling

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Manages interval-based fetching of chat statistics. Ensures polling is started only once and can be stopped cleanly.

```javascript
rtStatsPolling(chatId = this.currentChatId) {
  if (!this.statsInterval) {
    this.statsInterval = setInterval(() => this.fetchStats(), 200);
  }
  this.fetchStats();
  if (chatId === this.currentChatId) {
    this.recentStats = {
      avg_prefill_tps: 0,
      avg_generation_tps: 0,
      thinking_time: 0,
      total_time: 0,
      prompt_tokens: 0,
      thinking_tokens: 0,
      total_tokens: 0,
    };
  }
},
stopStatsPolling() {
  if (this.statsInterval) {
    clearInterval(this.statsInterval);
    this.statsInterval = null;
  }
}
```

--------------------------------

### Get Variant Profile Label

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves the profile label for a variant message.

```javascript
variantProfileLabel(msg) {
  return this.resolveMessageProfile(msg) || '';
}
```

--------------------------------

### Get Variant Tab Label

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Generates a label for a variant tab, truncating long names.

```javascript
variantTabLabel(variant, index) {
  const meta = variant.meta || {};
  const name = meta.modelDisplay || variant.model || `Response ${index + 1}`;
  return name.length > 28 ? name.slice(0, 26) + '…' : name;
}
```

--------------------------------

### Get Variant Message Model Label

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves the display label for a variant message's model.

```javascript
variantModelLabel(msg) {
  const meta = msg.meta || {};
  return meta.modelDisplay || msg.model || this.currentModel || window.t('chat.assistant_label');
}
```

--------------------------------

### Clear All Chat History

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Prompts the user for confirmation before clearing all chat history, sessions, and starting a new chat.

```javascript
async clearAllHistory() {
      if (!confirm(window.t('chat.confirm_clear_all'))) return;
      this.stopAllStreams();
      this.chatHistory = [];
      this.chatSessions = {};
      this.streamSessions = {};
      this.saveChatHistory();
      await this.startNewChat();
    }
```

--------------------------------

### Apply Theme and System Listener

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Applies the selected theme (light, dark, or auto) and sets up a listener for system theme changes when 'auto' is selected.

```javascript
applyTheme() {
 // Clean up existing listener
 if (this.systemThemeListener) {
 window.matchMedia('(prefers-color-scheme: dark)').removeEventListener('change', this.systemThemeListener);
 this.systemThemeListener = null;
 }
 if (this.theme === 'auto') {
 // Detect system theme
 const prefersDark = window.matchMedia('(prefers-color-scheme: dark)').matches;
 this.activeTheme = prefersDark ? 'dark' : 'light';
 // Add listener for system theme changes
 this.systemThemeListener = (e) => {
 this.activeTheme = e.matches ? 'dark' : 'light';
 document.documentElement.setAttribute('data-theme', this.activeTheme);
 const lightTheme = document.getElementById('hljs-light-theme');
 const darkTheme = document.getElementById('hljs-dark-theme');
 if (lightTheme && darkTheme) {
 lightTheme.disabled = this.activeTheme === 'dark';
 darkTheme.disabled = this.activeTheme === 'light';
 }
 };
 window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', this.systemThemeListener);
 } else {
 // Use explicit theme
 this.activeTheme = this.theme;
 }
 document.documentElement.setAttribute('data-theme', this.activeTheme);
 const lightTheme = docu
```

--------------------------------

### Get Count of Stripped Images

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Returns an array representing the count of stripped (unresolved) image placeholders in the content.

```javascript
getStrippedImageCount(content) {
      if (!Array.isArray(content)) return [];
      const count = content.filter(p => p.type === 'image_url' && !p.image_url?.url).length;
      return Array.from({ length: count }, (_, i) => i);
    }
```

--------------------------------

### oQ Pipeline Steps

Source: https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md

The oQ pipeline focuses on streaming and involves loading tensors via mmap, applying model sanitize, measuring sensitivity, building a budget plan, quantizing and sharding, and saving configuration and tokenizer.

```text
1. Load tensors via mmap
2. Apply model sanitize
3. Measure per-layer sensitivity (temporary model load)
4. Build budget plan
5. Per-tensor quantize + shard flush
6. Save config + tokenizer
```

--------------------------------

### Get Copyable Markdown from Message

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves and formats the markdown content of a message for copying, cleaning up any 'thinking' indicators.

```javascript
getCopyableMarkdown(msg) {
      if (!msg) return '';
      const raw = typeof msg.content === 'string' ? msg.content : this.getTextContent(msg.content);
      if (!raw) return '';
      const { cleanText } = this.extractThinking(raw);
      return this.normalizeCopyText(cleanText);
    }
```

--------------------------------

### Get Turn Variants At User Index

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves all assistant messages (variants) associated with a specific user message index.

```javascript
getTurnVariantsAt(messages, userIndex) {
  if (!messages[userIndex] || messages[userIndex].role !== 'user') return [];
  const variants = [];
  for (let i = userIndex + 1; i < messages.length && messages[i].role !== 'user'; i++) {
    const m = messages[i];
    if (m.role === 'assistant' && m._ui !== false) variants.push(m);
  }
  return variants;
}
```

--------------------------------

### Select and Sync Model Settings

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Handles model selection, synchronizing UI settings to the session and saving settings for the previous model before switching.

```javascript
async selectModel(modelId) {
  const session = this.getChatSession(this.currentChatId, true);
  const gatewayId = this.resolveGatewayModelId(modelId);
  try {
    if (session?.model && session.model !== gatewayId) {
      this.syncSessionModelSettingsFromUi(session);
      this.saveModelSettingsForModel(session, session.model);
    }
    this.currentModel = gatewayId;
    if (session) {
      session.model = gatewayId;
      if (!this.loadModelS
```

--------------------------------

### Restart oMLX Server

Source: https://github.com/jundot/omlx/blob/main/README.md

Restarts the oMLX managed background server. This command is typically used on macOS or systems with a Homebrew installation.

```bash
omlx restart
```

--------------------------------

### Manage oMLX Server with CLI

Source: https://github.com/jundot/omlx/blob/main/README.md

Control the oMLX background server using CLI commands. Also shows how to run the server in the foreground with a specified model directory.

```bash
# Managed background server (macOS app or Homebrew install)
omlx start
omlx stop
omlx restart

# Foreground server attached to this terminal
omlx serve --model-dir ~/models
```

--------------------------------

### Stop oMLX Server

Source: https://github.com/jundot/omlx/blob/main/README.md

Stops the oMLX managed background server. This command is typically used on macOS or systems with a Homebrew installation.

```bash
omlx stop
```

--------------------------------

### Set Theme

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Sets the chat theme, saves it to local storage, and applies it.

```javascript
setTheme(theme) {
 this.theme = theme;
 localStorage.setItem('omlx-chat-theme', this.theme);
 this.applyTheme();
 }
```

--------------------------------

### Get Document File Information

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Extracts information about document files present in the content, including filename and whether data is available.

```javascript
getDocumentFiles(content) {
      if (!Array.isArray(content)) return [];
      return content
        .filter(p => p.type === 'file' && p.file)
        .map(p => ({ filename: p.file.filename || window.t('chat.document_not_available'), hasData: Boolean(p.file.file_data || p.file.data) }));
    }
```

--------------------------------

### DFlash Workflow Stages

Source: https://github.com/jundot/omlx/blob/main/docs/experimental/dflash_mlx_integration.md

Illustrates the sequential steps involved in the DFlash speculative decoding process, from prompt prefilling to repeating the generation cycle.

```text
1. PREFILL: target model processes entire prompt, captures hidden states
2. DRAFT:   draft model generates block of 16 tokens in parallel (block diffusion)
3. VERIFY:  target model verifies all 16 in one forward pass
4. ACCEPT:  greedy prefix match — longest matching prefix is committed
5. REPLAY:  cache rollback via tape replay for hybrid (GatedDeltaNet) models
6. REPEAT:  until max_tokens or EOS
```

--------------------------------

### Prefill Polling Management

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Manages the interval for polling the prefill status of a model. Starts and stops the polling process for a given chat session.

```javascript
startPrefillPolling(chatId) {
  const stream = this.getStreamSession(chatId, true);
  this.stopPrefillPolling(chatId);
  stream._prefillInterval = setInterval(() => this._pollPrefill(chatId), 500);
}
```

```javascript
stopPrefillPolling(chatId) {
  const stream = this.getStreamSession(chatId, false);
  if (stream?._prefillInterval) {
    clearInterval(stream._prefillInterval);
    stream._prefillInterval = null;
  }
}
```

--------------------------------

### Apply Generation Snapshot

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Applies generation settings from a snapshot. Handles thinking budget and related parameters.

```javascript
async applyGenerationSnapshot(generation) {
  const s = this.modelSettings;
  if (generation.enable_thinking === false) { s.enable_thinking = false; } else if (generation.thinking_budget != null) { s.enable_thinking = true; } const budget = generation.thinking_budget ?? generation.thinking_budget_tokens;
  if (budget != null) {
    s.enable_thinking = true;
    s.thinking_budget_enabled = true;
    s.thinking_budget_tokens = this.normalizeThinkingBudgetTokens(budget);
  } else if (s.enable_thinking === true) {
    s.thinking_budget_enabled = false;
    s.thinking_budget_tokens = null;
  } else {
    s.thinking_budget_enabled = false;
    s.thinking_budget_tokens = null;
  }
  this.modelSettings = s;
}
```

--------------------------------

### Get Text Content from Message

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Extracts plain text content from a message object, handling both string content and array-based content types.

```javascript
getTextContent(content) {
      if (typeof content === 'string') return content;
      if (Array.isArray(content)) {
        return content.filter(p => p.type === 'text').map(p => p.text).join('\n');
      }
      return '';
    }
```

--------------------------------

### Snapshot Generation Settings

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Captures current generation settings, including scalar parameters and thinking budget configuration.

```javascript
snapshotGenerationSettings() {
  const gen = {};
  const s = this.modelSettings;
  const scalarKeys = [
    'temperature',
    'max_tokens',
    'top_p',
    'top_k',
    'min_p',
    'repetition_penalty',
    'presence_penalty'
  ];
  for (const key of scalarKeys) {
    if (s[key] != null) gen[key] = s[key];
  }
  if (s.enable_thinking === true) {
    gen.chat_template_kwargs = { enable_thinking: true };
    if (s.thinking_budget_enabled) {
      gen.thinking_budget = this.normalizeThinkingBudgetTokens(s.thinking_budget_tokens);
    }
  } else if (s.enable_thinking === false) {
    gen.chat_template_kwargs = { enable_thinking: false };
  }
  return Object.keys(gen).length ? gen : null;
}
```

--------------------------------

### Get API Key from Local Storage

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves the API key stored in local storage. Returns the key if found, otherwise returns null.

```javascript
getApiKey() {
  return localStorage.getItem('omlx_chat_api_key') || '';
}
```

--------------------------------

### Add License Header

Source: https://github.com/jundot/omlx/blob/main/docs/CONTRIBUTING.md

Ensure all source files include the Apache 2.0 license identifier as a comment at the beginning of the file.

```python
# SPDX-License-Identifier: Apache-2.0
```

--------------------------------

### Execute Tool Calls

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Processes and executes tool calls in parallel. Records tool start times and handles potential errors during execution or argument parsing.

```javascript
const toolCalls = Object.values(toolCallsMap);
if (toolCalls.length > 0) {
  // Log the resolved tool set once — summarise when count is large
  const names = toolCalls.map(t => t.function.name).filter(Boolean);
  const uniqueNames = [...new Set(names)];
  const label = toolCalls.length > 3 ? `Calling ${uniqueNames.slice(0, 2).join(', ')} +${toolCalls.length - 2} more` : `Calling ${names.join(', ')}`;
  if (names.length) this.setEngineStatus(context.chatId, { text: label, icon: 'wrench' });

  // Store the assistant tool_calls turn (hidden from chat UI)
  chatSession.messages.push({
    id: this.newMessageId(),
    role: 'assistant',
    content: stream.streamingContent || null,
    reasoning_content: stream.streamingThinking || null,
    model: context.model,
    tool_calls: toolCalls,
    _profile: context._profile,
    _ui: false,
  });

  // Record per-tool start times for elapsed tracking
  toolCalls.forEach((tc) => {
    stream._toolTimers[tc.function.name] = Date.now();
  });

  // Execute all tools in parallel
  const results = await Promise.all(toolCalls.map(async (tc, i) => {
    const toolName = tc.function.name;
    let args = {};
    try {
      args = JSON.parse(tc.function.arguments || '{}');
    } catch (e) {}

    try {
      const timeoutSignal = AbortSignal.timeout?.(this.TOOL_TIMEOUT_MS);
      const { signal: combinedSignal, dispose: disposeCombinedSignal } = this.combineAbortSignals([
        stream.ab
      ]);
      // ... rest of the tool execution logic
    } catch (e) {
      // ... error handling
    }
  }));
}
```

--------------------------------

### Initialize Chat View

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Ensures chat elements are ready and initializes rendering and copy button functionalities.

```javascript
this.$nextTick(() => {
 const container = document.getElementById('messagesContainer');
 if (!container) return;
 this.addCodeCopyButtons(container);
 this.renderPendingMath(container);
 this.computeTimelineDots();
 this.computeTimelineActive();
 });
```

--------------------------------

### Adjust Max Concurrent Requests

Source: https://github.com/jundot/omlx/blob/main/README.md

Starts the oMLX server and sets the maximum number of concurrent requests to 16. The default is 8. Models should be in the specified directory.

```bash
omlx serve --model-dir ~/models --max-concurrent-requests 16
```

--------------------------------

### Initialize Chat Application State

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Sets up the initial state for the chat application, including API key status, UI preferences, chat history, model availability, streaming settings, and editing states.

```javascript
function chatApp() {
    return {
        // API Key
        apiKeySet: false,
        apiKeyInput: '',
        // UI State
        isMobile: window.innerWidth < 768,
        sidebarOpen: window.innerWidth >= 768,
        showLeftToggle: window.innerWidth < 768,
        theme: localStorage.getItem('omlx-chat-theme') || 'auto',
        allowSvg: localStorage.getItem('omlx-chat-allow-svg') === 'true',
        activeTheme: 'light',
        // Will be updated by applyTheme
        systemThemeListener: null,
        // Chat State
        currentChatId: null,
        chatHistory: [],
        _thinkingDomSeq: 0,
        _adminModelsList: null,
        _adminModelsListPromise: null,
        _thinkingBudgetBeforeInput: null,
        _thinkingBudgetStepping: false,
        chatSessions: {},
        messages: [],
        inputMessage: '',
        // Model State
        availableModels: [],
        currentModel: null,
        // Streaming State
        streamSessions: {},
        // Scroll state
        autoScrollEnabled: true,
        isUserScrolling: false,
        scrollTimeout: null,
        thinkingAutoScroll: true,
        thinkingScrollTimeout: null,
        thinkingScrollPosition: 0,
        // Save scroll position before DOM update
        // Edit State
        editingIndex: null,
        editContent: '',
        editImages: [],
        // Preserve images during edit
        renamingChatId: null,
        renameDraft: '',
        // Image Upload State
        uploadImages: [],
        // Array of { id, base64 } for pending images (max 10)
        uploadDocuments: [],
        // Array of { id, filename, mimeType, base64 }
        markitdownSettings: {
            enabled: true,
            max_file_size_mb: 25,
            max_files_per_request: 5,
        },
        isDragOver: false,
        // drag-and-drop state
        // Image Modal State
        imageModal: {
            show: false,
            src: ''
        },
        // VLM Detection State
        modelTypeMap: {},
        // { modelId: "llm"|"vlm"|"embedding"|"reranker" }
        aliasToGateway: {},
        // { aliasName: gatewayId } maps alias to directory name
        // Session Prompt
        systemPrompt: '',
        // Right Sidebar Tab
        rightSidebarTab: 'settings',
        // PROFILE Section State
        promptProfiles: [],
        activePromptProfile: null,
        promptDirty: false,
        editingPromptProfileName: false,
        newProfileNameDraft: '',
        profileDeleteConfirm: null,
        // MODEL SETTINGS Section State (shared prompt profiles from PROFILE tab)
        // MCP Tool Call Limits
        MAX_TOOL_DEPTH: 10,
        // Max recursive streamResponse calls for tool loops
        TOOL_TIMEOUT_MS: 30000,
        // Per-tool execution timeout (ms)
        // MCP tools & prefill polling
        mcpTools: [],
        // OpenAI-format tool definitions loaded from /v1/mcp/tools
        // Right Sidebar State
        rightSidebarOpen: window.innerWidth >= 1200,
        showRightToggle: window.innerWidth < 1200,
        modelSettingsDirty: false,
        speculativeEngine: 'none',
        speculativeDraftModel: '',
        isMtpCompatible: false,
        isVlm: false,
        recentStats: {
            avg_prefill_tps: 0,
            avg_generation_tps: 0,
            thinking_time: 0,
            total_time: 0
        },
        statsInterval: null,
        // Update check state
        updateAvailable: false,
        latestVersion: null,
        releaseUrl: null,
        versionHover: false,
        _updateCheckTimer: null,
        // Timeline navigation rail
        timelineDots: [],
        // [ { index, topPx } ]
        timelineActive: -1,
        timelineTooltip: {
            show: false,
            content: '',
            x: 0,
            y: 0
        },
        modelSettings: {
            temperature: null,
            max_tokens: null,
            top_p: null,
            top_k: null,
            min_p: null,
            repetition_penalty: null,
            presence_penalty: null,
            enable_thinking: null,
            thinking_budget_enabled: false,
            thinking_budget_tokens: null
        },
        defaultModelSettings() {
            return {
                temperature: null,
                max_tokens: null,
                top_p: null,
                top_k: null,
                min_p: null,
                repetition_penalty: null,
                presence_penalty: null,
                enable_thinking: null,
                thinking_budget_enabled: false,
                thinking_budget_tokens: null
            };
        },
        thinkingModeValue() {
            if (this.modelSettings.enable_thinking === false) return 'off';
            if (this.modelSettings.enable_thinking === true) {
                return this.modelSettings.thinking_budget_enabled ? 'on_limit' : 'on_unlimit';
            }
            return 'auto';
        },
        normalizeThinkingBudgetTokens(value) {
            if (value === '' || value == null) return 4096;
            const n = Number(value);
            if (!Number.isFinite(n)) return 4096;
            if (n <= 0) return 1;
            return Math.floor(n);
        },
        stepThinkingBudgetTokens(direction) {
            if (this.thinkingModeValue() !== 'on_limit') return;
            const step = 1024;
            let v = Number(this.modelSettings.thinking_budget_tokens);
            if (!Number.isFinite(v) || v <= 0) v = 0;
            if (direction > 0) {
                this.modelSettings.thinking_budget_tokens = v <= 0 ? 1 : v + step;
            } else if (v <= 1) {
                this.modelSettings.thinking_budget_tokens = 1;
            } else if (v <= step) {
                this.modelSettings.thinking_budget_tokens = 1;
            } else {
                this.modelSettings.thin
```

--------------------------------

### Get Active Variant Chain

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Retrieves the active variant chain from turn messages. Returns the last segment if no active variant ID is provided or found.

```javascript
getActiveVariantChain(turnMessages, activeVariantId) {
  if (!turnMessages.length) return [];
  const segments = this.splitTurnSegments(turnMessages);
  if (segments.length <= 1) return turnMessages;
  if (activeVariantId) {
    const match = segments.find(s => s.some(m => m.id === activeVariantId));
    if (match) return match;
  }
  return segments[segments.length - 1];
}
```

--------------------------------

### Add a New Prompt Profile

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Creates a new prompt profile with a unique name, persists it, and sets it as the active profile. Focuses the input for the new name.

```javascript
addNewPromptProfile() {
  let baseName = 'New Profile';
  let counter = 1;
  let name = baseName;
  while (this.promptProfiles.some(p => p.name === name)) {
    name = `${baseName} (${counter})`;
    counter++;
  }
  this.promptProfiles.push({ name, content: '' });
  this._persistPromptProfiles();
  this.activePromptProfile = name;
  this.systemPrompt = '';
  this.promptDirty = false;
  this.newProfileNameDraft = name;
  this.editingPromptProfileName = true;
  this.$nextTick(() => {
    const input = document.querySelector('input[x-model="newProfileNameDraft"]');
    if (input) input.focus();
  });
  const session = this.getChatSession(this.currentChatId, true);
  if (session) session.systemPrompt = '';
}
```

--------------------------------

### Select and Apply a Prompt Profile

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Sets the system prompt and active profile based on the selected profile name. Handles 'Custom' selection to clear association.

```javascript
selectPromptProfile(name) {
  if (!name) {
    // Custom — keep current prompt, just clear association
    this.activePromptProfile = null;
    const session = this.getChatSession(this.currentChatId, true);
    if (session) session.activeProfile = null;
    return;
  }
  const profile = this.promptProfiles.find(p => p.name === name);
  if (profile) {
    this.systemPrompt = profile.content;
    this.activePromptProfile = name;
    this.promptDirty = false;
    // Sync to current session
    const session = this.getChatSession(this.currentChatId, true);
    if (session) {
      session.systemPrompt = profile.content;
      session.activeProfile = this.activePromptProfile;
    }
  }
}
```

--------------------------------

### Build Swift macOS Bundle

Source: https://github.com/jundot/omlx/blob/main/packaging/README.md

Commands to build the full Swift macOS bundle. The --no-rebuild-donor option can be used to reuse the existing _export/ directory, potentially speeding up the build.

```bash
apps/omlx-mac/Scripts/build.sh release
```

```bash
apps/omlx-mac/Scripts/build.sh release --no-rebuild-donor
```

--------------------------------

### Run Specific Test File

Source: https://github.com/jundot/omlx/blob/main/docs/CONTRIBUTING.md

Execute tests for a particular file, such as 'tests/test_config.py'. Use the -v flag for verbose output.

```bash
pytest tests/test_config.py -v
```

--------------------------------

### Load Models and Capabilities

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Asynchronously loads models and their capabilities. Includes error handling for model loading failures.

```javascript
setTimeout(async () => { try { await this.loadModels(); if (this.currentModel) { await this.loadModelCapabilities(this.currentModel); } } catch (e) { console.error('Failed to refresh models on visibility:', e); } }, 300);
```

--------------------------------

### Apply Theme Immediately

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Applies the user's selected theme to the document's root element immediately to prevent a flash of unstyled content. It reads the theme from local storage or defaults to the system's preferred color scheme.

```javascript
// Apply theme immediately before render to prevent flash (match applyTheme / Alpine default)
(function () {
  const stored = localStorage.getItem('omlx-chat-theme');
  let theme = stored || 'auto';
  if (theme === 'auto') {
    theme = window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
  }
  document.documentElement.setAttribute('data-theme', theme);
})();
```

--------------------------------

### Start Editing a Message

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Initiates the message editing process for a given index. Prevents editing if a chat is streaming or another message is already being edited. Stores the original content and images.

```javascript
if (this.isCurrentChatStreaming() || this.editingIndex !== null) return;
this.editingIndex = index;
this.editContent = this.getTextContent(this.messages[index].content);
// Preserve images from the original message
this.editImages = this.getImageUrls(this.messages[index].content);
```

--------------------------------

### Configure oMLX Service Defaults

Source: https://github.com/jundot/omlx/blob/main/README.md

Customize oMLX service behavior by setting environment variables or running `omlx serve` once to persist settings. The service defaults to `~/.omlx/models` and port 8000.

```bash
# Example of setting environment variables (not directly in source, but implied by text)
# export OMLX_MODEL_DIR=/your/path
# export OMLX_PORT=9000

omlx serve --model-dir /your/path
```

--------------------------------

### Build and Run macOS App

Source: https://github.com/jundot/omlx/blob/main/README.md

Build a release version of the oMLX macOS application. This script stages a runnable .app bundle, including Python layers managed by venvstacks.

```bash
# Stage a runnable oMLX.app (xcodebuild + venvstacks Python layers + ad-hoc sign)
apps/omlx-mac/Scripts/build.sh release

# Result lands at apps/omlx-mac/build/Stage/oMLX.app
open apps/omlx-mac/build/Stage/oMLX.app

# Force a fresh venvstacks rebuild (otherwise it's cached by fingerprint)
apps/omlx-mac/Scripts/build.sh release --rebuild-donor
```

--------------------------------

### Get Active Variant For User

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Determines the currently active assistant variant for a given user message index. Falls back to the last variant if no specific active ID is set.

```javascript
getActiveVariantForUser(messages, userIndex) {
  const variants = [];
  for (let i = userIndex + 1; i < messages.length && messages[i].role !== 'user'; i++) {
    const m = messages[i];
    if (m.role === 'assistant' && m._ui !== false) variants.push(m);
  }
  if (!variants.length) return null;
  const activeId = messages[userIndex]._activeVariantId;
  if (activeId) {
    return variants.find(v => v.id === activeId) || variants[variants.length - 1];
  }
  return variants[variants.length - 1];
}
```

--------------------------------

### Initiate Renaming of a Prompt Profile

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Sets up the state for renaming the currently active prompt profile.

```javascript
startRenamingPromptProfile() {
  if (!this.activePromptProfile) return;
  this.newProfileNameDraft = this.activePromptProfile;
  this.editingPromptProfileName = true;
}
```

--------------------------------

### GPTQ Column-wise Quantization Algorithm

Source: https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md

Illustrates the GPTQ algorithm's approach to quantizing weights column by column, including error compensation for remaining columns. This method minimizes actual output error.

```pseudocode
For each column i:
    q[i] = round_to_grid(w[i])
    error = (w[i] - q[i]) / H_inv[i, i]
    w[i+1:] -= error * H_inv[i, i+1:]    # compensate remaining columns
```

--------------------------------

### Get API Turn Messages

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Filters and formats messages for API calls, handling user turns and their associated variants. Can exclude variants at a specific user index or regenerate a chain.

```javascript
getApiTurnMessages(messages, excludeVariantsAtUserIndex = null, regenChainId = null) {
  const result = [];
  let i = 0;
  while (i < messages.length) {
    const msg = messages[i];
    if (msg.role !== 'user') {
      i++;
      continue;
    }
    const userIdx = i;
    result.push(msg);
    i++;
    const turn = [];
    while (i < messages.length && messages[i].role !== 'user') {
      turn.push(messages[i]);
      i++;
    }
    if (userIdx === excludeVariantsAtUserIndex) {
      if (regenChainId) {
        result.push(...this.getVariantChainFrom(turn, regenChainId));
      }
    } else {
      result.push(...this.getActiveVariantChain(turn, msg._activeVariantId));
    }
  }
  return result;
}
```

--------------------------------

### Load Image File

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Initiates the loading of an image file. Includes a check to ensure the file type is indeed an image.

```javascript
loadImageFile(file) {
  if (!file.type.startsWith('image/')) {
    alert(window.t('chat.error.invalid_im
```

--------------------------------

### Apply Session Model Settings

Source: https://github.com/jundot/omlx/blob/main/omlx/admin/templates/chat.html

Applies model settings from a session object to the current instance. It merges default settings with session settings and normalizes thinking budget tokens.

```javascript
applySessionModelSettings(session) {
    if (!session?.modelSettings) return false;
    this.modelSettings = {
      ...this.defaultModelSettings(),
      ...this.cloneData(session.modelSettings),
    };
    if (this.modelSettings.thinking_budget_enabled) {
      this.modelSettings.thinking_budget_tokens = this.normalizeThinkingBudgetTokens(
        this.modelSettings.thinking_budget_tokens
      );
    }
    this.speculativeEngine = session.speculativeEngine ?? 'none';
    this.speculativeDraftModel = session.speculativeDraftModel ?? '';
    this.modelSettingsDirty = false;
    return true;
  }
```

--------------------------------

### Clone oMLX Repository

Source: https://github.com/jundot/omlx/blob/main/docs/CONTRIBUTING.md

Clone your forked oMLX repository to your local machine. Navigate into the cloned directory.

```bash
git clone https://github.com/<your-username>/omlx.git
cd omlx
```