### Install Dependencies Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/README.md Installs project dependencies using uv or pip. ```bash uv sync ``` ```bash pip install -e . ``` -------------------------------- ### Fast-dLLM MLX Benchmark CLI Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Example usage of the fast-dLLM benchmark CLI, including options for model, generation parameters, warmup, response printing, and output file formats (CSV and JSON). ```bash # Fast-dLLM benchmark with warmup and response printing uv run python -m benchmarks.fast_dllm_mlx_benchmark \ --model mlx-community/DiffuCoder-7B-cpGRPO-8bit \ --trust-remote-code \ --max-new-tokens 128 \ --steps 20 \ --block-length 32 \ --threshold 0.9 \ --temperature 0.0 \ --warmup \ --print-response \ --csv results_fast.csv \ --json results_fast.json ``` -------------------------------- ### Dream MLX Benchmark CLI Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Example usage of the Dream MLX benchmark CLI, demonstrating configuration for model, generation parameters, compilation, and output file formats. ```bash # Dream baseline benchmark uv run python -m benchmarks.dream_mlx_benchmark \ --model mlx-community/DiffuCoder-7B-cpGRPO-8bit \ --trust-remote-code \ --max-new-tokens 128 \ --steps 20 \ --use-compile \ --csv results_dream.csv \ --json results_dream.json ``` -------------------------------- ### Run Qwen MLX LM Benchmark Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/README.md Executes the Qwen MLX LM benchmark with specified model and generation parameters. Supports temperature and top-p sampling. ```bash uv run python -m benchmarks.qwen_mlx_lm_benchmark \ --model mlx-community/Qwen2.5-Coder-7B-Instruct-8bit \ --max-new-tokens 128 \ --temp 0.0 \ --top-p 1.0 \ --warmup ``` -------------------------------- ### Benchmark CLI with Custom Prompt File Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Shows how to run the fast-dLLM benchmark using a custom prompt file, specifying generation parameters. ```bash # Custom prompt file (plain text, one prompt per line) uv run python -m benchmarks.fast_dllm_mlx_benchmark \ --model mlx-community/DiffuCoder-7B-cpGRPO-8bit \ --trust-remote-code \ --prompt-file my_prompts.txt \ --max-new-tokens 64 \ --steps 16 \ --block-length 32 ``` -------------------------------- ### Benchmark Utilities: Loading Prompts Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Demonstrates loading prompts from default locations or custom JSON files using the `load_prompts` utility. ```python from pathlib import Path from benchmarks.utils import load_prompts, BenchmarkResult, write_csv, write_json, print_summary # Load default prompts from prompts/*.txt prompts = load_prompts(None) for p in prompts: print(p.prompt_type, "| ", p.text[:60]) # coding | Write a Python function that ... # math | Solve: if x^2 - 5x + 6 = 0, find x. # Load from a custom JSON file: ["prompt1", "prompt2", ...] custom = load_prompts(Path("my_prompts.json")) ``` -------------------------------- ### Run Fast-dLLM MLX Benchmark Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/README.md Executes the Fast-dLLM MLX benchmark with specified model and generation parameters. Use --print-response to see generated output. ```bash uv run python -m benchmarks.fast_dllm_mlx_benchmark \ --model mlx-community/DiffuCoder-7B-cpGRPO-8bit \ --trust-remote-code \ --max-new-tokens 128 \ --steps 20 \ --block-length 32 \ --threshold 0.9 \ --warmup ``` ```bash --print-response ``` -------------------------------- ### DualKVCache and make_dual_prompt_cache Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt DualKVCache extends mlx_lm's KVCache to support both standard append (replace=False) and in-place block replacement (replace=True, position=). The in-place path overwrites a fixed-position slice without reallocating, enabling the Fast-dLLM pattern of caching the full prompt+generation context once and cheaply refreshing a block during denoising sub-steps. make_dual_prompt_cache allocates one DualKVCache per model layer. ```APIDOC ## `DualKVCache` and `make_dual_prompt_cache` — Dual-mode KV cache `DualKVCache` extends `mlx_lm`'s `KVCache` to support both standard append (`replace=False`) and in-place block replacement (`replace=True, position=`). The in-place path overwrites a fixed-position slice without reallocating, enabling the Fast-dLLM pattern of caching the full prompt+generation context once and cheaply refreshing a block during denoising sub-steps. `make_dual_prompt_cache` allocates one `DualKVCache` per model layer. ```python import mlx.core as mx from fast_dllm_mlx import load from fast_dllm_mlx.cache import DualKVCache, make_dual_prompt_cache model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) # Allocate one cache per layer caches = make_dual_prompt_cache(model) print(len(caches), type(caches[0])) # 28 # Full-sequence prefill (standard append mode) x = mx.array([[1, 2, 3, 4, 5]]) # [batch=1, seq=5] logits = model(x, mask="full", cache=caches) # caches[i].offset == 5 after this call # In-place replacement for block at position 3 (len-2 block) x_block = mx.array([[6, 7]]) # replacement block [batch=1, block=2] logits2 = model( x_block, mask="full", cache=caches, cache_position=3, replace_cache=True, ) # Keys/values at positions [3, 5) are overwritten in-place; .offset stays at 5 ``` ``` -------------------------------- ### Dream MLX Inference: Standard and Streaming Generation Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Demonstrates loading the model and tokenizer, initializing the DreamGenerator, and performing both standard text generation and step-by-step streaming generation with different unmasking algorithms. ```python from dream_mlx import load, DreamGenerationConfig, DreamGenerator, stream_diffusion_generate model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) generator = DreamGenerator(model=model, tokenizer=tokenizer) # Standard generation with maskgit_plus strategy text = generator.generate( "Write a bubble sort in Python.", generation_config=DreamGenerationConfig( max_new_tokens=128, steps=20, alg="maskgit_plus", temperature=0.0, use_compile=True, ), ) print(text) # Streaming: watch tokens get filled in step by step for step_response in stream_diffusion_generate( model, tokenizer, "Explain gradient descent.", max_new_tokens=64, steps=10, alg="origin", temperature=0.4, ): print(f"[{step_response.finish_reason}] {step_response.text[:80]}") # [step_0/10] [MASK][MASK][MASK]... # [step_1/10_masked_50] Gradient[MASK] descent[MASK]... # ... # [complete] Gradient descent is an iterative optimisation algorithm... ``` -------------------------------- ### Benchmark Utilities: Persisting and Summarizing Results Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Illustrates constructing `BenchmarkResult` objects and using `write_csv`, `write_json`, and `print_summary` to save and display benchmark outcomes. ```python # Construct and persist results results = [ BenchmarkResult( model_label="fast-dllm-mlx", model_id="mlx-community/DiffuCoder-7B-cpGRPO-8bit", device="mlx", prompt_index=1, prompt_type="coding", prompt_chars=45, prompt_tokens=12, generated_tokens=128, time_to_first_token_s=None, generation_tokens_per_s=None, overall_tokens_per_s=47.3, total_time_s=2.71, ) ] write_csv(Path("out.csv"), results) write_json(Path("out.json"), results) print_summary(results) # Benchmark summary ``` -------------------------------- ### load Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Loads a model and tokenizer from a Hugging Face repository ID or a local path. It downloads weights, instantiates the Dream Model, and returns the paired AutoTokenizer. The pad_token is set to eos_token if the tokenizer lacks an explicit pad token. ```APIDOC ## `load` — Load model and tokenizer Resolves a Hugging Face repo ID or local path, downloads weights, instantiates the Dream `Model`, and returns the paired `AutoTokenizer`. Sets `pad_token` to `eos_token` when the tokenizer has no explicit pad token. ```python from fast_dllm_mlx import load # Load from Hugging Face Hub (downloads on first call, cached afterwards) model, tokenizer = load( "mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True, ) # Load from a local directory model, tokenizer = load( "/path/to/local/model", tokenizer_config={"use_fast": True}, model_config={"dtype": "float16"}, lazy=False, ) print(type(model)) # print(tokenizer.vocab_size) # e.g. 152064 ``` ``` -------------------------------- ### Run Dream MLX Benchmark Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/README.md Executes the Dream MLX benchmark with specified model and generation parameters. Includes compilation optimization. ```bash uv run python -m benchmarks.dream_mlx_benchmark \ --model mlx-community/DiffuCoder-7B-cpGRPO-8bit \ --trust-remote-code \ --max-new-tokens 128 \ --steps 20 \ --use-compile ``` -------------------------------- ### DreamGenerationConfig Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Dataclass for bundling generation settings for the Fast-dLLM path. It includes parameters for output budget, denoising iterations, block size, confidence threshold, sampling methods, and compilation options. ```APIDOC ## `DreamGenerationConfig` — Generation hyperparameters (fast_dllm_mlx) Dataclass that bundles all generation settings for the Fast-dLLM path. Key fields: `max_new_tokens` / `max_length` for output budget; `steps` for total denoising iterations; `block_length` for the semi-autoregressive block size (must evenly divide `max_new_tokens`); `threshold` for the confidence gate; `temperature`, `top_p`, `top_k` for sampling; `dual_cache` must remain `True` (the only supported mode); `use_compile` enables `mx.compile` for the greedy path. ```python from fast_dllm_mlx import DreamGenerationConfig # Greedy, compiled — fastest on Apple Silicon fast_config = DreamGenerationConfig( max_new_tokens=128, steps=20, # steps must be divisible by (max_new_tokens / block_length) block_length=32, # 128 / 32 = 4 blocks, 20 / 4 = 5 steps per block threshold=0.9, # commit tokens with >= 90% confidence in parallel temperature=0.0, # greedy dual_cache=True, use_compile=True, ) # Stochastic with nucleus sampling stochastic_config = DreamGenerationConfig( max_new_tokens=256, steps=40, block_length=32, threshold=0.85, temperature=0.4, top_p=0.95, use_compile=False, # compile not used when temperature > 0 ) ``` ``` -------------------------------- ### Python HTTP Rate Limiter (Token Bucket) Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/prompts/coding.txt An HTTP rate limiter implementation in Python using the token bucket algorithm. Shows the core class and a demonstration script. ```Python import time class TokenBucketRateLimiter: def __init__(self, capacity: int, refill_rate: float): self.capacity = capacity self.refill_rate = refill_rate self.tokens = capacity self.last_refill_time = time.time() def consume(self, tokens_to_consume: float = 1.0) -> bool: self._refill() if self.tokens >= tokens_to_consume: self.tokens -= tokens_to_consume return True return False def _refill(self): now = time.time() time_elapsed = now - self.last_refill_time tokens_to_add = time_elapsed * self.refill_rate self.tokens = min(self.capacity, self.tokens + tokens_to_add) self.last_refill_time = now # Example Usage: # Allow 5 requests per second, with a burst capacity of 10 rate_limiter = TokenBucketRateLimiter(capacity=10, refill_rate=5) for i in range(15): if rate_limiter.consume(): print(f"Request {i+1}: Allowed") else: print(f"Request {i+1}: Denied (Rate Limited)") # Simulate some delay between requests time.sleep(0.1) print("\nWaiting for tokens to refill...") time.sleep(2) # Wait for 2 seconds for i in range(5): if rate_limiter.consume(): print(f"Request {i+16}: Allowed") else: print(f"Request {i+16}: Denied (Rate Limited)") time.sleep(0.1) ``` -------------------------------- ### Dual-Mode KV Cache for Efficient Generation Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Employ `DualKVCache` and `make_dual_prompt_cache` for advanced KV caching. This supports standard append mode and in-place block replacement, crucial for the Fast-dLLM pattern of caching prompts and efficiently refreshing generation blocks. ```python import mlx.core as mx from fast_dllm_mlx import load from fast_dllm_mlx.cache import DualKVCache, make_dual_prompt_cache model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) # Allocate one cache per layer caches = make_dual_prompt_cache(model) print(len(caches), type(caches[0])) # Full-sequence prefill (standard append mode) x = mx.array([[1, 2, 3, 4, 5]]) # [batch=1, seq=5] logits = model(x, mask="full", cache=caches) # caches[i].offset == 5 after this call # In-place replacement for block at position 3 (len-2 block) x_block = mx.array([[6, 7]]) # replacement block [batch=1, block=2] logits2 = model( x_block, mask="full", cache=caches, cache_position=3, replace_cache=True, ) # Keys/values at positions [3, 5) are overwritten in-place; .offset stays at 5 ``` -------------------------------- ### Load Model and Tokenizer Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Loads a model and tokenizer from a Hugging Face repository ID or a local path. Supports custom tokenizer and model configurations, and lazy loading. ```python from fast_dllm_mlx import load # Load from Hugging Face Hub (downloads on first call, cached afterwards) model, tokenizer = load( "mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True, ) # Load from a local directory model, tokenizer = load( "/path/to/local/model", tokenizer_config={"use_fast": True}, model_config={"dtype": "float16"}, lazy=False, ) print(type(model)) # print(tokenizer.vocab_size) # e.g. 152064 ``` -------------------------------- ### Stream Diffusion Generation with Per-Completion Response Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Use `stream_diffusion_generate` for streaming generation that yields a `DreamGenerationResponse` upon completion. This response includes timing metrics and peak memory usage, making it suitable for benchmarking or when structured output is needed. ```python from fast_dllm_mlx import load, stream_diffusion_generate, DreamGenerationConfig model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, ) for response in stream_diffusion_generate(model, tokenizer, "Sort a list in Python.", generation_config=config): print(response.text) print(f"Prompt tokens : {response.prompt_tokens}") print(f"Generated : {response.generation_tokens} tokens") print(f"Gen TPS : {response.generation_tps:.1f}") print(f"Peak memory : {response.peak_memory:.2f} GB") print(f"Finish reason : {response.finish_reason}") ``` -------------------------------- ### Python LRU Cache Implementation Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/prompts/coding.txt A complete Python LRU cache class from scratch. Includes method definitions, error handling, and demonstrates insertion, lookup, update, and eviction. ```Python from collections import OrderedDict class LRUCache: def __init__(self, capacity: int): self.cache = OrderedDict() self.capacity = capacity def get(self, key: int) -> int: if key not in self.cache: return -1 else: self.cache.move_to_end(key) return self.cache[key] def put(self, key: int, value: int) -> None: if key in self.cache: self.cache.move_to_end(key) elif len(self.cache) >= self.capacity: self.cache.popitem(last=False) self.cache[key] = value # Example Usage: lru_cache = LRUCache(2) lru_cache.put(1, 1) lru_cache.put(2, 2) print(lru_cache.get(1)) # returns 1 lru_cache.put(3, 3) # evicts key 2 print(lru_cache.get(2)) # returns -1 (not found) lru_cache.put(4, 4) # evicts key 1 print(lru_cache.get(1)) # returns -1 (not found) print(lru_cache.get(3)) # returns 3 print(lru_cache.get(4)) # returns 4 ``` -------------------------------- ### diffusion_generate_step Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Core stateless function that runs the complete Fast-dLLM denoising loop. Pads the prompt to `max_length` with mask tokens, then for each semi-autoregressive block performs a full-context prefill via `DualKVCache` followed by `steps_per_block` replacement sub-steps, committing confident tokens in parallel. Requires `dual_cache=True` and that `max_new_tokens` is divisible by `block_length`, and `steps` by the resulting `num_blocks`. ```APIDOC ## `diffusion_generate_step` — Low-level block-denoising kernel (fast_dllm_mlx) Core stateless function that runs the complete Fast-dLLM denoising loop. Pads the prompt to `max_length` with mask tokens, then for each semi-autoregressive block performs a full-context prefill via `DualKVCache` followed by `steps_per_block` replacement sub-steps, committing confident tokens in parallel. Requires `dual_cache=True` and that `max_new_tokens` is divisible by `block_length`, and `steps` by the resulting `num_blocks`. ```python import mlx.core as mx from fast_dllm_mlx import load, DreamGenerationConfig from fast_dllm_mlx.generate import diffusion_generate_step model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) prompt_ids = tokenizer.encode("What is 2+2?", add_special_tokens=True) prompt = mx.array(prompt_ids)[None, :] # [1, prompt_len] config = DreamGenerationConfig( max_new_tokens=32, steps=8, # 32 / 32 = 1 block, 8 steps per block block_length=32, threshold=0.9, temperature=0.0, dual_cache=True, use_compile=True, ) output = diffusion_generate_step(prompt, model, config) # [1, max_length] generated_ids = output[0, prompt.shape[1]:].tolist() print(tokenizer.decode(generated_ids, skip_special_tokens=True)) # 4 ``` ``` -------------------------------- ### DreamGenerationConfig for Fast-dLLM Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Configures generation hyperparameters for the Fast-dLLM path. Adjust `max_new_tokens`, `steps`, `block_length`, and `threshold` for performance and output quality. `use_compile` enables MLX compilation for greedy decoding. ```python from fast_dllm_mlx import DreamGenerationConfig # Greedy, compiled — fastest on Apple Silicon fast_config = DreamGenerationConfig( max_new_tokens=128, steps=20, # steps must be divisible by (max_new_tokens / block_length) block_length=32, # 128 / 32 = 4 blocks, 20 / 4 = 5 steps per block threshold=0.9, # commit tokens with >= 90% confidence in parallel temperature=0.0, # greedy dual_cache=True, use_compile=True, ) # Stochastic with nucleus sampling stochastic_config = DreamGenerationConfig( max_new_tokens=256, steps=40, block_length=32, threshold=0.85, temperature=0.4, top_p=0.95, use_compile=False, # compile not used when temperature > 0 ) ``` -------------------------------- ### Low-Level Block-Denoising Kernel Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Use `diffusion_generate_step` for the core stateless Fast-dLLM denoising loop. This function handles prompt padding, full-context prefill, and replacement sub-steps for confident token generation. Ensure `dual_cache=True` and divisibility constraints for `max_new_tokens` and `steps`. ```python import mlx.core as mx from fast_dllm_mlx import load, DreamGenerationConfig from fast_dllm_mlx.generate import diffusion_generate_step model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) prompt_ids = tokenizer.encode("What is 2+2?", add_special_tokens=True) prompt = mx.array(prompt_ids)[None, :] # [1, prompt_len] config = DreamGenerationConfig( max_new_tokens=32, steps=8, # 32 / 32 = 1 block, 8 steps per block block_length=32, threshold=0.9, temperature=0.0, dual_cache=True, use_compile=True, ) output = diffusion_generate_step(prompt, model, config) # [1, max_length] generated_ids = output[0, prompt.shape[1]:].tolist() print(tokenizer.decode(generated_ids, skip_special_tokens=True)) ``` -------------------------------- ### Object-Oriented Generation Wrapper with DreamGenerator Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Utilize `DreamGenerator` as a stateful wrapper for `model` and `tokenizer`. Its `generate` method is the primary integration point for application code and benchmarks, delegating to appropriate generation functions. ```python from fast_dllm_mlx import DreamGenerator, DreamGenerationConfig, load model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) generator = DreamGenerator(model=model, tokenizer=tokenizer) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, use_compile=True, ) result = generator.generate( "Implement binary search in Python.", generation_config=config, ) print(result) ``` -------------------------------- ### Python Topological Sort with Cycle Detection Source: https://github.com/macpaw/fast-dllm-mlx/blob/main/prompts/coding.txt A Python solution for topological sorting of a directed graph, including cycle detection. Features type hints and time complexity analysis. ```Python from collections import defaultdict, deque from typing import List, Dict, Tuple def topological_sort(graph: Dict[str, List[str]]) -> Tuple[List[str], bool]: """Performs topological sort on a directed graph and detects cycles. Args: graph: A dictionary representing the graph where keys are nodes and values are lists of adjacent nodes. Returns: A tuple containing a list of nodes in topological order and a boolean indicating if a cycle was detected. """ in_degree = {node: 0 for node in graph} for node in graph: for neighbor in graph[node]: in_degree[neighbor] = in_degree.get(neighbor, 0) + 1 queue = deque([node for node in graph if in_degree[node] == 0]) result = [] while queue: node = queue.popleft() result.append(node) for neighbor in graph.get(node, []): in_degree[neighbor] -= 1 if in_degree[neighbor] == 0: queue.append(neighbor) if len(result) == len(graph): return result, False # No cycle detected else: return result, True # Cycle detected # Example Usage: graph1 = { 'A': ['B', 'C'], 'B': ['D'], 'C': ['D'], 'D': [] } sorted_nodes1, has_cycle1 = topological_sort(graph1) print(f"Graph 1 - Sorted: {sorted_nodes1}, Has Cycle: {has_cycle1}") graph2 = { 'A': ['B'], 'B': ['C'], 'C': ['A'] # Cycle } sorted_nodes2, has_cycle2 = topological_sort(graph2) print(f"Graph 2 - Sorted: {sorted_nodes2}, Has Cycle: {has_cycle2}") graph3 = { '5': ['2', '0'], '4': ['0', '1'], '2': ['3'], '3': ['1'], '1': [], '0': [] } sorted_nodes3, has_cycle3 = topological_sort(graph3) print(f"Graph 3 - Sorted: {sorted_nodes3}, Has Cycle: {has_cycle3}") ``` -------------------------------- ### diffusion_generate Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Performs single-call text generation using the Fast-dLLM path. It encodes the prompt, runs the generation steps, decodes the output, and returns the final string. Accepts `DreamGenerationConfig` or keyword arguments that are merged into the configuration. ```APIDOC ## `diffusion_generate` — Single-call text generation (fast_dllm_mlx) Encodes the prompt (applying the chat template when available), runs `diffusion_generate_step`, decodes the generated token slice, and returns the final string. Accepts `DreamGenerationConfig` or ad-hoc `**kwargs` that are merged into the config. ```python from fast_dllm_mlx import load, diffusion_generate, DreamGenerationConfig model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, use_compile=True, ) text = diffusion_generate(model, tokenizer, "Write a Python function that reverses a string.", generation_config=config) print(text) # def reverse_string(s: str) -> str: # return s[::-1] # kwargs shorthand (merged into default DreamGenerationConfig) text2 = diffusion_generate( model, tokenizer, "Explain diffusion language models in one sentence.", max_new_tokens=64, steps=16, block_length=32, threshold=0.9, temperature=0.0, ) print(text2) ``` ``` -------------------------------- ### Single-Call Text Generation with diffusion_generate Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Generates text using the Fast-dLLM path by encoding prompts, performing denoising steps, and decoding the output. Accepts `DreamGenerationConfig` or keyword arguments for configuration. ```python from fast_dllm_mlx import load, diffusion_generate, DreamGenerationConfig model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, use_compile=True, ) text = diffusion_generate(model, tokenizer, "Write a Python function that reverses a string.", generation_config=config) print(text) # def reverse_string(s: str) -> str: # return s[::-1] # kwargs shorthand (merged into default DreamGenerationConfig) text2 = diffusion_generate( model, tokenizer, "Explain diffusion language models in one sentence.", max_new_tokens=64, steps=16, block_length=32, threshold=0.9, temperature=0.0, ) print(text2) ``` -------------------------------- ### DreamGenerator Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Stateful wrapper holding a `model` and `tokenizer` pair. Its `generate` method delegates to `diffusion_generate` (fast_dllm_mlx) or `diffusion_generate` (dream_mlx) depending on which module it was constructed from. Intended as the primary integration surface in application code and benchmarks. ```APIDOC ## `DreamGenerator` — Object-oriented generation wrapper Stateful wrapper holding a `model` and `tokenizer` pair. Its `generate` method delegates to `diffusion_generate` (fast_dllm_mlx) or `diffusion_generate` (dream_mlx) depending on which module it was constructed from. Intended as the primary integration surface in application code and benchmarks. ```python from fast_dllm_mlx import DreamGenerator, DreamGenerationConfig, load model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) generator = DreamGenerator(model=model, tokenizer=tokenizer) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, use_compile=True, ) result = generator.generate( "Implement binary search in Python.", generation_config=config, ) print(result) # def binary_search(arr, target): # lo, hi = 0, len(arr) - 1 # ... ``` ``` -------------------------------- ### stream_diffusion_generate Source: https://context7.com/macpaw/fast-dllm-mlx/llms.txt Generator that runs the fast diffusion pipeline and yields a single DreamGenerationResponse upon completion, carrying timing metrics and peak memory usage. Useful for benchmarking or when callers need structured output rather than a bare string. ```APIDOC ## `stream_diffusion_generate` — Streaming generation with per-completion response (fast_dllm_mlx) Generator that runs the same fast diffusion pipeline and yields a single `DreamGenerationResponse` upon completion, carrying timing metrics and peak memory usage. Useful for benchmarking or when callers need structured output rather than a bare string. ```python from fast_dllm_mlx import load, stream_diffusion_generate, DreamGenerationConfig model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-8bit", trust_remote_code=True) config = DreamGenerationConfig( max_new_tokens=128, steps=20, block_length=32, threshold=0.9, temperature=0.0, ) for response in stream_diffusion_generate(model, tokenizer, "Sort a list in Python.", generation_config=config): print(response.text) print(f"Prompt tokens : {response.prompt_tokens}") print(f"Generated : {response.generation_tokens} tokens") print(f"Gen TPS : {response.generation_tps:.1f}") print(f"Peak memory : {response.peak_memory:.2f} GB") print(f"Finish reason : {response.finish_reason}") # Output example: # sorted_list = sorted(my_list) # Prompt tokens : 18 # Generated : 128 tokens # Gen TPS : 47.3 # Peak memory : 8.21 GB # Finish reason : complete ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.