"
# Fixture test: ❌ fails if "
" is tokenized as ERROR instead of NAME_TAG
assert tokens[1].type == TokenType.NAME_TAG
```
--------------------------------
### Define Complete SyntaxPalette Scheme in Python
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/custom-palettes.md
This example demonstrates the structure of the `SyntaxPalette` dataclass, defining colors for a comprehensive set of semantic roles including control flow, data literals, identifiers, documentation, and feedback. It also shows how to set style modifiers like bold and italic for specific elements. The `SyntaxPalette` class is imported from `rosettes.themes`.
```python
from rosettes.themes import SyntaxPalette
palette = SyntaxPalette(
# Required fields
name="ocean-dark",
background="#0a192f",
text="#8892b0",
# Control & Structure
control_flow="#ff79c6", # if, for, while, return
declaration="#bd93f9", # def, class, let, const
import_="#ff79c6", # import, from, use
# Data & Literals
string="#50fa7b", # "hello", 'world'
number="#f1fa8c", # 42, 3.14
boolean="#bd93f9", # True, False
# Identifiers
type_="#8be9fd", # int, str, MyClass
function="#50fa7b", # function names
variable="#f8f8f2", # variable names
constant="#bd93f9", # CONSTANTS
# Documentation
comment="#6272a4", # # comments
docstring="#6272a4", # """docstrings"""
# Feedback (diffs, errors)
error="#ff5555",
warning="#ffb86c",
added="#50fa7b",
removed="#ff5555",
# Additional
muted="#6272a4", # less important elements
punctuation="#f8f8f2", # brackets, commas
operator="#ff79c6", # +, -, *, /
attribute="#50fa7b", # @decorator, .attribute
namespace="#f8f8f2", # module.submodule
tag="#ff79c6", # HTML/XML tags
regex="#f1fa8c", # regular expressions
escape="#bd93f9", # escape sequences
# Style modifiers
bold_control=True, # bold keywords
bold_declaration=True, # bold def/class
bold_type=False, # bold type names (optional)
italic_comment=True, # italic comments
italic_docstring=True, # italic docstrings
italic_variable=False, # italic variables (optional)
)
```
--------------------------------
### Python Language Keywords and Constructs
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-test-hardening.md
Demonstrates fundamental Python keywords like if/elif/else, for loops with continue/break, function and class definitions, exception handling (try/except), and import statements. It also includes an example of an asynchronous function.
```python
if True:
pass
elif False:
pass
else:
pass
for x in range(10):
continue
break
def func():
return None
class MyClass:
pass
try:
raise Exception
except Exception:
pass
import os
from sys import path
async def async_func():
await something()
```
--------------------------------
### Install Rosettes using pip
Source: https://github.com/lbliii/rosettes/blob/main/site/content/releases/0.2.0.md
Installs the Rosettes package with a version greater than or equal to 0.2.0. Requires Python 3.14 or higher.
```bash
pip install rosettes>=0.2.0
```
--------------------------------
### Basic Syntax Highlighting with Rosettes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/releases/0.1.0.md
Demonstrates basic syntax highlighting using the `highlight` function from the Rosettes library. It shows how to generate HTML output by default and how to specify terminal output.
```python
from rosettes import highlight
# HTML output (default)
html = highlight("def hello(): print('world')", "python")
# Terminal output
ansi = highlight("def hello(): print('world')", "python", formatter="terminal")
```
--------------------------------
### Retrieve Formatter Instance (Python)
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/formatters/_index.md
Demonstrates how to get an instance of a specific formatter using its name with the `get_formatter()` function and access its properties.
```python
from rosettes import get_formatter
formatter = get_formatter("terminal")
print(formatter.name) # 'terminal'
```
--------------------------------
### Dockerfile: Multi-Stage Build for Frontend and Nginx
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Demonstrates a multi-stage Docker build. The first stage builds a Node.js frontend application, and the second stage copies the built artifacts into an Nginx image for serving.
```dockerfile
FROM node:20 AS frontend
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=frontend /app/dist /usr/share/nginx/html
```
--------------------------------
### Verify Migration and HTML Highlighting
Source: https://context7.com/lbliii/rosettes/llms.txt
Demonstrates how to generate HTML-formatted code snippets using the highlight function. It verifies that the output contains expected Pygments-compatible CSS classes.
```python
html = highlight("def foo(): pass", "python", css_class_style="pygments")
assert 'def' in html
assert 'foo' in html
```
--------------------------------
### Execute Fixture Generation Script
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Commands to run the fixture generation script for specific languages or to update the entire suite. Requires the uv package manager.
```bash
uv run python scripts/generate_fixtures.py --language java
uv run python scripts/generate_fixtures.py --update
```
--------------------------------
### Kotlin Basics: Data Class and Main Function
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Defines a data class `User` in Kotlin and a `main` function to demonstrate its usage. It highlights Kotlin's concise syntax for properties and string templating.
```kotlin
data class User(
val id: Int,
val name: String,
val email: String?
)
fun main() {
val user = User(1, "Alice", "alice@example.com")
println("Hello, ${user.name}!")
}
```
--------------------------------
### Running Parallel Highlighting Benchmark
Source: https://github.com/lbliii/rosettes/blob/main/README.md
Shows how to execute a Python script to benchmark the parallel highlighting capabilities of Rosettes. This helps in understanding the performance scaling on multi-core systems.
```bash
python benchmarks/benchmark_parallel.py
```
--------------------------------
### Swift Basics: User Struct and UserService
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Defines a Codable User struct and a UserService class in Swift for fetching user data from an API. It uses async/await for asynchronous operations and URLSession for network requests.
```swift
import Foundation
struct User: Codable {
let id: Int
var name: String
var email: String?
}
class UserService {
func fetchUser(id: Int) async throws -> User {
let url = URL(string: "https://api.example.com/users/(id)")!
let (data, _) = try await URLSession.shared.data(from: url)
return try JSONDecoder().decode(User.self, from: data)
}
}
```
--------------------------------
### Access lexers and formatters
Source: https://context7.com/lbliii/rosettes/llms.txt
Demonstrates how to retrieve specific lexer instances for granular tokenization control and how to manage formatter registries for different output formats.
```python
from rosettes import get_lexer, get_formatter, list_formatters
# Lexer usage
lexer = get_lexer("python")
for token in lexer.tokenize("x = 1"):
print(f"{token.type}: {token.value!r}")
# Formatter usage
formatters = list_formatters()
html_formatter = get_formatter("html")
```
--------------------------------
### Adding and styling line numbers
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/highlighting/line-highlighting.md
Shows how to enable line numbering using the show_linenos parameter and how to style the resulting line numbers using CSS.
```python
html = highlight(code, "python", show_linenos=True)
```
```css
.rosettes .lineno {
color: #6272a4;
user-select: none; /* Don't include in copy */
padding-right: 1em;
text-align: right;
min-width: 2em;
display: inline-block;
}
```
--------------------------------
### Configure Advanced HTML Output in Python
Source: https://context7.com/lbliii/rosettes/llms.txt
Shows how to use HtmlFormatter with custom CSS classes, line highlighting, and line numbering configurations.
```python
from rosettes import highlight, HighlightConfig
from rosettes.formatters import HtmlFormatter
config = HighlightConfig(
hl_lines=frozenset({1, 2}),
show_linenos=True,
css_class="my-code"
)
formatter = HtmlFormatter(config=config, css_class_style="pygments")
html = highlight("def foo():\n pass", "python", formatter=formatter)
```
--------------------------------
### Elixir Pipe Operator for Data Transformation
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Illustrates the use of the Elixir pipe operator (|>) to chain function calls for data transformation. This example filters positive numbers, doubles them, and then reduces the result to a sum.
```elixir
result =
data
|> Enum.filter(&(&1 > 0))
|> Enum.map(&(&1 * 2))
|> Enum.reduce(0, &+/2)
```
--------------------------------
### Direct Lexer Access for Python
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/raw-tokens.md
Shows how to access the lexer directly using `get_lexer()` for more control. It demonstrates streaming tokenization with `lexer.tokenize()` and a faster method `lexer.tokenize_fast()` which returns only token types and values.
```python
from rosettes import get_lexer
lexer = get_lexer("python")
# Streaming tokenization (iterator)
for token in lexer.tokenize("x = 1"):
print(token)
# Fast path (no position tracking)
for token_type, value in lexer.tokenize_fast("x = 1"):
print(f"{token_type}: {value!r}")
```
--------------------------------
### Perform Basic Syntax Highlighting
Source: https://github.com/lbliii/rosettes/blob/main/README.md
Demonstrates how to generate HTML output from source code strings. Supports optional features like line numbering and highlighting specific line ranges.
```python
from rosettes import highlight
# Basic highlighting
html = highlight("def foo(): pass", "python")
# With line numbers
html = highlight(code, "python", show_linenos=True)
# Highlight specific lines
html = highlight(code, "python", hl_lines={2, 3, 4})
```
--------------------------------
### Tokenize Python Code
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/raw-tokens.md
Demonstrates how to use the `tokenize()` function to get a list of `Token` objects for Python code. It iterates through the tokens and prints their type, value, line, and column. This is useful for basic code inspection.
```python
from rosettes import tokenize
tokens = tokenize("x = 1 + 2", "python")
for token in tokens:
print(f"{token.type.name:20} {token.value!r:10} L{token.line}:C{token.column}")
```
--------------------------------
### Use Custom Formatter Instance with Python
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/custom-formatter.md
Illustrates how to pass a custom formatter instance directly to the `highlight` function in Python. This allows for flexible output formatting.
```python
from rosettes import highlight
formatter = MarkdownFormatter()
output = highlight("def foo(): pass", "python", formatter=formatter)
```
--------------------------------
### Ruby Blocks and Iterators
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Demonstrates the use of blocks and iterators in Ruby, including .each for iteration, .map for transformation, and File.open with a block for file handling. These are common patterns for processing collections and files.
```ruby
items.each do |item|
puts item
end
items.map { |x| x * 2 }
File.open("file.txt") do |f|
f.each_line { |line| process(line) }
end
```
--------------------------------
### Migrate syntax highlighting from Pygments to Rosettes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/about/comparison.md
This snippet demonstrates the transition from the Pygments library to Rosettes. It highlights the simplified API of Rosettes while maintaining compatibility with existing Pygments CSS themes.
```python
# Before (Pygments)
from pygments import highlight
from pygments.lexers import get_lexer_by_name
from pygments.formatters import HtmlFormatter
lexer = get_lexer_by_name("python")
formatter = HtmlFormatter()
html = highlight(code, lexer, formatter)
# After (Rosettes)
from rosettes import highlight
html = highlight(code, "python", css_class_style="pygments")
```
--------------------------------
### Get File Extension by Language in Python
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-test-hardening.md
A Python utility function that maps a given programming language string to its corresponding file extension. It uses a dictionary lookup and provides a default '.txt' extension if the language is not found.
```python
def get_extension(language: str) -> str:
"""Get file extension for language."""
extensions = {
"python": ".py",
"javascript": ".js",
"typescript": ".ts",
"rust": ".rs",
"go": ".go",
"yaml": ".yaml",
"json": ".json",
"php": ".php",
# ... add others
}
return extensions.get(language, ".txt")
```
--------------------------------
### CSS for Language-Specific Styling in Rosettes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/styling/custom-themes.md
Demonstrates how to apply language-specific styles in Rosettes using the `data-language` attribute. Examples show how to style decorators in Python, keyword types in Rust, and JSON keys, allowing for tailored highlighting based on the programming language.
```css
/* Python-specific */
.rosettes[data-language="python"] .syntax-decorator {
color: #f5c2e7;
font-weight: bold;
}
/* Rust-specific */
.rosettes[data-language="rust"] .syntax-keyword-type {
color: #fab387;
}
/* JSON keys */
.rosettes[data-language="json"] .syntax-attribute {
color: #89b4fa;
}
```
--------------------------------
### Register, List, and Get Palettes in Python
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/custom-palettes.md
This snippet illustrates the core palette registry operations in Rosettes. It shows how to register a palette using `register_palette`, list all available palettes with `list_palettes`, and retrieve a specific palette by its name using `get_palette`. These functions are all part of the `rosettes.themes` module.
```python
from rosettes.themes import register_palette, list_palettes, get_palette
# Assuming my_palette is already defined
# register_palette(my_palette)
print(list_palettes())
# ['bengal-tiger', 'bengal-snow-lynx', 'dracula', 'monokai', ...]
palette = get_palette("monokai")
print(palette.background) # #272822
```
--------------------------------
### Ruby Symbols and Method Arguments
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Illustrates Ruby's symbol syntax for hash keys and method arguments, including splat operators (*args) for variable arguments and keyword arguments (**kwargs), along with block passing (&block).
```ruby
options = {
name: "test",
:legacy => "value",
enabled: true
}
def method(arg, *args, **kwargs, &block)
yield if block_given?
end
```
--------------------------------
### Dark Theme CSS for Semantic Classes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/styling/_index.md
Provides a basic CSS stylesheet for styling Rosettes-generated HTML with semantic classes. This example includes styles for the main container, preformatted text, and common syntax elements like keywords, functions, strings, comments, and numbers, creating a dark theme appearance.
```css
/* Dark theme basics */
.rosettes {
background: #282a36;
padding: 1em;
border-radius: 4px;
overflow-x: auto;
}
.rosettes pre {
margin: 0;
font-family: 'Fira Code', monospace;
}
.syntax-keyword { color: #ff79c6; }
.syntax-function { color: #50fa7b; }
.syntax-string { color: #f1fa8c; }
.syntax-comment { color: #6272a4; }
.syntax-number { color: #bd93f9; }
```
--------------------------------
### Complete CSS Theme Example for Rosettes (Catppuccin Mocha)
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/styling/custom-themes.md
Presents a full CSS theme for Rosettes inspired by the Catppuccin Mocha color scheme. It defines CSS variables for each color and then applies them to the core `.rosettes` container and various syntax elements like keywords, functions, and strings. This provides a comprehensive and visually appealing theme.
```css
/* Catppuccin Mocha for Rosettes */
.rosettes {
--ctp-rosewater: #f5e0dc;
--ctp-flamingo: #f2cdcd;
--ctp-pink: #f5c2e7;
--ctp-mauve: #cba6f7;
--ctp-red: #f38ba8;
--ctp-maroon: #eba0ac;
--ctp-peach: #fab387;
--ctp-yellow: #f9e2af;
--ctp-green: #a6e3a1;
--ctp-teal: #94e2d5;
--ctp-sky: #89dceb;
--ctp-sapphire: #74c7ec;
--ctp-blue: #89b4fa;
--ctp-lavender: #b4befe;
--ctp-text: #cdd6f4;
--ctp-subtext1: #bac2de;
--ctp-subtext0: #a6adc8;
--ctp-overlay2: #9399b2;
--ctp-overlay1: #7f849c;
--ctp-overlay0: #6c7086;
--ctp-surface2: #585b70;
--ctp-surface1: #45475a;
--ctp-surface0: #313244;
--ctp-base: #1e1e2e;
--ctp-mantle: #181825;
--ctp-crust: #11111b;
background: var(--ctp-base);
color: var(--ctp-text);
padding: 1em;
border-radius: 8px;
overflow-x: auto;
}
.syntax-keyword { color: var(--ctp-mauve); }
.syntax-keyword-constant { color: var(--ctp-peach); }
.syntax-function { color: var(--ctp-blue); }
.syntax-class { color: var(--ctp-yellow); }
.syntax-string { color: var(--ctp-green); }
.syntax-number { color: var(--ctp-peach); }
.syntax-comment { color: var(--ctp-overlay0); font-style: italic; }
.syntax-operator { color: var(--ctp-sky); }
.syntax-decorator { color: var(--ctp-pink); }
.syntax-builtin { color: var(--ctp-red); }
```
--------------------------------
### Highlight and Tokenize Code with Rosettes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/highlighting/_index.md
Demonstrates the basic usage of the highlight and tokenize functions to process source code. The highlight function returns an HTML string, while tokenize returns a list of token objects for custom analysis.
```python
from rosettes import highlight, tokenize
# Get HTML output
html = highlight("def foo(): pass", "python")
# Get raw tokens
tokens = tokenize("def foo(): pass", "python")
for token in tokens:
print(f"{token.type}: {token.value!r}")
```
--------------------------------
### Highlight code with HTML formatter
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/formatters/html.md
Demonstrates the basic usage of the highlight function to generate HTML output. It shows both the default behavior and explicit formatter selection.
```python
from rosettes import highlight
html = highlight(code, "python")
# or explicitly:
html = highlight(code, "python", formatter="html")
```
--------------------------------
### Highlight Code Snippet in Python
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/custom-formatter.md
Demonstrates basic code highlighting using the `highlight` function from the rosettes library. It takes the code string, language, and an optional formatter as input.
```python
from rosettes import highlight
output = highlight("x = 1", "python", formatter=MarkdownFormatter())
```
--------------------------------
### Parallel Syntax Highlighting with Rosettes
Source: https://github.com/lbliii/rosettes/blob/main/site/content/releases/0.1.0.md
Illustrates parallel syntax highlighting using the `highlight_many` function for processing multiple code blocks concurrently. This is useful for improving performance when dealing with numerous code snippets.
```python
from rosettes import highlight_many
blocks = [
("def foo(): pass", "python"),
("const x = 1;", "javascript"),
]
results = highlight_many(blocks)
```
--------------------------------
### Running Mutation Tests
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-test-hardening.md
Command to execute mutation tests using 'mutmut'. It targets the 'src/rosettes/lexers/' directory to assess the quality of tests specifically for the lexer components.
```bash
uv run mutmut run --paths-to-mutate=src/rosettes/lexers/
```
--------------------------------
### Scala Functional Programming Patterns
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Demonstrates Scala features including case classes, Future-based asynchronous services, pattern matching, implicit classes for type enrichment, and for-comprehensions for monadic composition.
```scala
case class User(id: Long, name: String, email: Option[String])
def process(value: Any): String = value match {
case i: Int if i > 0 => s"Positive: $i"
case _ => "Unknown"
}
implicit class RichString(s: String) {
def isPalindrome: Boolean = s == s.reverse
}
for {
user <- findUser(id)
profile <- fetchProfile(user.id)
} yield UserDetails(user, profile)
```
--------------------------------
### Perl Scripting and Regex
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-fixture-expansion.md
Covers basic Perl syntax, regular expression operations for matching and substitution, and the use of references for complex data structures like anonymous arrays and hashes.
```perl
my $name = "World";
say "Hello, $name!";
if ($text =~ /pattern/) { print "Match!\n"; }
$text =~ s/old/new/g;
my $scalar_ref = \$scalar;
my $anon_array = [1, 2, 3];
$array_ref->[0];
```
--------------------------------
### GitHub Actions Workflow for Testing
Source: https://github.com/lbliii/rosettes/blob/main/plan/rfc-test-hardening.md
This workflow defines the CI process for the Rosettes project, using GitHub Actions to checkout code, set up the 'uv' environment, and run different pytest suites. It includes steps for fast tests, property tests, and optional slow tests.
```yaml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v4
- name: Run fast tests
run: uv run pytest tests/ -m "not slow and not property"
- name: Run property tests
run: uv run pytest tests/ -m property --hypothesis-seed=0
- name: Run slow tests (optional)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: uv run pytest tests/ -m slow
```
--------------------------------
### Print Hello function in Python
Source: https://github.com/lbliii/rosettes/blob/main/tests/fixtures/markdown/basics.md
A simple function that prints the string 'Hello' to the console. This serves as a basic entry point or test function for the project.
```python
def hello():
print("Hello")
```
--------------------------------
### Create Markdown Fenced Formatter
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/extending/custom-formatter.md
An implementation of the Formatter protocol that wraps token output within Markdown fenced code blocks.
```python
@dataclass(frozen=True, slots=True)
class MarkdownFormatter:
@property
def name(self) -> str:
return "markdown"
def format(self, tokens, config=None):
lang = config.data_language if config else ""
yield f"```{lang}\n"
for token in tokens:
yield token.value
yield "\n```"
def format_fast(self, tokens, config=None):
lang = config.data_language if config else ""
yield f"```{lang}\n"
for _, value in tokens:
yield value
yield "\n```"
def format_string(self, tokens, config=None):
return "".join(self.format(tokens, config))
def format_string_fast(self, tokens, config=None):
return "".join(self.format_fast(tokens, config))
```
--------------------------------
### Advanced HTML formatter configuration
Source: https://github.com/lbliii/rosettes/blob/main/site/content/docs/formatters/html.md
Illustrates how to instantiate a custom HtmlFormatter with specific HighlightConfig settings for line highlighting and custom CSS classes.
```python
from rosettes import highlight
from rosettes.formatters import HtmlFormatter
from rosettes import HighlightConfig
# Custom configuration
config = HighlightConfig(
hl_lines={1, 2},
hl_line_class="my-highlight",
lineno_class="my-lineno"
)
formatter = HtmlFormatter(config=config, css_class_style="pygments")
html = highlight(code, "python", formatter=formatter)
```