### Basic De-Unicode Conversion Source: https://github.com/kornelski/deunicode/blob/main/src/mapping.txt Demonstrates the basic usage of the deunicode function to convert a string with diacritics to its plain ASCII equivalent. This is the most common use case for the library. ```python from deunicode import deunicode text_with_diacritics = "Crème brûlée" plain_text = deunicode(text_with_diacritics) print(plain_text) # Output: Creme brulee ``` -------------------------------- ### Zero-Copy Conversion with deunicode_with_tofu_cow Source: https://context7.com/kornelski/deunicode/llms.txt Employ `deunicode_with_tofu_cow` for efficient, zero-copy conversion. It returns a `Cow`, avoiding allocation for ASCII-only input and allocating only when necessary for non-ASCII input. ```rust use deunicode::deunicode_with_tofu_cow; use std::borrow::Cow; // ASCII-only input returns borrowed string (no allocation) let result = deunicode_with_tofu_cow("hello world", "?"); assert!(matches!(result, Cow::Borrowed(_))); assert_eq!(result, "hello world"); // Non-ASCII input returns owned string let result = deunicode_with_tofu_cow("héllo wörld", "?"); assert!(matches!(result, Cow::Owned(_))); assert_eq!(result, "hello world"); // Mixed content with valid check mark let result = deunicode_with_tofu_cow("\u{2713} [x]", "?"); assert_eq!(result, "OK [x]"); ``` -------------------------------- ### Convert Unicode String to ASCII with deunicode Source: https://context7.com/kornelski/deunicode/llms.txt Use `deunicode` for basic transliteration of entire strings. Unknown characters are replaced with `[?]` by default. Requires the `alloc` feature. ```rust use deunicode::deunicode; // Basic transliteration examples assert_eq!(deunicode("Æneid"), "AEneid"); assert_eq!(deunicode("étude"), "etude"); assert_eq!(deunicode("café"), "cafe"); // Chinese characters - romanized to Mandarin assert_eq!(deunicode("北亰"), "Bei Jing"); assert_eq!(deunicode("北亰city"), "Bei Jing city"); assert_eq!(deunicode("技术").to_lowercase(), "ji shu"); // Japanese assert_eq!(deunicode("げんまい茶"), "genmaiCha"); // Various scripts assert_eq!(deunicode("ᔕᓇᓇ"), "shanana"); // Canadian Aboriginal assert_eq!(deunicode("ᏔᎵᏆ"), "taliqua"); // Cherokee assert_eq!(deunicode("अभिजीत"), "abhijiit"); // Devanagari assert_eq!(deunicode("অভিজীত"), "abhijiit"); // Bengali // Emoji support assert_eq!(deunicode("🦄☣"), "unicorn biohazard"); assert_eq!(deunicode("✓"), "OK"); // Preserves spaces correctly assert_eq!(deunicode(" spaces "), " spaces "); assert_eq!(deunicode(" two spaces "), " two spaces "); ``` -------------------------------- ### Display Trait Implementation for AsciiCharsIter Source: https://context7.com/kornelski/deunicode/llms.txt The `AsciiCharsIter` implements the `Display` trait, enabling direct formatting in strings without intermediate allocations. Unknown characters are rendered as the Unicode Replacement Character. ```rust use deunicode::AsciiChars; // Direct Display implementation for formatting let iter = "🐶".ascii_chars(); let output = format!("What's up {}!", iter); // Outputs: "What's up dog!" // Chaining with other format arguments let message = format!( "Welcome to {} - Status: {}", "東京".ascii_chars(), "✓".ascii_chars() ); // Outputs: "Welcome to Dong Jing - Status: OK" // Use in println! directly println!("Emoji test: {}", "🦄🌟💻".ascii_chars()); // Outputs: "Emoji test: unicorn glowing_star laptop" ``` -------------------------------- ### deunicode_with_tofu - Custom Placeholder for Unknown Characters Source: https://context7.com/kornelski/deunicode/llms.txt Converts Unicode to ASCII with a custom replacement string for unknown characters. ```APIDOC ## POST /deunicode_with_tofu ### Description Converts a Unicode string to its ASCII representation, using a specified string to replace unknown characters. ### Method POST ### Endpoint /deunicode_with_tofu ### Parameters #### Request Body - **input** (string) - Required - The Unicode string to convert. - **tofu** (string) - Required - The string to use as a replacement for unknown characters. ### Request Example ```json { "input": "Hello \u{FFFFF} World", "tofu": "?" } ``` ### Response #### Success Response (200) - **output** (string) - The ASCII transliterated string with unknown characters replaced. #### Response Example ```json { "output": "Hello ? World" } ``` ``` -------------------------------- ### deunicode - Convert Unicode String to ASCII Source: https://context7.com/kornelski/deunicode/llms.txt Converts entire Unicode strings to ASCII. Unknown characters are replaced with '[?]' by default. Requires the 'alloc' feature. ```APIDOC ## POST /deunicode ### Description Converts a Unicode string to its ASCII representation. Unknown characters are replaced with `[?]` by default. ### Method POST ### Endpoint /deunicode ### Request Body - **input** (string) - Required - The Unicode string to convert. ### Request Example ```json { "input": "Æneid" } ``` ### Response #### Success Response (200) - **output** (string) - The ASCII transliterated string. #### Response Example ```json { "output": "AEneid" } ``` ``` -------------------------------- ### Iterator-Based ASCII Conversion with AsciiChars Source: https://context7.com/kornelski/deunicode/llms.txt Use the `ascii_chars()` iterator for streaming Unicode to ASCII conversion without intermediate allocations. It can be directly used in format strings. Unknown characters are mapped to a default or can be handled with `map`. ```rust use deunicode::AsciiChars; // Iterator-based conversion let ascii_parts: Vec<_> = "🦄☣".ascii_chars().flatten().collect(); assert_eq!(ascii_parts, vec!["unicorn ", "biohazard"]); // Chinese character iteration with spacing let chars: Vec<_> = "中国".ascii_chars().flatten().collect(); assert_eq!(chars, vec!["Zhong ", "Guo"]); // Direct use in format strings (no temporary String allocated) let formatted = format!("Status: {}", "✓".ascii_chars()); assert_eq!(formatted, "Status: OK"); let greeting = format!("Hello {}", "世界".ascii_chars()); // Outputs: "Hello Shi Jie" // to_ascii_lossy() for simple string conversion let result = "héllo wörld".to_ascii_lossy(); assert_eq!(result, "hello world"); // Handle unknown characters with map let safe_ascii: String = "Test\u{FFFFF}Data" .ascii_chars() .map(|ch| ch.unwrap_or("?")) .collect(); ``` -------------------------------- ### deunicode_with_tofu_cow - Zero-Copy Conversion When Possible Source: https://context7.com/kornelski/deunicode/llms.txt Returns a `Cow` that avoids allocation when the input is already ASCII. This is the most efficient option when working with strings that may already be ASCII. ```APIDOC ## POST /deunicode_with_tofu_cow ### Description Converts a Unicode string to its ASCII representation, returning a borrowed string if the input is already ASCII, otherwise an owned string. Uses a specified string to replace unknown characters. ### Method POST ### Endpoint /deunicode_with_tofu_cow ### Parameters #### Request Body - **input** (string) - Required - The Unicode string to convert. - **tofu** (string) - Required - The string to use as a replacement for unknown characters. ### Request Example ```json { "input": "héllo wörld", "tofu": "?" } ``` ### Response #### Success Response (200) - **output** (string) - The ASCII transliterated string. This will be a borrowed string if the input was ASCII, or an owned string otherwise. #### Response Example ```json { "output": "hello world" } ``` ``` -------------------------------- ### Custom Placeholder for Unknown Characters with deunicode_with_tofu Source: https://context7.com/kornelski/deunicode/llms.txt Use `deunicode_with_tofu` to specify a custom replacement string for unknown characters. The term "tofu" refers to the common rectangular placeholder character. ```rust use deunicode::deunicode_with_tofu; // Replace unknown characters with custom string let unknown_char = std::char::from_u32(61849).unwrap().to_string(); assert_eq!(deunicode_with_tofu(&unknown_char, "?"), "?"); assert_eq!(deunicode_with_tofu(&unknown_char, ""), ""); assert_eq!(deunicode_with_tofu(&unknown_char, "[UNKNOWN]"), "[UNKNOWN]"); // Use Unicode Replacement Character as placeholder let result = deunicode_with_tofu("Hello \u{FFFFF} World", "\u{FFFD}"); // Unknown chars become the replacement character // Known characters still convert normally assert_eq!(deunicode_with_tofu("Æneid", "?"), "AEneid"); ``` -------------------------------- ### Handling Specific Characters Source: https://github.com/kornelski/deunicode/blob/main/src/mapping.txt Shows how the deunicode function handles various accented characters and special symbols. It attempts to find the closest ASCII representation for each character. ```python from deunicode import deunicode text = "Héllö Wörld! © 2023" print(deunicode(text)) # Output: Hello World! (c) 2023 ``` -------------------------------- ### Basic Unicode to ASCII Transliteration Source: https://github.com/kornelski/deunicode/blob/main/README.md Use the `deunicode` function to convert various Unicode characters and symbols into their ASCII equivalents. This function handles common cases including emojis. ```rust use deunicode::deunicode; assert_eq!(deunicode("Æneid"), "AEneid"); assert_eq!(deunicode("étude"), "etude"); assert_eq!(deunicode("北亰"), "Bei Jing"); assert_eq!(deunicode("ᔕᓇᓇ"), "shanana"); assert_eq!(deunicode("げんまい茶"), "genmaiCha"); assert_eq!(deunicode("🦄☣"), "unicorn biohazard"); ``` -------------------------------- ### no_std Compatibility with Character-Level Conversion Source: https://context7.com/kornelski/deunicode/llms.txt Deunicode supports `no_std` environments by disabling the default `alloc` feature. In this mode, only character-level functions like `deunicode_char` are available, allowing manual ASCII output construction. ```rust // In Cargo.toml: // [dependencies] // deunicode = { version = "1.6", default-features = false } use deunicode::deunicode_char; // Character-level conversion works without alloc fn transliterate_char(c: char) -> &'static str { deunicode_char(c).unwrap_or("?") } assert_eq!(transliterate_char('ñ'), "n"); assert_eq!(transliterate_char('北'), "Bei "); // Build ASCII output manually in no_std fn to_ascii_no_alloc(input: &str, output: &mut [u8]) -> usize { let mut pos = 0; for ch in input.chars() { if let Some(ascii) = deunicode_char(ch) { for &b in ascii.as_bytes() { if pos < output.len() { output[pos] = b; pos += 1; } } } } pos } ``` -------------------------------- ### Using Deunicode with Non-ASCII Characters Source: https://github.com/kornelski/deunicode/blob/main/src/mapping.txt Illustrates the function's ability to process strings containing a mix of ASCII and non-ASCII characters, converting the non-ASCII ones appropriately. ```python from deunicode import deunicode text = "你好世界, ça va?" print(deunicode(text)) # Output: 你好世界, ca va? ``` -------------------------------- ### Convert Single Unicode Character to ASCII with deunicode_char Source: https://context7.com/kornelski/deunicode/llms.txt Use `deunicode_char` to transliterate individual Unicode characters. It returns `Option<&'static str>`, yielding `None` for unknown characters. This is useful for custom transliteration logic. ```rust use deunicode::deunicode_char; // Single character transliteration assert_eq!(deunicode_char('Æ'), Some("AE")); assert_eq!(deunicode_char('é'), Some("e")); assert_eq!(deunicode_char('ñ'), Some("n")); // Chinese characters include trailing space for word separation assert_eq!(deunicode_char('北'), Some("Bei ")); assert_eq!(deunicode_char('亰'), Some("Jing ")); // Syllabic scripts assert_eq!(deunicode_char('ᔕ'), Some("sha")); // ASCII characters map to themselves assert_eq!(deunicode_char('a'), Some("a")); assert_eq!(deunicode_char('Z'), Some("Z")); assert_eq!(deunicode_char('5'), Some("5")); // Unknown characters return None assert_eq!(deunicode_char(std::char::from_u32(0x1FFFF).unwrap()), None); assert_eq!(deunicode_char(std::char::from_u32(0x10FFFF).unwrap()), None); ``` -------------------------------- ### deunicode_char - Convert Single Character Source: https://context7.com/kornelski/deunicode/llms.txt Transliterates a single Unicode character to its ASCII equivalent. Returns `Option<&'static str>` - `None` for unknown characters, `Some` for mapped characters. ```APIDOC ## GET /deunicode_char ### Description Transliterates a single Unicode character to its ASCII equivalent. Returns `None` for unknown characters. ### Method GET ### Endpoint /deunicode_char ### Query Parameters - **char** (string) - Required - The single Unicode character to convert. ### Request Example ``` /deunicode_char?char=%C3%86 ``` ### Response #### Success Response (200) - **result** (string | null) - The ASCII equivalent of the character, or null if the character is unknown. #### Response Example ```json { "result": "AE" } ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.