### unescape_bytes_in Examples Source: https://docs.rs/htmlize/1.1.0/htmlize/fn.unescape_bytes_in.html Demonstrates the behavior of `unescape_bytes_in` with different contexts and entity formats. Note the distinct handling of entities without trailing semicolons in attribute contexts. ```rust use htmlize::* assert!(unescape_bytes_in(&b"×"[..], Context::General) == "×".as_bytes()); assert!(unescape_bytes_in(&b"×"[..], Context::Attribute) == "×".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::General) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::Attribute) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::General) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::Attribute) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×="[..], Context::General) == "×=".as_bytes()); assert!(unescape_bytes_in(&b"×="[..], Context::Attribute) == "×=".as_bytes()); assert!(unescape_bytes_in(&b"×#"[..], Context::General) == "×#".as_bytes()); assert!(unescape_bytes_in(&b"×#"[..], Context::Attribute) == "×#".as_bytes()); ``` -------------------------------- ### unescape_in Examples Source: https://docs.rs/htmlize/1.1.0/htmlize/fn.unescape_in.html Demonstrates how `unescape_in` handles different entities and contexts. Note that in attribute context, named entities without trailing semicolons are not expanded when followed by an alphanumeric character or '='. ```rust use htmlize::{unescape_in, Context}; assert!(unescape_in("×", Context::General) == "×"); assert!(unescape_in("×", Context::Attribute) == "×"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×=", Context::General) == "×="); assert!(unescape_in("×=", Context::Attribute) == "×="); assert!(unescape_in("×#", Context::General) == "×#"); assert!(unescape_in("×#", Context::Attribute) == "×#"); ``` -------------------------------- ### Unescape Bytes with Different Entities Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/mod.rs.html Demonstrates the unescaping of a specific HTML entity ('×') in both general and attribute contexts. This example shows that '×' is unescaped in both cases. ```rust assert!(unescape_bytes_in(&b"×"[..], Context::General) == "×".as_bytes()); assert!(unescape_bytes_in(&b"×"[..], Context::Attribute) == "×".as_bytes()); ``` -------------------------------- ### Matcher trait and implementations for entity matching - Rust Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Defines the `Matcher` trait for entity matching algorithms and provides an example implementation for `Matchgen` with `ContextAttribute`. ```rust pub trait Matcher { /// Match an entity at the beginning of `iter`. Either: /// /// * It finds a match: returns `Some(expansion)` and `iter` is updated to /// point to the next character after the entity. /// * It doesn’t find a match: returns `None` and `iter` is updated to /// point to the next character than could plausibly start an entity /// (not necessarily b'&', though; the only guarantee is that we didn’t /// skip a potential entity). /// /// This version uses matchgen instead of the `ENTITIES` map. It is faster /// at runtime but slower to build. fn match_entity<'a>(iter: &'a mut slice::Iter) -> Option>; } #[cfg(feature = "unescape_fast")] impl Matcher for (Matchgen, ContextAttribute) { fn match_entity<'a>( iter: &'a mut slice::Iter, ) -> Option> { assert_peek_eq(iter, Some(b'&'), "match_entity() expected '&'"); } } ``` -------------------------------- ### Escape and Unescape Strings with htmlize Source: https://docs.rs/htmlize/1.1.0/htmlize Demonstrates basic usage of escape and unescape functions. Ensure the 'unescape' feature is enabled for unescape functionality. ```rust use htmlize::{escape_attribute, escape_text}; assert!(escape_attribute("ab & < > \" '") == "ab & < > " '"); assert!(escape_text("ab & < > \" '") == "ab & < > \" '"); ``` ```rust assert!(htmlize::unescape("3 × 4 > 10") == "3 × 4 > 10"); ``` -------------------------------- ### Basic Escaping and Unescaping Source: https://docs.rs/htmlize/1.1.0/htmlize/index.html Demonstrates the basic usage of `escape_attribute`, `escape_text`, and `unescape` functions. ```APIDOC ## Basic Escaping and Unescaping ### Description This snippet shows how to use the primary `escape` and `unescape` functions for handling HTML entities. ### Functions - `htmlize::escape_attribute(text: &str) -> String` - `htmlize::escape_text(text: &str) -> String` - `htmlize::unescape(text: &str) -> String` ### Request Example ```rust use htmlize::{escape_attribute, escape_text}; assert!(escape_attribute("ab & < > \" '") == "ab & < > " '"); assert!(escape_text("ab & < > \" '") == "ab & < > \" '"); // Requires 'unescape' or 'unescape_fast' feature // assert!(htmlize::unescape("3 × 4 > 10") == "3 × 4 > 10"); ``` ### Response Example ```rust // For escape_attribute and escape_text: // "ab & < > " '" // For unescape (with feature enabled): // "3 × 4 > 10" ``` ``` -------------------------------- ### Advanced Unescaping with Context Source: https://docs.rs/htmlize/1.1.0/htmlize/index.html Explains the use of `unescape_in` and `unescape_bytes_in` for context-aware unescaping. ```APIDOC ## Advanced Unescaping with Context ### Description This snippet covers the `unescape_in` and `unescape_bytes_in` functions, which allow specifying the context (attribute or general text) for unescaping. ### Functions - `htmlize::unescape_in(text: &str, context: htmlize::Context) -> String` - `htmlize::unescape_bytes_in(bytes: &[u8], context: htmlize::Context) -> Vec` ### Parameters #### Path Parameters - `context` (htmlize::Context) - Required - The context for unescaping ('Attribute' or 'Text'). ### Request Example ```rust use htmlize::Context; let attribute_value = "data-name='John Doe'"; let unescaped_attr = htmlize::unescape_in(attribute_value, Context::Attribute); assert_eq!(unescaped_attr, "data-name='John Doe'"); let text_value = "<b>Bold Text</b>"; let unescaped_text = htmlize::unescape_in(text_value, Context::Text); assert_eq!(unescaped_text, "Bold Text"); ``` ### Response Example ```rust // For unescape_in with Context::Attribute: // "data-name='John Doe'" // For unescape_in with Context::Text: // "Bold Text" ``` ``` -------------------------------- ### Find Longest Alphanumeric Candidate Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Advances an iterator to consume the longest possible sequence of ASCII alphanumeric characters starting after an '&'. This is used for identifying potential HTML entity candidates. ```rust /// Advance `iter` to consume longest possible candidate (alphanumeric only). #[cfg(feature = "unescape")] fn find_longest_candidate(iter: &mut slice::Iter) { use crate::ENTITY_MAX_LENGTH; assert_next_eq(iter, Some(b'&'), PEEK_MATCH_ERROR); // Start at 1 since we got the '&'. for _ in 1..ENTITY_MAX_LENGTH { if let Some(c) = peek(iter) { if c.is_ascii_alphanumeric() { iter.next(); continue; } } break; } } ``` -------------------------------- ### Byte-based Escaping Source: https://docs.rs/htmlize/1.1.0/htmlize/index.html Illustrates the usage of byte-slice versions of the escaping functions. ```APIDOC ## Byte-based Escaping ### Description This snippet demonstrates how to use the `_bytes` variants of the escaping functions, which return byte slices (`&[u8]`) instead of `String`. ### Functions - `htmlize::escape_attribute_bytes(text: &str) -> Vec` - `htmlize::escape_text_bytes(text: &str) -> Vec` ### Request Example ```rust use htmlize::escape_text_bytes; let escaped_bytes = escape_text_bytes("Hello & World"); assert_eq!(escaped_bytes, b"Hello & World"); ``` ### Response Example ```rust // For escape_text_bytes: // b"Hello & World" ``` ``` -------------------------------- ### unescape_in Source: https://docs.rs/htmlize/1.1.0/index.html Expands all valid HTML entities within a given context for strings. ```APIDOC ## unescape_in ### Description Expand all valid entities in a given context. ### Parameters - **input** (string) - The string containing HTML entities. - **context** (string) - The context in which to unescape entities (e.g., 'attribute', 'text'). ``` -------------------------------- ### Include Generated Entity Data Source: https://docs.rs/htmlize/1.1.0/src/htmlize/entities.rs.html This snippet shows how to include the automatically generated entity data from the build script into the Rust code. This is typically done once at the beginning of the module. ```rust //! # Information about entities //! //! Everything here is automatically generated by build.rs. include!(concat!(env!("OUT_DIR"), "/entities.rs")); ``` -------------------------------- ### unescape Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/mod.rs.html Expands all valid HTML entities in a string. This is suitable for general text content outside of HTML attributes. ```APIDOC ## unescape ### Description Expand all valid entities. This is appropriate to use on any text outside of an attribute. See [`unescape_in()`] for more information. To work with bytes (`[u8]`) instead of strings, see [`unescape_bytes_in()`]. ### Method `fn unescape<'a, S: Into>>(escaped: S) -> Cow<'a, str>` ### Parameters * `escaped` (S): The input string with HTML entities to be unescaped. ### Response * `Cow<'a, str>`: The unescaped string. ### Request Example ```rust assert!(htmlize::unescape("1×2<3") == "1×2<3"); ``` ``` -------------------------------- ### Test All Entities Decoding Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Compares the output of unescaping a comprehensive source of entities against its expected expanded form. ```rust const ALL_SOURCE: &str = include_str!("../../tests/corpus/all-entities-source.txt"); const ALL_EXPANDED: &str = include_str!("../../tests/corpus/all-entities-expanded.txt"); test_both!(all_entities, unescape(ALL_SOURCE) == ALL_EXPANDED); ``` -------------------------------- ### Testing HTML Escaping Functions Source: https://docs.rs/htmlize/1.1.0/src/htmlize/escape.rs.html This macro simplifies testing various HTML escaping functions with different input strings and expected outputs. It covers text, attribute, and all-quotes escaping for both string and byte slice variants. ```rust macro_rules! test { ($name:ident, $($test:tt)+) => { #[test] fn $name() { #![allow(clippy::string_lit_as_bytes)] assert!($($test)+); } }; } // Test all escape functions macro_rules! test_all { ($name:ident, $in:expr, $out:expr) => { paste! { test!([], escape_text($in) == $out); test!( [], escape_attribute($in) == $out ); test!( [], escape_all_quotes($in) == $out ); test!( [], escape_text_bytes($in.as_bytes()) == $out.as_bytes() ); test!( [], escape_attribute_bytes($in.as_bytes()) == $out.as_bytes() ); test!( [], escape_all_quotes_bytes($in.as_bytes()) == $out.as_bytes() ); } }; } test_all!(none, "", ""); test_all!(clean, "clean", "clean"); test_all!(lt_gt, "< >", "< >"); test_all!(amp, "&", "&amp;"); test_all!(prefix_amp, "prefix&", "prefix&"); test_all!(emoji_amp, "☺️&☺️", "☺️&☺️"); test_all!( special_clean, "Björk and Борис OBrien ❤️, \"love beats hate\"", "Björk and Борис OBrien ❤️, \"love beats hate\"" ); test!( escape_text_quotes, escape_text("He said, \"That's mine.\"") == "He said, \"That's mine.\"" ); test!( escape_attribute_quotes, ``` -------------------------------- ### unescape_in Source: https://docs.rs/htmlize/1.1.0/htmlize Expands all valid HTML entities within a string in a given context. ```APIDOC ## unescape_in ### Description Expand all valid entities in a given context. ### Function Signature ```rust fn unescape_in(s: &str, context: EscapeContext) -> String ``` ### Alias `unescape_fast` ``` -------------------------------- ### Test Slow Path Invalid UTF-8 Handling Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the behavior of the slow unescape path when encountering invalid UTF-8 byte sequences, confirming they are preserved. ```rust #[cfg(feature = "unescape")] #[test] fn slow_invalid_utf8() { assert!( unescape_bytes_in((Phf, ContextGeneral), &b"\xa1"[..]) == &b"\xa1"[..] ); } ``` -------------------------------- ### unescape_bytes_in Source: https://docs.rs/htmlize/1.1.0/htmlize/fn.unescape_bytes_in.html Expands all valid HTML entities in a given byte slice based on the provided context. This function is available when the `unescape` or `unescape_fast` crate features are enabled. It follows the WHATWG specification for entity expansion, with special handling for named entities within attribute contexts. ```APIDOC ## unescape_bytes_in ### Description Expands all valid entities in a given context. `context` may be: * `Context::General`: use the rules for text outside of an attribute. This is usually what you want. * `Context::Attribute`: use the rules for attribute values. This uses the algorithm described in the WHATWG spec. In attributes, named entities without trailing semicolons are treated differently. They not expanded if they are followed by an alphanumeric character or or `=". ### Function Signature ```rust pub fn unescape_bytes_in<'a, S: Into>>( escaped: S, context: Context, ) -> Cow<'a, [u8]> ``` ### Parameters * `escaped` (S): The byte slice containing HTML entities to be unescaped. `S` must implement `Into>`. * `context` (Context): Specifies the parsing context, either `Context::General` or `Context::Attribute`. ### Available Features * `unescape` * `unescape_fast` ### Examples ```rust use htmlize::* assert!(unescape_bytes_in(&b"×"[..], Context::General) == "×".as_bytes()); assert!(unescape_bytes_in(&b"×"[..], Context::Attribute) == "×".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::General) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::Attribute) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::General) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×X"[..], Context::Attribute) == "×X".as_bytes()); assert!(unescape_bytes_in(&b"×="[..], Context::General) == "×=".as_bytes()); assert!(unescape_bytes_in(&b"×="[..], Context::Attribute) == "×=".as_bytes()); assert!(unescape_bytes_in(&b"×#"[..], Context::General) == "×#".as_bytes()); assert!(unescape_bytes_in(&b"×#"[..], Context::Attribute) == "×#".as_bytes()); ``` ### Related Functions * `unescape_in()`: For working with `String` instead of bytes. ``` -------------------------------- ### Macro to Test Both Unescape and Unescape Attribute Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html This macro simplifies testing by calling the `test!` macro for both `unescape` and `unescape_attribute` functions with the same inputs and expected outputs. It's useful for ensuring consistent behavior across different unescaping contexts. ```rust macro_rules! test_both { ($name:ident, unescape ($input:expr) == $expected:expr) => { paste! { test!($name, unescape($input) == $expected); test!([], unescape_attribute($input) == $expected); } }; } ``` -------------------------------- ### unescape_bytes_in Source: https://docs.rs/htmlize/1.1.0/index.html Expands all valid HTML entities within a given context for byte strings. ```APIDOC ## unescape_bytes_in ### Description Expand all valid entities in a given context. ### Parameters - **input** (bytes) - The byte string containing HTML entities. - **context** (string) - The context in which to unescape entities (e.g., 'attribute', 'text'). ``` -------------------------------- ### unescape_in Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/mod.rs.html Expands valid HTML entities based on the provided context (General or Attribute). This function implements the algorithm described in the WHATWG spec. ```APIDOC ## unescape_in ### Description Expand all valid entities in a given context. `context` may be: * `Context::General`: use the rules for text outside of an attribute. This is usually what you want. * `Context::Attribute`: use the rules for attribute values. This uses the [algorithm described] in the WHATWG spec. In attributes, [named entities] without trailing semicolons are not expanded when followed by an alphanumeric character or `=`. To work with bytes (`[u8]`) instead of strings, see [`unescape_bytes_in()`]. ### Method `fn unescape_in<'a, S: Into>>(escaped: S, context: Context) -> Cow<'a, str>` ### Parameters * `escaped` (S): The input string with HTML entities to be unescaped. * `context` (Context): The context in which the unescaping should occur (`Context::General` or `Context::Attribute`). ### Response * `Cow<'a, str>`: The unescaped string. ### Request Example ```rust use htmlize::{unescape_in, Context}; # use assert2::check as assert; assert!(unescape_in("×", Context::General) == "×"); assert!(unescape_in("×", Context::Attribute) == "×"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×=", Context::General) == "×="); assert!(unescape_in("×=", Context::Attribute) == "×="); assert!(unescape_in("×#", Context::General) == "×#"); assert!(unescape_in("×#", Context::Attribute) == "×#"); ``` ``` -------------------------------- ### Test Fast Path Invalid UTF-8 Handling Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Checks how the fast unescape path handles invalid UTF-8 byte sequences, ensuring they are passed through unchanged. ```rust #[cfg(feature = "unescape_fast")] #[test] fn fast_invalid_utf8() { assert!( unescape_bytes_in((Matchgen, ContextGeneral), &b"\xa1"[..]) == &b"\xa1"[..] ); } ``` -------------------------------- ### Test Numeric Entity Decoding for Euro Symbol Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the correct decoding of the numeric entity for the Euro symbol (0x80). ```rust #[test] fn correct_numeric_entity_euro() { match correct_numeric_entity(0x80) { Cow::Borrowed(s) => assert!(s == "\u{20AC}".as_bytes()), Cow::Owned(_) => panic!("expected borrowed"), } } ``` -------------------------------- ### Basic HTML Unescaping Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/mod.rs.html Use `unescape` for general text outside of HTML attributes. It converts all valid HTML entities. ```rust use htmlize::{unescape, unescape_in, Context}; # use assert2::check as assert; assert!(unescape("1×2<3") == "1×2<3"); assert!(unescape_in("1×2<3", Context::Attribute) == "1×2<3"); assert!(unescape_in("3 × 5 < 16", Context::Attribute) == "3 × 5 < 16"); ``` -------------------------------- ### Test Numeric Entity Decoding for 'z' Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the decoding of the numeric entity for the character 'z', expecting an owned string. ```rust #[test] fn correct_numeric_entity_z() { match correct_numeric_entity(b'z'.into()) { Cow::Borrowed(_) => panic!("expected owned"), Cow::Owned(ref s) => assert!(s == b"z"), } } ``` -------------------------------- ### unescape_bytes_in Source: https://docs.rs/htmlize/1.1.0/htmlize Expands all valid HTML entities within a byte string in a given context. ```APIDOC ## unescape_bytes_in ### Description Expand all valid entities in a given context. ### Function Signature ```rust fn unescape_bytes_in(s: &[u8], context: EscapeContext) -> Vec ``` ### Alias `unescape_fast` ``` -------------------------------- ### Test Slow Path Attribute Context Invalid UTF-8 Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Confirms that the slow unescape path correctly processes invalid UTF-8 byte sequences within an attribute context, leaving them as is. ```rust #[cfg(feature = "unescape")] #[test] fn slow_attribute_invalid_utf8() { assert!( unescape_bytes_in((Phf, ContextAttribute), &b"\xa1"[..]) == &b"\xa1"[..] ); } ``` -------------------------------- ### unescape_in Function Source: https://docs.rs/htmlize/1.1.0/htmlize/fn.unescape_in.html Expands all valid entities in a given context. The context can be `Context::General` for text outside attributes or `Context::Attribute` for attribute values. This function follows the WHATWG specification for entity expansion. ```APIDOC ## unescape_in ### Description Expands all valid entities in a given context. `context` may be: * `Context::General`: use the rules for text outside of an attribute. This is usually what you want. * `Context::Attribute`: use the rules for attribute values. This uses the algorithm described in the WHATWG spec. In attributes, named entities without trailing semicolons are not expanded when followed by an alphanumeric character or `=`. Available on `crate features`unescape` or `unescape_fast` only. ### Signature ```rust pub fn unescape_in<'a, S: Into>>( escaped: S, context: Context, ) -> Cow<'a, str> ``` ### Parameters #### Path Parameters - **escaped** (S): The input string with HTML entities to be unescaped. - **context** (Context): The context in which to unescape entities (`Context::General` or `Context::Attribute`). ### Examples ```rust use htmlize::{unescape_in, Context}; assert!(unescape_in("×", Context::General) == "×"); assert!(unescape_in("×", Context::Attribute) == "×"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×X", Context::General) == "×X"); assert!(unescape_in("×X", Context::Attribute) == "×X"); assert!(unescape_in("×=", Context::General) == "×="); assert!(unescape_in("×=", Context::Attribute) == "×="); assert!(unescape_in("×#", Context::General) == "×#"); assert!(unescape_in("×#", Context::Attribute) == "×#"); ``` ### See Also - `unescape_bytes_in()` for working with bytes. ``` -------------------------------- ### Test Correct Hexadecimal and Decimal HTML Entities Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the correct decoding of hexadecimal and decimal HTML entities, including those with leading zeros and Unicode characters. ```rust test_both!(correct_hex_leading_zero, unescape("z") == "z"); test_both!(correct_hex_leading_zero_zero, unescape("z") == "z"); test_both!(correct_dec, unescape("z") == "z"); test_both!(correct_dec_leading_zero, unescape("z") == "z"); test_both!(correct_dec_leading_zero_zero, unescape("z") == "z"); test_both!(correct_hex_unicode, unescape("⇒") == "⇒"); ``` -------------------------------- ### Test Fast Path Attribute Context Invalid UTF-8 Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Examines the fast unescape path's handling of invalid UTF-8 in an attribute context, ensuring byte sequences are not altered. ```rust #[cfg(feature = "unescape_fast")] #[test] fn fast_attribute_invalid_utf8() { assert!( unescape_bytes_in((Matchgen, ContextAttribute), &b"\xa1"[..]) == &b"\xa1"[..] ); } ``` -------------------------------- ### Test Invalid Hexadecimal and Decimal Entity Formats Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Ensures that malformed hexadecimal and decimal entities are handled correctly, returning the original string. ```rust test_both!(hex_instead_of_dec, unescape("&#a0;") == "&#a0;"); test_both!(invalid_hex_lowerx, unescape("&#xZ;") == "&#xZ;"); test_both!(invalid_hex_upperx, unescape("&#XZ;") == "&#XZ;"); ``` -------------------------------- ### Validate Bare Entities Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Ensures that no bare HTML entity is a prefix for another bare entity. This prevents ambiguity during unescaping. ```rust let all_bare: Vec<_> = ALL_SOURCE .split_ascii_whitespace() .filter(|entity| entity.ends_with(';')) .collect(); for bare in &all_bare { check!( all_bare .iter() .find(|other| other.starts_with(bare) && *other != bare) == None, "No bare entity may be a prefix for another bare entity" ); } ``` -------------------------------- ### Define HTML escape functions for text and bytes Source: https://docs.rs/htmlize/1.1.0/src/htmlize/escape.rs.html This macro generates paired functions for escaping strings and byte strings. It handles character-to-entity mapping and efficient searching within the input. ```rust macro_rules! find_u8_body { ($slice:expr, $ch1:literal $(,)?) => { memchr::memchr($ch1, $slice) }; ($slice:expr, $ch1:literal, $ch2:literal $(,)?) => { memchr::memchr2($ch1, $ch2, $slice) }; ($slice:expr, $ch1:literal, $ch2:literal, $ch3:literal $(,)?) => { memchr::memchr3($ch1, $ch2, $ch3, $slice) }; ($slice:expr, $($ch:literal),+) => { $slice.iter().position(|c| matches!(c, $($ch)|+)) }; } /// Generate string and byte string versions of an escape function. macro_rules! escape_fn { ( $(#[$meta:meta])* $vis:vis fn $name:ident; $(#[$bytes_meta:meta])* $bytes_vis:vis fn $bytes_name:ident; { $($ch:literal => $entity:literal,)+ } ) => { paste! { $(#[$meta])* $vis fn $name<'a, S: Into>>(input: S) -> Cow<'a, str> { let input = input.into(); match [<$name _bytes_internal>](input.as_bytes()) { Some(output) => String::from_utf8(output).unwrap().into(), None => input, } } $(#[$bytes_meta])* $bytes_vis fn $bytes_name<'a, S: Into>>(input: S) -> Cow<'a, [u8]> { let input = input.into(); match [<$name _bytes_internal>](&*input) { Some(output) => output.into(), None => input, } } #[inline(always)] fn [<$name _bytes_internal>](raw: &[u8]) -> Option> { #[inline] fn find_u8(haystack: &[u8]) -> Option { find_u8_body!(haystack, $($ch),+) } #[inline] const fn map_u8(c: u8) -> &'static [u8] { match c { $( $ch => $entity, )+ // This should never happen, but using unreachable!() // actually makes other parts of the function slower. _ => b"", } } if let Some(i) = find_u8(raw) { let mut output: Vec = Vec::with_capacity(raw.len().saturating_mul(2)); output.extend_from_slice(&raw[..i]); output.extend_from_slice(map_u8(raw[i])); // i is a valid index, so it can't be usize::MAX. debug_assert!(i < usize::MAX); #[allow(clippy::arithmetic_side_effects)] let mut remainder = &raw[i+1..]; while let Some(i) = find_u8(remainder) { output.extend_from_slice(&remainder[..i]); output.extend_from_slice(map_u8(remainder[i])); // i is a valid index, so it can't be usize::MAX. debug_assert!(i < usize::MAX); #[allow(clippy::arithmetic_side_effects)] let n = i + 1; // Work around https://github.com/rust-lang/rust/issues/15701 remainder = &remainder[n..]; } output.extend_from_slice(&remainder); Some(output) } else { None } } } } } ``` -------------------------------- ### Unescape Functions Source: https://docs.rs/htmlize/1.1.0/src/htmlize/lib.rs.html Functions for decoding HTML entities back into raw strings. These require the `unescape` or `unescape_fast` feature to be enabled. ```APIDOC ## unescape ### Description Decodes HTML entities in a string. This is the general-purpose unescaping function. ### Feature Requires `unescape` or `unescape_fast` feature. ### Signature `pub fn unescape(s: &str) -> String` ### Example ```rust // Requires the 'unescape' or 'unescape_fast' feature // assert!(htmlize::unescape("3 × 4 > 10") == "3 × 4 > 10"); ``` ## unescape_attribute ### Description Decodes HTML entities specifically for attribute values. ### Feature Requires `unescape` or `unescape_fast` feature. ### Signature `pub fn unescape_attribute(s: &str) -> String` ## unescape_in ### Description Decodes HTML entities based on the provided context (text or attribute). ### Feature Requires `unescape` or `unescape_fast` feature. ### Parameters #### Path Parameters - **context** (`htmlize::UnescapeContext`) - Required - Specifies whether to unescape for text or attribute context. ### Signature `pub fn unescape_in(s: &str, context: UnescapeContext) -> String` ## unescape_bytes_in ### Description Decodes HTML entities from a byte slice based on the provided context. ### Feature Requires `unescape` or `unescape_fast` feature. ### Parameters #### Path Parameters - **context** (`htmlize::UnescapeContext`) - Required - Specifies whether to unescape for text or attribute context. ### Signature `pub fn unescape_bytes_in(s: &[u8], context: UnescapeContext) -> Vec` ``` -------------------------------- ### Internal Test Macros for Unescaping Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html These macros define tests for both fast and slow unescaping functions, handling general text and attribute contexts. They are used internally for comprehensive testing. ```rust macro_rules! test { ($name:ident, unescape ($($input:tt)+) == $expected:expr) => { paste! { #[cfg(feature = "unescape_fast")] #[test] fn []() { assert!(unescape_in((Matchgen, ContextGeneral), $($input)+) == $expected); } #[cfg(feature = "unescape")] #[test] fn []() { assert!(unescape_in((Phf, ContextGeneral), $($input)+) == $expected); } } }; ($name:ident, unescape_attribute ($($input:tt)+) == $expected:expr) => { paste! { #[cfg(feature = "unescape_fast")] #[test] fn []() { assert!(unescape_in((Matchgen, ContextAttribute), $($input)+) == $expected); } #[cfg(feature = "unescape")] #[test] fn []() { assert!(unescape_in((Phf, ContextAttribute), $($input)+) == $expected); } } }; } ``` -------------------------------- ### Test Bare Hexadecimal and Decimal Entities Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Checks how the unescape function handles bare hexadecimal and decimal entities, including cases where they are followed by other characters or are incomplete. ```rust test_both!(bare_hex_char, unescape("zz") == "zz"); test_both!(bare_hex_entity, unescape("z<") == "z<"); test_both!(bare_hex_end, unescape("z") == "z"); test_both!(bare_dec_char, unescape("zz") == "zz"); test_both!(bare_dec_entity, unescape("z<") == "z<"); test_both!(bare_dec_end, unescape("z") == "z"); ``` -------------------------------- ### Test Macro for Unescape Function Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html A macro for testing both the fast and slow versions of an unescape function. It takes the function name, input, and expected output. ```rust // Test fast and slow versions of a function. macro_rules! test { ($name:ident, unescape ($($input:tt)+) == $expected:expr) => { paste! { #[cfg(feature = "unescape_fast")] ``` -------------------------------- ### Test Bare Empty Numeric Entities Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the behavior of the unescape function with bare, empty numeric entities, ensuring they are not incorrectly decoded. ```rust test_both!(bare_empty_numeric_char, unescape("&#z") == "&#z"); test_both!(bare_empty_numeric_entity, unescape("&#<") == "&#<"); test_both!(bare_empty_numeric_end, unescape("&#") == "&#"); ``` -------------------------------- ### Macro for conditional compilation with feature gates Source: https://docs.rs/htmlize/1.1.0/src/htmlize/lib.rs.html This macro is used to conditionally compile items based on feature flags, particularly for documentation generation on docs.rs. It applies metadata and cfg attributes to the enclosed items. ```rust macro_rules! feature { ( #![$meta:meta] $($item:item)* ) => { $( #[cfg($meta)] #[cfg_attr(docsrs, doc(cfg($meta)))] $item )* }; } ``` -------------------------------- ### unescape Source: https://docs.rs/htmlize/1.1.0/htmlize/all.html Unescapes HTML entities within a string. ```APIDOC ## unescape ### Description Unescapes HTML entities within a string, converting them back to their original characters. ### Function Signature `fn unescape(input: &str) -> String` ### Parameters * **input** (string) - The string containing HTML entities to unescape. ### Returns A new string with HTML entities unescaped. ``` -------------------------------- ### Correct Numeric Entity Expansion Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Calculates the correct byte slice expansion for a given numeric HTML entity value. It handles specific error conditions like null characters, out-of-range Unicode values, surrogate pairs, and control characters, mapping them to their specified expansions or the replacement character. ```rust /// Calculate the expansion for a numeric entity (after parsing it). /// /// See #[allow(clippy::match_same_arms)] fn correct_numeric_entity(number: u32) -> Cow<'static, [u8]> { match number { // null-character-reference parse error: 0x00 => REPLACEMENT_CHAR_BYTES.into(), // character-reference-outside-unicode-range parse error: 0x11_0000.. => REPLACEMENT_CHAR_BYTES.into(), // https://infra.spec.whatwg.org/#surrogate // surrogate-character-reference parse error: 0xD800..=0xDFFF => REPLACEMENT_CHAR_BYTES.into(), // control-character-reference parse error exceptions: 0x80 => "\u{20AC}".as_bytes().into(), // EURO SIGN (€) 0x82 => "\u{201A}".as_bytes().into(), // SINGLE LOW-9 QUOTATION MARK (‚) 0x83 => "\u{0192}".as_bytes().into(), // LATIN SMALL LETTER F WITH HOOK (ƒ) 0x84 => "\u{201E}".as_bytes().into(), // DOUBLE LOW-9 QUOTATION MARK („) 0x85 => "\u{2026}".as_bytes().into(), // HORIZONTAL ELLIPSIS (…) 0x86 => "\u{2020}".as_bytes().into(), // DAGGER (†) 0x87 => "\u{2021}".as_bytes().into(), // DOUBLE DAGGER (‡) 0x88 => "\u{02C6}".as_bytes().into(), // MODIFIER LETTER CIRCUMFLEX ACCENT (ˆ) 0x89 => "\u{2030}".as_bytes().into(), // PER MILLE SIGN (‰) 0x8A => "\u{0160}".as_bytes().into(), // LATIN CAPITAL LETTER S WITH CARON (Š) 0x8B => "\u{2039}".as_bytes().into(), // SINGLE LEFT-POINTING ANGLE QUOTATION MARK (‹) 0x8C => "\u{0152}".as_bytes().into(), // LATIN CAPITAL LIGATURE OE (Œ) 0x8E => "\u{017D}".as_bytes().into(), // LATIN CAPITAL LETTER Z WITH CARON (Ž) 0x91 => "\u{2018}".as_bytes().into(), // LEFT SINGLE QUOTATION MARK (‘) 0x92 => "\u{2019}".as_bytes().into(), // RIGHT SINGLE QUOTATION MARK (’) 0x93 => "\u{201C}".as_bytes().into(), // LEFT DOUBLE QUOTATION MARK (“) 0x94 => "\u{201D}".as_bytes().into(), // RIGHT DOUBLE QUOTATION MARK (”) 0x95 => "\u{2022}".as_bytes().into(), // BULLET (•) 0x96 => "\u{2013}".as_bytes().into(), // EN DASH (–) 0x97 => "\u{2014}".as_bytes().into(), // EM DASH (—) 0x98 => "\u{02DC}".as_bytes().into(), // SMALL TILDE (˜) 0x99 => "\u{2122}".as_bytes().into(), // TRADE MARK SIGN (™) 0x9A => "\u{0161}".as_bytes().into(), // LATIN SMALL LETTER S WITH CARON (š) 0x9B => "\u{203A}".as_bytes().into(), // SINGLE RIGHT-POINTING ANGLE QUOTATION MARK (›) 0x9C => "\u{0153}".as_bytes().into(), // LATIN SMALL LIGATURE OE (œ) 0x9E => "\u{017E}".as_bytes().into(), // LATIN SMALL LETTER Z WITH CARON (ž) } } ``` -------------------------------- ### Test Special Entity Decoding Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html Verifies the decoding of special entities like null, bullet points, and spaces, including combinations of different entity types. ```rust test_both!(special_entity_null, unescape("�") == "\u{fffd}"); test_both!(special_entity_bullet, unescape("•") == "•"); test_both!( special_entity_bullets, unescape("••••") == "••••" ); test_both!(special_entity_space, unescape(" ") == " "); ``` -------------------------------- ### Test Cases for Unescape Function Source: https://docs.rs/htmlize/1.1.0/src/htmlize/unescape/internal.rs.html These tests cover various scenarios for the `unescape` function, including handling of partial entities, invalid entities, mixed valid and invalid entities, and long entities. They verify correct conversion of HTML entities to their corresponding characters. ```rust test_both!(almost_entity, unescape("&time") == "&time"); ``` ```rust test_both!(exact_times, unescape("×") == "×"); ``` ```rust test_both!(exact_timesb, unescape("⊠") == "⊠"); ``` ```rust test_both!(bare_times_end, unescape("×") == "×"); ``` ```rust test_both!(bare_times_bang, unescape("×!") == "×!"); ``` ```rust test!(bare_entity_char, unescape("×a") == "×a"); ``` ```rust test!(other_entity, unescape("⨱") == "⨱"); // To confirm next test ``` ```rust test!(bare_entity_almost_other, unescape("×bar") == "×bar"); ``` ```rust test!( bare_entity_long_suffix, unescape("×barrrrrr") == "×barrrrrr" ); ``` ```rust test!(bare_entity_equal, unescape("×=") == "×="); ``` ```rust test!(bare_entity_char_semicolon, unescape("×a;") == "×a;"); ``` ```rust test!(bare_entity_equal_semicolon, unescape("×=;") == "×=;"); ``` ```rust test_both!(bare_entity_entity, unescape("×<") == "×<"); ``` ```rust test!(bare_entity_char_is_prefix, unescape("×b") == "×b"); ``` ```rust test!( bare_entity_char_is_prefix_entity, unescape("×b<") == "×b<" ); ``` ```rust test_both!(empty, unescape("") == ""); ``` ```rust test_both!(no_entities, unescape("none") == "none"); ``` ```rust test_both!(only_ampersand, unescape("&") == "&"); ``` ```rust test_both!(empty_entity, unescape("&;") == "&;"); ``` ```rust test_both!(invalid_entity, unescape("&time;") == "&time;"); ``` ```rust test_both!(middle_invalid_entity, unescape(" &time; ") == " &time; "); ``` ```rust test_both!( mixed_valid_invalid_entities, unescape("&time; & &time; & &time;") == "&time; & &time; & &time;" ); ``` ```rust test_both!(middle_entity, unescape(" & ") == " & "); ``` ```rust test_both!(extra_ampersands, unescape("&&&") == "&&&"); ``` ```rust test_both!(two_entities, unescape("AND && and") == "AND && and"); ``` ```rust test_both!( long_valid_entity, unescape("∳") == "∳" ); ``` ```rust test_both!( long_invalid_entity, unescape("&CounterClockwiseContourIntegralX;") == "&CounterClockwiseContourIntegralX;" ); ``` ```rust test_both!( very_long_invalid_entity, unescape("&aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa;") == "&aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa;" ); ``` ```rust test_both!(correct_hex_lowerx_lower, unescape("z") == "z"); ``` ```rust test_both!(correct_hex_lowerx_upper, unescape("z") == "z"); ``` ```rust test_both!(correct_hex_upperx_lower, unescape("z") == "z"); ``` ```rust test_both!(correct_hex_upperx_upper, unescape("z") == "z"); ```