### Rust Struct Initialization: By Value vs By Reference Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/style_guide.md Demonstrates idiomatic Rust for struct initialization. The 'GOOD' example shows returning a new struct instance by value, leveraging Rust's optimizations, while the 'BAD' example unnecessarily clones a reference. ```rust struct ListFormat { locale: Locale } impl ListFormat { // BAD pub fn try_new(locale: &Locale) -> Result { Ok(Self { locale: locale.clone() }) } // GOOD pub fn try_new(locale: Locale) -> Result { Ok(Self { locale }) } } ``` -------------------------------- ### Install ICU4X and Create Entry Point Source: https://github.com/unicode-org/icu4x/blob/main/ffi/npm/README.md Installs the ICU4X NPM package and creates a new JavaScript module file for application entry. This is the initial setup step for using ICU4X in a project. ```shell npm add icu touch index.mjs ``` -------------------------------- ### Verify Rust and Cargo Installation Source: https://github.com/unicode-org/icu4x/wiki/Introduction-to-ICU4X-for-Rust This snippet shows how to verify the installation of Git, Rust, and Cargo by checking their version numbers in the terminal. It's a prerequisite for following the tutorial. ```bash user@host:~/projects/icu$ git --version git version 2.27.0 user@host:~/projects/icu$ cargo --version cargo 1.47.0 (f3c7e066a 2020-08-28) ``` -------------------------------- ### Rust: Using `try_new` with Default Options Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/style_guide.md A minimal example showing the usage of a `try_new` constructor with default options. This pattern is common when a struct requires configuration but has sensible defaults. The `.expect()` call handles potential construction failures. ```rust fn main() { // Assuming 'locale' is defined elsewhere and 'MyStruct' has a 'try_new' method. // let locale = ...; // let s = MyStruct::try_new(locale, MyStructOptions::default()).expect("Construction failed."); } ``` -------------------------------- ### Decimal Formatting Example in C++ Source: https://github.com/unicode-org/icu4x/blob/main/examples/cpp/README.md A C++ example demonstrating how to use ICU4X for decimal formatting. It includes initializing logging, creating locale and formatter objects, formatting a decimal number, and verifying the output against an expected string. ```cpp #include #include #include #include int main() { // For basic logging Logger::init_simple_logger(); // Create a locale object representing Bangla std::unique_ptr locale = Locale::from_string("bn").ok().value(); std::cout << "Running test for locale " << locale->to_string() << std::endl; // Create a formatter object with the appropriate settings std::unique_ptr formatter = DecimalFormatter::create_with_grouping_strategy( *locale.get(), DecimalGroupingStrategy::Auto).ok().value(); // Create a decimal representing the number 1,000,007 std::unique_ptr decimal = Decimal::from(1000007); // Format it to a string std::string out = formatter->format(*decimal.get()); // Report formatted value std::cout << "Formatted value is " << out << std::endl; if (out != "১০,০০,০০৭") { std::cout << "Output does not match expected output" << std::endl; return 1; } return 0; } ``` -------------------------------- ### Install icu4x-datagen Tool Source: https://github.com/unicode-org/icu4x/blob/main/tutorials/date-picker-data.md Installs the `icu4x-datagen` command-line tool, which is used to generate locale data. Ensure Rust and Cargo are installed beforehand. ```bash cargo install icu4x-datagen ``` -------------------------------- ### Demonstrate Unambiguous Imports and Usage - Rust Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/style_guide.md Provides a comprehensive example of how ICU4X's naming conventions facilitate clear and unambiguous imports and usage of types from different modules, even when multiple structs from various components are used together. It showcases `use` statements and subsequent variable instantiations. ```rust use icu_locale_core::Locale; use icu_datetime::{DateTimeFormat, DateTimeLength, skeleton::{Skeleton, SkeletonField}}; use icu_list::ListFormat; fn format() -> Result { let loc: Locale = "ru".parse()?; let mut dt_ops = Default::default(); dt_opts.date_length = DateTimeLength::Long; let dtf = DateTimeFormat::try_new(loc.clone(), dt_opts)?; let lf = ListFormat::try_new(loc, Default::default())?; assert_eq!(Skeleton::from_length(dt_opts.date_length).get(0), Some(SkeletonField::YYYY)); } ``` -------------------------------- ### Enable Memory Benchmarking Feature in Examples Source: https://github.com/unicode-org/icu4x/blob/main/tools/benchmark/README.md This command demonstrates how to enable the `benchmark_memory` feature for an example in ICU4X. This feature integrates dhat-rs instrumentation, allowing for memory profiling directly within the example's execution. The output includes memory statistics and information on viewing the collected data. ```bash cargo run --example code_line_diff --features icu_benchmark_macros/benchmark_memory ``` -------------------------------- ### GitHub Actions Artifact Download Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/ci_build.md Demonstrates downloading artifacts in GitHub Actions. This example downloads previously uploaded benchmark performance data for use in a subsequent job, specifying the path and artifact name. ```yaml - name: Download previous content destined for GH pages uses: actions/download-artifact@v2 with: path: ./copy-to-ext-repo/benchmarks/perf name: benchmark-perf ``` -------------------------------- ### JSON Data File Example Source: https://github.com/unicode-org/icu4x/blob/main/documents/design/data_pipeline.md Illustrates a JSON data file structure containing localized month names, which can be mapped to specific keys by a DataProvider. This serves as an example of raw data that ICU4X can process and serve. ```json { "gregorian_months": [ "January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December" ] } ``` -------------------------------- ### JSON Key-Value Example for Data Provider Source: https://github.com/unicode-org/icu4x/blob/main/documents/proposals/pr002-hunkspace.md Illustrates a key-value pair structure for localizable resources within the ICU4X data provider. This example showcases a custom schema with dimensions like 'source', 'app', 'message_id', and 'locale'. It's useful for understanding how non-CLDR data can be represented. ```json { "Key": { "source" = "fuchsia.dev", "app"="weirdy-birds", "message_id"=42, "locale"="en-US" }, "Value": "Hello, {user}" } ``` -------------------------------- ### Install and Use icu4x-datagen CLI Source: https://github.com/unicode-org/icu4x/blob/main/provider/icu4x-datagen/README.md Installs the `icu4x-datagen` tool using Cargo and demonstrates its usage for generating data files. It specifies markers, locales, output format, and output file. ```bash cargo install icu4x-datagen icu4x-datagen --markers all --locales de en-AU --format blob --out data.postcard ``` -------------------------------- ### Install ICU4X Rust Dependencies Source: https://github.com/unicode-org/icu4x/blob/main/tutorials/date-picker.md This snippet shows how to create a new Rust binary project and add `icu` as a dependency, preparing the environment for ICU4X integration. ```console cargo new --bin tutorial cd tutorial cargo add icu ``` -------------------------------- ### Install LLVM 18 via Apt Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/rust_versions.md Installs LLVM version 18 on Ubuntu-based systems using the `apt` package manager. This is the command used within ICU4X CI on GitHub Actions runners utilizing `ubuntu-latest`. ```shell apt install llvm-18 ``` -------------------------------- ### ICU4X: Complete Usage Example in Rust Source: https://context7.com/unicode-org/icu4x/llms.txt A comprehensive Rust example demonstrating the integration of multiple ICU4X internationalization components. It showcases date formatting, decimal formatting, plural rules, list formatting, case mapping, normalization, collation, and word segmentation using various ICU4X modules. ```rust use icu::calendar::Date; use icu::casemap::CaseMapper; use icu::collator::{Collator, options::*}; use icu::datetime::{DateTimeFormatter, fieldsets::YMD}; use icu::decimal::{DecimalFormatter, input::Decimal}; use icu::list::ListFormatter; use icu::locale::{locale, langid}; use icu::normalizer::ComposingNormalizerBorrowed; use icu::plurals::{PluralRules, PluralCategory}; use icu::segmenter::WordSegmenter; fn main() { // Date formatting let dtf = DateTimeFormatter::try_new( locale!("es-MX").into(), YMD::long() ).unwrap(); let date = Date::try_new_iso(2024, 10, 20).unwrap(); println!("Date: {}", dtf.format(&date)); // Output: "Date: 20 de octubre de 2024" // Decimal formatting let dec_fmt = DecimalFormatter::try_new( locale!("de").into(), Default::default() ).unwrap(); let number = Decimal::from(1234567); println!("Number: {}", dec_fmt.format(&number)); // Output: "Number: 1.234.567" // Plural rules let plural_rules = PluralRules::try_new_cardinal( locale!("ru").into(), Default::default() ).unwrap(); let category = plural_rules.category_for(3); println!("3 is {:?} in Russian", category); // Output: "3 is Few in Russian" // List formatting let list_fmt = ListFormatter::try_new_and( locale!("en").into(), Default::default() ).unwrap(); let items = ["apples", "oranges", "bananas"]; println!("List: {}", list_fmt.format_to_string(items.iter())); // Output: "List: apples, oranges, and bananas" // Case mapping let case_mapper = CaseMapper::new(); println!("Upper: {}", case_mapper.uppercase_to_string( "hello world", &langid!("en") )); // Output: "Upper: HELLO WORLD" // Normalization let nfc = ComposingNormalizerBorrowed::new_nfc(); println!("Normalized: {}", nfc.normalize("café")); // Output: "Normalized: café" // Collation let mut options = CollatorOptions::default(); options.strength = Some(Strength::Primary); let collator = Collator::try_new( locale!("sv").into(), options ).unwrap(); println!("Compare 'ä' vs 'a': {:?}", collator.compare("ä", "a") ); // Output: "Compare 'ä' vs 'a': Greater" // Segmentation let segmenter = WordSegmenter::new_auto(Default::default()); let text = "Hello world!"; let breaks: Vec = segmenter.segment_str(text).collect(); println!("Word breaks at: {:?}", breaks); // Output: "Word breaks at: [0, 5, 6, 11, 12]" } ``` -------------------------------- ### GitHub Actions Conditional Step Execution (Upload Artifact) Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/ci_build.md Shows how to conditionally upload artifacts using the `if` key in GitHub Actions. This example uploads benchmark performance data only when a push event occurs on the `main` branch. ```yaml - name: Upload updated benchmark data (merge to main only) if: github.event_name == 'push' && github.ref == 'refs/heads/main' uses: actions/upload-artifact@v2 with: path: ./benchmarks/perf/** # use wildcard pattern to preserve dir structure of uploaded files name: benchmark-perf ``` -------------------------------- ### Initialize a new Rust Binary Application Source: https://github.com/unicode-org/icu4x/wiki/Introduction-to-ICU4X-for-Rust This command initializes a new binary Rust application named 'myapp' within the specified project directory. It creates the necessary file structure, including the main application file './src/main.rs'. ```bash cd ~/projects/icu cargo init --bin myapp ``` -------------------------------- ### Build Rust Docs with Private Items Source: https://github.com/unicode-org/icu4x/blob/main/utils/zerotrie/README.md This bash command illustrates how to generate Rust documentation for a project, including private items, all features, and without external dependencies. The `--open` flag will attempt to open the generated documentation in a web browser. ```bash cargo doc --document-private-items --all-features --no-deps --open ``` -------------------------------- ### Rust: Implementing Default and Non-exhaustive for Options Structs Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/style_guide.md Demonstrates how to implement the `Default` trait and use the `#[non_exhaustive]` attribute for options structs in Rust. This pattern simplifies construction and allows for future extensions without breaking changes. The example shows a `MyStructOptions` struct with default values and a `MyStruct` that uses these options. ```rust #[derive(Default)] #[non_exhaustive] struct MyStructOptions { pub min_fraction_digits: usize, pub max_fraction_digits: usize, } impl Default for MyStructOptions { pub fn default() -> Self { Self { min_fraction_digits: 3, max_fraction_digits: 5, } } } impl MyStructOptions { // Additional helper methods for setting the options, // or validating the consistency of the options. } struct MyStruct { locale: Locale, fraction_digits: Range, } impl MyStruct { pub fn new(locale: Locale, options: MyStructOptions) -> Result { ... }; } fn main() { let mut options = MyStructOptions::default(); // Assuming options has a field like `max_fraction_digits` // options.max_fraction_digits = 10; // Optional debug time validation of the options // debug_assert!(options.validate()); // let s = MyStruct::try_new(locale, options).expect("Construction failed."); // Note: try_new is not defined in the provided snippet, assuming it's a placeholder } ``` -------------------------------- ### DataProvider Mapping Pseudocode Example Source: https://github.com/unicode-org/icu4x/blob/main/documents/design/data_pipeline.md Demonstrates pseudocode for a DataProvider mapping, showing how different keys (e.g., DATE_SYM_JAN_V1, DATE_SYM_MONTHS_V1) can be resolved to specific entries within a JSON data structure like gregorian_months. This allows a single data source to serve multiple data key versions efficiently. ```pseudocode switch (key) { case DATE_SYM_JAN_V1: return json["gregorian_months"][0]; case DATE_SYM_FEB_V1: return json["gregorian_months"][1]; case DATE_SYM_MONTHS_V1: return json["gregorian_months"]; } ``` -------------------------------- ### Example Changelog Entry (Markdown) Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/release.md This markdown snippet shows the expected format for an out-of-cycle changelog entry for a specific ICU4X crate. It includes the crate name, version, and a brief description of the fix, referencing the associated issue number. ```markdown - `databake`: 0.1.5 - Fixed [#3356](https://github.com/unicode-org/icu4x/pull/3356), adding `allow` for clippy false-positives ``` -------------------------------- ### Build WebAssembly CodePointTrie Wrapper (Bash) Source: https://github.com/unicode-org/icu4x/blob/main/components/collections/codepointtrie_builder/cpp/README.md This script demonstrates how to build the `ucptrie_wrap.wat` WebAssembly module using `make`. It requires a local copy of ICU4C sources and specific development packages. The output is a WebAssembly Text file. ```bash make clean make ICU4C_SOURCE=/path/to/icu4c/source ucptrie_wrap.wat ``` -------------------------------- ### Install LLVM 18 via Homebrew Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/rust_versions.md Installs LLVM version 18 using the Homebrew package manager on macOS (OSX). This ensures that the required LLVM version is available for development and testing on Apple systems. ```shell brew install llvm@18 ``` -------------------------------- ### Build and Generate Slimmed Data Pack (Console) Source: https://github.com/unicode-org/icu4x/blob/main/tutorials/date-picker-data.md This console command sequence shows how to build the ICU4X project in release mode and then use the `icu4x-datagen` tool with the `--markers-for-bin` flag to generate a smaller data pack (`.blob` file). This is achieved by only including markers relevant to the specified locales and format, leveraging the optimizations made in the code. ```bash cargo build --release icu4x-datagen --markers-for-bin target/release/tutorial --locales ccp --format blob --out ccp_smallest.blob ``` -------------------------------- ### Publish Components with Cargo Source: https://github.com/unicode-org/icu4x/wiki/Release-Process This snippet demonstrates the command used to publish each component and meta-component of the project to the cargo registry. It's a crucial step in making the release available to users. ```shell cargo publish ``` -------------------------------- ### Install LLVM Toolset via Yum/DNF Source: https://github.com/unicode-org/icu4x/blob/main/documents/process/rust_versions.md Installs the LLVM toolset using package managers on Red Hat Enterprise Linux (RHEL) or Rocky Linux. This command is applicable for older RHEL versions using `yum` and RHEL 9+ using `dnf`. ```shell yum module install llvm-toolset ``` ```shell dnf install llvm-toolset ``` -------------------------------- ### Prepend Operation Example in Model B Source: https://github.com/unicode-org/icu4x/blob/main/documents/research/format_to_parts.md Illustrates the quadratic inefficiency of the prepend operation in Model B due to the need to update all subsequent field pointers. This example shows how prepending a string requires updating existing field metadata. ```text Current string: "20" with fields: [{ field: "day", start: 0, end: 2 }] String to prepend: "November " with field "month" After prepend and field update: [{ field: "month", start: 0, end: 9 }, { field: "day", start: 9, end: 11 }] ``` -------------------------------- ### Running ICU4X Overview Benchmarks with Cargo Source: https://github.com/unicode-org/icu4x/wiki/Benchmarking This command executes the default overview benchmarks for ICU4X components. These are intended to provide a quick performance check during development. Ensure you are in the project's root directory for this command to work correctly. ```bash cargo bench ``` -------------------------------- ### Running Detailed ICU4X Benchmarks with Cargo and Features Source: https://github.com/unicode-org/icu4x/wiki/Benchmarking This command allows running detailed benchmarks for a specific component (e.g., 'operands' in this example) by enabling the 'bench' feature. This is useful for in-depth performance analysis when specific parts of a component need to be investigated. ```bash cargo bench operands --features bench ``` -------------------------------- ### Rust Example: Format Date with ICU4X Source: https://github.com/unicode-org/icu4x/blob/main/README.md This Rust code snippet demonstrates how to format a date using the ICU4X library. It requires the 'icu' crate as a dependency. The example shows how to initialize a DateTimeFormatter for a specific locale and date, then format a Date object, asserting the expected output. ```toml [dependencies] icu = "2.0.0" ``` ```rust use icu::calendar::Date; use icu::datetime::{DateTimeFormatter, fieldsets::YMD}; use icu::locale::locale; let dtf = DateTimeFormatter::try_new( locale!("es").into(), YMD::long() ) .expect("locale should be present in compiled data"); let date = Date::try_new_iso(2020, 9, 12).expect("date should be valid"); let formatted_date = dtf.format(&date).to_string(); assert_eq!( formatted_date, "12 de septiembre de 2020" ); ``` -------------------------------- ### Basic Cargo.toml with Compiled Data Source: https://github.com/unicode-org/icu4x/blob/main/examples/cargo/README.md This snippet shows the most basic Cargo.toml setup for ICU4X, including the main 'icu' dependency. It allows the use of all stable ICU4X components with a default set of locales compiled into the library. No additional environment variables or features are needed for this basic setup. ```toml [dependencies] icu = "2.0.0" ``` -------------------------------- ### Initialize FsDataProvider in Rust Source: https://github.com/unicode-org/icu4x/wiki/Introduction-to-ICU4X-for-Rust This Rust code demonstrates how to initialize a file-system data provider (`FsDataProvider`) for ICU4X. It takes a path to the directory containing the generated ICU4X data and returns a `Result` which is unwrapped using `expect`. This provider is essential for accessing locale data. ```rust use icu_provider_fs::FsDataProvider; fn main() { let _provider = FsDataProvider::try_new("~/projects/icu/icu4x-data") .expect("Failed to initialize Data Provider."); } ``` -------------------------------- ### Run Memory Benchmarks with icu_benchmark_memory Source: https://github.com/unicode-org/icu4x/blob/main/tools/benchmark/README.md This tool uses dhat-rs to collect memory instrumentation data from examples. It can be used to run specific examples and output memory usage statistics to stderr and a JSON file for analysis. When run in CI, data is collected into charts for tracking memory changes over time. ```shell cargo run --package icu_benchmark_memory cargo run --package icu_benchmark_memory -- icu_locale_core/filter_langids icu_datetime/work_log cargo run --package icu_benchmark_memory -- --os macos-latest icu_datetime/work_log ``` -------------------------------- ### Custom ULE Type Implementation Example in Rust Source: https://github.com/unicode-org/icu4x/blob/main/utils/zerovec/design_doc.md Illustrates how to create a custom ULE type in Rust for a struct named `Foo`. It shows the definition of the struct and its corresponding `FooULE` representation, which uses `#[repr(C, packed)]` for memory layout control. The example highlights the necessity of custom `AsULE` implementation for type shuttling and `ULE` implementation for byte stream validation. ```rust // Implements AsULE struct Foo { field1: u32, field2: char, field3: i16 } // Implements ULE #[repr(C, packed)] struct FooULE { field1: u32::ULE, field2: char::ULE, field3: i16::ULE, } ``` -------------------------------- ### Use Custom ULE Types in ZeroVec/VarZeroVec with Bincode Source: https://github.com/unicode-org/icu4x/blob/main/utils/zerovec/README.md This example demonstrates the usage of custom ULE (Unsized Like Encoded) types within ZeroVec, VarZeroVec, and ZeroMap, and their serialization/deserialization using Bincode. It defines custom struct types for fixed-size ('Date') and variable-sized ('Person') data, along with their corresponding ULE representations. The example shows how to populate these collections and then serialize and deserialize a struct containing them. ```rust use zerovec::{ZeroVec, VarZeroVec, ZeroMap}; use std::borrow::Cow; use zerovec::ule::encode_varule_to_box; // custom fixed-size ULE type for ZeroVec #[zerovec::make_ule(DateULE)] #[derive(Copy, Clone, PartialEq, Eq, Ord, PartialOrd, serde::Serialize, serde::Deserialize)] struct Date { y: u64, m: u8, d: u8 } // custom variable sized VarULE type for VarZeroVec #[zerovec::make_varule(PersonULE)] #[zerovec::derive(Serialize, Deserialize)] // add Serde impls to PersonULE #[derive(Clone, PartialEq, Eq, Ord, PartialOrd, serde::Serialize, serde::Deserialize)] struct Person<'a> { birthday: Date, favorite_character: char, #[serde(borrow)] name: Cow<'a, str>, } #[derive(serde::Serialize, serde::Deserialize)] struct Data<'a> { #[serde(borrow)] important_dates: ZeroVec<'a, Date>, // note: VarZeroVec always must reference the ULE type directly #[serde(borrow)] important_people: VarZeroVec<'a, PersonULE>, #[serde(borrow)] birthdays_to_people: ZeroMap<'a, Date, PersonULE> } let person1 = Person { birthday: Date { y: 1990, m: 9, d: 7}, favorite_character: 'π', name: Cow::from("Kate") }; let person2 = Person { birthday: Date { y: 1960, m: 5, d: 25}, favorite_character: '冇', name: Cow::from("Jesse") }; let important_dates = ZeroVec::alloc_from_slice(&[Date { y: 1943, m: 3, d: 20}, Date { y: 1976, m: 8, d: 2}, Date { y: 1998, m: 2, d: 15}]); let important_people = VarZeroVec::from(&[&person1, &person2]); let mut birthdays_to_people: ZeroMap = ZeroMap::new(); // ".insert_var_v()" is slightly more convenient over ".insert()" for custom ULE types birthdays_to_people.insert_var_v(&person1.birthday, &person1); birthdays_to_people.insert_var_v(&person2.birthday, &person2); let data = Data { important_dates, important_people, birthdays_to_people }; let bincode_bytes = bincode::serialize(&data) .expect("Serialization should be successful"); assert_eq!(bincode_bytes.len(), 160); let deserialized: Data = bincode::deserialize(&bincode_bytes) .expect("Deserialization should be successful"); assert_eq!(deserialized.important_dates.get(0).unwrap().y, 1943); assert_eq!(&deserialized.important_people.get(1).unwrap().name, "Jesse"); assert_eq!(&deserialized.important_people.get(0).unwrap().name, "Kate"); assert_eq!(&deserialized.birthdays_to_people.get(&person1.birthday).unwrap().name, "Kate"); } // feature = serde and derive ``` -------------------------------- ### HTML UI Declaration for Localization Source: https://github.com/unicode-org/icu4x/blob/main/documents/proposals/pr002-hunkspace.md An example of an HTML UI declaration file used in Firefox, specifying localization links for different parts of the user interface. ```html ```