### SortOptions Creation Examples Source: https://docs.rs/arrow/latest/arrow/compute/struct.SortOptions.html Examples demonstrating how to create and configure SortOptions. ```APIDOC ## §Example creation ```rust // configure using explicit initialization let options = SortOptions { descending: false, nulls_first: true, }; // Default is ASC NULLs First assert_eq!(options, SortOptions::default()); assert_eq!(options.to_string(), "ASC NULLS FIRST"); // Configure using builder APIs let options = SortOptions::default() .desc() .nulls_first(); assert_eq!(options.to_string(), "DESC NULLS FIRST"); // configure using explicit field values let options = SortOptions::default() .with_descending(false) .with_nulls_first(false); assert_eq!(options.to_string(), "ASC NULLS LAST"); ``` ``` -------------------------------- ### starts_with Example Source: https://docs.rs/arrow/latest/arrow/compute/kernels/comparison/fn.starts_with.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E An example demonstrating the usage of the `starts_with` function with `StringArray` and `BooleanArray`. ```APIDOC ## Example ```rust let strings = StringArray::from(vec!["arrow-rs", "arrow-rs", "arrow-rs", "Parquet"]); let patterns = StringArray::from(vec!["arr", "arrow", "arrow-cpp", "p"]); let result = starts_with(&strings, &patterns).unwrap(); assert_eq!(result, BooleanArray::from(vec![true, true, false, false])); ``` ``` -------------------------------- ### Example Usage Source: https://docs.rs/arrow/latest/arrow/array/struct.RecordBatchIterator.html?search=std%3A%3Avec Example demonstrating how to create and use a RecordBatchIterator. ```APIDOC ```rust let a: ArrayRef = Arc::new(Int32Array::from(vec![1, 2])); let b: ArrayRef = Arc::new(StringArray::from(vec!["a", "b"])); let record_batch = RecordBatch::try_from_iter(vec![ ("a", a), ("b", b), ]).unwrap(); let batches: Vec = vec![record_batch.clone(), record_batch.clone()]; let mut reader = RecordBatchIterator::new(batches.into_iter().map(Ok), record_batch.schema()); assert_eq!(reader.schema(), record_batch.schema()); assert_eq!(reader.next().unwrap().unwrap(), record_batch); ``` ``` -------------------------------- ### starts_with Example Usage Source: https://docs.rs/arrow/latest/arrow/compute/kernels/comparison/fn.starts_with.html?search= Demonstrates how to use the starts_with kernel with StringArrays. It checks if elements in the 'strings' array start with corresponding patterns in the 'patterns' array. ```rust let strings = StringArray::from(vec!["arrow-rs", "arrow-rs", "arrow-rs", "Parquet"]); let patterns = StringArray::from(vec!["arr", "arrow", "arrow-cpp", "p"]); let result = starts_with(&strings, &patterns).unwrap(); assert_eq!(result, BooleanArray::from(vec![true, true, false, false])); ``` -------------------------------- ### UnionArray Creation Examples Source: https://docs.rs/arrow/latest/arrow/array/struct.UnionArray.html?search= Demonstrates how to create both dense and sparse UnionArrays with example data. ```APIDOC ## Examples ### Create a dense UnionArray `[1, 3.2, 34]` ```rust // Example code for creating a dense UnionArray ``` ### Create a sparse UnionArray `[1, 3.2, 34]` ```rust // Example code for creating a sparse UnionArray ``` ``` -------------------------------- ### IntervalYearMonthArray Example Source: https://docs.rs/arrow/latest/arrow/array/array/type.IntervalYearMonthArray.html?search=std%3A%3Avec Example demonstrating how to create and initialize an IntervalYearMonthArray. ```APIDOC ## §Example ```rust let array = IntervalYearMonthArray::from(vec![ 2, // 2 months 25, // 2 years and 1 month -1 // -1 months ]); ``` ``` -------------------------------- ### take Function Example Source: https://docs.rs/arrow/latest/arrow/compute/kernels/take/fn.take.html?search= An example demonstrating how to use the `take` function to select specific elements from a `StringArray` using a `UInt32Array` for indices. ```APIDOC ### Examples ```rust let values = StringArray::from(vec!["zero", "one", "two"]); // Take items at index 2, and 1: let indices = UInt32Array::from(vec![2, 1]); let taken = take(&values, &indices, None).unwrap(); let taken = taken.as_string::(); assert_eq!(*taken, StringArray::from(vec!["two", "one"])); ``` ``` -------------------------------- ### Example Searches Source: https://docs.rs/arrow/latest/arrow/buffer/struct.Buffer.html?search= Provides examples of how to perform searches within the buffer. ```APIDOC ## Example Searches ### Description Demonstrates common search patterns. ### Examples * `std::vec` * `u32 -> bool` * `Option, (T -> U) -> Option` ``` -------------------------------- ### BooleanBufferBuilder Example Source: https://docs.rs/arrow/latest/arrow/array/struct.BooleanBufferBuilder.html?search=u32+-%3E+bool Example demonstrating the creation and usage of BooleanBufferBuilder. ```APIDOC ## Example ```rust let mut builder = BooleanBufferBuilder::new(10); builder.append(true); builder.append(false); builder.append_n(3, true); // append 3 trues let buffer = builder.build(); assert_eq!(buffer.len(), 5); // 5 bits appended assert_eq!(buffer.values(), &[0b00011101_u8]); // packed bits ``` ``` -------------------------------- ### shift Function Examples Source: https://docs.rs/arrow/latest/arrow/compute/kernels/window/fn.shift.html?search= Examples demonstrating how to use the `shift` function for right, left, and zero shifts. ```APIDOC ## Examples ```rust let a: Int32Array = vec![Some(1), None, Some(4)].into(); // shift array 1 element to the right let res = shift(&a, 1).unwrap(); let expected: Int32Array = vec![None, Some(1), None].into(); assert_eq!(res.as_ref(), &expected); // shift array 1 element to the left let res = shift(&a, -1).unwrap(); let expected: Int32Array = vec![None, Some(4), None].into(); assert_eq!(res.as_ref(), &expected); // shift array 0 element, although not recommended let res = shift(&a, 0).unwrap(); let expected: Int32Array = vec![Some(1), None, Some(4)].into(); assert_eq!(res.as_ref(), &expected); // shift array 3 element to the right let res = shift(&a, 3).unwrap(); let expected: Int32Array = vec![None, None, None].into(); assert_eq!(res.as_ref(), &expected); ``` ``` -------------------------------- ### ListViewArray Construction Example Source: https://docs.rs/arrow/latest/arrow/array/array/type.ListViewArray.html This example demonstrates how to create a ListViewArray from a vector of optional vectors of optional integers. ```APIDOC ## §Example ```rust let data = vec![ Some(vec![Some(0), Some(1), Some(2)]), None, Some(vec![Some(3), None, Some(5)]), Some(vec![Some(6), Some(7)]), ]; let list_array = ListViewArray::from_iter_primitive::(data); println!("{:?}", list_array); ``` ``` -------------------------------- ### Get BooleanBuffer Offset Source: https://docs.rs/arrow/latest/arrow/buffer/struct.BooleanBuffer.html?search=u32+-%3E+bool Returns the starting bit offset of the BooleanBuffer within its underlying storage. This is useful when dealing with buffers that do not start at bit 0. ```rust let buffer = BooleanBuffer::from_bitwise_unary_op(&[0u8], 4, 8, |a| a); assert_eq!(buffer.offset(), 4); ``` -------------------------------- ### Get Stream Position Source: https://docs.rs/arrow/latest/arrow/array/type.ArrayDataRef.html?search=std%3A%3Avec Returns the current seek position from the start of the stream. ```APIDOC ## fn stream_position(&mut self) -> Result ### Description Returns the current seek position from the start of the stream. ### Method `stream_position` ### Response #### Success Response (Result) - `Ok(u64)`: The current position in the stream. - `Err(Error)`: An error occurred while getting the stream position. ``` -------------------------------- ### Buffer Creation and Conversion Examples Source: https://docs.rs/arrow/latest/arrow/buffer/struct.Buffer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates how to create a Buffer from a Vec or bytes::Bytes, and how to convert a Buffer back into a Vec. ```APIDOC ## Example: Create a `Buffer` from a `Vec` (without copying) ```rust let vec: Vec = vec![1, 2, 3]; let buffer = Buffer::from(vec); ``` ## Example: Convert a `Buffer` to a `Vec` (without copying) Use `Self::into_vec` to convert a `Buffer` back into a `Vec` if there are no other references and the types are aligned correctly. ```rust // convert the buffer back into a Vec of u32 // note this will fail if the buffer is shared or not aligned correctly let vec: Vec = buffer.into_vec().unwrap(); ``` ## Example: Create a `Buffer` from a `bytes::Bytes` (without copying) `bytes::Bytes` is a common type in the Rust ecosystem for shared memory regions. You can create a buffer from a `Bytes` instance using the `From` implementation, also without copying. ```rust let bytes = bytes::Bytes::from("hello"); let buffer = Buffer::from(bytes); ``` ``` -------------------------------- ### Get Current Stream Position Source: https://docs.rs/arrow/latest/arrow/array/array/type.ArrayRef.html?search= Returns the current byte offset from the start of the stream. ```rust fn stream_position(&mut self) -> Result ``` -------------------------------- ### Get Zero-Copy Slice of ScalarBuffer Source: https://docs.rs/arrow/latest/arrow/buffer/struct.ScalarBuffer.html?search=u32+-%3E+bool Returns a zero-copy slice of the buffer with a specified length and starting offset. ```rust pub fn slice(&self, offset: usize, len: usize) -> ScalarBuffer ``` -------------------------------- ### SortOptions Creation and Configuration Source: https://docs.rs/arrow/latest/arrow/compute/kernels/sort/struct.SortOptions.html?search= Examples demonstrating how to create and configure SortOptions using explicit initialization, default values, and builder APIs. ```APIDOC ## §Example creation ```rust // configure using explicit initialization let options = SortOptions { descending: false, nulls_first: true, }; // Default is ASC NULLs First assert_eq!(options, SortOptions::default()); assert_eq!(options.to_string(), "ASC NULLS FIRST"); // Configure using builder APIs let options = SortOptions::default() .desc() .nulls_first(); assert_eq!(options.to_string(), "DESC NULLS FIRST"); // configure using explicit field values let options = SortOptions::default() .with_descending(false) .with_nulls_first(false); assert_eq!(options.to_string(), "ASC NULLS LAST"); ``` ``` -------------------------------- ### Getting ArrayData Offset Source: https://docs.rs/arrow/latest/arrow/array/struct.ArrayData.html Returns the offset of this `ArrayData`. This indicates the starting position within the underlying buffers. ```rust pub const fn offset(&self) -> usize ``` -------------------------------- ### Example of using BatchCoalescer Source: https://docs.rs/arrow/latest/arrow/compute/kernels/coalesce/struct.BatchCoalescer.html?search=std%3A%3Avec Demonstrates how to create, push batches to, and retrieve completed batches from a BatchCoalescer. Ensure the target batch size is met before retrieving a completed batch. ```rust use arrow_array::record_batch; use arrow_select::coalesce::{BatchCoalescer}; let batch1 = record_batch!(("a", Int32, [1, 2, 3])).unwrap(); let batch2 = record_batch!(("a", Int32, [4, 5])).unwrap(); // Create a `BatchCoalescer` that will produce batches with at least 4 rows let target_batch_size = 4; let mut coalescer = BatchCoalescer::new(batch1.schema(), 4); // push the batches coalescer.push_batch(batch1).unwrap(); // only pushed 3 rows (not yet 4, enough to produce a batch) assert!(coalescer.next_completed_batch().is_none()); coalescer.push_batch(batch2).unwrap(); // now we have 5 rows, so we can produce a batch let finished = coalescer.next_completed_batch().unwrap(); // 4 rows came out (target batch size is 4) let expected = record_batch!(("a", Int32, [1, 2, 3, 4])).unwrap(); assert_eq!(finished, expected); // Have no more input, but still have an in-progress batch assert!(coalescer.next_completed_batch().is_none()); // We can finish the batch, which will produce the remaining rows coalescer.finish_buffered_batch().unwrap(); let expected = record_batch!(("a", Int32, [5])).unwrap(); assert_eq!(coalescer.next_completed_batch().unwrap(), expected); // The coalescer is now empty assert!(coalescer.next_completed_batch().is_none()); ``` -------------------------------- ### AsFd Trait Implementation for Arc Source: https://docs.rs/arrow/latest/arrow/datatypes/type.SchemaRef.html Allows implementing traits that require `AsFd` on `Arc`. This example shows a basic setup for such an implementation. ```rust use std::net::UdpSocket; use std::sync::Arc; trait MyTrait: AsFd {} // Assuming AsFd is a trait that exists impl MyTrait for Arc {} impl MyTrait for Box {} ``` -------------------------------- ### BinaryRunBuilder Example Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryRunBuilder.html?search= Demonstrates creating and populating a BinaryRunBuilder, then finishing it into a RunArray. Shows appending values, nulls, and extending with multiple values, followed by assertions on the resulting array's run ends and values. ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded data is binary values. let mut builder = BinaryRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value(b"abc"); builder.append_null(); builder.extend([Some(b"def"), Some(b"def"), Some(b"abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &BinaryArray = av.as_binary(); assert_eq!(ava.value(0), b"abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), b"def"); assert_eq!(ava.value(3), b"abc"); ``` -------------------------------- ### Example: Creating and Populating a BinaryDictionaryBuilder Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryDictionaryBuilder.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates how to create a `BinaryDictionaryBuilder`, append binary values (including duplicates and nulls), and finish it into a `DictionaryArray`. It also shows how to access and verify the keys and values of the resulting array. ```rust // Create a dictionary array indexed by bytes whose values are binary. // It can thus hold up to 256 distinct binary values. let mut builder = BinaryDictionaryBuilder::::new(); // The builder builds the dictionary value by value builder.append(b"abc").unwrap(); builder.append_null(); builder.append(b"def").unwrap(); builder.append(b"def").unwrap(); builder.append(b"abc").unwrap(); let array = builder.finish(); assert_eq!( array.keys(), &Int8Array::from(vec![Some(0), None, Some(1), Some(1), Some(0)]) ); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &BinaryArray = av.as_any().downcast_ref::().unwrap(); assert_eq!(ava.value(0), b"abc"); assert_eq!(ava.value(1), b"def"); ``` -------------------------------- ### Substring by Character Example Source: https://docs.rs/arrow/latest/arrow/compute/kernels/substring/fn.substring_by_char.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates basic usage of substring_by_char with a start index and a specified length. Handles None values in the input array. ```rust let array = StringArray::from(vec![Some("arrow"), None, Some("Γ ⊢x:T")]); let result = substring_by_char(&array, 1, Some(4)).unwrap(); assert_eq!(result, StringArray::from(vec![Some("rrow"), None, Some(" ⊢x:")])); ``` -------------------------------- ### Construction Examples Source: https://docs.rs/arrow/latest/arrow/array/array/type.Int64Array.html?search= Demonstrates various ways to construct an Int64Array, including from Vec>, Vec, and iterators. ```APIDOC ## Construction ```rust // Create from Vec> let arr = Int64Array::from(vec![Some(1), None, Some(2)]); // Create from Vec let arr = Int64Array::from(vec![1, 2, 3]); // Create iter/collect let arr: Int64Array = std::iter::repeat(42).take(10).collect(); ``` See `PrimitiveArray` for more information and examples ``` -------------------------------- ### Get a mutable iterator over slice elements Source: https://docs.rs/arrow/latest/arrow/buffer/struct.MutableBuffer.html Returns an iterator that yields mutable references to each element, allowing modification. Iterates from start to end. ```rust let x = &mut [1, 2, 4]; for elem in x.iter_mut() { *elem += 2; } assert_eq!(x, &[3, 4, 6]); ``` -------------------------------- ### Example: Creating and Appending to BufferBuilder Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.BufferBuilder.html?search= Demonstrates creating a BufferBuilder, appending individual elements and slices, and finishing to create a Buffer. ```rust let mut builder = BufferBuilder::::new(100); builder.append_slice(&[42, 43, 44]); builder.append(45); let buffer = builder.finish(); assert_eq!(unsafe { buffer.typed_data::() }, &[42, 43, 44, 45]); ``` -------------------------------- ### Get Start Physical Index Source: https://docs.rs/arrow/latest/arrow/array/array/struct.RunArray.html?search= Retrieves the physical index at which the current array slice begins. This is equivalent to calling `get_start_physical_index` on the underlying `RunEndBuffer`. ```rust pub fn get_start_physical_index(&self) -> usize ``` -------------------------------- ### take Source: https://docs.rs/arrow/latest/arrow/datatypes/type.FieldRef.html?search=u32+-%3E+bool Creates an adapter that reads at most a specified number of bytes. ```APIDOC ## take ### Description Creates an adapter which will read at most `limit` bytes from it. ### Method `take(self, limit: u64) -> Take` ### Constraints `where Self: Sized` ``` -------------------------------- ### Get BooleanBuffer Offset Source: https://docs.rs/arrow/latest/arrow/buffer/struct.BooleanBuffer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Returns the starting bit offset of the BooleanBuffer within its underlying storage. This is useful when dealing with sub-sections of a larger bit buffer. ```rust let buffer = BooleanBuffer::from_slice(4, &[0b11001100u8]); // Offset of 4 bits assert_eq!(buffer.offset(), 4); ``` -------------------------------- ### take Source: https://docs.rs/arrow/latest/arrow/array/array/type.ArrayRef.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Creates an adapter which will read at most `limit` bytes from it. ```APIDOC ## fn take(limit: u64) ### Description Creates an adapter which will read at most `limit` bytes from it. ### Method (Implicitly called on a `Read` instance) ### Parameters - **limit** (u64): The maximum number of bytes to read. ### Response A `Take` adapter that limits the number of bytes read. ``` -------------------------------- ### Get Element Lengths with Zero for Nulls Source: https://docs.rs/arrow/latest/arrow/array/array/struct.GenericByteViewArray.html?search= The `lengths` method returns an iterator over the length of each element. The example demonstrates how to map null values to a length of 0. ```rust pub fn lengths(&self) -> impl ExactSizeIterator + Clone ``` ```rust use arrow_data::ByteView; fn lengths_with_zero_for_nulls(view: &StringViewArray) -> impl Iterator { view.lengths() .enumerate() .map(|(index, length)| if view.is_null(index) { 0 } else { length }) } ``` -------------------------------- ### Example Usage of BinaryViewBuilder Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryViewBuilder.html?search=u32+-%3E+bool Demonstrates how to create a BinaryViewBuilder, append string values and nulls, and then finish the builder to create a BinaryViewArray. Includes an assertion to verify the created array's content. ```rust use arrow_array::BinaryViewArray; let mut builder = BinaryViewBuilder::new(); builder.append_value("hello"); builder.append_null(); builder.append_value("world"); let array = builder.finish(); let expected: Vec> = vec![Some(b"hello"), None, Some(b"world")]; let actual: Vec<_> = array.iter().collect(); assert_eq!(expected, actual); ``` -------------------------------- ### Get Bit Chunks Iterator Source: https://docs.rs/arrow/latest/arrow/buffer/struct.Buffer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Returns a `BitChunks` iterator for processing bits in larger chunks, starting at arbitrary bit offsets. Both `offset` and `length` are measured in bits. ```rust pub fn bit_chunks(&self, offset: usize, len: usize) -> BitChunks<'_> ``` -------------------------------- ### GenericByteViewArray::lengths Source: https://docs.rs/arrow/latest/arrow/array/struct.GenericByteViewArray.html Returns an iterator over the length of each array element. Null values are represented by their underlying byte length, not zero. An example is provided to show how to get zero for nulls. ```APIDOC ## pub fn lengths(&self) -> impl ExactSizeIterator + Clone Return an iterator over the length of each array element, including null values. Null values length would equal to the underlying bytes length and NOT 0 Example of getting 0 for null values ``` use arrow_data::ByteView; fn lengths_with_zero_for_nulls(view: &StringViewArray) -> impl Iterator { view.lengths() .enumerate() .map(|(index, length)| if view.is_null(index) { 0 } else { length }) } ``` ``` -------------------------------- ### BinaryRunBuilder Usage Example Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryRunBuilder.html?search=u32+-%3E+bool Demonstrates how to create, append values to, and finish a BinaryRunBuilder to create a RunArray. ```APIDOC ## BinaryRunBuilder Builder for `RunArray` of `BinaryArray`. ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded data is binary values. let mut builder = BinaryRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value(b"abc"); builder.append_null(); builder.extend([Some(b"def"), Some(b"def"), Some(b"abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &BinaryArray = av.as_binary(); assert_eq!(ava.value(0), b"abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), b"def"); assert_eq!(ava.value(3), b"abc"); ``` ``` -------------------------------- ### fn take(self, limit: u64) -> Take Source: https://docs.rs/arrow/latest/arrow/array/array/type.ArrayRef.html Creates an adapter which will read at most `limit` bytes from it. ```APIDOC ## fn take(self, limit: u64) -> Take ### Description Creates an adapter which will read at most `limit` bytes from it. ### Method `take` ``` -------------------------------- ### GenericListArray Slicing Example Source: https://docs.rs/arrow/latest/arrow/array/array/struct.GenericListArray.html?search= Illustrates the effect of slicing a ListArray. Note that the Values array remains unchanged, and Offsets do not start at 0 or cover all values in the Values array after slicing. ```plaintext ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ─ ─ ┐ │ ╔═══╗ │ ╔═══╗ ╔═══╗ ║ ║ Not used │ ║ 1 ║ ║ A ║ │ 0 │ ╚═══╝ ┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ╠═══╣ ╠═══╣ │ [] (empty) │ │ (3,3) │ │ 1 │ │ 3 │ │ ║ 1 ║ ║ B ║ │ 1 │ ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ╠═══╣ ╠═══╣ │ NULL │ │ (3,3) │ │ 0 │ │ 3 │ │ ║ 1 ║ ║ C ║ │ 2 │ ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ╚═══╝ ╚═══╝ │ [D] │ │ (3,4) │ │ 1 │ │ 3 │ │ │ 1 │ │ D │ │ 3 │ └─────────────┘ └───────┘ │ └───┘ ├───┤ ╔═══╗ ╔═══╗ │ 4 │ │ ║ 0 ║ ║ ? ║ │ 4 │ │ └───┘ ╠═══╣ ╠═══╣ │ ║ 1 ║ ║ F ║ │ 5 │ │ Validity ╚═══╝ ╚═══╝ Logical Logical (nulls) Offsets │ Values │ │ Values Offsets │ (Array) └ ─ ─ ─ ─ ─ ─ ┘ │ (offsets[i], offsets[i+1]) │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ``` -------------------------------- ### Getting Raw Pointer Range with `as_ptr_range` Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.OffsetBufferBuilder.html?search= Returns a `Range` of two raw pointers representing the start and one-past-the-end of the slice. Useful for C-style APIs and checking pointer containment within the slice. ```rust let a = [1, 2, 3]; let x = &a[1] as *const _; let y = &5 as *const _; assert!(a.as_ptr_range().contains(&x)); assert!(!a.as_ptr_range().contains(&y)); ``` -------------------------------- ### Example: Using collect Source: https://docs.rs/arrow/latest/arrow/array/type.UInt16DictionaryArray.html?search= Demonstrates how to create a UInt16DictionaryArray using the collect method and verifies its keys and values. ```APIDOC ## Example: Using `collect` ```rust let array: UInt16DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &UInt16Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` ``` -------------------------------- ### Get Element Offset in Slice Source: https://docs.rs/arrow/latest/arrow/buffer/struct.MutableBuffer.html Finds the index of a given element reference within a slice. Returns None if the element reference does not point to the start of an element. This method uses pointer arithmetic and does not compare elements. ```rust let nums: &[u32] = &[1, 7, 1, 1]; let num = &nums[2]; assert_eq!(num, &1); assert_eq!(nums.element_offset(num), Some(2)); ``` -------------------------------- ### Construction Examples Source: https://docs.rs/arrow/latest/arrow/array/array/type.Int64Array.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates various ways to construct an Int64Array, from Vec, iterators, and collecting iterators. ```APIDOC ## Construction ```rust // Create from Vec> let arr = Int64Array::from(vec![Some(1), None, Some(2)]); // Create from Vec let arr = Int64Array::from(vec![1, 2, 3]); // Create iter/collect let arr: Int64Array = std::iter::repeat(42).take(10).collect(); ``` ``` -------------------------------- ### Get Bit Slice Source: https://docs.rs/arrow/latest/arrow/buffer/struct.Buffer.html Returns a `Buffer` slice starting at a specific bit `offset` with a given `len` (in bits). If the offset is byte-aligned, it's a shallow clone; otherwise, a new buffer is allocated and filled with the specified bits. ```rust pub fn bit_slice(&self, offset: usize, len: usize) -> Buffer Returns a slice of this buffer starting at a certain bit offset. If the offset is byte-aligned the returned buffer is a shallow clone, otherwise a new buffer is allocated and filled with a copy of the bits in the range. ``` -------------------------------- ### Example: Creating and Using RecordBatchIterator Source: https://docs.rs/arrow/latest/arrow/array/struct.RecordBatchIterator.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates how to create a RecordBatchIterator from a vector of RecordBatches and a schema. It shows how to verify the schema and iterate through the batches. ```rust let a: ArrayRef = Arc::new(Int32Array::from(vec![1, 2])); let b: ArrayRef = Arc::new(StringArray::from(vec!["a", "b"])); let record_batch = RecordBatch::try_from_iter(vec![ ("a", a), ("b", b), ]).unwrap(); let batches: Vec = vec![record_batch.clone(), record_batch.clone()]; let mut reader = RecordBatchIterator::new(batches.into_iter().map(Ok), record_batch.schema()); assert_eq!(reader.schema(), record_batch.schema()); assert_eq!(reader.next().unwrap().unwrap(), record_batch); ``` -------------------------------- ### Iterate subslices from the end, split by predicate Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.OffsetBufferBuilder.html?search= Use `rsplit` to get an iterator over subslices separated by elements matching a predicate, starting from the end. The matched element is excluded. Handles leading/trailing matches by producing empty slices. ```rust let slice = [11, 22, 33, 0, 44, 55]; let mut iter = slice.rsplit(|num| *num == 0); assert_eq!(iter.next().unwrap(), &[44, 55]); assert_eq!(iter.next().unwrap(), &[11, 22, 33]); assert_eq!(iter.next(), None); ``` ```rust let v = &[0, 1, 1, 2, 3, 5, 8]; let mut it = v.rsplit(|n| *n % 2 == 0); assert_eq!(it.next().unwrap(), &[]); assert_eq!(it.next().unwrap(), &[3, 5]); assert_eq!(it.next().unwrap(), &[1, 1]); assert_eq!(it.next().unwrap(), &[]); assert_eq!(it.next(), None); ``` -------------------------------- ### take Source: https://docs.rs/arrow/latest/arrow/array/type.DynComparator.html?search=u32+-%3E+bool Creates an adapter that reads at most a specified number of bytes from Box. ```APIDOC ## fn take ### Description Creates an adapter which will read at most `limit` bytes from it. ### Parameters - `limit`: `u64` - The maximum number of bytes to read. ### Returns - `Take` - A `Read` adapter that limits the number of bytes read. ``` -------------------------------- ### Split Mutable Slice into N Parts from End Source: https://docs.rs/arrow/latest/arrow/buffer/struct.MutableBuffer.html Use `rsplitn_mut` to get an iterator over mutable subslices separated by elements matching a predicate, limited to `n` items, starting from the end. The matched element is not included. The last subslice contains the remainder. ```rust let mut s = [10, 40, 30, 20, 60, 50]; for group in s.rsplitn_mut(2, |num| *num % 3 == 0) { group[0] = 1; } assert_eq!(s, [1, 40, 30, 20, 60, 1]); ``` -------------------------------- ### IntervalMonthDayNanoArray Example Source: https://docs.rs/arrow/latest/arrow/array/type.IntervalMonthDayNanoArray.html?search=std%3A%3Avec Example demonstrating the creation and usage of IntervalMonthDayNanoArray. ```APIDOC ## §Example ```rust use arrow_array::types::IntervalMonthDayNano; let array = IntervalMonthDayNanoArray::from(vec![ IntervalMonthDayNano::new(1, 2, 1000), // 1 month, 2 days, 1 nanosecond IntervalMonthDayNano::new(12, 1, 0), // 12 months, 1 days, 0 nanoseconds IntervalMonthDayNano::new(0, 0, 12 * 1000 * 1000), // 0 days, 12 milliseconds ]); ``` ``` -------------------------------- ### BinaryRunBuilder Usage Example Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryRunBuilder.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates creating a BinaryRunBuilder, appending binary values and nulls, and finishing the array. It also shows how to access and assert the contents of the resulting BinaryArray. ```rust pub type BinaryRunBuilder = GenericByteRunBuilder>; ``` ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded data is binary values. let mut builder = BinaryRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value(b"abc"); builder.append_null(); builder.extend([Some(b"def"), Some(b"def"), Some(b"abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &BinaryArray = av.as_binary(); assert_eq!(ava.value(0), b"abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), b"def"); assert_eq!(ava.value(3), b"abc"); ``` -------------------------------- ### StringRunBuilder Usage Example Source: https://docs.rs/arrow/latest/arrow/array/type.StringRunBuilder.html?search=u32+-%3E+bool Demonstrates how to create, populate, and finish a StringRunBuilder to produce a RunArray of Strings. ```APIDOC ## StringRunBuilder Builder for `RunArray` of `StringArray` ### Example ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded values are Strings. use arrow::array::StringRunBuilder; use arrow::datatypes::Int16Type; use arrow::array::StringArray; let mut builder = StringRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value("abc"); builder.append_null(); builder.extend([Some("def"), Some("def"), Some("abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &StringArray = av.as_string::(); assert_eq!(ava.value(0), "abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), "def"); assert_eq!(ava.value(3), "abc"); ``` ``` -------------------------------- ### Example: Using collect Source: https://docs.rs/arrow/latest/arrow/array/array/type.UInt8DictionaryArray.html?search= Demonstrates how to create a UInt8DictionaryArray using the `collect` method and verifies its contents. ```APIDOC ## Example: Using `collect` ```rust let array: UInt8DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &UInt8Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` See `DictionaryArray` for more information and examples ``` -------------------------------- ### Construction Examples Source: https://docs.rs/arrow/latest/arrow/array/array/type.Int32Array.html Demonstrates various ways to construct an Int32Array, including from Vec>, Vec, and by collecting from an iterator. ```APIDOC ## Construction ```rust // Create from Vec> let arr = Int32Array::from(vec![Some(1), None, Some(2)]); // Create from Vec let arr = Int32Array::from(vec![1, 2, 3]); // Create iter/collect let arr: Int32Array = std::iter::repeat(42).take(10).collect(); ``` ``` -------------------------------- ### and_not Example Source: https://docs.rs/arrow/latest/arrow/compute/fn.and_not.html?search= Example demonstrating the usage of the `and_not` function with `BooleanArray`. ```APIDOC ## Example ```rust let a = BooleanArray::from(vec![Some(false), Some(true), None]); let b = BooleanArray::from(vec![Some(true), Some(true), Some(false)]); let andn_ab = and_not(&a, &b).unwrap(); assert_eq!(andn_ab, BooleanArray::from(vec![Some(false), Some(false), None])); // It's equal to and(left, not(right)) assert_eq!(andn_ab, and(&a, ¬(&b).unwrap()).unwrap()); ``` ``` -------------------------------- ### StringRunBuilder Usage Example Source: https://docs.rs/arrow/latest/arrow/array/builder/type.StringRunBuilder.html?search=std%3A%3Avec Demonstrates how to create, populate, and finish a StringRunBuilder to produce a RunArray. ```APIDOC ## StringRunBuilder ### Description Builder for `RunArray` of `StringArray`. ### Example ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded values are Strings. use arrow::array::builder::StringRunBuilder; use arrow::datatypes::Int16Type; let mut builder = StringRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value("abc"); builder.append_null(); builder.extend([Some("def"), Some("def"), Some("abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &arrow::array::StringArray = av.as_string::(); assert_eq!(ava.value(0), "abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), "def"); assert_eq!(ava.value(3), "abc"); ``` ``` -------------------------------- ### SortOptions Creation and Manipulation Source: https://docs.rs/arrow/latest/arrow/compute/struct.SortOptions.html?search=std%3A%3Avec Examples demonstrating how to create and manipulate SortOptions using explicit initialization, builder patterns, and the `!` operator. ```APIDOC ## §Example creation ```rust // configure using explicit initialization let options = SortOptions { descending: false, nulls_first: true, }; // Default is ASC NULLs First assert_eq!(options, SortOptions::default()); assert_eq!(options.to_string(), "ASC NULLS FIRST"); // Configure using builder APIs let options = SortOptions::default() .desc() .nulls_first(); assert_eq!(options.to_string(), "DESC NULLS FIRST"); // configure using explicit field values let options = SortOptions::default() .with_descending(false) .with_nulls_first(false); assert_eq!(options.to_string(), "ASC NULLS LAST"); ``` ## §Example operations It is also possible to negate the sort options using the `!` operator. ```rust use arrow_schema::SortOptions; let options = !SortOptions::default(); assert_eq!(options.to_string(), "DESC NULLS LAST"); ``` ``` -------------------------------- ### PrimitiveRunBuilder Search Examples Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.PrimitiveRunBuilder.html?search= Provides example searches that can be performed with PrimitiveRunBuilder. ```APIDOC ## PrimitiveRunBuilder Search ### Description This section provides example searches that can be used with the PrimitiveRunBuilder. ### Example Searches * `std::vec` * `u32 -> bool` * `Option, (T -> U) -> Option` ``` -------------------------------- ### Example: Using collect Source: https://docs.rs/arrow/latest/arrow/array/array/type.Int8DictionaryArray.html?search=u32+-%3E+bool Demonstrates how to create an Int8DictionaryArray using the collect method and verifies its keys and values. ```APIDOC ## Example: Using `collect` ```rust let array: Int8DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &Int8Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` ``` -------------------------------- ### NullBuilder Example Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.NullBuilder.html?search= An example demonstrating how to use NullBuilder to create a NullArray. ```APIDOC ## Example Create a `NullArray` from a `NullBuilder` ```rust let mut b = NullBuilder::new(); b.append_empty_value(); b.append_null(); b.append_nulls(3); b.append_empty_values(3); let arr = b.finish(); assert_eq!(8, arr.len()); assert_eq!(0, arr.null_count()); ``` ``` -------------------------------- ### Example: Using collect Source: https://docs.rs/arrow/latest/arrow/array/array/type.UInt8DictionaryArray.html Demonstrates how to create a UInt8DictionaryArray using the `collect` method. ```APIDOC ## §Example: Using `collect` ```rust let array: UInt8DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &UInt8Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` ``` -------------------------------- ### Example Usage of BinaryViewBuilder Source: https://docs.rs/arrow/latest/arrow/array/builder/type.BinaryViewBuilder.html Demonstrates how to create a BinaryViewBuilder, append string values and nulls, and finish the array. It then asserts the correctness of the built array against expected values. ```rust use arrow_array::BinaryViewArray; let mut builder = BinaryViewBuilder::new(); builder.append_value("hello"); builder.append_null(); builder.append_value("world"); let array = builder.finish(); let expected: Vec> = vec![Some(b"hello"), None, Some(b"world")]; let actual: Vec<_> = array.iter().collect(); assert_eq!(expected, actual); ``` -------------------------------- ### GenericByteRunBuilder Search Examples Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.GenericByteRunBuilder.html?search= Illustrates example searches that can be performed with GenericByteRunBuilder. ```APIDOC ## Search Examples Example searches: * std::vec * u32 -> bool * Option, (T -> U) -> Option ``` -------------------------------- ### type_id - Get Type ID Source: https://docs.rs/arrow/latest/arrow/array/types/struct.IntervalDayTime.html?search=std%3A%3Avec Gets the TypeId of the IntervalDayTime instance. ```APIDOC ## fn type_id(&self) -> TypeId ### Description Gets the `TypeId` of `self`. ### Method `type_id` ### Parameters - **&self**: A reference to the IntervalDayTime instance. ``` -------------------------------- ### Example: Using collect Source: https://docs.rs/arrow/latest/arrow/array/array/type.UInt32DictionaryArray.html?search=std%3A%3Avec Demonstrates how to create a UInt32DictionaryArray using the collect method. ```APIDOC ## §Example: Using `collect` ```rust let array: UInt32DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &UInt32Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` ``` -------------------------------- ### SortOptions Operations Example Source: https://docs.rs/arrow/latest/arrow/compute/struct.SortOptions.html Example showing how to negate SortOptions using the '!' operator. ```APIDOC ## §Example operations It is also possible to negate the sort options using the `!` operator. ```rust use arrow_schema::SortOptions; let options = !SortOptions::default(); assert_eq!(options.to_string(), "DESC NULLS LAST"); ``` ``` -------------------------------- ### Partitioning a RecordBatch Source: https://docs.rs/arrow/latest/arrow/compute/fn.partition.html?search= Demonstrates how to partition a RecordBatch based on the first two columns and verifies the resulting ranges. ```rust let batch = RecordBatch::try_from_iter(vec![ ("x", Arc::new(Int64Array::from(vec![1, 1, 1, 1, 2, 3])) as ArrayRef), ("y", Arc::new(Int64Array::from(vec![1, 2, 2, 2, 1, 1])) as ArrayRef), ("z", Arc::new(StringArray::from(vec!["A", "B", "C", "D", "E", "F"])) as ArrayRef), ]).unwrap(); // Partition on first two columns let ranges = partition(&batch.columns()[..2]).unwrap().ranges(); let expected = vec![ (0..1), (1..4), (4..5), (5..6), ]; assert_eq!(ranges, expected); ``` -------------------------------- ### StringRunBuilder Usage Example Source: https://docs.rs/arrow/latest/arrow/array/builder/type.StringRunBuilder.html?search= Demonstrates how to create, populate, and finish a StringRunBuilder to produce a RunArray of StringArray. ```APIDOC ## StringRunBuilder ### Description Builder for `RunArray` of `StringArray`. ### Example ```rust // Create a run-end encoded array with run-end indexes data type as `i16`. // The encoded values are Strings. use arrow::array::builder::StringRunBuilder; use arrow::datatypes::Int16Type; let mut builder = StringRunBuilder::::new(); // The builder builds the dictionary value by value builder.append_value("abc"); builder.append_null(); builder.extend([Some("def"), Some("def"), Some("abc")]); let array = builder.finish(); assert_eq!(array.run_ends().values(), &[1, 2, 4, 5]); // Values are polymorphic and so require a downcast. let av = array.values(); let ava: &StringArray = av.as_string::(); assert_eq!(ava.value(0), "abc"); assert!(av.is_null(1)); assert_eq!(ava.value(2), "def"); assert_eq!(ava.value(3), "abc"); ``` ``` -------------------------------- ### Example: Using `collect` Source: https://docs.rs/arrow/latest/arrow/array/array/type.UInt8DictionaryArray.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Demonstrates how to create a UInt8DictionaryArray by collecting an iterator of string slices. ```APIDOC ## §Example: Using `collect` ``` let array: UInt8DictionaryArray = vec!["a", "a", "b", "c"].into_iter().collect(); let values: Arc = Arc::new(StringArray::from(vec!["a", "b", "c"])); assert_eq!(array.keys(), &UInt8Array::from(vec![0, 0, 1, 2])); assert_eq!(array.values(), &values); ``` ``` -------------------------------- ### PrimitiveDictionaryBuilder Search Examples Source: https://docs.rs/arrow/latest/arrow/array/builder/struct.PrimitiveDictionaryBuilder.html?search= Provides examples of search queries that can be performed on PrimitiveDictionaryBuilder. ```APIDOC ## Search Examples This section provides example searches that can be performed. ### Example Searches * `std::vec` * `u32 -> bool` * `Option, (T -> U) -> Option` ``` -------------------------------- ### Construction Examples Source: https://docs.rs/arrow/latest/arrow/array/array/type.Decimal128Array.html?search= Demonstrates various ways to construct a Decimal128Array. ```APIDOC ## Construction ```rust // Create from Vec> let arr = Decimal128Array::from(vec![Some(1), None, Some(2)]); // Create from Vec let arr = Decimal128Array::from(vec![1, 2, 3]); // Create iter/collect let arr: Decimal128Array = std::iter::repeat(42).take(10).collect(); ``` ```