Bigtools

https://docs.rs/bigtools/0.5.6/
Admin
Bigtools is a high-performance Rust library that provides a high-level API for reading and writing...
Tokens:161,305
Snippets:777
Trust Score:9.7
Update:11 months ago
Show doc for...
Tokens:
Raw
### Create bigWig file for writing in Rust

Source: https://docs.rs/bigtools/0.5.6/bigtools/index

Provides an example of creating a new bigWig file for writing using `BigWigWrite::create_file`. This function initializes the necessary structures for writing bigWig data, typically requiring an async runtime for efficient I/O operations.

```Rust
use bigtools::BigWigWrite;

// Assuming 'writer' is a type that implements Write and Seek
// let mut bigwig_writer = BigWigWrite::create_file(writer).expect("Failed to create bigWig file");

// For file paths, use BigWigWrite::create_file_path("path/to/new/file.bw").expect(...);
```

--------------------------------

### Rust TempFileBufferWriter State Transition Example

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/tempfilebuffer

Illustrates a state transition within the `update` method of `TempFileBufferWriter`. This specific part shows how data from an in-memory buffer is written to a newly provided real file, changing the buffer state to `BufferState::Real`.

```rust
BufferState::InMemory(data) => {
    let real_file = self.real_file.swap(None).take();
    if let Some(mut new_file) = real_file {
        new_file.write_all(&data)?;
        self.buffer_state = BufferState::Real(new_file);
    }
}
```

--------------------------------

### Write Chromosome Tree and R-Tree Index

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bbiwrite

This function writes the chromosome tree and R-tree index to a file. It first determines the start of the chromosome index, writes the chromosome tree, then determines the start of the R-tree index and writes it. It returns the data size, the start offset of the chromosome index, the start offset of the R-tree index, and the total number of sections.

```rust
    let chrom_index_start = file.tell()?;
    write_chrom_tree(file, chrom_sizes, &chrom_ids)?;

    let index_start = file.tell()?;
    let (nodes, levels, total_sections) = get_rtreeindex(sections_iter, &options);
    write_rtreeindex(file, nodes, levels, total_sections, &options)?;

    Ok((data_size, chrom_index_start, index_start, total_sections))
}
```

--------------------------------

### Create and Write BigWig File Example (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigwrite

Demonstrates how to create a bigWig file from bedgraph data. It involves setting up input data using BedParserStreamingIterator, defining chromosome sizes, initializing a Tokio runtime, and then using BigWigWrite to write the data.

```rust
# use std::collections::HashMap;
# use std::error::Error;
# use std::path::PathBuf;
# use std::fs::File;
# use bigtools::BigWigWrite;
# use bigtools::beddata::BedParserStreamingIterator;
# fn main() -> Result<(), Box<dyn Error>> {
# let mut dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
# dir.push("resources/test");
# let mut bedgraph_in = dir.clone();
# bedgraph_in.push("single_chrom.bedGraph");
// First, set up our input data. Here, we're using the `BedParserStreamingIterator`.
let bedgraph_file: File = File::open(bedgraph_in)?;
let vals = BedParserStreamingIterator::from_bedgraph_file(bedgraph_file, false);

// Then, we need to know what the chromosome sizes are. This can be read in from a file, but here we
// just construct a map for ease.
let mut chrom_map = HashMap::new();
chrom_map.insert("chr17".to_string(), 83257441);

// We also need a `Runtime` to spawn processing on.
let runtime = tokio::runtime::Builder::new_multi_thread()
    .worker_threads(6)
    .build()
    .expect("Unable to create runtime.");

// Finally, we can create a `BigWigWrite` with a file to write to. We'll use a temporary file.
let tempfile = tempfile::NamedTempFile::new()?;
let out = BigWigWrite::create_file(tempfile.path(), chrom_map).unwrap();
// Then write.
out.write(vals, runtime)?;
# Ok(())
# }
```

--------------------------------

### Rust TempFileBufferWriter Temporary File Copy Example

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/tempfilebuffer

Shows the logic for copying data from a temporary file buffer to a real file when a `switch` operation occurs. This involves seeking to the beginning of the temporary file and using `io::copy` for efficient transfer.

```rust
BufferState::Temp(ref mut file) => {
    let real_file = self.real_file.swap(None).take();
    if let Some(mut new_file) = real_file {
        file.seek(io::SeekFrom::Start(0))?;

        io::copy(file, &mut new_file)?;
        self.buffer_state = BufferState::Real(new_file);
    }
}
```

--------------------------------

### Encode Zoom Section with Compression (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bbiwrite

Encodes a section of zoom records into binary data, with an option for compression using the `libdeflater` crate. It calculates the start and end points of the section and writes chromosome, start, end, bases covered, and summary statistics for each item.

```rust
pub(crate) async fn encode_zoom_section(
    compress: bool,
    items_in_section: Vec<ZoomRecord>,
) -> io::Result<(SectionData, usize)> {
    use libdeflater::{CompressionLvl, Compressor};

    let mut bytes = Vec::with_capacity(items_in_section.len() * 32);

    let start = items_in_section[0].start;
    let end = items_in_section[items_in_section.len() - 1].end;

    let chrom = items_in_section[0].chrom;
    for item in items_in_section.iter() {
        bytes.write_u32::<NativeEndian>(item.chrom)?;
        bytes.write_u32::<NativeEndian>(item.start)?;
        bytes.write_u32::<NativeEndian>(item.end)?;
        bytes.write_u32::<NativeEndian>(item.summary.bases_covered as u32)?;
        bytes.write_f32::<NativeEndian>(item.summary.min_val as f32)?;
    }

    if compress {
        let mut compressor = Compressor::new(CompressionLvl::BestSpeed);
        let compressed_bytes = compressor.deflate_exact(&bytes)?;
        Ok((SectionData { bytes: compressed_bytes, start, end, chrom }, bytes.len()))
    } else {
        Ok((SectionData { bytes, start, end, chrom }, bytes.len()))
    }
}
```

--------------------------------

### Setup Chromosome Data Channels in Rust

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bbiwrite

Initializes channels and futures for processing data for a specific chromosome. This function prepares the necessary communication conduits and asynchronous tasks required for handling chromosome-specific data processing, including temporary storage and future results.

```rust
fn setup_chrom<
    W: Write + Seek + Send + 'static,
    BBIWriteOptions: Clone,
    Runtime: Send + Sync + 'static,
>( 
    send: &mut tokio::sync::mpsc::UnboundedSender<(
        tokio::sync::mpsc::Receiver<BBIDataProcessoringInputSectionChannel>,
        TempFileBuffer<BufWriter<W>>,
        tokio::task::JoinHandle<Result<(usize, usize), ProcessDataError>>,
        Vec<TempZoomInfo>,
    )>,
    options: &BBIWriteOptions,
    runtime: &Runtime,
    zoom_sizes: &Vec<u32>,
) -> (
    Vec<(u32, BBIDataProcessoringInputSectionChannel)>,
    BBIDataProcessoringInputSectionChannel,
) {
    let (ftx, sections_handle, buf, section_receiver) = 
        future_channel(options.channel_size, runtime.handle(), options.inmemory);

    let (zoom_infos, zooms_channels) = { 
        let mut zoom_infos = Vec::with_capacity(zoom_sizes.len());
        let mut zooms_channels = Vec::with_capacity(zoom_sizes.len());

        for size in zoom_sizes {
            let (ftx, data_write_future, buf, section_receiver) = 
                future_channel(options.channel_size, runtime.handle(), options.inmemory);
            let zoom_info = TempZoomInfo {
                resolution: *size,
                data_write_future,
                data: buf,
                sections: section_receiver,
            };
            zoom_infos.push(zoom_info);
            zooms_channels.push((*size, ftx));
        }
        (zoom_infos, zooms_channels)
    };

    send.unbounded_send((section_receiver, buf, sections_handle, zoom_infos))
        .expect("Expected to always send.");

    (zooms_channels, ftx)
}
```

--------------------------------

### RemoteFile Reopen Implementation in Rust

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/remote_file

Implements the `Reopen` trait for `RemoteFile`, providing a method to create a new `RemoteFile` instance pointing to the same URL. This is useful for re-establishing a connection or starting from the beginning of the file.

```rust
impl Reopen for RemoteFile {
    fn reopen(&self) -> io::Result<RemoteFile> {
        Ok(RemoteFile {
            url: self.url.clone(),
            current_position: 0,
            current: None,
            cache: None,
        })
    }
}
```

--------------------------------

### Rust TempFileBufferWriter Internal Logic Example

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/tempfilebuffer

This snippet demonstrates the core logic within the `TempFileBufferWriter`'s `write` method, specifically how it handles different `BufferState` variants to direct the write operation. It ensures that before writing, the buffer state is updated.

```rust
loop {
    match self.buffer_state {
        BufferState::NotStarted => unreachable!(),
        BufferState::InMemory(ref mut data) => return data.write(buf),
        BufferState::Temp(ref mut file) => return file.write(buf),
        BufferState::Real(ref mut file) => return file.write(buf),
    }
}
```

--------------------------------

### Create FileView Instance - Rust

Source: https://docs.rs/bigtools/0.5.6/bigtools/utils/file/file_view/struct

Implements a constructor `new` for the `FileView` struct. This function takes a `File` object, a start offset, and an end offset to create a view into a specific portion of the file. The end offset is truncated to the file's actual end.

```rust
pub fn new(file: File, start: u64, end: u64) -> Result<FileView>
```

--------------------------------

### Implement BigWig Average Over Bed - Rust

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/cli/bigwigaverageoverbed

The main function `bigwigaverageoverbed` orchestrates the process of calculating bigWig statistics over BED regions. It handles argument parsing, file opening, output stream setup, and delegates processing to parallel or sequential execution paths based on the number of threads.

```rust
pub fn bigwigaverageoverbed(
    args: BigWigAverageOverBedArgs,
) -> Result<(), Box<dyn Error + Send + Sync>> {
    let bigwigpath = args.bigwig;
    let bedinpath = args.bedin;
    let bedoutpath = args.output;
    let add_min_max = args.min_max;

    let reopen = ReopenableFile {
        file: File::open(&bigwigpath)?,
        path: bigwigpath.into(),
    };
    let mut inbigwig = BigWigRead::open(reopen)?.cached();

    let outbed = File::create(bedoutpath)?;
    let mut bedoutwriter: BufWriter<File> = BufWriter::new(outbed);

    let name = match args.namecol.as_deref() {
        Some("interval") => Name::Interval,
        Some("none") => Name::None,
        Some(col) => {
            let col = col.parse::<usize>();
            match col {
                Ok(col) if col > 0 => Name::Column(col - 1),
                Ok(_) => return Err(io::Error::new(
                    io::ErrorKind::InvalidData,
                    "Invalid name column option. Column values are one-indexed, so should not be zero.",
                ).into()),
                Err(_) => return Err(io::Error::new(
                    io::ErrorKind::InvalidData,
                    "Invalid name column option. Allowed options are `interval`, `none`, or an one-indexed integer value for a given column.",
                ).into()),
            }
        }
        None => Name::Column(3),
    };

    let nthreads: usize = args.nthreads;
    let parallel = nthreads > 1;

    if parallel {
        let chunks = split_file_into_chunks_by_size(File::open(&bedinpath)?, nthreads as u64)?;

        fn process_chunk<R: Reopen + BBIFileRead>(
            start: u64,
            end: u64,
            bedinpath: String,
            name: Name,
            add_min_max: bool,
            inbigwig: &mut BigWigRead<R>,
        ) -> Result<File, Box<dyn Error + Send + Sync>> {
            let mut tmp = tempfile::tempfile()?;

            let chrom_bed_file = File::open(bedinpath)?;
            let chrom_bed_file = FileView::new(chrom_bed_file, start, end)?;
            let mut bed_stream = BedFileStream {
                bed: StreamingLineReader::new(BufReader::new(chrom_bed_file)),
                parse: parse_bed,
            };

            loop {
                let (chrom, entry) = match bed_stream.next() {
                    None => break,
                    Some(Err(e)) => {
                        return Err(e.into());
                    }

```

--------------------------------

### Open and Read Remote BigWig Intervals (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/remote_file

Demonstrates opening a remote BigWig file and retrieving interval data. It uses the `RemoteFile` and `BigWigRead` structs. This example fetches data for a specific interval on 'chr17' at a zoom level of 2048.

```rust
    #[ignore]
    #[test]
    fn test_remote2() {
        let f = RemoteFile::new("http://hgdownload.soe.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeMapability/wgEncodeCrgMapabilityAlign100mer.bigWig");
        let mut remote = BigWigRead::open(f).unwrap();

        let interval = remote.get_zoom_interval("chr17", 0, 36996442, 2048);
        let _: Vec<_> = interval.unwrap().collect();
    }
```

--------------------------------

### Get Interval (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Retrieves an iterator for BigWig values intersecting a specified genomic interval. This method takes ownership of the `BigWigRead` object. It requires the chromosome name, start, and end positions as input.

```rust
pub fn get_interval_move(
        mut self,
        chrom_name: &str,
        start: u32,
        end: u32,
    ) -> Result<BigWigIntervalIter<R, BigWigRead<R>>, BBIReadError> {
        let chrom = self.info.chrom_id(chrom_name)?;
        let cir_tree = self.full_data_cir_tree()?;
        let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
        Ok(BigWigIntervalIter {
            r: std::marker::PhantomData,
            bigwig: self,
            known_offset: 0,
            blocks: blocks.into_iter(),
            vals: None,
            chrom,
            start,
            end,
        })
    }
```

--------------------------------

### Create bigBed file for writing in Rust

Source: https://docs.rs/bigtools/0.5.6/bigtools/index

Shows the process of creating a new bigBed file for writing using `BigBedWrite::create_file`. This sets up the writer for bigBed data, and often involves using an async runtime for handling compression and I/O.

```Rust
use bigtools::BigBedWrite;

// Assuming 'writer' is a type that implements Write and Seek
// let mut bigbed_writer = BigBedWrite::create_file(writer).expect("Failed to create bigBed file");

// For file paths, use BigBedWrite::create_file_path("path/to/new/file.bb").expect(...);
```

--------------------------------

### Get Values (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Extracts BigWig values between specified start and end positions for a given chromosome. Positions without data are represented as `std::f32::NAN`. This method requires mutable access to the `BigWigRead` object.

```rust
pub fn values(
        &mut self,
        chrom_name: &str,
        start: u32,
        end: u32,
    ) -> Result<Vec<f32>, BBIReadError> {
        let chrom = self.info.chrom_id(chrom_name)?;
        let cir_tree = self.full_data_cir_tree()?;
        let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
        let mut values = vec![std::f32::NAN; (end - start) as usize];
        use crate::utils::tell::Tell;
        let mut known_offset = self.reader().raw_reader().tell()?;
        for block in blocks {
            let block_values = get_block_values(self, block, &mut known_offset, chrom, start, end)?;
            let block_values = match block_values {
                Some(v) => v,
                None => continue,
            };
            for block_value in block_values {
                let block_value_start = (block_value.start - start) as usize;
                let block_value_end = (block_value.end - start) as usize;
                for i in &mut values[block_value_start..block_value_end] {
                    *i = block_value.value
                }
            }
        }
        Ok(values)
    }
```

--------------------------------

### Get Intersecting Intervals from BigBed

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedread

Returns an iterator for `BedEntry`s that intersect with a specified chromosomal range (chromosome name, start, and end). This method requires a mutable reference to `BigBedRead` and performs searches within the CIR tree.

```Rust
impl<R: BBIFileRead> BigBedRead<R> {
    /// For a given chromosome, start, and end, returns an `Iterator` of the
    /// intersecting `BedEntry`s. The resulting iterator takes a mutable reference
    /// of this `BigBedRead`.
    pub fn get_interval<'a>(
        &'a mut self,
        chrom_name: &str,
        start: u32,
        end: u32,
    ) -> Result<BigBedIntervalIter<R, &'a mut BigBedRead<R>>, BBIReadError> {
        let cir_tree = self.full_data_cir_tree()?;
        let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
        // TODO: this is only for asserting that the chrom is what we expect
        let chrom_ix = self
            .info()
            .chrom_info
            .iter()
            // ... rest of the function would follow
```

--------------------------------

### Initialize BigBedWrite with a File Path

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedwrite

Creates a new `BigBedWrite` instance by creating a file at the specified path. It requires a path and a HashMap of chromosome sizes. This is a convenience constructor for direct file output.

```rust
impl BigBedWrite<File> {
    pub fn create_file(
        path: impl AsRef<Path>,
        chrom_sizes: HashMap<String, u32>,
    ) -> io::Result<Self> {
        let out = File::create(path)?;
        Ok(BigBedWrite::new(out, chrom_sizes))
    }
}
```

--------------------------------

### Get Interval Move (BigBedRead)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedread

Retrieves an iterator of BigBed entries that intersect a given genomic interval. This method consumes the BigBedRead instance and requires chromosome name, start, and end positions. It is similar to `get_interval` but takes ownership.

```rust
pub fn get_interval_move(
    mut self,
    chrom_name: &str,
    start: u32,
    end: u32,
) -> Result<BigBedIntervalIter<R, BigBedRead<R>>, BBIReadError> {
    let cir_tree = self.full_data_cir_tree()?;
    let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
    // TODO: this is only for asserting that the chrom is what we expect
    let chrom_ix = self
        .info()
        .chrom_info
        .iter()
        .find(|&x| x.name == chrom_name)
        .unwrap()
        .id;
    Ok(BigBedIntervalIter {
        r: std::marker::PhantomData,
        bigbed: self,
        known_offset: 0,
        blocks: blocks.into_iter(),
        vals: None,
        expected_chrom: chrom_ix,
        start,
        end,
    })
}
```

--------------------------------

### BigWigWrite General Constructor and Pre-write Logic (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigwrite

Provides the general `new` constructor for `BigWigWrite` and the `write_pre` function. `write_pre` writes blank headers and placeholders for summary and data offsets, returning the file position after these initial writes.

```rust
impl<W: Write + Seek + Send + 'static> BigWigWrite<W> {
    pub fn new(out: W, chrom_sizes: HashMap<String, u32>) -> Self {
        BigWigWrite {
            out,
            chrom_sizes,
            options: BBIWriteOptions::default(),
        }
    }

    fn write_pre(file: &mut BufWriter<W>) -> Result<(u64, u64, u64), ProcessDataError> {
        write_blank_headers(file)?;

        let total_summary_offset = file.tell()?;
        file.write_all(&[0; 40])?;

        let full_data_offset = file.tell()?;

        // Total items
        // Unless we know the vals ahead of time, we can't estimate total sections ahead of time.
        // Even then simply doing "(vals.len() as u32 + ITEMS_PER_SLOT - 1) / ITEMS_PER_SLOT"
        // underestimates because sections are split by chrom too, not just size.
        // Skip for now, and come back when we write real header + summary.
        file.write_u64::<NativeEndian>(0)?;

        let pre_data = file.tell()?;

        Ok((total_summary_offset, full_data_offset, pre_data))
    }
}
```

--------------------------------

### Get Zoom Interval Move (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Retrieves an iterator for zoom record intervals intersecting a specified genomic range and reduction level, taking ownership of the `BigWigRead` object. It requires chromosome name, start, end positions, and the reduction level.

```rust
pub fn get_zoom_interval_move<'a>( 
        mut self,
        chrom_name: &str,
        start: u32,
        end: u32,
        reduction_level: u32,
    ) -> Result<ZoomIntervalIter<BigWigRead<R>, BigWigRead<R>>, ZoomIntervalError> {
        let cir_tree = self.zoom_cir_tree(reduction_level)?;
        let chrom = self.info.chrom_id(chrom_name)?;
        let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
        Ok(ZoomIntervalIter::new(
            self,
            blocks.into_iter(),
            chrom,
            start,
            end,
        ))
    }
```

--------------------------------

### Rust StreamingLineReader Implementation and Usage

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/file/streaming_linereader

Implements a StreamingLineReader for efficiently reading lines from a BufRead source. It provides a constructor and a read method. Includes unit tests to demonstrate reading from a file.

```Rust
use std::io::{self, BufRead};

#[derive(Debug)]
pub struct StreamingLineReader<B> {
    current_line: String,
    buf_read: B,
}

impl<B: BufRead> StreamingLineReader<B> {
    pub fn new(bf: B) -> StreamingLineReader<B> {
        StreamingLineReader {
            current_line: String::new(),
            buf_read: bf,
        }
    }

    pub fn read(&mut self) -> Option<io::Result<&'_ str>> {
        self.current_line.clear();
        match self.buf_read.read_line(&mut self.current_line) {
            Ok(size) if size == 0 => None,
            Ok(_) => Some(Ok(self.current_line.trim_end())),
            Err(e) => Some(Err(e)),
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use std::fs::File;
    use std::io::{self, BufReader};
    use std::path::PathBuf;

    #[test]
    fn test_works() -> io::Result<()> {
        let mut dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
        dir.push("resources/test");
        dir.push("small.bedGraph");
        let f = File::open(dir)?;
        let bf = BufReader::new(f);
        let mut slr = StreamingLineReader::new(bf);
        assert_eq!("chr17\t1\t100\t0.5", slr.read().unwrap().unwrap());
        assert_eq!("chr17\t101\t200\t0.5", slr.read().unwrap().unwrap());
        assert_eq!("chr17\t201\t300\t0.5", slr.read().unwrap().unwrap());
        assert_eq!("chr18\t1\t100\t0.5", slr.read().unwrap().unwrap());
        assert_eq!("chr18\t101\t200\t0.5", slr.read().unwrap().unwrap());
        assert_eq!("chr19\t1\t100\t0.5", slr.read().unwrap().unwrap());
        assert!(slr.read().is_none());
        Ok(())
    }
}
```

--------------------------------

### Get Zoom Interval (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Returns an iterator for zoom record intervals intersecting a specified genomic range and reduction level. This method borrows the `BigWigRead` object mutably. It requires the chromosome name, start, end positions, and the reduction level.

```rust
pub fn get_zoom_interval<'a>( 
        &'a mut self,
        chrom_name: &str,
        start: u32,
        end: u32,
        reduction_level: u32,
    ) -> Result<ZoomIntervalIter<BigWigRead<R>, &'a mut BigWigRead<R>>, ZoomIntervalError> {
        let cir_tree = self.zoom_cir_tree(reduction_level)?;
        let chrom = self.info.chrom_id(chrom_name)?;
        let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
        Ok(ZoomIntervalIter::new(
            self,
            blocks.into_iter(),
            chrom,
            start,
            end,
        ))
    }
```

--------------------------------

### Iterate Over BigWig Genomic Intervals in Rust

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Provides a method to get an iterator for genomic intervals within a specified range (chromosome, start, end). This function requires a mutable reference to `BigWigRead` and returns a `Result` containing either a `BigWigIntervalIter` or a `BBIReadError`.

```rust
/// For a given chromosome, start, and end, returns an `Iterator` of the
/// intersecting `Value`s. The resulting iterator takes a mutable reference
/// of this `BigWigRead`.
pub fn get_interval<'a>(
    &'a mut self,
    chrom_name: &str,
    start: u32,
    end: u32,
) -> Result<BigWigIntervalIter<R, &'a mut BigWigRead<R>>, BBIReadError> {
    let chrom = self.info.chrom_id(chrom_name)?;
    let cir_tree = self.full_data_cir_tree()?;
    let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
    Ok(BigWigIntervalIter {
        r: std::marker::PhantomData,
        bigwig: self,
        known_offset: 0,
        blocks: blocks.into_iter(),
        vals: None,
    })
}
```

--------------------------------

### Get Interval (BigBedRead)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedread

Retrieves an iterator of BigBed entries that intersect a given genomic interval. This method takes a mutable reference to BigBedRead and requires chromosome name, start, and end positions. It utilizes a CIR tree for efficient searching.

```rust
pub fn get_interval<'a>(
    &'a mut self,
    chrom_name: &str,
    start: u32,
    end: u32,
) -> Result<BigBedIntervalIter<R, &'a mut BigBedRead<R>>, BBIReadError> {
    let cir_tree = self.full_data_cir_tree()?;
    let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
    // TODO: this is only for asserting that the chrom is what we expect
    let chrom_ix = self
        .info()
        .chrom_info
        .iter()
        .find(|&x| x.name == chrom_name)
        .unwrap()
        .id;
    Ok(BigBedIntervalIter {
        r: std::marker::PhantomData,
        bigbed: self,
        known_offset: 0,
        blocks: blocks.into_iter(),
        vals: None,
        expected_chrom: chrom_ix,
        start,
        end,
    })
}
```

--------------------------------

### Writing BigWig Files in Rust

Source: https://docs.rs/bigtools/0.5.6/index

Shows the process of writing new bigWig files using the `bigtools` crate. It involves creating a `BigWigWrite` instance and using `BigWigWrite::write` with a data source and runtime.

```Rust
use bigtools::{BigWigWrite, BBIDataSource, BBIWriteOptions, BBIProcessError};
use tokio::runtime::Runtime;
use std::collections::HashMap;

fn write_bigwig<S: BBIDataSource>(data_source: S, chrom_sizes: HashMap<String, u32>, runtime: &Runtime) -> Result<(), BBIProcessError> {
    let options = BBIWriteOptions::default();
    let mut writer = BigWigWrite::create_file("output.bw", options)?;

    writer.write(data_source, chrom_sizes, runtime)?;
    Ok(())
}
```

--------------------------------

### Get Zoom Interval Move (BigBedRead)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedread

Retrieves an iterator of zoom records that intersect a given genomic interval and reduction level. This method consumes the BigBedRead instance and requires chromosome name, start, end, and reduction level. It's a consuming version of `get_zoom_interval`.

```rust
pub fn get_zoom_interval_move<'a>(
    mut self,
    chrom_name: &str,
    start: u32,
    end: u32,
    reduction_level: u32,
) -> Result<ZoomIntervalIter<BigBedRead<R>, BigBedRead<R>>, ZoomIntervalError> {
    let cir_tree = self
        .zoom_cir_tree(reduction_level)
        .map_err(|_| ZoomIntervalError::ReductionLevelNotFound)?;

    let chrom = self.info.chrom_id(chrom_name)?;

    let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
    Ok(ZoomIntervalIter::new(
        self,
        blocks.into_iter(),
        chrom,
        start,
        end,
    ))
}
```

--------------------------------

### Open and Read BigWig File - Rust

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Demonstrates how to open a bigWig file using `BigWigRead::open_file`. It then shows how to retrieve chromosome information, summary data (bases covered and maximum value), and read specific interval data. This example requires a valid bigWig file and specifies the necessary imports.

```rust
# use std::error::Error;
# use std::path::PathBuf;
# use bigtools::BigWigRead;
# use bigtools::BBIRead;
# fn main() -> Result<(), Box<dyn Error>> {
# let mut dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
# dir.push("resources/test");
# let mut bigwig = dir.clone();
# bigwig.push("valid.bigWig");
// First, we open a bigWig using a file name.
let mut bwread = BigWigRead::open_file(bigwig)?;

// Then, we could get the chromosomes and lengths
let chroms = bwread.chroms();
assert_eq!(chroms.len(), 1);
assert_eq!(chroms[0].length, 83257441);

// We can get summary data
let summary = bwread.get_summary()?;
assert_eq!(summary.bases_covered, 137894);
assert_eq!(summary.max_val, 14254.0);

// Or we can read data from an interval
let first_interval = bwread
    .get_interval("chr17", 0, 59899)?
    .next()
    .unwrap()
    .unwrap();
assert_eq!(first_interval.start, 59898);
assert_eq!(first_interval.end, 59899);
assert_eq!(first_interval.value, 0.06792);
# Ok(())
# }
```

--------------------------------

### Initialize BigWigZoomsProcess

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigwrite

Initializes the BigWigZoomsProcess struct, preparing it to handle BigWig data with zoom levels. It sets up temporary zoom item storage, zoom item structures, and configuration options.

```rust
impl<W: Write + Seek + Send + 'static> BBIDataProcessorCreate for BigWigZoomsProcess<W> {
    type I = ZoomsInternalProcessData<W>;
    type Out = ZoomsInternalProcessedData<W>;
    fn create(internal_data: Self::I) -> Self {
        let ZoomsInternalProcessData(temp_zoom_items, zooms_channels, chrom_id, options, runtime) =
            internal_data;

        let zoom_items: Vec<ZoomItem> = zooms_channels
            .into_iter()
            .map(|(size, channel)| ZoomItem {
                size,
                live_info: None,
                records: Vec::with_capacity(options.items_per_slot as usize),
                channel,
            })
            .collect();

        BigWigZoomsProcess {
            temp_zoom_items,
            chrom_id,
            options,
            runtime,
            zoom_items,
        }
    }
    fn destroy(self) -> Self::Out {
        let BigWigZoomsProcess { zoom_items, .. } = self;

        for zoom_item in zoom_items.iter() {
            debug_assert!(zoom_item.live_info.is_none());
            debug_assert!(zoom_item.records.is_empty());
        }

        ZoomsInternalProcessedData(self.temp_zoom_items)
    }
}
```

--------------------------------

### Get Zoom Interval (BigBedRead)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedread

Retrieves an iterator of zoom records that intersect a given genomic interval and reduction level. This method takes a mutable reference to BigBedRead and requires chromosome name, start, end, and reduction level. It uses CIR trees for zoom data.

```rust
pub fn get_zoom_interval<'a>(
    &'a mut self,
    chrom_name: &str,
    start: u32,
    end: u32,
    reduction_level: u32,
) -> Result<ZoomIntervalIter<BigBedRead<R>, &'a mut BigBedRead<R>>, ZoomIntervalError> {
    let cir_tree = self
        .zoom_cir_tree(reduction_level)
        .map_err(|_| ZoomIntervalError::ReductionLevelNotFound)?;

    let chrom = self.info.chrom_id(chrom_name)?;

    let blocks = search_cir_tree(&self.info, &mut self.read, cir_tree, chrom_name, start, end)?;
    Ok(ZoomIntervalIter::new(
        self,
        blocks.into_iter(),
        chrom,
        start,
        end,
    ))
}
```

--------------------------------

### Setup Future Channel for Data Processing

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bbiwrite

This function sets up a channel pipeline for processing BigBed data. It creates an in-memory or temporary file buffer, spawns an asynchronous task to write data sections using the `write_data` function, and returns the input channel sender, the join handle for the data writing task, the buffer, and a receiver for processed sections.

```rust
pub(crate) fn future_channel<R: Write + Send + 'static>(
    channel_size: usize,
    runtime: &Handle,
    inmemory: bool,
) -> (
    BBIDataProcessoringInputSectionChannel,
    tokio::task::JoinHandle<Result<(usize, usize), ProcessDataError>>,
    TempFileBuffer<R>,
    crossbeam_channel::Receiver<Section>,
) {
    let (ftx, frx) = channel(channel_size);
    let (buf, write) = TempFileBuffer::new(inmemory);
    let file = BufWriter::new(write);

    let (section_sender, section_receiver) = unbounded();
    let sections_remote = write_data(file, section_sender, frx);
    let sections_handle = runtime.spawn(sections_remote);
    (ftx, sections_handle, buf, section_receiver)
}
```

--------------------------------

### Get Current Position

Source: https://docs.rs/bigtools/0.5.6/bigtools/utils/file/reopen/struct

Gets the current position within a seekable reader.

```APIDOC
## fn tell(&mut self) -> Result<u64, Error>

### Description
Gets the current position of the reader.

### Method
*Implicitly part of a seekable reader interface, specific method not defined.*

### Endpoint
*Not applicable for this function.*

### Parameters
#### Path Parameters
*None*

#### Query Parameters
*None*

#### Request Body
*None*

### Request Example
*Not applicable for this function.*

### Response
#### Success Response (200)
- **position** (u64) - The current position in the reader.

#### Response Example
*None specified.*

```

--------------------------------

### Writing BigBed Files in Rust

Source: https://docs.rs/bigtools/0.5.6/index

Details how to write new bigBed files with the `bigtools` crate. It outlines the use of `BigBedWrite::create_file` and `BigBedWrite::write`, requiring a data source, chromosome sizes, and a runtime.

```Rust
use bigtools::{BigBedWrite, BBIDataSource, BBIWriteOptions, BBIProcessError};
use tokio::runtime::Runtime;
use std::collections::HashMap;

fn write_bigbed<S: BBIDataSource>(data_source: S, chrom_sizes: HashMap<String, u32>, runtime: &Runtime) -> Result<(), BBIProcessError> {
    let options = BBIWriteOptions::default();
    let mut writer = BigBedWrite::create_file("output.bb", options)?;

    writer.write(data_source, chrom_sizes, runtime)?;
    Ok(())
}
```

--------------------------------

### Initialize BigBedWrite with a Generic Writer

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigbedwrite

Creates a new `BigBedWrite` instance with a generic writer that implements `Write`, `Seek`, and `Send`. This allows writing to various output streams beyond just files, such as network sockets or in-memory buffers.

```rust
impl<W: Write + Seek + Send + 'static> BigBedWrite<W> {
    pub fn new(out: W, chrom_sizes: HashMap<String, u32>) -> Self {
        BigBedWrite {
            out,
            chrom_sizes,
            options: BBIWriteOptions::default(),
            autosql: None,
        }
    }
}
```

--------------------------------

### Rust: Function to Fill Gaps and Pad Start/End in Value Iterator

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/fill

This Rust function, `fill_start_to_end`, extends the gap-filling functionality by also padding the start and end of the sequence with zero-value `Value` entries. It takes an iterator of `io::Result<Value>`, a start position, and an end position. If the start is greater than the end of the first value, it's ignored. This ensures the output covers a specific range.

```rust
/// Fills any space between `Value`s with `0.0`s. This will also pad the start and end with `0.0`s if they do not exist.
/// Note: Output values will not be merged if any input Values are `0.0`
///
/// If the start > the end of the first value, it will be ignored.
pub fn fill_start_to_end<I>(
    iter: I,
    start: u32,
    end: u32,
) -> impl Iterator<Item = io::Result<Value>> + Send
where
    I: Iterator<Item = io::Result<Value>> + Send,
{
    FillValues {
        iter,
        last_val: None,
        expected_end: Some(end),
        last_end: start,
    }
}
```

--------------------------------

### Rust BigBed to Bed Conversion with Arguments

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/utils/cli/bigbedtobed

Defines the main function 'bigbedtobed' that parses command-line arguments and orchestrates the conversion process. It handles file opening, argument validation, and calls specific writing functions based on whether overlap BED is provided or if single-threaded operation is required.

```rust
use std::error::Error;
use std::fs::File;
use std::io::{self, BufReader, Write};
use std::path::Path;

use clap::Parser;
use futures::{SinkExt, StreamExt};
use tokio::runtime;
use ufmt::uwrite;

use crate::utils::reopen::{Reopen, SeekableRead};
use crate::utils::streaming_linereader::StreamingLineReader;
use crate::utils::tempfilebuffer::{TempFileBuffer, TempFileBufferWriter};
use crate::{BBIReadError, BigBedRead, ChromInfo};

#[derive(Clone, Debug, PartialEq, Parser)]
#[command(
    name = "bigbedtobed",
    about = "Converts an input bigBed to a bed.",
    long_about = "Converts an input bigBed to a bed. Can be multi-threaded for substantial speedups. Note for roughly each core, one temporary file will be opened."
)]
pub struct BigBedToBedArgs {
    /// the bigbed to get convert to bed
    pub big_bed: String,

    /// the path of the bed to output to
    pub bed: String,

    /// If set, restrict output to given chromosome
    #[arg(long)]
    pub chrom: Option<String>,

    /// If set, restrict output to regions greater than or equal to it
    #[arg(long)]
    pub start: Option<u32>,

    /// If set, restrict output to regions less than it
    #[arg(long)]
    pub end: Option<u32>,

    /// If set, restrict output to regions overlapping the bed file
    pub overlap_bed: Option<String>,

    /// Set the number of threads to use. This tool will nearly always benefit from more cores (<= # chroms). Note: for parts of the runtime, the actual usage may be nthreads+1
    #[arg(short = 't', long)]
    #[arg(default_value_t = 6)]
    pub nthreads: usize,

    /// Do not create temporary files for intermediate data. (Only applicable when using multiple threads.)
    /// By default, approximately one temporary file will be opened for each core.
    #[arg(long)]
    #[arg(default_value_t = false)]
    pub inmemory: bool,

    /// If set, outputs the values for a given zoom level. The format will be
    /// <chrom> <zoom_start>    <zoom_end>  <total_items> <bases_covered>   <min_val>   <max_val>   <sum>   <sum_squares>
    #[arg(long)]
    pub zoom: Option<u32>,
}

pub fn bigbedtobed(args: BigBedToBedArgs) -> Result<(), Box<dyn Error>> {
    let bigbedpath = args.big_bed;
    let bedpath = args.bed;

    let nthreads = args.nthreads;

    let bigbed = BigBedRead::open_file(&bigbedpath)?;
    let bed = File::create(bedpath)?;

    if (args.start.is_some() || args.end.is_some()) && args.chrom.is_none() {
        eprintln!("Cannot specify --start or --end without specifying --chrom.");
        return Ok(());
    }

    if args.chrom.is_some() && args.overlap_bed.is_some() {
        eprintln!("Cannot specify both --overlap-bed and interval to overlap.");
        return Ok(());
    }

    match args.overlap_bed {
        Some(overlap_bed) => {
            if !Path::exists(&Path::new(&overlap_bed)) {
                eprintln!("Overlap bed file does not exist.");
                return Ok(());
            }
            let overlap_bed = File::open(overlap_bed)?;
            write_bed_from_bed(bigbed, bed, overlap_bed)?;
        }
        None => {
            // Right now, we don't offload decompression to separate threads,
            // so specifying `chrom` effectively means single-threaded
            if nthreads == 1 || args.chrom.is_some() || args.zoom.is_some() {
                write_bed_singlethreaded(bigbed, bed, args.chrom, args.start, args.end, args.zoom)?;
            } else {
                write_bed(bigbed, bed, args.inmemory, nthreads)?;
            }
        }
    }

    Ok(())
}

```

--------------------------------

### Parse BedGraph Section Data (Rust)

Source: https://docs.rs/bigtools/0.5.6/src/bigtools/bbi/bigwigread

Parses bedGraph data within a BigWig file, extracting chromosome start, end, and value for each item. It handles endianness for reading u32 and f32 values and filters data based on provided start and end ranges.

```Rust
481        1 => {
482            assert!(bytes.len() >= (item_count as usize) * 12);
483            for i in 0..(item_count as usize) {
484                let istart = i * 12;
485                let block_item_data: &[u8; 12] = bytes[istart..istart + 12].try_into().unwrap();
486                // bedgraph
487                let (chrom_start, chrom_end, value) = match bigwig.info.header.endianness {
488                    Endianness::Big => {
489                        let chrom_start = u32::from_be_bytes([
490                            block_item_data[0],
491                            block_item_data[1],
492                            block_item_data[2],
493                            block_item_data[3],
494                        ]);
495                        let chrom_end = u32::from_be_bytes([
496                            block_item_data[4],
497                            block_item_data[5],
498                            block_item_data[6],
499                            block_item_data[7],
500                        ]);
501                        let value = f32::from_be_bytes([
502                            block_item_data[8],
503                            block_item_data[9],
504                            block_item_data[10],
505                            block_item_data[11],
506                        ]);
507                        (chrom_start, chrom_end, value)
508                    }
509                    Endianness::Little => {
510                        let chrom_start = u32::from_le_bytes([
511                            block_item_data[0],
512                            block_item_data[1],
513                            block_item_data[2],
514                            block_item_data[3],
515                        ]);
516                        let chrom_end = u32::from_le_bytes([
517                            block_item_data[4],
518                            block_item_data[5],
519                            block_item_data[6],
520                            block_item_data[7],
521                        ]);
522                        let value = f32::from_le_bytes([
523                            block_item_data[8],
524                            block_item_data[9],
525                            block_item_data[10],
526                            block_item_data[11],
527                        ]);
528                        (chrom_start, chrom_end, value)
529                    }
530                };
531                let mut value = Value {
532                    start: chrom_start,
533                    end: chrom_end,
534                    value,
535                };
536                if value.end > start && value.start < end {
537                    value.start = value.start.max(start);
538                    value.end = value.end.min(end);
539                    values.push(value)
540                }
541            }
542        }
```
Tokens:
Raw