### Build and Install Rusty Blockparser Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Clone the repository, build the project with Cargo, and run tests. Verify the installation by checking the help command. ```bash git clone https://github.com/gcarq/rusty-blockparser.git cd rusty-blockparser cargo build --release cargo test ./target/release/rusty-blockparser --help ``` -------------------------------- ### Example Tx Out CSV Row Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Provides a sample row from the `tx_out-*.csv` file generated by the `csvdump` command. ```csv 4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b;0;5000000000;4104678afdb0...88ac;1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa ``` -------------------------------- ### Example UTXO Dump Session Output Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Illustrates the typical log output during the execution of the `unspentcsvdump` command. ```log [6:02:53] INFO - main: Starting rusty-blockparser v0.12.0 ... [6:02:53] INFO - index: Reading index from ~/.bitcoin/blocks/index ... [6:02:54] INFO - index: Got longest chain with 639626 blocks ... [6:02:54] INFO - parser: Parsing Bitcoin blockchain (range=0..) ... [6:02:54] INFO - callback: Using `unspentcsvdump` with dump folder: /path/to/output ... [10:29:19] INFO - parser: Done. Processed 639626 blocks in 266.43 minutes. Dumped all 639626 blocks: -> transactions: 549390991 -> inputs: 1347165535 -> outputs: 1359449320 ``` -------------------------------- ### Multi-Coin Support Examples Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Parse different cryptocurrency blockchains by specifying the coin type and data directory. Supports Bitcoin, Litecoin, Dogecoin, Namecoin, and more. ```bash # Parse Bitcoin mainnet (default) ./rusty-blockparser simplestats ``` ```bash # Parse Bitcoin testnet3 ./rusty-blockparser -c testnet3 -d ~/.bitcoin/testnet3 simplestats ``` ```bash # Parse Litecoin ./rusty-blockparser -c litecoin -d ~/.litecoin/blocks unspentcsvdump /output/ ``` ```bash # Parse Dogecoin ./rusty-blockparser -c dogecoin -d ~/.dogecoin/blocks balances /output/ ``` ```bash # Parse Namecoin (supports AuxPow merged mining) ./rusty-blockparser -c namecoin csvdump /output/ ``` -------------------------------- ### Example Block CSV Row Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Provides a sample row from the `blocks-*.csv` file generated by the `csvdump` command. ```csv 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f;0;1;285;0000000000000000000000000000000000000000000000000000000000000000;4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b;1231006505;486604799;2083236893 ``` -------------------------------- ### Example UTXO Dump Output Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Shows the format of the output CSV file generated by the `unspentcsvdump` command. ```csv txid;indexOut;height;value;address fff2525b8931402dd09222c50775608f75787bd2b87e56995a7bdd30f79702c4;0;100000;556000000;1JqDybm2nWTENrHvMyafbSXXtTk5Uv5QAn ``` -------------------------------- ### Example Transaction CSV Row Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Provides a sample row from the `transactions-*.csv` file generated by the `csvdump` command. ```csv 4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b;000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f;1;0 ``` -------------------------------- ### Example Full CSV Dump Output Structure Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Details the structure of the CSV files generated by the `csvdump` command for blocks, transactions, inputs, and outputs. ```csv blocks-{start}-{end}.csv: block_hash;height;version;blocksize;hashPrev;hashMerkleRoot;nTime;nBits;nNonce transactions-{start}-{end}.csv: txid;hashBlock;version;lockTime tx_in-{start}-{end}.csv: txid;hashPrevOut;indexPrevOut;scriptSig;sequence tx_out-{start}-{end}.csv: txid;indexOut;value;scriptPubKey;address ``` -------------------------------- ### Example: Unspent CSV Dump Source: https://github.com/gcarq/rusty-blockparser/blob/master/README.md Demonstrates how to perform an unspent CSV dump of the Bitcoin blockchain using rusty-blockparser. This command processes the entire blockchain and outputs unspent transaction outputs to a specified directory. ```bash # ./blockparser unspentcsvdump /path/to/dump/ [6:02:53] INFO - main: Starting rusty-blockparser v0.7.0 ... [6:02:53] INFO - index: Reading index from ~/.bitcoin/blocks/index ... [6:02:54] INFO - index: Got longest chain with 639626 blocks ... [6:02:54] INFO - blkfile: Reading files from ~/.bitcoin/blocks ... [6:02:54] INFO - parser: Parsing Bitcoin blockchain (range=0..) ... [6:02:54] INFO - callback: Using `unspentcsvdump` with dump folder: /path/to/dump ... [6:03:04] INFO - parser: Status: 130885 Blocks processed. (left: 508741, avg: 13088 blocks/sec) ... [10:28:47] INFO - parser: Status: 639163 Blocks processed. (left: 463, avg: 40 blocks/sec) [10:28:57] INFO - parser: Status: 639311 Blocks processed. (left: 315, avg: 40 blocks/sec) [10:29:07] INFO - parser: Status: 639452 Blocks processed. (left: 174, avg: 40 blocks/sec) [10:29:17] INFO - parser: Status: 639596 Blocks processed. (left: 30, avg: 40 blocks/sec) [10:29:19] INFO - parser: Done. Processed 639626 blocks in 266.43 minutes. (avg: 40 blocks/sec) [10:32:01] INFO - callback: Done. Dumped all 639626 blocks: -> transactions: 549390991 -> inputs: 1347165535 -> outputs: 1359449320 [10:32:01] INFO - main: Fin. ``` -------------------------------- ### Implement Custom Blockchain Callback in Rust Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Implements the `Callback` trait in Rust to create custom blockchain analysis tools. This example defines a callback to count transactions and sum their values. ```rust use clap::{Arg, ArgMatches, Command}; use crate::blockchain::proto::block::Block; use crate::callbacks::Callback; use crate::common::Result; pub struct MyCallback { total_value: u64, tx_count: u64, } impl Callback for MyCallback { /// Define CLI subcommand and arguments fn build_subcommand() -> Command where Self: Sized { Command::new("mycallback") .about("Custom blockchain analysis") .version("0.1") .arg(Arg::new("output-file") .help("Output file path") .index(1) .required(true)) } /// Initialize callback with CLI arguments fn new(matches: &ArgMatches) -> Result where Self: Sized { let _output = matches.get_one::("output-file").unwrap(); Ok(MyCallback { total_value: 0, tx_count: 0 }) } /// Called before parsing begins fn on_start(&mut self, block_height: u64) -> Result<()> { info!(target: "callback", "Starting analysis from block {} ...", block_height); Ok(()) } /// Called for each block in order fn on_block(&mut self, block: &Block, block_height: u64) -> Result<()> { for tx in &block.txs { self.tx_count += 1; // Access transaction data for output in &tx.value.outputs { self.total_value += output.out.value; // Access script pattern and address if let Some(addr) = &output.script.address { debug!(target: "callback", "Output to {} at height {}", addr, block_height); } } } Ok(()) } /// Called after all blocks processed fn on_complete(&mut self, block_height: u64) -> Result<()> { info!(target: "callback", "Processed {} txs, total value: {} satoshis up to block {}", self.tx_count, self.total_value, block_height); Ok(()) } /// Control progress display fn show_progress(&self) -> bool { true } } ``` -------------------------------- ### Build Rusty Blockparser from Source Source: https://github.com/gcarq/rusty-blockparser/blob/master/README.md Instructions for cloning the repository, building the release version of rusty-blockparser, and running tests. Building with `--release` is crucial for optimal performance. ```bash git clone https://github.com/gcarq/rusty-blockparser.git cd rusty-blockparser cargo build --release cargo test ./target/release/rusty-blockparser --help ``` -------------------------------- ### Rusty Blockparser Basic Usage Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Illustrates the general command-line structure and available global options and subcommands for the parser. ```bash rusty-blockparser [OPTIONS] [COMMAND] # Available commands: # unspentcsvdump - Dumps the unspent outputs to CSV file # csvdump - Dumps the whole blockchain into CSV files # simplestats - Shows various Blockchain stats # balances - Dumps all addresses with non-zero balance to CSV file # opreturn - Shows embedded OP_RETURN data that is representable as UTF8 # Global options: # --verify Verifies merkle roots and block hashes # -v Increases verbosity (Info=0, Debug=1, Trace=2) # -c, --coin Specify blockchain (bitcoin, testnet3, namecoin, litecoin, dogecoin, myriadcoin, unobtanium, noteblockchain) # -d, --blockchain-dir Sets blockchain directory with blk.dat files # -s, --start Starting block height (inclusive) # -e, --end Ending block height (inclusive) ``` -------------------------------- ### Rusty Blockparser CLI Usage Source: https://github.com/gcarq/rusty-blockparser/blob/master/README.md Displays available commands and options for the rusty-blockparser command-line interface. Use this to understand the tool's capabilities and parameters. ```bash Usage: rusty-blockparser [OPTIONS] [COMMAND] Commands: unspentcsvdump Dumps the unspent outputs to CSV file csvdump Dumps the whole blockchain into CSV files simplestats Shows various Blockchain stats balances Dumps all addresses with non-zero balance to CSV file opreturn Shows embedded OP_RETURN data that is representable as UTF8 help Print this message or the help of the given subcommand(s) Options: --verify Verifies merkle roots and block hashes -v... Increases verbosity level. Info=0, Debug=1, Trace=2 (default: 0) -c, --coin Specify blockchain coin (default: bitcoin) [possible values: bitcoin, testnet3, namecoin, litecoin, dogecoin, myriadcoin, unobtanium, noteblockchain] -d, --blockchain-dir Sets blockchain directory which contains blk.dat files (default: ~/.bitcoin/blocks) -s, --start Specify starting block for parsing (inclusive) -e, --end Specify last block for parsing (inclusive) (default: all known blocks) -h, --help Print help -V, --version Print version ``` -------------------------------- ### Export Address Balances Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Extracts all addresses with non-zero balances and their total holdings. Memory usage is approximately 18GB. Balance values are in satoshis. ```bash # Export all address balances ./rusty-blockparser balances /path/to/output/ ``` ```bash # Parse Dogecoin balances ./rusty-blockparser -c dogecoin balances /path/to/output/ ``` -------------------------------- ### Register coin in FromStr implementation Source: https://github.com/gcarq/rusty-blockparser/blob/master/README.md Add the new coin to the match statement in the FromStr implementation for CoinType. ```rust "bitcoin" => Ok(CoinType::from(Bitcoin)), ... "nocoinium" => Ok(CoinType::from(NoCoinium)), ... ``` -------------------------------- ### Export Full Blockchain to CSV Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Exports the entire blockchain into four CSV files: blocks, transactions, inputs, and outputs. This command generates a very large amount of data. ```bash ./rusty-blockparser csvdump /path/to/output/ ``` ```bash ./rusty-blockparser -c litecoin -d ~/.litecoin/blocks csvdump /path/to/output/ ``` -------------------------------- ### Implement Custom Coin Support with `Coin` Trait Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Implement the `Coin` trait to add support for new Bitcoin-fork cryptocurrencies. Define essential parameters like magic bytes, version IDs, genesis block hash, and default data directory. ```rust use bitcoin::hashes::sha256d; use std::path::{Path, PathBuf}; use std::str::FromStr; // Define coin struct pub struct MyCoin; impl Coin for MyCoin { fn name(&self) -> String { String::from("MyCoin") } // Magic bytes identify blocks in blk.dat files // Found in chainparams.cpp: pchMessageStart[0-3] (reversed) fn magic(&self) -> u32 { 0xd9b4bef9 // Example: Bitcoin mainnet magic } // Address version byte for Base58Check encoding // Found in chainparams.cpp: base58Prefixes[PUBKEY_ADDRESS] fn version_id(&self) -> u8 { 0x00 // Bitcoin mainnet prefix } // Genesis block hash from chainparams.cpp: consensus.hashGenesisBlock fn genesis(&self) -> sha256d::Hash { sha256d::Hash::from_str( "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" ).unwrap() } // Optional: Enable AuxPow for merged mining (Namecoin, Dogecoin) fn aux_pow_activation_version(&self) -> Option { Some(0x620102) // Block version activating AuxPow } // Default blockchain data directory fn default_folder(&self) -> PathBuf { Path::new(".mycoin").join("blocks") } } // Register in CoinType::from_str() in types.rs: // "mycoin" => Ok(CoinType::from(MyCoin)), // Add to coins array in main.rs command(): // let coins = ["bitcoin", ..., "mycoin"]; ``` -------------------------------- ### Show Blockchain Statistics Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Displays comprehensive blockchain statistics including transaction counts, fees, volumes, and averages. Can specify a block range or verify data integrity. ```bash # Show blockchain statistics ./rusty-blockparser simplestats ``` ```bash # Statistics for specific block range ./rusty-blockparser -s 0 -e 100000 simplestats ``` ```bash # Verify data integrity while collecting stats ./rusty-blockparser --verify simplestats ``` -------------------------------- ### Load CSV Data into MySQL Tables Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Loads data from CSV files into the respective MySQL tables. Ensure the file path is correct and fields are terminated by ';'. Hexadecimal fields are converted using unhex(). ```sql -- Bulk load CSV data LOAD DATA INFILE '/path/to/dump/blocks-0-639626.csv' INTO TABLE blocks FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n' (@hash, height, version, blocksize, @hashPrev, @hashMerkleRoot, nTime, nBits, nNonce) SET hash = unhex(@hash), hashPrev = unhex(@hashPrev), hashMerkleRoot = unhex(@hashMerkleRoot); ``` -------------------------------- ### Implement Coin trait for custom coin Source: https://github.com/gcarq/rusty-blockparser/blob/master/README.md Define the blockchain parameters for a new coin by implementing the Coin trait in src/blockchain/parser/types.rs. ```rust //The name here should be the same case as defined in the pub struct line impl Coin for NoCoinium { fn name(&self) -> String { // This is primarily for display. Use same case as before String::from("NoCoinium") } fn magic(&self) -> u32 { // Magic bytes are a string of hex characters that prefix messages in the chain. // To find this value, look for the fields pchMessageStart[0-3] in the file chainparams.cpp under CMainParams // The value to be used here is 0x + pchMessageStart[3] + pchMessageStart[2] + pchMessageStart[1] + pchMessageStart[0] // i.e. string the values in reverse. 0xd9b4bef9 } fn version_id(&self) -> u8 { // Version ID is used to identify the address prefix for Base58 encoding of the public address // Found this using the stackoverflow comment - https://bitcoin.stackexchange.com/questions/62781/litecoin-constants-and-prefixes // Again with chainparams.cpp and CMainParams, look for base58Prefixes[PUBKEY_ADDRESS]. Convert the decimal value to Hex and add it here 0x00 } fn genesis(&self) -> sha256d::Hash { // This is the Genesis Block hash - Get the value from consensus.hashGenesisBlock, again found in chainparams.cpp sha256d::Hash::from_str("000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f").unwrap() } fn default_folder(&self) -> PathBuf { // This is the folder from the user's home folder to where the blocks files are found // Note the case here. It is not CamelCase as most coin directories are lower case. However, use the actual folder name // from your coin implementation. Path::new(".nocoinium").join("blocks") } } ``` -------------------------------- ### Dump Unspent Transaction Outputs (UTXOs) to CSV Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Extracts all unspent transaction outputs (UTXOs) with their addresses to a CSV file. Requires significant memory for the full Bitcoin blockchain. ```bash ./rusty-blockparser unspentcsvdump /path/to/output/ ``` ```bash ./rusty-blockparser -d /custom/bitcoin/blocks unspentcsvdump /path/to/output/ ``` ```bash ./rusty-blockparser -s 0 -e 500000 unspentcsvdump /path/to/output/ ``` -------------------------------- ### Extract OP_RETURN Data Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Displays all embedded OP_RETURN data that can be represented as valid UTF-8 text. Can parse a specific block range. ```bash # Show all OP_RETURN messages ./rusty-blockparser opreturn ``` ```bash # Parse specific range for OP_RETURN data ./rusty-blockparser -s 300000 -e 400000 opreturn ``` -------------------------------- ### Add Indexes to MySQL Tables Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Adds necessary indexes to the MySQL tables after bulk loading to improve query performance. Indexes are added for efficient lookups on hashes, transaction IDs, and addresses. ```sql -- Add indexes after bulk load ALTER TABLE `blocks` ADD UNIQUE KEY (`hash`); ALTER TABLE `transactions` ADD KEY (`txid`); ALTER TABLE `tx_in` ADD KEY (`hashPrevOut`, `indexPrevOut`); ALTER TABLE `tx_out` ADD KEY (`txid`, `indexOut`), ADD KEY (`address`); ``` -------------------------------- ### Create MySQL Database Schema Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Defines the schema for storing blockchain data in MySQL, including tables for blocks, transactions, and outputs. Ensure the character set and collation are set to ascii for compatibility. ```sql -- Create database schema CREATE SCHEMA IF NOT EXISTS `btc_blockchain` CHARACTER SET ascii COLLATE ascii_bin; USE `btc_blockchain`; -- Create blocks table CREATE TABLE `blocks` ( `id` int(4) unsigned AUTO_INCREMENT NOT NULL, `hash` binary(32) NOT NULL, `height` int(10) unsigned NOT NULL, `version` int(11) NOT NULL, `blocksize` int(10) unsigned NOT NULL, `hashPrev` binary(32) NOT NULL, `hashMerkleRoot` binary(32) NOT NULL, `nTime` int(10) unsigned NOT NULL, `nBits` int(10) unsigned NOT NULL, `nNonce` int(10) unsigned NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; -- Create transactions table CREATE TABLE `transactions` ( `id` int(4) unsigned AUTO_INCREMENT NOT NULL, `txid` binary(32) NOT NULL, `hashBlock` binary(32) NOT NULL, `version` int(11) unsigned NOT NULL, `lockTime` int(10) unsigned NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; -- Create tx_out table CREATE TABLE `tx_out` ( `id` int(4) unsigned AUTO_INCREMENT NOT NULL, `txid` binary(32) NOT NULL, `indexOut` int(10) unsigned NOT NULL, `value` bigint(8) unsigned NOT NULL, `scriptPubKey` blob NOT NULL, `address` varchar(36) DEFAULT NULL, `unspent` bit DEFAULT TRUE NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; -- Create tx_in table CREATE TABLE `tx_in` ( `id` int(10) unsigned AUTO_INCREMENT NOT NULL, `txid` binary(32) NOT NULL, `hashPrevOut` binary(32) NOT NULL, `indexPrevOut` int(10) unsigned NOT NULL, `scriptSig` blob NOT NULL, `sequence` int(10) unsigned NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; ``` -------------------------------- ### Mark Spent Outputs in MySQL Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Updates the `unspent` status in the `tx_out` table to FALSE for outputs that have been spent. This is determined by matching outputs with inputs in the `tx_in` table. ```sql -- Mark spent outputs UPDATE `tx_out` o, `tx_in` i SET o.unspent = FALSE WHERE o.txid = i.hashPrevOut AND o.indexOut = i.indexPrevOut; ``` -------------------------------- ### Process Parsed Block Data Source: https://context7.com/gcarq/rusty-blockparser/llms.txt Access and process detailed information from a parsed `Block` structure, including its header, transactions, inputs, and outputs. Includes verification of the merkle root. ```rust use crate::blockchain::proto::block::Block; use crate::blockchain::proto::tx::{EvaluatedTx, TxInput, EvaluatedTxOut}; use crate::blockchain::proto::Hashed; // Block structure contains: // - size: u32 Block size in bytes // - header: Hashed Block header with hash // - tx_count: VarUint Number of transactions // - txs: Vec> Parsed transactions fn process_block(block: &Block, height: u64) { // Access block header println!("Block hash: {}", block.header.hash); println!("Previous block: {}", block.header.value.prev_hash); println!("Merkle root: {}", block.header.value.merkle_root); println!("Timestamp: {}", block.header.value.timestamp); println!("Difficulty bits: {}", block.header.value.bits); println!("Nonce: {}", block.header.value.nonce); println!("Block size: {} bytes", block.size); // Verify merkle root if let Err(e) = block.verify_merkle_root() { eprintln!("Invalid merkle root: {}", e); } // Process transactions for tx in &block.txs { println!("TxID: {}", tx.hash); println!("Version: {}", tx.value.version); println!("Lock time: {}", tx.value.locktime); // Check if coinbase transaction if tx.value.is_coinbase() { let reward = block::get_base_reward(height); println!("Coinbase reward: {} satoshis", reward); } // Process inputs for input in &tx.value.inputs { println!(" Input - Previous tx: {}:{}", input.outpoint.txid, input.outpoint.index); } // Process outputs for (i, output) in tx.value.outputs.iter().enumerate() { println!(" Output {} - Value: {} satoshis", i, output.out.value); println!(" Script pattern: {:?}", output.script.pattern); if let Some(addr) = &output.script.address { println!(" Address: {}", addr); } } } } ``` ```rust // Block reward calculation (halves every 210,000 blocks) pub const fn get_base_reward(block_height: u64) -> u64 { (50 * 100000000) >> (block_height / 210000) } // Block 0-209999: 5000000000 satoshis (50 BTC) // Block 210000-419999: 2500000000 satoshis (25 BTC) // Block 420000-629999: 1250000000 satoshis (12.5 BTC) // Block 630000+: 625000000 satoshis (6.25 BTC) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.