### Install Tokei via Winget on Windows Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Windows using Winget. ```console winget install XAMPPRocky.tokei ``` -------------------------------- ### Install Tokei via Scoop on Windows Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Windows using Scoop. ```console scoop install tokei ``` -------------------------------- ### Install Tokei via Nix/NixOS Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Nix or NixOS systems. ```console nix-env -i tokei ``` -------------------------------- ### Install Tokei via NetBSD Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on NetBSD systems. ```console pkgin install tokei ``` -------------------------------- ### Install Tokei via OpenSUSE Zypper Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on OpenSUSE systems. ```console sudo zypper install tokei ``` -------------------------------- ### TOML Configuration File Example Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md An example of a TOML configuration file for Tokei. This file demonstrates various keys that can be used to customize analysis behavior. ```toml columns = 80 hidden = false treat_doc_strings_as_comments = true sort = "code" types = ["Rust", "Python"] ``` -------------------------------- ### Install Tokei via FreeBSD Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on FreeBSD systems. ```console pkg install tokei ``` -------------------------------- ### Rust Code Example Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/INDEX.md This is a basic example of Rust code, demonstrating the initialization of a `Languages` object. It is intended to show the format of code examples within the documentation. ```rust // This is a Rust code example let mut languages = Languages::new(); ``` -------------------------------- ### Tokei Installation with Serialization Features Source: https://github.com/xampprocky/tokei/blob/master/README.md Install Tokei with specific features enabled for different serialization formats like 'all', 'cbor', or 'yaml'. ```console cargo install tokei --features all cargo install tokei --features cbor cargo install tokei --features yaml ``` -------------------------------- ### Install Tokei via Void Linux Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Void Linux systems. ```console sudo xbps-install tokei ``` -------------------------------- ### Install Tokei via Cargo Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei using the Cargo package manager. ```console cargo install tokei ``` -------------------------------- ### Install Tokei via Fedora DNF Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Fedora systems. ```console sudo dnf install tokei ``` -------------------------------- ### Install cargo-fuzz Source: https://github.com/xampprocky/tokei/blob/master/fuzz/README.md Install the cargo-fuzz tool, which is required to run fuzzing jobs. ```bash cargo install cargo-fuzz ``` -------------------------------- ### Install Tokei via Homebrew on macOS Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on macOS using Homebrew. ```console brew install tokei ``` -------------------------------- ### Rust Test File Example with Header Source: https://github.com/xampprocky/tokei/blob/master/CONTRIBUTING.md An example of a Rust test file, including the required header format and a basic Rust program structure. ```rust //! 48 lines 36 code 6 comments 6 blanks //! ```rust //! fn main () { //! // Comment //! //! println!("Hello World!"); //! } //! ``` /* /**/ */ fn main() { let start = r##"/*##\" "##; // comment loop { if x.len() >= 2 && x[0] == '*' && x[1] == '/' { // found the */ break; } } } fn foo<'a, 'b>(name: &'b str) { let this_ends = "a \"test/*.ยป; call1(); call2(); let this_does_not = /* a /* nested */ comment " */ "*/another /*test call3(); */; } fn foobar() { let does_not_start = // " "until here, test/* test"; // a quote: " let also_doesnt_start = /* " */ "until here, test,*/ test"; // another quote: " } fn foo() { let a = 4; // /* let b = 5; let c = 6; // */ } ``` -------------------------------- ### Install Tokei via Alpine Linux Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Alpine Linux systems. ```console apk add tokei ``` -------------------------------- ### Load Configuration from File Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/usage-examples.md Load Tokei configuration programmatically from default configuration files. This example shows how to initialize a Config object and access its properties. ```rust use tokei::Config; let config = Config::from_config_files(); // Loads from ~/.config/tokei.toml, ~/tokei.toml, ./tokei.toml // (in order of precedence) println!("Hidden files: {:?}", config.hidden); println!("Language filter: {:?}", config.types); ``` -------------------------------- ### Install Tokei via Arch Linux Package Manager Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on Arch Linux systems. ```console pacman -S tokei ``` -------------------------------- ### Default Configuration Usage Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Demonstrates how to use the default configuration to get language statistics for a given directory. ```rust use tokei::{Config, Languages}; let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &Config::default()); ``` -------------------------------- ### Analyze Multiple Directories Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/README.md This example demonstrates how to analyze code statistics across multiple directories simultaneously. It also shows how to specify directories to ignore. ```rust use tokei::{Config, Languages}; let mut languages = Languages::new(); languages.get_statistics( &["src", "tests", "examples"], &["target"], &Config::default() ); ``` -------------------------------- ### Install Tokei via Conda Source: https://github.com/xampprocky/tokei/blob/master/README.md Use this command to install Tokei on systems with Conda, from the conda-forge channel. ```console conda install -c conda-forge tokei ``` -------------------------------- ### Rust Multi-line Comment Example Source: https://github.com/xampprocky/tokei/blob/master/CONTRIBUTING.md Demonstrates multi-line comments in Rust, enclosed by '/*' and '*/'. ```rust let x = /* There is a reason for this comment, I swear! */ 10; ``` -------------------------------- ### Install Tokei via MacPorts on macOS Source: https://github.com/xampprocky/tokei/blob/master/README.md Use these commands to update MacPorts and install Tokei on macOS. ```console sudo port selfupdate sudo port install tokei ``` -------------------------------- ### Accessing Language Properties Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Use `LanguageType` methods to query syntax rules for any supported language. This example demonstrates how to get line comments for Rust and Python. ```Rust use tokei::LanguageType; let rust = LanguageType::Rust; let python = LanguageType::Python; println!("Rust line comments: {:?}", rust.line_comments()); println!("Python line comments: {:?}", python.line_comments()); ``` -------------------------------- ### Build Tokei from Source using Cargo Source: https://github.com/xampprocky/tokei/blob/master/README.md Build and install Tokei from its Git repository using Cargo. Requires a recent Rust compiler. ```console cargo install --git https://github.com/XAMPPRocky/tokei.git tokei ``` -------------------------------- ### Line Comment Syntax and Usage Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Demonstrates how to retrieve and use line comment syntax for various languages. The usage example shows parsing a Rust code string and counting comments and code lines. ```Rust pub fn line_comments(self) -> &'static [&'static str] ``` ```Rust use tokei::LanguageType; let code = r#"// This line is a comment let x = 42; // This part is code "#; let stats = LanguageType::Rust.parse_from_str(code, &Config::default()); println!("Comments: {}", stats.comments); // 1 (first line) println!("Code: {}", stats.code); // 1 (second line with both code and comment) ``` -------------------------------- ### Rust Line Comment Example Source: https://github.com/xampprocky/tokei/blob/master/CONTRIBUTING.md Illustrates the use of single-line comments in Rust, which are defined by '//'. ```rust let x = 5; // default x position let y = 0; // default y position ``` -------------------------------- ### Create a tokei.toml Config File Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/usage-examples.md Define custom configurations for Tokei in a toml file. This example filters by specific languages, includes hidden files, sorts by code lines, and treats docstrings as comments. ```toml # Count only Rust and Python types = ["Rust", "Python"] # Include hidden files hidden = true # Sort by code lines sort = "code" # Treat docstrings as comments treat_doc_strings_as_comments = true ``` -------------------------------- ### Create New Report Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/report.md Initializes a new Report instance for a given file path with zero statistics. Use this when starting to collect stats for a new file. ```rust use std::path::PathBuf; use tokei::Report; let report = Report::new(PathBuf::from("src/main.rs")); assert_eq!(report.stats.lines(), 0); ``` -------------------------------- ### Analyze a File on Disk Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/usage-examples.md Analyze a specific file on disk by providing its path. This example shows how to handle potential errors during file analysis. ```rust use tokei::{Config, LanguageType}; use std::path::PathBuf; let path = PathBuf::from("src/main.rs"); match LanguageType::Rust.parse(path.clone(), &Config::default()) { Ok(report) => { println!("Code: {}", report.stats.code); println!("Comments: {}", report.stats.comments); } Err((err, path)) => { eprintln!("Failed to analyze {:?}: {}", path, err); } } ``` -------------------------------- ### Initialize New Language Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language.md Constructs a new, empty Language instance with all counters initialized to zero. Use this to start accumulating statistics for a language. ```rust use tokei::Language; let mut rust = Language::new(); assert!(rust.is_empty()); ``` -------------------------------- ### Custom Configuration Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/configuration.md Create a custom Tokei configuration by overriding default values. This example sets specific options like `hidden`, `treat_doc_strings_as_comments`, `types`, and `sort`. ```rust use tokei::{Config, LanguageType, Sort}; let config = Config { hidden: Some(true), treat_doc_strings_as_comments: Some(true), types: Some(vec![LanguageType::Rust, LanguageType::Python]), sort: Some(Sort::Code), ..Config::default() }; ``` -------------------------------- ### Tokei Ignore File Syntax Examples Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/configuration.md Illustrates the syntax used in Tokei ignore files, which follows the .gitignore pattern. This includes comments, ignoring files and directories, using wildcards, and negation. ```ignore # Comments start with # # Patterns are case-sensitive # Ignore a file target/debug # Ignore a directory and its contents node_modules/ # Ignore a pattern in any directory **/*.tmp # Ignore with wildcards *.log *.bak *~ # Negation (un-ignore) !important.tmp # Ignore all .d files except main.d *.d !main.d ``` ```ignore # Build artifacts target/ dist/ build/ # Dependencies node_modules/ vendor/ # IDE and editor files .idea/ .vscode/ *.swp *.swo # Temporary files *.tmp *.bak ~* # OS files .DS_Store Thumbs.db ``` -------------------------------- ### Create New CodeStats Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/code-stats.md Initializes a new CodeStats instance with all counters set to zero. Useful for starting fresh statistics. ```rust use tokei::CodeStats; let stats = CodeStats::new(); assert_eq!(stats.lines(), 0); ``` -------------------------------- ### Process Files with Custom Callback Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/usage-examples.md Analyze files using a custom callback function for each file processed. This example counts files and prints details for each, demonstrating the use of `for_each_fn` in the Config. ```rust use tokei::{Config, LanguageType, Report}; use std::sync::{Arc, Mutex}; let file_count = Arc::new(Mutex::new(0)); let file_count_clone = Arc::clone(&file_count); let config = Config { for_each_fn: Some(|lang_type: LanguageType, report: Report| { let mut count = file_count_clone.lock().unwrap(); *count += 1; println!( "File {}: {} ({} lines)", count, report.name.display(), report.stats.lines() ); }), ..Config::default() }; let mut languages = tokei::Languages::new(); languages.get_statistics(&["."], &[], &config); let final_count = *file_count.lock().unwrap(); println!("Total files analyzed: {}", final_count); ``` -------------------------------- ### Loading Configuration from Files Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Illustrates how to load Tokei's configuration directly from configuration files. ```rust use tokei::{Config, Languages}; let config = Config::from_config_files(); let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &config); ``` -------------------------------- ### Get Single-Line Comment Delimiters Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Retrieves the single-line comment markers for a given language. For example, `//` for Rust and `#` for Python. ```rust use tokei::LanguageType; let rust = LanguageType::Rust; assert_eq!(rust.line_comments(), &["//"]); let python = LanguageType::Python; assert_eq!(python.line_comments(), &["#"]); ``` -------------------------------- ### Get Documentation String Delimiters Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Retrieves the delimiters for documentation strings (docstrings). By default, these are counted as code, but can be configured to be counted as comments. ```Rust pub fn doc_quotes(self) -> &'static [(&'static str, &'static str)] ``` ```Rust use tokei::{Config, LanguageType}; let code = r#"\n""\nThis is a docstring.\nIt spans multiple lines.\n"""\n\ndef hello():\n """Function docstring"""\n print("Hello")\n"#; // Default: docstrings counted as code let default_stats = LanguageType::Python.parse_from_str(code, &Config::default()); println!("Code (docstrings as code): {}", default_stats.code); // With treat_doc_strings_as_comments: docstrings counted as comments let config = Config { treat_doc_strings_as_comments: Some(true), ..Config::default() }; let comment_stats = LanguageType::Python.parse_from_str(code, &config); println!("Comments (docstrings as comments): {}", comment_stats.comments); ``` -------------------------------- ### Find Files with Most Code Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/usage-examples.md Identify and display the top 5 files with the most code lines for each programming language. This example sorts languages by code lines and iterates through the top files. ```rust use tokei::{Config, Languages, Sort}; let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &Config::default()); for (lang_type, mut lang) in languages { lang.sort_by(Sort::Code); println!("\nTop files in {}:", lang_type.name()); for (i, report) in lang.reports.iter().take(5).enumerate() { println!( "{}. {} ({} lines)", i + 1, report.name.display(), report.stats.code ); } } ``` -------------------------------- ### Get Verbatim String Delimiters Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Retrieves the delimiters for verbatim strings, which do not process escape sequences. This is useful for paths or regular expressions. ```Rust pub fn verbatim_quotes(self) -> &'static [(&'static str, &'static str)] ``` ```Rust use tokei::LanguageType; let csharp = LanguageType::CSharp; println!("Verbatim quotes: {:?}", csharp.verbatim_quotes()); // [("@\"", "\")] ``` -------------------------------- ### Receive and Process Message Source: https://github.com/xampprocky/tokei/blob/master/tests/embedding/file_triggeringprincipal_frame_1.html Handles incoming messages, validating the 'start' command. If valid, it proceeds to click a link within the subframe. If invalid, it removes the listener and notifies the parent frame of failure. This is crucial for controlling test flow and ensuring correct message origins. ```javascript function receiveMessage(event) { // make sure to get the right start command, otherwise // let the parent know and fail the test if (event.data.start !== "startTest") { window.removeEventListener("message", receiveMessage); window.parent.postMessage({triggeringPrincipalURI: "false"}, "*"); } // click the link to navigate the subframe document.getElementById("testlink").click(); } ``` -------------------------------- ### Get Basic String Literal Delimiters Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Retrieves the delimiters used for basic string literals in a given language. These are typically double quotes, single quotes, or backticks. ```Rust pub fn quotes(self) -> &'static [(&'static str, &'static str)] ``` ```Rust use tokei::LanguageType; let python = LanguageType::Python; println!("Quotes: {:?}", python.quotes()); // [("\"", "\"), ("'", "'")] // Tokens inside strings are not counted as comments let code = r#" s = \"This is not a # comment\"\n# This is a comment\n"#; let stats = LanguageType::Python.parse_from_str(code, &Config::default()); println!("Comments: {}", stats.comments); // 1 (only the actual comment line) ``` -------------------------------- ### Multi-Line Comment Syntax and Usage Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Shows how to retrieve multi-line comment delimiters for different languages. The usage example parses a Rust code string containing a multi-line comment. ```Rust pub fn multi_line_comments(self) -> &'static [(&'static str, &'static str)] ``` ```Rust use tokei::LanguageType; let code = r#"/* This is a multi-line comment */ let x = 42; "#; let stats = LanguageType::Rust.parse_from_str(code, &Config::default()); println!("Comments: {}", stats.comments); // 2 ``` -------------------------------- ### Nested Comment Support and Retrieval Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/language-syntax-guide.md Explains how to check if a language supports nested multi-line comments and retrieve their specific delimiters. Examples show checking support for Rust and C, and retrieving nested comment pairs for D. ```Rust pub fn allows_nested(self) -> bool ``` ```Rust pub fn nested_comments(self) -> &'static [(&'static str, &'static str)] ``` ```Rust use tokei::LanguageType; let rust = LanguageType::Rust; println!("Allows nested: {}", rust.allows_nested()); // true let c = LanguageType::C; println!("Allows nested: {}", c.allows_nested()); // false let d = LanguageType::D; println!("Nested comments: {:?}", d.nested_comments()); // [("/*", "*/")] ``` -------------------------------- ### Load Tokei Configuration from Files Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/configuration.md Loads Tokei configuration settings from default configuration files. ```rust use tokei::{Config, Languages}; let config = Config::from_config_files(); let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &config); ``` -------------------------------- ### Load Configuration from Files Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/configuration.md Load Tokei configuration settings from standard configuration files. Later files in the search order override settings from earlier ones. ```rust use tokei::Config; let config = Config::from_config_files(); ``` -------------------------------- ### C# Verbatim String Literal Example Source: https://github.com/xampprocky/tokei/blob/master/CONTRIBUTING.md Provides an example of a verbatim string literal in C#, which allows backslashes to be treated as literal characters. ```csharp const string BasePath = @"C:\"; ``` -------------------------------- ### Custom Configuration for Tokei Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Shows how to create and apply a custom configuration, enabling specific language types, hiding files, and sorting reports. ```rust use tokei::{Config, Languages, LanguageType, Sort}; let config = Config { treat_doc_strings_as_comments: Some(true), hidden: Some(true), types: Some(vec![LanguageType::Rust, LanguageType::Python]), sort: Some(Sort::Code), ..Config::default() }; let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &config); ``` -------------------------------- ### UserDict/List/String Imports (Python 2 & 3) Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Illustrates importing `UserDict`, `UserList`, and `UserString`, showing differences between Python 2, Python 3, and compatibility options using `future` and `six` libraries. ```python # Python 2 only: from UserDict import UserDict from UserList import UserList from UserString import UserString # Python 3 only: from collections import UserDict, UserList, UserString # Python 2 and 3: alternative 1 from future.moves.collections import UserDict, UserList, UserString # Python 2 and 3: alternative 2 from six.moves import UserDict, UserList, UserString # Python 2 and 3: alternative 3 from future.standard_library import install_aliases install_aliases() from collections import UserDict, UserList, UserString ``` -------------------------------- ### Get documentation string delimiters for a LanguageType Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Use `doc_quotes()` to get the delimiters for documentation strings. Python, for instance, uses both triple double quotes and triple single quotes. ```rust use tokei::LanguageType; let python = LanguageType::Python; assert_eq!(python.doc_quotes(), &[("\"\"\"", "\"\"\""), ("'''", "'''")]); ``` -------------------------------- ### quotes() Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Gets the string literal delimiters for a given language. ```APIDOC ## quotes() ### Description Returns the string literal delimiters for this language. ### Method `quotes(self) -> &'static [(&'static str, &'static str)]` ### Returns Array of (open, close) pairs (e.g., `[("\"", "\"")]` for double quotes). ### Example ```rust use tokei::LanguageType; let c = LanguageType::C; assert_eq!(c.quotes(), &[("\"", "\"")]); ``` ``` -------------------------------- ### Python 2 File IO vs Python 3 io.open Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Demonstrates file opening and reading in Python 2 and Python 3, including encoding handling. ```python # Python 2 only f = open('myfile.txt') data = f.read() # as a byte string text = data.decode('utf-8') # Python 2 and 3: alternative 1 from io import open f = open('myfile.txt', 'rb') data = f.read() # as bytes text = data.decode('utf-8') # unicode, not bytes # Python 2 and 3: alternative 2 from io import open f = open('myfile.txt', encoding='utf-8') text = f.read() # unicode, not bytes ``` -------------------------------- ### doc_quotes() Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Gets the documentation string delimiters for a language, like Python's triple quotes. ```APIDOC ## doc_quotes() ### Description Returns documentation string delimiters (e.g., Python's triple quotes). ### Method `doc_quotes(self) -> &'static [(&'static str, &'static str)]` ### Returns Array of (open, close) pairs for doc strings. ### Example ```rust use tokei::LanguageType; let python = LanguageType::Python; assert_eq!(python.doc_quotes(), &[("""", """"), ("'''", "'''")]); ``` ``` -------------------------------- ### Build Docker Image with Earthly Source: https://github.com/xampprocky/tokei/blob/master/README.md Build the Tokei Docker image using Earthly. This is the first step before running Tokei in a Docker container. ```bash earthly +docker ``` -------------------------------- ### Get Language Display Name Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Retrieves the display name of a language using the `name()` method. Useful for user-facing output. ```rust use tokei::LanguageType; let bash = LanguageType::Bash; assert_eq!(bash.name(), "BASH"); let rust = LanguageType::Rust; assert_eq!(rust.name(), "Rust"); ``` -------------------------------- ### Load Config from Files Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Loads configuration from multiple TOML files with increasing precedence. Settings in later files override earlier ones. Unset options will be `None`. ```rust use tokei::Config; let config = Config::from_config_files(); // Loads from ~/.config/tokei.toml, ~/.tokeirc, ./tokei.toml, ./.tokeirc ``` -------------------------------- ### Analyze Code String Directly Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/README.md Parse a given string of code to get statistics for lines of code, comments, and blanks. ```rust use tokei::{Config, LanguageType}; let code = r#"\nfn main() {\n // This is a comment\n println!(\"Hello, world!\");\n}\n"#; let stats = LanguageType::Rust.parse_from_str(code, &Config::default()); println!("Lines of code: {}", stats.code); println!("Lines of comments: {}", stats.comments); println!("Blank lines: {}", stats.blanks); ``` -------------------------------- ### Execute File in Python 2 and 3 Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Demonstrates two methods for executing the content of a Python file. The first uses `past.builtins.execfile`, while the second uses `exec` with `compile` and `open`. Be aware of potential `SyntaxError` with the second method when using `import *` and `bare exec`. ```python # Python 2 and 3: alternative 1 from past.builtins import execfile execfile('myfile.py') ``` ```python # Python 2 and 3: alternative 2 exec(compile(open('myfile.py').read())) # This can sometimes cause this: # SyntaxError: function ... uses import * and bare exec ... # See https://github.com/PythonCharmers/python-future/issues/37 ``` -------------------------------- ### Looping Over Byte Strings in Python 2 Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Iterate over byte strings directly in Python 2 to get individual byte characters. ```python # Python 2 only: for bytechar in 'byte-string with high-bit chars like \xf9': ... ``` -------------------------------- ### Get nested comment delimiters for a LanguageType Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md The `nested_comments()` method returns delimiters for nested comments if supported. Currently, only the D language has this feature. ```rust use tokei::LanguageType; let d = LanguageType::D; assert_eq!(d.nested_comments(), &[("/+", "+/")]); ``` -------------------------------- ### Queue Imports (Python 2 & 3) Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Demonstrates importing `Queue`, `heapq`, and `deque` from the `Queue` module in Python 2 and the `queue` module in Python 3, with a compatible import using the `future` library. ```python # Python 2 only: from Queue import Queue, heapq, deque # Python 2 and 3 (after ``pip install future``): from queue import Queue, heapq, deque ``` -------------------------------- ### Detect Language from File Path Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/README.md Determine the programming language of a file based on its path and then parse the file to get code statistics. ```rust use tokei::{Config, LanguageType}; let lang = LanguageType::from_path("src/main.rs", &Config::default()); assert_eq!(lang, Some(LanguageType::Rust)); let stats = lang.unwrap().parse( std::path::PathBuf::from("src/main.rs"), &Config::default() )?; ``` -------------------------------- ### Config::from_config_files Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Loads configuration from TOML files found in standard locations: the user's config directory, home directory, and the current directory. Settings from files with higher precedence override those from lower precedence files. ```APIDOC ## `from_config_files()` ### Description Loads configuration from up to three files, in order of increasing precedence: Config directory, Home directory, and Current directory. Within each directory, `tokei.toml` takes precedence over `.tokeirc`. Settings in later files override settings in earlier files. ### Returns A merged `Config` from all available files, with unset options being `None`. ### Example ```rust use tokei::Config; let config = Config::from_config_files(); ``` ### TOML Configuration File Example ```toml columns = 80 hidden = false treat_doc_strings_as_comments = true sort = "code" types = ["Rust", "Python"] ``` ``` -------------------------------- ### CodeStats::new() Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/code-stats.md Creates a new, empty CodeStats instance with all counters initialized to zero. This is the starting point for accumulating code statistics. ```APIDOC ## CodeStats::new() ### Description Creates a new blank `CodeStats` with all counters initialized to zero. ### Returns A new `CodeStats` instance. ### Example ```rust use tokei::CodeStats; let stats = CodeStats::new(); assert_eq!(stats.lines(), 0); ``` ``` -------------------------------- ### Raising Exceptions with Traceback Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Illustrates how to raise exceptions with a traceback. Provides Python 2 only syntax and two Python 2 and 3 compatible options using `six.reraise` or `future.utils.raise_with_traceback`. ```python # Python 2 only: traceback = sys.exc_info()[2] raise ValueError, "dodgy value", traceback ``` ```python # Python 3 only: raise ValueError("dodgy value").with_traceback() ``` ```python # Python 2 and 3: option 1 from six import reraise as raise_ # or from future.utils import raise_ traceback = sys.exc_info()[2] raise_(ValueError, "dodgy value", traceback) ``` ```python # Python 2 and 3: option 2 from future.utils import raise_with_traceback raise_with_traceback(ValueError("dodgy value")) ``` -------------------------------- ### Tokei CLI Usage Source: https://github.com/xampprocky/tokei/blob/master/README.md Displays the general usage syntax for the Tokei command-line tool, including available flags and options. ```bash USAGE: tokei [FLAGS] [OPTIONS] [--] [input]... FLAGS: -f, --files Will print out statistics on individual files. -h, --help Prints help information --hidden Count hidden files. -l, --languages Prints out supported languages and their extensions. --no-ignore Don't respect ignore files (.gitignore, .ignore, etc.). This implies --no-ignore-parent, --no-ignore-dot, and --no-ignore-vcs. --no-ignore-dot Don't respect .ignore and .tokeignore files, including those in parent directories. --no-ignore-parent Don't respect ignore files (.gitignore, .ignore, etc.) in parent directories. --no-ignore-vcs Don't respect VCS ignore files (.gitignore, .hgignore, etc.), including those in parent directories. -V, --version Prints version information -v, --verbose Set log output level: 1: to show unknown file extensions, 2: reserved for future debugging, 3: enable file level trace. Not recommended on multiple files OPTIONS: -c, --columns Sets a strict column width of the output, only available for terminal output. -e, --exclude ... Ignore all files & directories matching the pattern. -i, --input Gives statistics from a previous tokei run. Can be given a file path, or "stdin" to read from stdin. -o, --output Outputs Tokei in a specific format. Compile with additional features for more format support. [possible values: cbor, json, yaml] -s, --sort Sort languages based on column [possible values: files, lines, blanks, code, comments] -t, --type Filters output by language type, separated by a comma. i.e. -t=Rust,Markdown ARGS: ... The path(s) to the file or directory to be counted. ``` -------------------------------- ### Check if a LanguageType is literate Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Use `is_literate()` to determine if a language is primarily for documentation, where code is treated as comments. Markdown is an example of a literate language. ```rust use tokei::LanguageType; let markdown = LanguageType::Markdown; assert!(markdown.is_literate()); let rust = LanguageType::Rust; assert!(!rust.is_literate()); ``` -------------------------------- ### Default Configuration Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/configuration.md Initialize Tokei with default configuration settings. All options are set to `None`, enabling standard ignore file behavior. ```rust use tokei::Config; let config = Config::default(); ``` -------------------------------- ### list() Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Provides a static list of all supported languages and their associated file extensions. ```APIDOC ## list() ### Description Returns all supported languages as a static list with their file extensions. ### Method `list() -> &'static [(Self, &'static [&'static str])]` ### Returns Array of `(LanguageType, extensions)` tuples for all supported languages. ### Example ```rust use tokei::LanguageType; for (language, extensions) in LanguageType::list() { println!("{:?}: {{:?}}", language.name(), extensions); } ``` ``` -------------------------------- ### Run Dockerized Tokei Source: https://github.com/xampprocky/tokei/blob/master/README.md Run the Tokei Docker image to analyze code. Mount the directory to analyze into the container at /src. ```bash docker run --rm -v /path/to/analyze:/src tokei . ``` -------------------------------- ### String Representation Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Compares the Python 2 only backtick syntax for getting a string representation with the Python 2 and 3 compatible `repr()` function. ```python `x` # Python 2 only ``` ```python repr(x) # Python 2 and 3 ``` -------------------------------- ### Python 2 exec with globals and locals vs Python 3 exec() with globals and locals Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Illustrates executing dynamic code with both global and local namespaces in Python 2 and 3. ```python # Python 2 only: l = locals() exec 'x = 10' in g, l ``` ```python # Python 2 and 3: exec('x = 10', g, l) ``` -------------------------------- ### Get shebang patterns for a LanguageType Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md The `shebangs()` method returns an array of shebang patterns used for file detection. For Bash, this is typically `#!/bin/bash`. ```rust use tokei::LanguageType; let bash = LanguageType::Bash; assert_eq!(bash.shebangs(), &[("#!/bin/bash")]); ``` -------------------------------- ### Python 2 file() vs Python 3 open() Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Shows the equivalent ways to open files in Python 2 using `file()` and in Python 3 using `open()`. ```python # Python 2 only: f = file(pathname) ``` ```python # Python 2 and 3: f = open(pathname) # But preferably, use this: from io import open f = open(pathname, 'rb') # if f.read() should return bytes # or f = open(pathname, 'rt') # if f.read() should return unicode text ``` -------------------------------- ### Report::new Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/report.md Creates a new Report instance for a given file path. Initializes with zero code statistics. ```APIDOC ## Report::new ### Description Creates a new `Report` from a file path with zero statistics. ### Method Signature `pub fn new(name: PathBuf) -> Self` ### Parameters #### Path Parameters - **name** (PathBuf) - The path to the file. ### Returns A new `Report` instance with default (zero) statistics. ### Example ```rust use std::path::PathBuf; use tokei::Report; let report = Report::new(PathBuf::from("src/main.rs")); assert_eq!(report.stats.lines(), 0); ``` ``` -------------------------------- ### Looping Over Byte Strings in Python 3 Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Iterate over byte strings in Python 3 to get integer representations of bytes, then convert to byte strings. ```python # Python 3 only: for myint in b'byte-string with high-bit chars like \xf9': bytechar = bytes([myint]) ``` -------------------------------- ### Get string literal delimiters for a LanguageType Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Use `quotes()` to retrieve the string literal delimiters for a given language. For C, this is typically double quotes. ```rust use tokei::LanguageType; let c = LanguageType::C; assert_eq!(c.quotes(), &[("\"", "\"")]); ``` -------------------------------- ### Import Everything from Tokei Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/exported-api.md Demonstrates importing all items from the Tokei crate using a glob import. ```rust use tokei::*; let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &Config::default()); ``` -------------------------------- ### TOML Configuration File Format Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Illustrates the structure and available keys for TOML configuration files used by Tokei. These keys map directly to `Config` struct fields. ```toml # Terminal output column width (ignored by library) columns = 120 # Include hidden files in analysis hidden = true # Do not respect any ignore files no_ignore = false # Do not respect parent directory ignore files no_ignore_parent = false # Do not respect .ignore and .tokeignore files no_ignore_dot = false # Do not respect VCS ignore files (.gitignore, etc.) no_ignore_vcs = false # Treat doc strings as comments treat_doc_strings_as_comments = true # Sort strategy: "lines", "code", "comments", "blanks", "files" sort = "code" # Only count these languages types = ["Rust", "Python", "JavaScript"] ``` -------------------------------- ### Summarize CodeStats Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/code-stats.md Merges statistics from embedded language blobs into the parent CodeStats. Use this to get an aggregated count across all languages within a file. ```rust use tokei::{CodeStats, LanguageType}; let mut stats = CodeStats::new(); stats.code = 100; // Create a child blob (e.g., JavaScript embedded in HTML) let mut js_stats = CodeStats::new(); js_stats.code = 20; stats.blobs.insert(LanguageType::JavaScript, js_stats); let summary = stats.summarise(); assert_eq!(summary.code, 120); // Parent code + child code ``` -------------------------------- ### Check if LanguageType allows nested multi-line comments Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Use `allows_nested()` to determine if a language supports nested multi-line comments. For example, Rust allows them, while C does not. ```rust use tokei::LanguageType; let rust = LanguageType::Rust; assert!(rust.allows_nested()); let c = LanguageType::C; assert!(!c.allows_nested()); ``` -------------------------------- ### Config::default Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/config.md Creates a new Config instance with sensible default settings. These defaults include respecting all ignore files, not counting hidden files by default, and applying no language filtering. ```APIDOC ## `default()` ### Description Creates a new `Config` with sensible defaults: respects all ignore files, counts hidden files as false, and applies no language filtering. ### Returns A new `Config` instance with default settings. ### Example ```rust use tokei::Config; let config = Config::default(); ``` ``` -------------------------------- ### Get verbatim string literal delimiters for a LanguageType Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md The `verbatim_quotes()` method returns delimiters for verbatim strings, such as C#'s `@"..."` format. ```rust use tokei::LanguageType; let csharp = LanguageType::CSharp; assert_eq!(csharp.verbatim_quotes(), &[("@\"", "\"")]); ``` -------------------------------- ### List all supported LanguageTypes and their extensions Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md The `list()` method provides a static list of all supported languages along with their associated file extensions. This is useful for iterating through all language options. ```rust use tokei::LanguageType; for (language, extensions) in LanguageType::list() { println!("{:?}: {:?}", language.name(), extensions); } ``` -------------------------------- ### Python 2 range vs Python 3 list(range()) Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Illustrates different methods for creating lists from ranges, considering Python 2 and 3 compatibility. ```python # Python 2 only: mylist = range(5) assert mylist == [0, 1, 2, 3, 4] ``` ```python # Python 2 and 3: forward-compatible: option 1 mylist = list(range(5)) # copies memory on Py2 assert mylist == [0, 1, 2, 3, 4] ``` ```python # Python 2 and 3: forward-compatible: option 2 from builtins import range mylist = list(range(5)) assert mylist == [0, 1, 2, 3, 4] ``` ```python # Python 2 and 3: option 3 from future.utils import lrange mylist = lrange(5) assert mylist == [0, 1, 2, 3, 4] ``` ```python # Python 2 and 3: backward compatible from past.builtins import range mylist = range(5) assert mylist == [0, 1, 2, 3, 4] ``` -------------------------------- ### Get Multi-Line Comment Delimiters Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language-type.md Retrieves the open and close delimiters for multi-line comments for a language. Returns an empty array if the language does not support multi-line comments. ```rust use tokei::LanguageType; let rust = LanguageType::Rust; assert_eq!(rust.multi_line_comments(), &[("/*", "*/")] ); let python = LanguageType::Python; assert_eq!(python.multi_line_comments(), &[]); ``` -------------------------------- ### Python 2 exec with globals vs Python 3 exec() with globals Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Shows how to execute dynamic code with specified global namespaces in Python 2 and 3. ```python # Python 2 only: g = globals() exec 'x = 10' in g ``` ```python # Python 2 and 3: g = globals() exec('x = 10', g) ``` -------------------------------- ### Filter by Specific Languages Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/README.md This example demonstrates how to configure Tokei to count only specific programming languages, such as Rust and Python. It modifies the default `Config` to include a `types` field. ```rust use tokei::{Config, Languages, LanguageType}; let config = Config { types: Some(vec![LanguageType::Rust, LanguageType::Python]), ..Config::default() }; let mut languages = Languages::new(); languages.get_statistics(&["."], &[], &config); // languages only contains Rust and Python ``` -------------------------------- ### Dictionary Values as List with `builtins.dict` (Python 2 and 3) Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Use `builtins.dict` and `list(heights.values())` to get a list of dictionary values, compatible with Python 2 and 3. ```python # Python 2 and 3: option 2 from builtins import dict heights = dict(Fred=175, Anne=166, Joe=192) valuelist = list(heights.values()) ``` -------------------------------- ### Configuring Sort Strategy in Code Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/sort.md Demonstrates setting the sort strategy directly within a `Config` struct using `Sort::Code`. ```rust use tokei::{Config, Sort}; let config = Config { sort: Some(Sort::Code), ..Config::default() }; ``` -------------------------------- ### Summarize All Languages Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/languages.md Creates a summary of all languages found, merging child language statistics into their parent languages. This is useful for getting overall statistics across all files analyzed. ```rust use tokei::Languages; let languages = Languages::new(); let total_stats = languages.total(); println!("Total lines of code: {}", total_stats.code); ``` -------------------------------- ### Calculate Total Lines Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/code-stats.md Calculates the sum of code, comment, and blank lines within a CodeStats instance. Used to get the total line count of a code blob. ```rust use tokei::CodeStats; let mut stats = CodeStats::new(); stats.code = 50; stats.comments = 10; stats.blanks = 5; assert_eq!(stats.lines(), 65); ``` -------------------------------- ### Flexible Path Inputs with AsRef Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/types.md Demonstrates using `AsRef` with `Languages::get_statistics` to accept various path input types such as string slices, references, owned strings, and path references. ```rust use tokei::Languages; let mut languages = Languages::new(); // All of these work: languages.get_statistics(&["src"], &[], &Config::default()); languages.get_statistics(&[String::from("src")], &[], &Config::default()); languages.get_statistics(&[std::path::Path::new("src")], &[], &Config::default()); languages.get_statistics(&[std::path::PathBuf::from("src")], &[], &Config::default()); ``` -------------------------------- ### itertools Imports (Python 2 & 3) Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Demonstrates importing functions from `itertools`, highlighting differences between Python 2 and Python 3, and providing compatibility options using `future` and `six` libraries. ```python # Python 2 only: from itertools import ifilterfalse, izip_longest # Python 3 only: from itertools import filterfalse, zip_longest # Python 2 and 3: alternative 1 from future.moves.itertools import filterfalse, zip_longest # Python 2 and 3: alternative 2 from six.moves import filterfalse, zip_longest # Python 2 and 3: alternative 3 from future.standard_library import install_aliases install_aliases() from itertools import filterfalse, zip_longest ``` -------------------------------- ### new() Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/language.md Constructs a new, empty `Language` instance with all counters initialized to zero. ```APIDOC ## `new()` ```rust pub fn new() -> Self ``` Constructs a new empty `Language` with all counters initialized to zero. **Returns:** A new `Language` instance. **Example:** ```rust use tokei::Language; let mut rust = Language::new(); assert!(rust.is_empty()); ``` ``` -------------------------------- ### Get Statistics for Paths Source: https://github.com/xampprocky/tokei/blob/master/_autodocs/api-reference/languages.md Populates the Languages struct with statistics about languages found in the provided paths. Recursively traverses directories and counts code, comments, and blank lines for each language detected. ```rust use tokei::{Config, Languages}; let mut languages = Languages::new(); languages.get_statistics(&["."], &[ ".git", "target"], &Config::default()); ``` -------------------------------- ### Database Module Compatibility Source: https://github.com/xampprocky/tokei/blob/master/tests/data/jupyter.ipynb Illustrates how to handle database modules like `dbm` across Python versions. The `future` library provides aliases for Python 3 compatibility, while `six.moves` offers some alternatives. ```python # Python 2 only import anydbm import whichdb import dbm import dumbdbm import gdbm ``` ```python # Python 2 and 3: alternative 1 from future import standard_library standard_library.install_aliases() import dbm import dbm.ndbm import dbm.dumb import dbm.gnu ``` ```python # Python 2 and 3: alternative 2 from future.moves import dbm from future.moves.dbm import dumb from future.moves.dbm import ndbm from future.moves.dbm import gnu ``` ```python # Python 2 and 3: alternative 3 from six.moves import dbm_gnu # (others not supported) ```