### Install and Set Up pre-commit Source: https://github.com/filyp/autocorrect/blob/master/CONTRIBUTING.md Install the pre-commit tool and set it up to run automatically before each commit. This ensures code style consistency. ```bash pip install pre-commit pre-commit install ``` -------------------------------- ### Install autocorrect Source: https://github.com/filyp/autocorrect/blob/master/README.md Install the autocorrect library using pip. ```bash pip install autocorrect ``` -------------------------------- ### Autocorrect with Other Languages Source: https://github.com/filyp/autocorrect/blob/master/README.md Initialize the Speller with a language code to use it for other supported languages. For example, 'pl' for Polish. ```python spell = Speller('pl') spell('ptaaki latatją kluczmm') ``` -------------------------------- ### Get Correction Candidates Source: https://context7.com/filyp/autocorrect/llms.txt Retrieve all possible corrections for a word along with their frequency scores using the `get_candidates` method. Higher scores indicate more common words. ```python from autocorrect import Speller spell = Speller() # Get candidates with frequency scores (higher = more common) candidates = spell.get_candidates("tehre") print(candidates) # Output: [(5437024, 'there'), (5860, 'terre')] candidates = spell.get_candidates("recieve") print(candidates) # Output: [(2345678, 'receive')] ``` -------------------------------- ### Get Correction Candidates Source: https://context7.com/filyp/autocorrect/llms.txt Details how to retrieve a list of possible corrections for a word along with their frequency scores using `get_candidates`. ```APIDOC ## Get Correction Candidates The `get_candidates` method returns all possible corrections for a word along with their frequency scores. Higher scores indicate more common words, helping choose the best correction. ### Request Example ```python from autocorrect import Speller spell = Speller() # Get candidates with frequency scores (higher = more common) candidates = spell.get_candidates("tehre") print(candidates) # Output: [(5437024, 'there'), (5860, 'terre')] candidates = spell.get_candidates("recieve") print(candidates) # Output: [(2345678, 'receive')] ``` ``` -------------------------------- ### Get Multiple Correction Candidates Source: https://github.com/filyp/autocorrect/blob/master/README.md Retrieve a list of possible corrections for a given word, along with their frequencies. Higher frequency indicates a more likely correction. ```python spell.get_candidates("tehre") ``` -------------------------------- ### List and Test Supported Languages Source: https://context7.com/filyp/autocorrect/llms.txt Demonstrates how to list all supported languages and test the autocorrect functionality for each. Language dictionaries are downloaded automatically on first use. ```python from autocorrect import Speller from autocorrect.constants import word_regexes, alphabets # List all supported languages print("Supported languages:", list(word_regexes.keys())) # Output: ['en', 'pl', 'ru', 'uk', 'tr', 'es', 'pt', 'cs', 'el', 'it', 'fr', 'vi'] # Language codes and examples: languages = { 'en': ('English', "I'm not sleapy"), 'pl': ('Polish', "ptaaki latatją"), 'ru': ('Russian', "убийста"), 'uk': ('Ukrainian', "положеннн"), 'tr': ('Turkish', "yanlş gidiyr"), 'es': ('Spanish', "Fenomenos meteorologicos"), 'pt': ('Portuguese', "contgio nanorada"), 'cs': ('Czech', "lingviskiku"), 'el': ('Greek', "άνθρωπει"), 'it': ('Italian', "ciap mamma"), 'fr': ('French', "bein gallerie"), 'vi': ('Vietnamese', "hiéu chau"), } # Test each language for code, (name, sample) in languages.items(): spell = Speller(code) corrected = spell(sample) print(f"{name} ({code}): '{sample}' -> '{corrected}'") # Check alphabet for a language print(f"English alphabet: {alphabets['en']}") print(f"Polish alphabet: {alphabets['pl']}") print(f"Greek alphabet: {alphabets['el']}") ``` -------------------------------- ### Speller Class Initialization Source: https://context7.com/filyp/autocorrect/llms.txt Demonstrates various ways to initialize the Speller class with different languages and performance options. ```APIDOC ## Speller Class Initialization The `Speller` class is the main interface for spelling correction. It accepts a language code, performance options, custom dictionaries, and OCR-specific settings to customize correction behavior. ### Request Example ```python from autocorrect import Speller # Basic English speller (default) spell = Speller() # Speller for a specific language spell_polish = Speller('pl') spell_spanish = Speller('es') spell_russian = Speller('ru') spell_french = Speller('fr') spell_italian = Speller('it') spell_turkish = Speller('tr') spell_portuguese = Speller('pt') spell_greek = Speller('el') spell_czech = Speller('cs') spell_ukrainian = Speller('uk') spell_vietnamese = Speller('vi') # Fast mode - only corrects single typos (much faster for real-time applications) spell_fast = Speller(fast=True) # OCR mode - only considers character replacements (no insertions/deletions) spell_ocr = Speller(only_replacements=True) # With word frequency threshold - filters out rare words spell_threshold = Speller(threshold=100) # Custom word frequency dictionary custom_words = {'customterm': 1000, 'anotherword': 500, 'hello': 10000} spell_custom = Speller(nlp_data=custom_words) ``` ``` -------------------------------- ### Configure Fast Mode for Real-Time Applications Source: https://context7.com/filyp/autocorrect/llms.txt Enable fast mode to prioritize speed by correcting only single typos, suitable for chatbots. ```python from autocorrect import Speller # Standard mode - handles double typos but slower spell_standard = Speller() result1 = spell_standard("consiousnes") print(result1) # Output: "consciousness" (takes ~200ms) # Fast mode - single typos only, much faster spell_fast = Speller(fast=True) result2 = spell_fast("sleapy") print(result2) # Output: "sleepy" (takes ~300μs) # Fast mode won't correct double typos result3 = spell_fast("consiousnes") print(result3) # Output: "consiousnes" (unchanged, but very fast) # Ideal for chatbots and real-time text processing spell_chatbot = Speller(lang='en', fast=True) def process_user_input(user_text): """Process user input with fast spelling correction.""" return spell_chatbot(user_text) # Process messages quickly message = "Helo, I woud like to ordr a pizza" corrected = process_user_input(message) print(corrected) # Output: "Hello, I would like to order a pizza" ``` -------------------------------- ### Autocorrect Sentences Source: https://context7.com/filyp/autocorrect/llms.txt Shows how to use the Speller instance to correct entire sentences, preserving punctuation and capitalization. ```APIDOC ## Autocorrect Sentences The Speller instance can be called directly on text strings to correct all words in a sentence. It preserves punctuation, capitalization patterns, and non-alphabetic characters while correcting misspelled words. ### Request Example ```python from autocorrect import Speller spell = Speller() # Correct full sentences text = "I'm not sleapy and tehre is no place I'm giong to." corrected = spell(text) print(corrected) # Output: "I'm not sleepy and there is no place I'm going to." # Using autocorrect_sentence method explicitly text2 = "There is no comin to consiousnes without pain." corrected2 = spell.autocorrect_sentence(text2) print(corrected2) # Output: "There is no coming to consciousness without pain." # Multi-language sentence correction spell_pl = Speller('pl') polish_text = "ptaaki latatją kluczmm" corrected_pl = spell_pl(polish_text) print(corrected_pl) # Output: "ptaki latają kluczem" # Spanish sentence correction spell_es = Speller('es') spanish_text = "Fenomenos meteorologicos son extructura" corrected_es = spell_es(spanish_text) print(corrected_es) # Output: "Fenómenos meteorológicos son estructura" ``` ``` -------------------------------- ### Initialize Speller Class Source: https://context7.com/filyp/autocorrect/llms.txt Instantiate the Speller class for various languages and configurations. Default is English. Options include fast mode, OCR mode, custom thresholds, and custom dictionaries. ```python from autocorrect import Speller # Basic English speller (default) spell = Speller() # Speller for a specific language spell_polish = Speller('pl') spell_spanish = Speller('es') spell_russian = Speller('ru') spell_french = Speller('fr') spell_italian = Speller('it') spell_turkish = Speller('tr') spell_portuguese = Speller('pt') spell_greek = Speller('el') spell_czech = Speller('cs') spell_ukrainian = Speller('uk') spell_vietnamese = Speller('vi') # Fast mode - only corrects single typos (much faster for real-time applications) spell_fast = Speller(fast=True) # OCR mode - only considers character replacements (no insertions/deletions) spell_ocr = Speller(only_replacements=True) # With word frequency threshold - filters out rare words spell_threshold = Speller(threshold=100) # Custom word frequency dictionary custom_words = {'customterm': 1000, 'anotherword': 500, 'hello': 10000} spell_custom = Speller(nlp_data=custom_words) ``` -------------------------------- ### Find Correction Threshold Source: https://github.com/filyp/autocorrect/blob/master/README.md Run the test suite to determine the optimal threshold for word correction. ```bash python test_all.py find_threshold ru ``` -------------------------------- ### Run Autocorrection Logic Tests Source: https://github.com/filyp/autocorrect/blob/master/CONTRIBUTING.md Execute quality and benchmark tests for autocorrection logic. Run this command before and after making changes to assess their impact. ```bash python test_all.py quality ; python test_all.py benchmark ``` -------------------------------- ### Custom Word Sets for Autocorrection Source: https://github.com/filyp/autocorrect/blob/master/README.md Initialize Speller with a custom dictionary of word frequencies using the nlp_data argument. ```python spell = Speller(nlp_data=your_word_frequency_dict) ``` -------------------------------- ### Manage Custom Word Dictionaries Source: https://context7.com/filyp/autocorrect/llms.txt Inject domain-specific vocabularies or modify the dictionary dynamically to improve correction accuracy. ```python from autocorrect import Speller # Use completely custom dictionary medical_terms = { 'acetaminophen': 50000, 'ibuprofen': 45000, 'paracetamol': 40000, 'aspirin': 60000, 'medication': 100000, 'prescription': 80000, 'diagnosis': 70000, 'symptom': 65000, 'treatment': 90000, } spell_medical = Speller(nlp_data=medical_terms) # Correct medical terms print(spell_medical.autocorrect_word("acetminophen")) # Output: "acetaminophen" # Modify default dictionary after initialization spell = Speller('en') # Add custom words spell.nlp_data['customterm'] = 10000 spell.nlp_data['brandname'] = 8000 # Now custom words are recognized print(spell.autocorrect_word("custmterm")) # Output: "customterm" # Combine with threshold to filter rare words spell_filtered = Speller(threshold=50) # Only words with frequency >= 50 are considered ``` -------------------------------- ### Retrieve Spelling Candidates and Check Existence Source: https://context7.com/filyp/autocorrect/llms.txt Retrieve potential corrections for a word or verify if a word exists in the dictionary. ```python candidates = spell.get_candidates("definately") print(candidates) # First candidate is most likely the intended word # Correct word returns itself with its frequency candidates = spell.get_candidates("hello") print(candidates) # Output: [(frequency, 'hello')] # Check if a specific word exists in dictionary spell_en = Speller() exists = spell_en.existing({'hello', 'xyznotaword'}) print(exists) # Output: {'hello'} ``` -------------------------------- ### Decompress Wikipedia Dump Source: https://github.com/filyp/autocorrect/blob/master/README.md Decompress the downloaded bzip2 Wikipedia archive file. ```bash bzip2 -d ruiwiki-latest-pages-articles.xml.bz2 ``` -------------------------------- ### Compress Word Count Data Source: https://github.com/filyp/autocorrect/blob/master/README.md Package the generated word_count.json into a tar.gz archive for the language. ```bash tar -zcvf autocorrect/data/ru.tar.gz word_count.json ``` -------------------------------- ### Build Custom Language Dictionaries Source: https://context7.com/filyp/autocorrect/llms.txt Uses the `count_words` function to create word frequency dictionaries from text corpora, essential for adding new language support. Supports custom encoding and output filenames. ```python from autocorrect.word_count import count_words # Count words from a text file to create frequency dictionary # Typically used with Wikipedia dumps or large text corpora # Example: Process a text file for Russian language # count_words('ruwiki-latest-pages-articles.xml', 'ru') # Creates word_count.json with word frequencies # For creating language support: # 1. Download Wikipedia dump: https://dumps.wikimedia.org/ # 2. Extract: bzip2 -d xxwiki-latest-pages-articles.xml.bz2 # 3. Process with count_words # 4. Package: tar -zcvf autocorrect/data/xx.tar.gz word_count.json # Custom encoding support # count_words('mytext.txt', 'en', encd='utf-8', out_filename='custom_freq.json') # The output JSON format: # { # "the": 5000000, # "and": 4500000, # "is": 4000000, # ... # } ``` -------------------------------- ### Apply Word Frequency Thresholds Source: https://context7.com/filyp/autocorrect/llms.txt Filter out rare words by setting a frequency threshold to improve correction relevance. ```python from autocorrect import Speller # Default speller - uses all words spell_all = Speller('en') # Speller with threshold - only common words (frequency >= 100) spell_common = Speller('en', threshold=100) # Speller with high threshold - very common words only spell_very_common = Speller('en', threshold=1000) # Useful for different use cases: # - Low threshold: Technical or specialized text # - High threshold: Casual/informal text correction ``` -------------------------------- ### Autocorrect Single Words Source: https://context7.com/filyp/autocorrect/llms.txt Explains how to correct individual words using the `autocorrect_word` method, including handling capitalization and unknown words. ```APIDOC ## Autocorrect Single Words The `autocorrect_word` method corrects individual words, handling capitalization automatically. It returns the original word unchanged if no correction is found or if the word is already correct. ### Request Example ```python from autocorrect import Speller spell = Speller() # Correct single words print(spell.autocorrect_word("accomodation")) # Output: "accommodation" print(spell.autocorrect_word("basicaly")) # Output: "basically" print(spell.autocorrect_word("benifit")) # Output: "benefit" print(spell.autocorrect_word("definately")) # Output: "definitely" # Capitalized words are preserved print(spell.autocorrect_word("Definately")) # Output: "Definitely" # Correct words are unchanged print(spell.autocorrect_word("correct")) # Output: "correct" # Unknown words are returned as-is print(spell.autocorrect_word("xyzabc")) # Output: "xyzabc" # Empty strings handled gracefully print(spell.autocorrect_word("")) # Output: "" # Special cases - uppercase and camelCase preserved print(spell.autocorrect_word("USA")) # Output: "USA" print(spell.autocorrect_word("camelCased")) # Output: "camelCased" ``` ``` -------------------------------- ### Count Words from Wikipedia Dump Source: https://github.com/filyp/autocorrect/blob/master/README.md Process the decompressed XML file to generate word counts for the specified language. ```python >>> from autocorrect.word_count import count_words >>> count_words('ruwiki-latest-pages-articles.xml', 'ru') ``` -------------------------------- ### Test Optimal Threshold for a Language Source: https://context7.com/filyp/autocorrect/llms.txt Tests different threshold values for a given language to find the optimal setting for spell correction. Requires a dictionary of test words with their typos. ```python def test_threshold(lang, test_words): """Test different thresholds to find optimal value.""" results = [] for threshold in [0, 10, 50, 100, 500, 1000]: spell = Speller(lang, threshold=threshold) correct = sum(1 for word, typo in test_words.items() if spell(typo) == word) results.append((threshold, correct, len(test_words))) return results # Test data test = {'hello': 'helo', 'world': 'wrold', 'python': 'pyhton'} print(test_threshold('en', test)) ``` -------------------------------- ### Autocorrect Only Replacements Source: https://github.com/filyp/autocorrect/blob/master/README.md Initialize Speller with only_replacements=True to focus on correcting replacement errors, often useful for OCR cleanup. ```python spell = Speller(only_replacements=True) ``` -------------------------------- ### Autocorrect Full Sentences Source: https://github.com/filyp/autocorrect/blob/master/README.md Use the Speller class to correct spelling errors in English sentences. Initialize Speller without arguments for English. ```python from autocorrect import Speller spell = Speller() spell("I'm not sleapy and tehre is no place I'm giong to.") ``` -------------------------------- ### Measure Correction Speed Source: https://github.com/filyp/autocorrect/blob/master/README.md Use %timeit to measure the execution time of the spell correction function for a given sentence. ```python %timeit spell("I'm not sleapy and tehre is no place I'm giong to.") ``` ```python %timeit spell("There is no comin to consiousnes without pain.") ``` -------------------------------- ### Fast Autocorrect Mode Source: https://github.com/filyp/autocorrect/blob/master/README.md Initialize Speller with fast=True for microsecond corrections. Note that words with double typos might not be corrected in this mode. ```python spell = Speller(fast=True) %timeit spell("There is no comin to consiousnes without pain.") ``` -------------------------------- ### Optimize for OCR Error Correction Source: https://context7.com/filyp/autocorrect/llms.txt Use only_replacements mode to focus on character substitution errors common in OCR output. ```python from autocorrect import Speller # OCR mode - only considers character replacements spell_ocr = Speller(only_replacements=True) # Common OCR errors (character substitutions) print(spell_ocr.autocorrect_word("heaith")) # Output: "health" (i->l) print(spell_ocr.autocorrect_word("mentat")) # Output: "mental" (t->l) print(spell_ocr.autocorrect_word("aiso")) # Output: "also" (i->l) print(spell_ocr.autocorrect_word("arder")) # Output: "order" (a->o) print(spell_ocr.autocorrect_word("shauld")) # Output: "should" (a->o) # Full document processing for OCR output def clean_ocr_text(ocr_text): """Clean up OCR output with replacement-only corrections.""" spell = Speller(only_replacements=True) return spell(ocr_text) ocr_output = "The heaith of the mentat patient was aiso improving" cleaned = clean_ocr_text(ocr_output) print(cleaned) # Output: "The health of the mental patient was also improving" ``` -------------------------------- ### Generate Word Typo Variations Source: https://context7.com/filyp/autocorrect/llms.txt Generates single and double typo variations for a given word using language-specific alphabets. Supports OCR-only mode for replacements. ```python from autocorrect.typos import Word # Generate single typos for a word word = Word("cat", "en") # Get all possible single-typo variations single_typos = list(word.typos()) print(f"Single typos for 'cat': {len(single_typos)} variations") # Includes: 'at', 'ct', 'ca' (deletions) # 'act', 'cta' (transpositions) # 'aat', 'bat', 'dat'... (replacements) # 'acat', 'bcat'..., 'caat', 'cabt'... (insertions) # Generate double typos (two errors) double_typos = list(word.double_typos()) print(f"Double typos for 'cat': {len(double_typos)} variations") # Language-specific alphabets are used word_polish = Word("kot", "pl") polish_typos = list(word_polish.typos()) # Uses Polish alphabet including: ę, ó, ą, ś, ł, ż, ź, ć, ń # OCR-only mode (replacements only) word_ocr = Word("test", "en", only_replacements=True) ocr_typos = list(word_ocr.typos()) print(f"OCR typos (replacements only): {len(ocr_typos)} variations") # Only includes replacement variants, not deletions/insertions ``` -------------------------------- ### Autocorrect Full Sentences Source: https://context7.com/filyp/autocorrect/llms.txt Correct all words within a sentence using the Speller instance directly or the `autocorrect_sentence` method. Punctuation, capitalization, and non-alphabetic characters are preserved. Supports multi-language correction. ```python from autocorrect import Speller spell = Speller() # Correct full sentences text = "I'm not sleapy and tehre is no place I'm giong to." corrected = spell(text) print(corrected) # Output: "I'm not sleepy and there is no place I'm going to." # Using autocorrect_sentence method explicitly text2 = "There is no comin to consiousnes without pain." corrected2 = spell.autocorrect_sentence(text2) print(corrected2) # Output: "There is no coming to consciousness without pain." # Multi-language sentence correction spell_pl = Speller('pl') polish_text = "ptaaki latatją kluczmm" corrected_pl = spell_pl(polish_text) print(corrected_pl) # Output: "ptaki latają kluczem" # Spanish sentence correction spell_es = Speller('es') spanish_text = "Fenomenos meteorologicos son extructura" corrected_es = spell_es(spanish_text) print(corrected_es) # Output: "Fenómenos meteorológicos son estructura" ``` -------------------------------- ### Autocorrect Single Words Source: https://context7.com/filyp/autocorrect/llms.txt Correct individual words using the `autocorrect_word` method. Capitalization is preserved, and correct or unknown words are returned as-is. Handles empty strings and special cases like all-caps or camelCase. ```python from autocorrect import Speller spell = Speller() # Correct single words print(spell.autocorrect_word("accomodation")) # Output: "accommodation" print(spell.autocorrect_word("basicaly")) # Output: "basically" print(spell.autocorrect_word("benifit")) # Output: "benefit" print(spell.autocorrect_word("definately")) # Output: "definitely" # Capitalized words are preserved print(spell.autocorrect_word("Definately")) # Output: "Definitely" # Correct words are unchanged print(spell.autocorrect_word("correct")) # Output: "correct" # Unknown words are returned as-is print(spell.autocorrect_word("xyzabc")) # Output: "xyzabc" # Empty strings handled gracefully print(spell.autocorrect_word("")) # Output: "" # Special cases - uppercase and camelCase preserved print(spell.autocorrect_word("USA")) # Output: "USA" print(spell.autocorrect_word("camelCased")) # Output: "camelCased" ``` -------------------------------- ### Edit Default Word Set Source: https://github.com/filyp/autocorrect/blob/master/README.md Modify the default word frequency data directly by accessing and editing the spell.nlp_data attribute after initialization. ```python spell.nlp_data ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.