### Configure Flexible Morphology Analysis (Informal & Diacritics) in Python Source: https://context7.com/loodos/zemberek-python/llms.txt This code example demonstrates the creation of a highly flexible TurkishMorphology analyzer that combines support for both informal language and the ability to ignore diacritics. It processes a list of test words, handling variations in spelling and formality, and prints the first analysis result for each word. ```python from zemberek.morphology import TurkishMorphology from zemberek.lexicon import RootLexicon # Combined configuration flexible_morphology = TurkishMorphology.builder(RootLexicon.get_default()) \ .use_informal_analysis() \ .ignore_diacritics_in_analysis_() \ .build() # Handles both informal and diacritics issues test_words = ["gelcek", "gidiyo", "yapcak", "okuyo"] print("\nFlexible analysis:") for word in test_words: results = flexible_morphology.analyze(word) if results.analysis_results: print(f"{word:10} -> {results.analysis_results[0].format_string()}") ``` -------------------------------- ### Extract Sentences from Turkish Paragraph with TurkishSentenceExtractor Source: https://context7.com/loodos/zemberek-python/llms.txt This Python code uses the `TurkishSentenceExtractor` to perform sentence boundary detection in Turkish text. It demonstrates extracting sentences from a given paragraph and measures the time taken. The example also shows how the extractor handles text containing abbreviations like 'Dr.' and 'Prof. Dr.' correctly, preventing them from being mistaken as sentence endings. ```python from zemberek import TurkishSentenceExtractor import time # Initialize sentence extractor extractor = TurkishSentenceExtractor() # Extract sentences from paragraph paragraph = """İnsanoğlu aslında ne para ne sevgi ne kariyer ne şöhret ne de çevre ile sonsuza dek mutlu olabilecek bir yapıya sahiptir. Dış kaynaklardan gelebilecek bu mutluluklar sadece belirli bir zaman için insanı mutlu kılıyor. Kişi bu kaynakları elde ettiği zaman belirli bir dönem için kendini iyi hissediyor, ancak alışma dönemine girdiği andan itibaren bu iyilik hali hızla tükeniyor. Mutlu olma sanatının özü bu değildir. Gerçek mutluluk, kişinin her türlü olaya ve duruma karşı kendini pozitif tutarak mutlu hissedebilmesi halidir.""" start = time.time() sentences = extractor.from_paragraph(paragraph) elapsed = time.time() - start print(f"Extracted {len(sentences)} sentences in {elapsed:.4f}s:") print("=" * 70) for i, sentence in enumerate(sentences, 1): print(f"{i}. {sentence}\n") # Handle mixed content with abbreviations mixed_text = "Dr. Ahmet Bey sabah saat 09:00'da geldi. Prof. Dr. Ayşe Hanım da oradaydı." sentences = extractor.from_paragraph(mixed_text) print("\nAbbreviation handling:") for sentence in sentences: print(f" - {sentence}") ``` -------------------------------- ### Configure Informal Morphology Analysis in Python Source: https://context7.com/loodos/zemberek-python/llms.txt This snippet demonstrates how to build a TurkishMorphology object configured to support informal word analysis. It utilizes the builder pattern and requires the RootLexicon. It takes informal word forms as input and outputs analysis results. ```python from zemberek.morphology import TurkishMorphology from zemberek.lexicon import RootLexicon # Create morphology with informal analysis support informal_morphology = TurkishMorphology.builder(RootLexicon.get_default()) \ .use_informal_analysis() \ .build() # Analyze informal word forms informal_word = "yapıyom" # informal form of "yapıyorum" (I'm doing) results = informal_morphology.analyze(informal_word) for result in results: print(result.format_string()) ``` -------------------------------- ### Configure Advanced Turkish Morphology with Zemberek Source: https://context7.com/loodos/zemberek-python/llms.txt This Python snippet introduces the advanced configuration options for Turkish morphology analysis using the zemberek library. It highlights the use of a builder pattern to customize morphology analysis, including settings for informal text processing, handling diacritics, and incorporating custom lexicons for more specialized linguistic analysis. ```python from zemberek import TurkishMorphology from zemberek.morphology.lexicon import RootLexicon # Example of advanced morphology configuration (code to be added) ``` -------------------------------- ### TurkishMorphology: Core Morphological Analysis in Python Source: https://context7.com/loodos/zemberek-python/llms.txt Demonstrates the core Turkish morphological analysis capabilities using TurkishMorphology. It covers single word analysis, sentence analysis, ambiguity resolution, and combined analysis. Initialization requires default lexicons. ```python from zemberek import TurkishMorphology # Initialize morphology instance with default lexicon morphology = TurkishMorphology.create_with_defaults() # Single word morphological analysis results = morphology.analyze("kalemin") for result in results: print(result.format_string()) # Output shows morphological breakdown with stems and suffixes # [kalem:Noun] kal:Noun+em:A3sg+Pnon+Gen # [kalem:Noun] kal:Noun+em:A3sg+P2sg+Nom # Sentence analysis with multiple word forms sentence = "Yarın kar yağacak." analysis = morphology.analyze_sentence(sentence) print("\nBefore disambiguation:") for word_analysis in analysis: print(f"Word: {word_analysis.inp}") for single_analysis in word_analysis: print(f" {single_analysis.format_string()}") # Ambiguity resolution using perceptron model disambiguated = morphology.disambiguate(sentence, analysis) print("\nAfter disambiguation:") for best in disambiguated.best_analysis(): print(best.format_string()) # Output: [yarın:Noun,Time] yarın:Noun,Time+A3sg+Pnon+Nom # [kar:Noun] kar:Noun+A3sg+Pnon+Nom # [yağ:Verb] yağ:Verb+Fut+A3sg # Combined analysis and disambiguation result = morphology.analyze_and_disambiguate("Kitapları okuyorum.") for analysis in result.best_analysis(): print(analysis.format_string()) ``` -------------------------------- ### Tokenize Turkish Text with TurkishTokenizer Source: https://context7.com/loodos/zemberek-python/llms.txt This Python code demonstrates tokenizing Turkish text using the `TurkishTokenizer` from the zemberek library. It shows how to use the default tokenizer to break text into tokens of various types (Word, Time, Punctuation, Email, etc.) and their positions. It also illustrates building a custom tokenizer that can ignore specific token types like punctuation. ```python from zemberek import TurkishTokenizer from zemberek.tokenization.token import Token # Use default tokenizer (ignores newlines and spaces) tokenizer = TurkishTokenizer.DEFAULT # Tokenize text with various token types text = "Saat 12:00. Email: user@example.com. Fiyat: %15. #Python @zemberek" tokens = tokenizer.tokenize(text) print("Token Analysis:") print("-" * 70) for token in tokens: print(f"Content: {token.content:20} | Type: {token.type_.name:20} | Pos: {token.start}-{token.end}") # Build custom tokenizer with specific token type filters custom_tokenizer = TurkishTokenizer.builder() \ .accept_all() \ .ignore_types([Token.Type.Punctuation, Token.Type.SpaceTab]) \ .build() tokens = custom_tokenizer.tokenize("Merhaba! Nasılsın?") print("\nCustom tokenizer (no punctuation):") for token in tokens: print(f" {token.content}") ``` -------------------------------- ### TurkishSpellChecker: Spelling Correction and Suggestions in Python Source: https://context7.com/loodos/zemberek-python/llms.txt Illustrates how to use TurkishSpellChecker for correcting misspelled Turkish words and providing suggestions. It requires an initialized TurkishMorphology instance and uses character graph matching and language models. ```python from zemberek import TurkishMorphology, TurkishSpellChecker # Initialize spell checker with morphology instance morphology = TurkishMorphology.create_with_defaults() spell_checker = TurkishSpellChecker(morphology) # Get spelling suggestions for misspelled words word = "okuyablirim" suggestions = spell_checker.suggest_for_word(word) print(f"{word} -> {' | '.join(suggestions)}") # Output: okuyablirim -> okuyabilirim | okuyabilir # Multiple word corrections misspelled_words = [ "tartısıyor", # correct: tartışıyor "knlıca", # correct: kınalıca "yapablrim", # correct: yapabilirim "kıredi", # correct: kredi "geldm", # correct: geldim "geliyom", # correct: geliyorum "asln" # correct: aslında ] for word in misspelled_words: suggestions = spell_checker.suggest_for_word(word) if suggestions: print(f"{word:15} => {suggestions[0]}") else: print(f"{word:15} => (no suggestions)") ``` -------------------------------- ### Normalize Informal Turkish Sentences with Zemberek Source: https://context7.com/loodos/zemberek-python/llms.txt This code snippet demonstrates how to normalize informal Turkish sentences using the zemberek-python library's normalizer. It handles spelling errors, informal suffixes, ASCII normalization, and word segmentation. The input is a list of informal Turkish strings, and the output is a list of normalized strings along with the time taken for each normalization. ```python from zemberek import Normalizer import time # Initialize the normalizer normalizer = Normalizer() informal_texts = [ "Yrn okua gidicem", # Tomorrow I'll go to school "Tmm, yarin havuza giricem ve aksama kadar yaticam :)", # OK, tomorrow I'll swim "ah aynen ya annemde fark ettı siz evinizden cıkmayın diyo", # Yes, my mom noticed too "gercek mı bu? Yuh! Artık unutulması bile beklenmiyo", # Is this real? "yok hocam kesınlıkle oyle birşey yok", # No sir, definitely nothing like that "email adresim zemberek_python@loodos.com", # my email address "Kredi başvrusu yapmk istiyrum.", # I want to apply for credit ] print("Normalization Results:") print("=" * 70) for text in informal_texts: start = time.time() normalized = normalizer.normalize(text) elapsed = time.time() - start print(f"Original: {text}") print(f"Normalized: {normalized}") print(f"Time: {elapsed:.4f}s\n") # Output handles: # - Spelling errors (yrn -> yarın, gidicem -> gideceğim) # - Informal suffixes (cıkmayın -> çıkmayın) # - ASCII normalization (siz -> siz) # - Word splitting/joining ``` -------------------------------- ### TurkishSentenceNormalizer: Normalizing Noisy Turkish Text in Python Source: https://context7.com/loodos/zemberek-python/llms.txt Shows the initialization of TurkishSentenceNormalizer for cleaning informal Turkish text from sources like social media. It handles spelling errors, informal suffixes, ASCII characters, and abbreviations, requiring a TurkishMorphology instance. ```python from zemberek import TurkishMorphology, TurkishSentenceNormalizer import time # Initialize normalizer with morphology instance morphology = TurkishMorphology.create_with_defaults() normalizer = TurkishSentenceNormalizer(morphology) ``` -------------------------------- ### Configure Diacritics-Tolerant Morphology Analysis in Python Source: https://context7.com/loodos/zemberek-python/llms.txt This snippet shows how to create a TurkishMorphology instance that ignores diacritics during analysis. This is useful for handling words where diacritical marks might be missing. It takes words with potentially missing diacritics as input. ```python from zemberek.morphology import TurkishMorphology from zemberek.lexicon import RootLexicon # Create morphology that ignores diacritics diacritics_tolerant = TurkishMorphology.builder(RootLexicon.get_default()) \ .ignore_diacritics_in_analysis_() \ .build() # Analyze words with missing diacritics word_without_diacritics = "kahvalti" # should be "kahvaltı" results = diacritics_tolerant.analyze(word_without_diacritics) for result in results: print(result.format_string()) # Recognizes "kahvalti" as "kahvaltı" (breakfast) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.