### Getting Lexeme Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Returns a list of all possible parses for a given word form. ```python def get_lexeme(self, form): return list(self._iter_lexeme(form)) ``` -------------------------------- ### Example Word Data and Parse Information Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Illustrates sample words with their associated parse information, represented as (paradigm number, form number) pairs. This data is used to construct the internal dictionary. ```text двор (103, 0) ёж (104, 0) дворник (101, 2) и (102, 2) ёжик (101, 2) и (102, 2) ``` -------------------------------- ### Iterate with Progress Bar Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Returns an iterator that displays progress using the tqdm package. Falls back to returning the original iterable if tqdm is not installed. ```python def with_progress(iterable, desc=None, total=None, leave=True): """ Return an iterator which prints the iteration progress using tqdm package. Return iterable intact if tqdm is not available. """ try: from tqdm import tqdm # workarounds for tqdm bugs def _it(iterable, desc, total, leave): if total is None: try: total = len(iterable) except Exception: total = 0 for el in tqdm(iterable, desc=desc, total=total, leave=leave): yield el if leave: print("") return _it(iterable, desc, total, leave) except ImportError: return iterable ``` -------------------------------- ### pymorphy2.shapes.restore_capitalization Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Adjusts the capitalization of a word to match that of an example word. If alignment fails, the remainder is lowercased. ```APIDOC ## pymorphy2.shapes.restore_capitalization ### Description Make the capitalization of the `word` be the same as in `example`. In the alignment fails, the reminder is lower-cased. ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Request Example ```python >>> pymorphy2.shapes.restore_capitalization('bye', 'Hello') 'Bye' >>> pymorphy2.shapes.restore_capitalization('half-an-hour', 'Minute') 'Half-An-Hour' >>> pymorphy2.shapes.restore_capitalization('usa', 'IEEE') 'USA' >>> pymorphy2.shapes.restore_capitalization('pre-world', 'anti-World') 'pre-World' >>> pymorphy2.shapes.restore_capitalization('123-do', 'anti-IEEE') '123-DO' >>> pymorphy2.shapes.restore_capitalization('123--do', 'anti--IEEE') '123--DO' >>> pymorphy2.shapes.restore_capitalization('foo-BAR-BAZ', 'Baz-Baz') 'Foo-Bar-baz' >>> pymorphy2.shapes.restore_capitalization('foo', 'foo-bar') 'foo' ``` ### Response #### Success Response (200) - type: string - description: The word with adjusted capitalization. ### Response Example ```json { "example": "AdjustedWord" } ``` ``` -------------------------------- ### Restore capitalization of a word Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Applies the capitalization pattern from an example word to a target word. Handles hyphenated words and mixed cases. If alignment fails, the remainder is lowercased. ```python >>> restore_capitalization('bye', 'Hello') 'Bye' >>> restore_capitalization('half-an-hour', 'Minute') 'Half-An-Hour' >>> restore_capitalization('usa', 'IEEE') 'USA' >>> restore_capitalization('pre-world', 'anti-World') 'pre-World' >>> restore_capitalization('123-do', 'anti-IEEE') '123-DO' >>> restore_capitalization('123--do', 'anti--IEEE') '123--DO' ``` ```python >>> restore_capitalization('foo-BAR-BAZ', 'Baz-Baz') 'Foo-Bar-baz' >>> restore_capitalization('foo', 'foo-bar') 'foo' ``` -------------------------------- ### Analyzer.iter_known_word_parses Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Returns an iterator over parses of known dictionary words that start with a specified prefix. Useful for exploring the dictionary content. ```APIDOC ## Analyzer.iter_known_word_parses ### Description Returns an iterator over parses of dictionary words that begin with a given prefix. If no prefix is provided, it iterates over all known words. ### Method `iter_known_word_parses(self, prefix="")` ### Parameters * **prefix** (string, optional) - The prefix to filter dictionary words. Defaults to an empty string, meaning all words. ### Returns An iterator yielding parses of known dictionary words. ``` -------------------------------- ### Getting Normalized Form Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Returns the normalized form of a given parsed word form. ```python def normalized(self, form): return next(self._iter_lexeme(form)) ``` -------------------------------- ### Number Analyzer (NumberAnalyzer) Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html Identifies and tags integer and real numbers. Integers get 'NUMB,intg' and real numbers get 'NUMB,real'. Note that this is distinct from 'NUMR' which is for number words like 'тридцать'. ```python class NumberAnalyzer(_ShapeAnalyzer): """ This analyzer marks integer numbers with "NUMB,int" or "NUMB,real" tags. Example: "12" -> NUMB,int; "12.4" -> NUMB,real .. note:: Don't confuse it with "NUMR": "тридцать" -> NUMR """ EXTRA_GRAMMEMES = ['NUMB', 'intg', 'real'] EXTRA_GRAMMEMES_CYR = ['ЧИСЛО', 'цел', 'вещ'] def init(self, morph): super(NumberAnalyzer, self).init(morph) self._tags = { 'intg': morph.TagClass('NUMB,intg'), 'real': morph.TagClass('NUMB,real'), } def check_shape(self, word, word_lower): try: int(word) return 'intg' except ValueError: try: float(word.replace(',', '.')) return 'real' except ValueError: pass return False def get_tag(self, word, shape): return self._tags[shape] ``` -------------------------------- ### Get Memory Usage Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Returns the memory usage of the current process in bytes. Requires the psutil Python package. ```python import os import heapq import itertools import codecs import json def get_mem_usage(): """ Return memory usage of the current process, in bytes. Requires psutil Python package. """ import psutil proc = psutil.Process(os.getpid()) try: return proc.memory_info().rss except AttributeError: # psutil < 2.x return proc.get_memory_info()[0] ``` -------------------------------- ### Iterate known word parses Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Returns an iterator over parses of dictionary words that start with a given prefix. If no prefix is provided, it iterates over all words in the dictionary. ```python def iter_known_word_parses(self, prefix=""): """ Return an iterator over parses of dictionary words that starts with a given prefix (default empty prefix means "all words"). """ # XXX: this method currently assumes that # units.DictionaryAnalyzer is the first analyzer unit. for word, tag, normal_form, para_id, idx in self.dictionary.iter_known_words(prefix): methods = ((self._units[0][0], word, para_id, idx),) parse = (word, tag, normal_form, 1.0, methods) if self._result_type is None: yield parse else: yield self._result_type(*parse) ``` -------------------------------- ### Parsing as Variable Both Parts Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Parses a hyphenated word when both the left and right parts can be inflected. This is used for examples like человек-гора, команд-участниц, компания-производитель. ```python def _parse_as_variable_both(self, left_parses, right_parses, seen): """ Step 2: if left and right can be parsed the same way, then it may be the case that both parts should be inflected. Examples: человек-гора, команд-участниц, компания-производитель """ result = [] right_features = [self._similarity_features(p[1]) for p in right_parses] # FIXME: quadratic algorithm for left_parse in left_parses: left_tag = left_parse[1] if left_tag._is_unknown(): continue left_feat = self._similarity_features(left_tag) for parse_index, right_parse in enumerate(right_parses): right_feat = right_features[parse_index] if left_feat != right_feat: continue left_methods = left_parse[4] right_methods = right_parse[4] new_methods_stack = ((self, left_methods, right_methods),) # tag parse = ( '-'.join((left_parse[0], right_parse[0])), # word left_tag, '-'.join((left_parse[2], right_parse[2])), # normal form left_parse[3] * self.score_multiplier, new_methods_stack ) result.append(parse) # add_parse_if_not_seen(parse, result, seen_right) return result ``` -------------------------------- ### Get TagClass Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Provides access to the TagClass used by the dictionary, typically pymorphy2.tagset.OpencorporaTag. ```python @property def TagClass(self): """ :rtype: pymorphy2.tagset.OpencorporaTag """ return self.dictionary.Tag ``` -------------------------------- ### Parsing as Fixed Left Part Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Assumes the left part of a hyphenated word is an immutable prefix and parses the right part. Examples: интернет-магазин, воздушно-капельный. ```python def _parse_as_fixed_left(self, right_parses, seen, left): """ Step 1: Assume that the left part is an immutable prefix. Examples: интернет-магазин, воздушно-капельный """ result = [] for fixed_word, tag, normal_form, score, right_methods in right_parses: if tag._is_unknown(): continue new_methods_stack = ((self, left, right_methods),) parse = ( '-'.join((left, fixed_word)), tag, '-'.join((left, normal_form)), score * self.score_multiplier, new_methods_stack ) result.append(parse) # add_parse_if_not_seen(parse, result, seen_left) return result ``` -------------------------------- ### Get lexeme for a form Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Retrieves the lexeme associated with a given parse form. This is useful for understanding the inflectional paradigm of a word. ```python def get_lexeme(self, form): """ Return the lexeme this parse belongs to. """ methods_stack = form[4] last_method = methods_stack[-1] result = last_method[0].get_lexeme(form) if self._result_type is None: return result return [self._result_type(*p) for p in result] ``` -------------------------------- ### Get normal forms of a word Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Returns a list of unique normal forms for a given word. It utilizes the parse method to find all possible forms and extracts their normal forms. ```python def normal_forms(self, word): """ Return a list of word normal forms. """ seen = set() result = [] for p in self.parse(word): normal_form = p[2] if normal_form not in seen: result.append(normal_form) seen.add(normal_form) return result ``` -------------------------------- ### Build Paradigm Info Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Retrieves paradigm information as a list of (prefix, tag, suffix) tuples for a given paradigm ID. ```python build_paradigm_info(_para_id_) ``` -------------------------------- ### Initialize OpenCorpora Dictionary Wrapper Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Instantiates the Dictionary wrapper for OpenCorpora dictionaries. Requires the path to the dictionary folder. ```python pymorphy2.opencorpora_dict.wrapper.Dictionary(_path_) ``` -------------------------------- ### HyphenatedWordsAnalyzer Init Method Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Initializes the analyzer with the morph object and sets up feature grammemes and prefix checking. ```python def init(self, morph): super(HyphenatedWordsAnalyzer, self).init(morph) Tag = morph.TagClass self._FEATURE_GRAMMEMES = (Tag.PARTS_OF_SPEECH | Tag.NUMBERS | Tag.CASES | Tag.PERSONS | Tag.TENSES) self._has_skip_prefix = PrefixMatcher(self.skip_prefixes).is_prefixed ``` -------------------------------- ### Extracting Word Paradigms Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Demonstrates the initial tokenization of a word ('хомяковый') into its lemma and associated grammatical tags. ```text СЛОВО ТЕГ хомяковый ADJF,Qual masc,sing,nomn хомякового ADJF,Qual masc,sing,gent ... хомяковы ADJS,Qual plur хомяковее COMP,Qual хомяковей COMP,Qual V-ej похомяковее COMP,Qual Cmp2 похомяковей COMP,Qual Cmp2,V-ej ``` -------------------------------- ### pymorphy2 CLI Options Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Lists the available options for the pymorphy2 command-line tool, including lemmatization, scoring, tagging, caching, and language selection. ```bash -l --lemmatize Include normal forms (lemmas) ``` ```bash -s --score Include non-contextual P(tag|word) scores ``` ```bash -t --tag Include tags ``` ```bash --thresh Drop all results with estimated P(tag|word) less than a threshold [default: 0.0] ``` ```bash --tokenized Assume that input text is already tokenized: one token per line. ``` ```bash -c --cache Cache size, in entries. Set it to 0 to disable cache; use 'unlim' value for unlimited cache size [default: 20000] ``` ```bash --lang Language to use. Allowed values: ru, uk [default: ru] ``` ```bash --dict Dictionary folder path ``` ```bash -v --verbose Be more verbose ``` ```bash -h --help Show this help ``` -------------------------------- ### pymorphy2 Command-Line Usage Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Shows the available commands and options for the pymorphy2 command-line interface. Use 'pymorphy -h' for detailed help. ```bash pymorphy parse [options] [] ``` ```bash pymorphy dict meta [--lang | --dict ] ``` ```bash pymorphy dict mem_usage [--lang | --dict ] [--verbose] ``` ```bash pymorphy -h | --help ``` ```bash pymorphy --version ``` -------------------------------- ### Initialize Grammemes and Incompatibilities Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html This method initializes class attributes like GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE based on provided grammeme data. It also expands EXTRA_INCOMPATIBLE sets. ```python # figure out parents & children children = collections.defaultdict(set) for index, (name, parent, alias, description) in enumerate(dict_grammemes): if parent: children[parent].add(name) if gr.get(parent, None): # parent's parent children[gr[parent]].add(name) # expand EXTRA_INCOMPATIBLE for grammeme, g_set in cls._EXTRA_INCOMPATIBLE.items(): for g in g_set.copy(): g_set.update(children[g]) # fill GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE for index, (name, parent, alias, description) in enumerate(dict_grammemes): cls._GRAMMEME_INDICES[name] = index incompatible = cls._EXTRA_INCOMPATIBLE.get(name, set()) incompatible = (incompatible | children[parent]) - set([name]) cls._GRAMMEME_INCOMPATIBLE[name] = frozenset(incompatible) ``` -------------------------------- ### Build Tag Info Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Returns the morphological tag as a string for a specific word index within a paradigm. ```python build_tag_info(_para_id_ , _idx_) ``` -------------------------------- ### Initialize Grammemes from Dictionary Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Class method to initialize class attributes like KNOWN_GRAMMEMES, _LAT2CYR, and _CYR2LAT from a provided dictionary of grammeme information. The input is expected to be a list of tuples. ```python @classmethod def _init_grammemes(cls, dict_grammemes): """ Initialize various class attributes with grammeme information obtained from XML dictionary. ``dict_grammemes`` is a list of tuples:: [ (name, parent, alias, description), ... ] """ cls.KNOWN_GRAMMEMES = set() cls._CYR2LAT = {} cls._LAT2CYR = {} for name, parent, alias, description in dict_grammemes: cls.add_grammemes_to_known(name, alias) gr = dict((name, parent) for (name, parent, alias, description) in dict_grammemes) ``` -------------------------------- ### Get Largest Elements by Key Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Returns a list of elements from an iterable that have one of the top 'n' key values. Useful for retrieving elements based on a specific criterion. ```python def largest_elements(iterable, key, n=1): """ Return a list of large elements of the ``iterable`` (according to ``key`` function). ``n`` is a number of top element values to consider; when n==1 (default) only largest elements are returned; when n==2 - elements with one of the top-2 values, etc. >>> s = [-4, 3, 5, 7, 4, -7] >>> largest_elements(s, abs) [7, -7] >>> largest_elements(s, abs, 2) [5, 7, -7] >>> largest_elements(s, abs, 3) [-4, 5, 7, 4, -7] """ it1, it2 = itertools.tee(iterable) top_keys = set(heapq.nlargest(n, set(map(key, it1)))) return [el for el in it2 if key(el) in top_keys] ``` -------------------------------- ### Split Word into Prefix and Suffix Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Returns all possible splits of a word into a prefix and suffix, respecting minimum reminder and maximum prefix length constraints. ```python def word_splits(word, min_reminder=3, max_prefix_length=5): """ Return all splits of a word (taking in account min_reminder and max_prefix_length). """ max_split = min(max_prefix_length, len(word)-min_reminder) split_indexes = range(1, 1+max_split) return [(word[:i], word[i:]) for i in split_indexes] ``` -------------------------------- ### Initialize Cyrillic Alias Map Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Initializes the _GRAMMEME_ALIAS_MAP by iterating through the dictionary of grammemes and storing the mapping from grammeme name to its Cyrillic alias. ```python @classmethod def _init_alias_map(cls, dict_grammemes): for name, parent, alias, description in dict_grammemes: cls._GRAMMEME_ALIAS_MAP[name] = alias ``` -------------------------------- ### Assert Grammemes Initialized Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Internal class method to ensure that the grammeme data structures have been initialized. Raises RuntimeError if not. ```python @classmethod def _assert_grammemes_initialized(cls): if not cls.KNOWN_GRAMMEMES: msg = "The class was not properly initialized." raise RuntimeError(msg) ``` -------------------------------- ### Write JSON to File Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Creates a file with the given object serialized to JSON. Sets default options for ensuring ASCII is false and indentation is 2. ```python def json_write(filename, obj, **json_options): """ Create file ``filename`` with ``obj`` serialized to JSON """ json_options.setdefault('ensure_ascii', False) json_options.setdefault('indent', 2) with codecs.open(filename, 'w', 'utf8') as f: json.dump(obj, f, **json_options) ``` -------------------------------- ### Get largest elements by key Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Returns a list of elements from an iterable that have the top 'n' largest values according to a specified key function. Defaults to returning the single largest element. ```python >>> s = [-4, 3, 5, 7, 4, -7] >>> largest_elements(s, abs) [7, -7] ``` ```python >>> largest_elements(s, abs, 2) [5, 7, -7] ``` ```python >>> largest_elements(s, abs, 3) [-4, 5, 7, 4, -7] ``` -------------------------------- ### HyphenatedWordsAnalyzer Initialization Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html Initializes the HyphenatedWordsAnalyzer with a list of prefixes to skip and an optional score multiplier. ```python def __init__(self, skip_prefixes, score_multiplier=0.75): self.score_multiplier = score_multiplier self.skip_prefixes = skip_prefixes ``` -------------------------------- ### PyMorphy2 Command-Line Interface Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html The PyMorphy2 command-line interface allows users to parse text, manage dictionaries, and access help information. ```APIDOC ## Command-Line Interface Usage: ``` pymorphy parse [options] [] pymorphy dict meta [--lang | --dict ] pymorphy dict mem_usage [--lang | --dict ] [--verbose] pymorphy -h | --help pymorphy --version ``` Options: ``` -l --lemmatize Include normal forms (lemmas) -s --score Include non-contextual P(tag|word) scores -t --tag Include tags --thresh Drop all results with estimated P(tag|word) less than a threshold [default: 0.0] --tokenized Assume that input text is already tokenized: one token per line. -c --cache Cache size, in entries. Set it to 0 to disable cache; use 'unlim' value for unlimited cache size [default: 20000] --lang Language to use. Allowed values: ru, uk [default: ru] --dict Dictionary folder path -v --verbose Be more verbose -h --help Show this help ``` ``` -------------------------------- ### Paradigm Structure: Prefix, Stem, Suffix Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Illustrates how a word's paradigm can be broken down into prefix, stem, and suffix components, with the stem being common across all forms. ```text ПРЕФИКС СТЕМ ХВОСТ ТЕГ хомяков ый ADJF,Qual masc,sing,nomn хомяков ого ADJF,Qual masc,sing,gent ... хомяков ы ADJS,Qual plur хомяков ее COMP,Qual хомяков ей COMP,Qual V-ej по хомяков ее COMP,Qual Cmp2 по хомяков ей COMP,Qual Cmp2,V-ej ``` -------------------------------- ### Initialize Cyrillic Grammemes Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Initializes class attributes for Cyrillic tags, including mapping internal grammemes to their Cyrillic aliases and updating GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE accordingly. ```python @classmethod def _init_grammemes(cls, dict_grammemes): """ Initialize various class attributes with grammeme information obtained from XML dictionary. """ cls._init_alias_map(dict_grammemes) super(CyrillicOpencorporaTag, cls)._init_grammemes(dict_grammemes) GRAMMEME_INDICES = collections.defaultdict(int) for name, idx in cls._GRAMMEME_INDICES.items(): GRAMMEME_INDICES[cls._from_internal_grammeme(name)] = idx cls._GRAMMEME_INDICES = GRAMMEME_INDICES GRAMMEME_INCOMPATIBLE = collections.defaultdict(set) for name, value in cls._GRAMMEME_INCOMPATIBLE.items(): GRAMMEME_INCOMPATIBLE[cls._from_internal_grammeme(name)] = set([ cls._from_internal_grammeme(gr) for gr in value ]) cls._GRAMMEME_INCOMPATIBLE = GRAMMEME_INCOMPATIBLE cls._NON_PRODUCTIVE_GRAMMEMES = set([ cls._from_internal_grammeme(gr) for gr in cls._NON_PRODUCTIVE_GRAMMEMES ]) ``` -------------------------------- ### Generate Combinations of All Lengths Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Returns an iterable with all possible combinations of items from an input iterable. Demonstrates generating combinations of increasing lengths. ```python def combinations_of_all_lengths(it): """ Return an iterable with all possible combinations of items from ``it``: >>> for comb in combinations_of_all_lengths('ABC'): ... print("".join(comb)) A B C AB AC BC ABC """ return itertools.chain( *(itertools.combinations(it, num+1) for num in range(len(it)))) ``` -------------------------------- ### Check if Grammeme is Known Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Class method to verify if a given grammeme is recognized by the system. Asserts that grammemes have been initialized. ```python @classmethod def grammeme_is_known(cls, grammeme): cls._assert_grammemes_initialized() return grammeme in cls.KNOWN_GRAMMEMES ``` -------------------------------- ### Building Tag Information Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Reconstructs grammatical tag information from paradigm data using paradigm ID and word index. It accesses the paradigm array and extracts the tag ID. ```python def _build_tag_info(self, para_id, idx): # получаем массив с данными парадигмы paradigm = self._dictionary.paradigms[para_id] # индексы грамматической информации начинаются со второй трети # массива с парадигмой tag_info_offset = len(paradigm) // 3 # получаем искомый индекс tag_id = paradigm[tag_info_offset + tag_id_index] # возвращаем соответствующую строку из таблицы с грамматической информацией return self._dictionary.gramtab[tag_id] ``` -------------------------------- ### Build Stem Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Calculates the stem of a word given its paradigm, index, and fixed word form. Useful for analyzing word structure. ```python build_stem(_paradigm_ , _idx_ , _fixed_word_) ``` -------------------------------- ### Less Than Comparison for Tags Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Compares if one tag is lexicographically less than another based on their grammeme tuples. ```python def __lt__(self, other): return self._grammemes_tuple < other._grammemes_tuple ``` -------------------------------- ### Format Keyword Arguments for Representation Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html Generates a string representation of keyword arguments, optionally hiding values for specified keys. Useful for debugging and logging. ```python def kwargs_repr(kwargs=None, dont_show_value=None): """ >>> kwargs_repr(dict(foo="123", a=5, x=8)) "a=5, foo='123', x=8" >>> kwargs_repr(dict(foo="123", a=5, x=8), dont_show_value=['foo']) 'a=5, foo=<...>, x=8' >>> kwargs_repr() '' """ kwargs = kwargs or {} dont_show_value = set(dont_show_value or []) return ", ".join( "%s=%s" % (k, repr(v) if k not in dont_show_value else "<...>") for k, v in sorted(kwargs.items())) ``` -------------------------------- ### Paradigm Data Structure Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Stores paradigms as arrays of integers, referencing suffixes, tag IDs, and prefix IDs. This structure is memory-efficient. ```python paradigms = [ array.array(" (обходом по графу); из этих ключей (из того, что за ) # получаем список кортежей [(para_id1, index1), (para_id2, index2), ...]. # # RecordDAWG из библиотек DAWG или DAWG-Python умеет это делать # одной командой (с возможностью нечеткого поиска для буквы Ё): para_data = self._dictionary.words.similar_items(word, self._ee) # fixed_word - это слово с исправленной буквой Ё, для которого был # проведен разбор. for fixed_word, parse in para_data: for para_id, idx in parse: # по информации о номере парадигмы и номере слова в # парадигме восстанавливаем нормальную форму слова и # грамматическую информацию. tag = self._build_tag_info(para_id, idx) normal_form = self._build_normal_form(para_id, idx, fixed_word) result.append( (fixed_word, tag, normal_form) ) ``` -------------------------------- ### pymorphy2.utils.with_progress Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Returns an iterator that displays iteration progress using the 'tqdm' package. If 'tqdm' is unavailable, the iterable is returned unchanged. ```APIDOC ## pymorphy2.utils.with_progress ### Description Return an iterator which prints the iteration progress using tqdm package. Return iterable intact if tqdm is not available. ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Request Example ```python # Example usage not provided in source ``` ### Response #### Success Response (200) - type: iterator - description: An iterator that yields progress updates. ### Response Example ```json { "example": "" } ``` ``` -------------------------------- ### OpenCorpora Dictionary Wrapper Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Utilities for working with OpenCorpora dictionaries, providing methods to build word forms, retrieve paradigm information, and check word existence. ```APIDOC ## Utilities for OpenCorpora Dictionaries _class_`pymorphy2.opencorpora_dict.wrapper.``Dictionary`(_path_)[source]¶ OpenCorpora dictionary wrapper class. `build_normal_form`(_para_id_ , _idx_ , _fixed_word_)[source]¶ Build a normal form. `build_paradigm_info`(_para_id_)[source]¶ Return a list of > (prefix, tag, suffix) tuples representing the paradigm. `build_stem`(_paradigm_ , _idx_ , _fixed_word_)[source]¶ Return word stem (given a word, paradigm and the word index). `build_tag_info`(_para_id_ , _idx_)[source]¶ Return tag as a string. `iter_known_words`(_prefix=''_)[source]¶ Return an iterator over `(word, tag, normal_form, para_id, idx)` tuples with dictionary words that starts with a given prefix (default empty prefix means “all words”). `word_is_known`(_word_ , _substitutes_compiled=None_)[source]¶ Check if a `word` is in the dictionary. To allow some fuzzyness pass `substitutes_compiled` argument; it should be a result of `DAWG.compile_replaces()`. This way you can e.g. handle ё letters replaced with е in the input words. Note Dictionary words are not always correct words; the dictionary also contains incorrect forms which are commonly used. So for spellchecking tasks this method should be used with extra care. ``` -------------------------------- ### DAWG Graph Representation Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html A DOT language representation of the Deterministic Acyclic Finite State Automaton (DAWG) graph. This graph encodes the words and their parse information, demonstrating how common prefixes are shared and data is compressed. ```dot digraph foo { rankdir=LR; size=9; node [shape = doublecircle]; 10 14; node [shape = circle]; 0 -> 2 [label=Д]; 0 -> 3 [label=Ё]; 1 -> 4 [label=О]; 2 -> 1 [label=В]; 3 -> 16 [label=Ж]; 4 -> 6 [label=Р]; 5 -> 8 [label=К]; 6 -> 7 [label=Н]; 6 -> 22 [label=sep]; 7 -> 5 [label=И]; 8 -> 9 [label=sep]; 9 -> 12 [label="103"]; 9 -> 15 [label="102"]; 12 -> 10 [label="2"]; 13 -> 14 [label="0"]; 15 -> 10 [label="2"]; 16 -> 32 [label=И]; 16 -> 54 [label=sep]; 17 -> 14 [label="2"]; 22 -> 13 [label="103"]; 32 -> 8 [label=К]; 54 -> 17 [label="104"]; } ``` -------------------------------- ### Base Shape Analyzer (_ShapeAnalyzer) Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html Abstract base class for shape-based analyzers. Subclasses must implement `check_shape` and `get_tag` methods. ```python # -*- coding: utf-8 -*- """ Analyzer units that analyzes non-word tokes ------------------------------------------- """ from __future__ import absolute_import, unicode_literals, division from pymorphy2.units.base import BaseAnalyzerUnit from pymorphy2.shapes import is_latin, is_punctuation, is_roman_number class _ShapeAnalyzer(BaseAnalyzerUnit): EXTRA_GRAMMEMES = [] EXTRA_GRAMMEMES_CYR = [] def __init__(self, score=0.9): self.score = score def init(self, morph): super(_ShapeAnalyzer, self).init(morph) for lat, cyr in zip(self.EXTRA_GRAMMEMES, self.EXTRA_GRAMMEMES_CYR): self.morph.TagClass.add_grammemes_to_known(lat, cyr) def parse(self, word, word_lower, seen_parses): shape = self.check_shape(word, word_lower) if not shape: return [] methods = ((self, word),) return [(word_lower, self.get_tag(word, shape), word_lower, self.score, methods)] def tag(self, word, word_lower, seen_tags): shape = self.check_shape(word, word_lower) if not shape: return [] return [self.get_tag(word, shape)] def get_lexeme(self, form): return [form] def normalized(self, form): return form # implement these 2 methods in a subclass: def check_shape(self, word, word_lower): raise NotImplementedError() def get_tag(self, word, shape): raise NotImplementedError() ``` -------------------------------- ### Inequality Comparison for Tags Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Compares two tag objects for inequality based on their grammeme tuples. ```python def __ne__(self, other): return self._grammemes_tuple != other._grammemes_tuple ``` -------------------------------- ### pymorphy2.utils.word_splits Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Generates all possible splits of a word, considering minimum reminder length and maximum prefix length. ```APIDOC ## pymorphy2.utils.word_splits ### Description Return all splits of a word (taking in account min_reminder and max_prefix_length). ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Request Example ```python # Example usage not provided in source ``` ### Response #### Success Response (200) - type: list of tuples - description: A list where each tuple represents a split of the word (prefix, suffix). ### Response Example ```json { "example": [["split1_prefix", "split1_suffix"], ["split2_prefix", "split2_suffix"]] } ``` ``` -------------------------------- ### Equality Comparison for Tags Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Compares two tag objects for equality based on their grammeme tuples. ```python def __eq__(self, other): return self._grammemes_tuple == other._grammemes_tuple ``` -------------------------------- ### Update Grammemes with Required and Incompatible Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Returns a new set of grammemes by adding required grammemes and removing any grammemes incompatible with the required ones. Raises ValueError for unknown required grammemes. ```python def updated_grammemes(self, required): """ Return a new set of grammemes with ``required`` grammemes added and incompatible grammemes removed. """ new_grammemes = self.grammemes | required for grammeme in required: if not self.grammeme_is_known(grammeme): raise ValueError("Unknown grammeme: %s" % grammeme) new_grammemes -= self._GRAMMEME_INCOMPATIBLE[grammeme] return new_grammemes ``` -------------------------------- ### Object Representation of Tag Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Returns the developer-friendly representation of the tag object. ```python def __repr__(self): return "OpencorporaTag('%s')" % self ``` -------------------------------- ### Cyrillic Grammemes Property Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Returns a frozenset of Cyrillic grammemes for a given tag. Caches the result for efficiency. ```python @property def grammemes_cyr(self): """ A frozenset with Cyrillic grammemes for this tag. """ if self._cyr_grammemes_cache is None: cyr_grammemes = [self._LAT2CYR[g] for g in self._grammemes_tuple] self._cyr_grammemes_cache = frozenset(cyr_grammemes) return self._cyr_grammemes_cache ``` -------------------------------- ### pymorphy2.utils.get_mem_usage Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Retrieves the memory usage of the current process in bytes. Requires the 'psutil' Python package. ```APIDOC ## pymorphy2.utils.get_mem_usage ### Description Return memory usage of the current process, in bytes. Requires psutil Python package. ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Request Example ```python # Example usage not provided in source ``` ### Response #### Success Response (200) - type: integer - description: Memory usage in bytes. ### Response Example ```json { "example": 123456789 } ``` ``` -------------------------------- ### Length of Tag Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Returns the number of grammemes in the tag. ```python def __len__(self): return len(self._grammemes_tuple) ``` -------------------------------- ### Greater Than Comparison for Tags Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Compares if one tag is lexicographically greater than another based on their grammeme tuples. ```python def __gt__(self, other): return self._grammemes_tuple > other._grammemes_tuple ``` -------------------------------- ### pymorphy2.utils.kwargs_repr Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html Generates a string representation of keyword arguments, with an option to hide values. ```APIDOC ## pymorphy2.utils.kwargs_repr ### Description Generates a string representation of keyword arguments. ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Request Example ```python >>> pymorphy2.utils.kwargs_repr(dict(foo="123", a=5, x=8)) "a=5, foo='123', x=8" >>> pymorphy2.utils.kwargs_repr(dict(foo="123", a=5, x=8), dont_show_value=['foo']) 'a=5, foo=<...>, x=8' >>> pymorphy2.utils.kwargs_repr() '' ``` ### Response #### Success Response (200) - type: string - description: A string representation of the keyword arguments. ### Response Example ```json { "example": "a=5, foo='123', x=8" } ``` ``` -------------------------------- ### Pickle Reduction for Tag Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Defines how the tag object is pickled, specifying the class and arguments needed for reconstruction. ```python def __reduce__(self): return self.__class__, (self._str,), None ``` -------------------------------- ### Tag a word Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Analyzes a word and returns a list of tags. This method is similar to parse but returns only the tags associated with the word. ```python def tag(self, word): res = [] seen = set() word_lower = word.lower() for analyzer, is_terminal in self._units: res.extend(analyzer.tag(word, word_lower, seen)) if is_terminal and res: break if self.prob_estimator is not not None: res = self.prob_estimator.apply_to_tags(word, word_lower, res) return res ``` -------------------------------- ### Word Storage in RecordDAWG Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html Words are stored in a RecordDAWG, mapping each word to its paradigm ID and index within that paradigm. ```python dawg.RecordDAWG 'word1': (para_id1, para_index1), 'word1': (para_id2, para_index2), 'word2': (para_id1, para_index1), ... ``` -------------------------------- ### Single Shape Analyzer (_SingleShapeAnalyzer) Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html Base class for analyzers that recognize a single, specific shape. It simplifies tag assignment. ```python class _SingleShapeAnalyzer(_ShapeAnalyzer): TAG_STR = None TAG_STR_CYR = None def init(self, morph): assert self.TAG_STR is not None assert self.TAG_STR_CYR is not None self.EXTRA_GRAMMEMES = self.TAG_STR.split(',') self.EXTRA_GRAMMEMES_CYR = self.TAG_STR_CYR.split(',') super(_SingleShapeAnalyzer, self).init(morph) self._tag = self.morph.TagClass(self.TAG_STR) def get_tag(self, word, shape): return self._tag ``` -------------------------------- ### Assert Grammemes are Known Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html Internal class method to check if all provided grammemes are known. Raises ValueError if any unknown grammemes are found. ```python @classmethod def _assert_grammemes_are_known(cls, grammemes): if not grammemes <= cls.KNOWN_GRAMMEMES: cls._assert_grammemes_initialized() unknown = grammemes - cls.KNOWN_GRAMMEMES unknown_repr = ", ".join(["'%s'" % g for g in sorted(unknown)]) raise ValueError("Grammemes are unknown: {%s}" % unknown_repr) ``` -------------------------------- ### Analyzer.tag Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html Analyzes a word and returns a list of possible tags. This method is similar to `parse` but focuses solely on the grammatical tags associated with the word. ```APIDOC ## Analyzer.tag ### Description Analyzes a word and returns a list of its possible grammatical tags. This method is a specialized version of `parse` that returns only the tag information. ### Method `tag(self, word)` ### Parameters * **word** (string) - The word to analyze. ### Returns List of grammatical tags associated with the word. ```