pymorphy2

https://pymorphy2.readthedocs.io
Admin
pymorphy2 is a Python morphological analyzer for Russian language that normalizes words, inflects...
Tokens:20,056
Snippets:151
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Getting Lexeme

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Returns a list of all possible parses for a given word form.

```python
def get_lexeme(self, form):
    return list(self._iter_lexeme(form))
```

--------------------------------

### Example Word Data and Parse Information

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Illustrates sample words with their associated parse information, represented as (paradigm number, form number) pairs. This data is used to construct the internal dictionary.

```text
двор    (103, 0)
ёж      (104, 0)
дворник (101, 2) и (102, 2)
ёжик    (101, 2) и (102, 2)


```

--------------------------------

### Iterate with Progress Bar

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Returns an iterator that displays progress using the tqdm package. Falls back to returning the original iterable if tqdm is not installed.

```python
def with_progress(iterable, desc=None, total=None, leave=True):
    """
    Return an iterator which prints the iteration progress using tqdm package.
    Return iterable intact if tqdm is not available.
    """
    try:
        from tqdm import tqdm

        # workarounds for tqdm bugs
        def _it(iterable, desc, total, leave):
            if total is None:
                try:
                    total = len(iterable)
                except Exception:
                    total = 0
            for el in tqdm(iterable, desc=desc, total=total, leave=leave):
                yield el
            if leave:
                print("")

        return _it(iterable, desc, total, leave)

    except ImportError:
        return iterable
```

--------------------------------

### pymorphy2.shapes.restore_capitalization

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Adjusts the capitalization of a word to match that of an example word. If alignment fails, the remainder is lowercased.

```APIDOC
## pymorphy2.shapes.restore_capitalization

### Description
Make the capitalization of the `word` be the same as in `example`.
In the alignment fails, the reminder is lower-cased.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Request Example
```python
>>> pymorphy2.shapes.restore_capitalization('bye', 'Hello')
'Bye'
>>> pymorphy2.shapes.restore_capitalization('half-an-hour', 'Minute')
'Half-An-Hour'
>>> pymorphy2.shapes.restore_capitalization('usa', 'IEEE')
'USA'
>>> pymorphy2.shapes.restore_capitalization('pre-world', 'anti-World')
'pre-World'
>>> pymorphy2.shapes.restore_capitalization('123-do', 'anti-IEEE')
'123-DO'
>>> pymorphy2.shapes.restore_capitalization('123--do', 'anti--IEEE')
'123--DO'
>>> pymorphy2.shapes.restore_capitalization('foo-BAR-BAZ', 'Baz-Baz')
'Foo-Bar-baz'
>>> pymorphy2.shapes.restore_capitalization('foo', 'foo-bar')
'foo'
```

### Response
#### Success Response (200)
- type: string
- description: The word with adjusted capitalization.

### Response Example
```json
{
  "example": "AdjustedWord"
}
```
```

--------------------------------

### Restore capitalization of a word

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Applies the capitalization pattern from an example word to a target word. Handles hyphenated words and mixed cases. If alignment fails, the remainder is lowercased.

```python
>>> restore_capitalization('bye', 'Hello')
'Bye'
>>> restore_capitalization('half-an-hour', 'Minute')
'Half-An-Hour'
>>> restore_capitalization('usa', 'IEEE')
'USA'
>>> restore_capitalization('pre-world', 'anti-World')
'pre-World'
>>> restore_capitalization('123-do', 'anti-IEEE')
'123-DO'
>>> restore_capitalization('123--do', 'anti--IEEE')
'123--DO'
```

```python
>>> restore_capitalization('foo-BAR-BAZ', 'Baz-Baz')
'Foo-Bar-baz'
>>> restore_capitalization('foo', 'foo-bar')
'foo'
```

--------------------------------

### Analyzer.iter_known_word_parses

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Returns an iterator over parses of known dictionary words that start with a specified prefix. Useful for exploring the dictionary content.

```APIDOC
## Analyzer.iter_known_word_parses

### Description
Returns an iterator over parses of dictionary words that begin with a given prefix. If no prefix is provided, it iterates over all known words.

### Method
`iter_known_word_parses(self, prefix="")`

### Parameters
* **prefix** (string, optional) - The prefix to filter dictionary words. Defaults to an empty string, meaning all words.

### Returns
An iterator yielding parses of known dictionary words.
```

--------------------------------

### Getting Normalized Form

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Returns the normalized form of a given parsed word form.

```python
def normalized(self, form):
    return next(self._iter_lexeme(form))
```

--------------------------------

### Number Analyzer (NumberAnalyzer)

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html

Identifies and tags integer and real numbers. Integers get 'NUMB,intg' and real numbers get 'NUMB,real'. Note that this is distinct from 'NUMR' which is for number words like 'тридцать'.

```python
class NumberAnalyzer(_ShapeAnalyzer):
    """
    This analyzer marks integer numbers with "NUMB,int" or "NUMB,real" tags.
    Example: "12" -> NUMB,int; "12.4" -> NUMB,real

    .. note::

        Don't confuse it with "NUMR": "тридцать" -> NUMR

    """
    EXTRA_GRAMMEMES = ['NUMB', 'intg', 'real']
    EXTRA_GRAMMEMES_CYR = ['ЧИСЛО', 'цел', 'вещ']

    def init(self, morph):
        super(NumberAnalyzer, self).init(morph)
        self._tags = {
            'intg': morph.TagClass('NUMB,intg'),
            'real': morph.TagClass('NUMB,real'),
        }

    def check_shape(self, word, word_lower):
        try:
            int(word)
            return 'intg'
        except ValueError:
            try:
                float(word.replace(',', '.'))
                return 'real'
            except ValueError:
                pass
        return False

    def get_tag(self, word, shape):
        return self._tags[shape]

```

--------------------------------

### Get Memory Usage

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Returns the memory usage of the current process in bytes. Requires the psutil Python package.

```python
import os
import heapq
import itertools
import codecs
import json


def get_mem_usage():
    """
    Return memory usage of the current process, in bytes.
    Requires psutil Python package.
    """
    import psutil
    proc = psutil.Process(os.getpid())
    try:
        return proc.memory_info().rss
    except AttributeError:
        # psutil < 2.x
        return proc.get_memory_info()[0]
```

--------------------------------

### Iterate known word parses

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Returns an iterator over parses of dictionary words that start with a given prefix. If no prefix is provided, it iterates over all words in the dictionary.

```python
    def iter_known_word_parses(self, prefix=""):
        """
        Return an iterator over parses of dictionary words that starts
        with a given prefix (default empty prefix means "all words").
        """

        # XXX: this method currently assumes that
        # units.DictionaryAnalyzer is the first analyzer unit.
        for word, tag, normal_form, para_id, idx in self.dictionary.iter_known_words(prefix):
            methods = ((self._units[0][0], word, para_id, idx),)
            parse = (word, tag, normal_form, 1.0, methods)
            if self._result_type is None:
                yield parse
            else:
                yield self._result_type(*parse)
```

--------------------------------

### Parsing as Variable Both Parts

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Parses a hyphenated word when both the left and right parts can be inflected. This is used for examples like человек-гора, команд-участниц, компания-производитель.

```python
def _parse_as_variable_both(self, left_parses, right_parses, seen):
    """
    Step 2: if left and right can be parsed the same way,
    then it may be the case that both parts should be inflected.
    Examples: человек-гора, команд-участниц, компания-производитель
    """
    result = []
    right_features = [self._similarity_features(p[1]) for p in right_parses]

    # FIXME: quadratic algorithm
    for left_parse in left_parses:

        left_tag = left_parse[1]

        if left_tag._is_unknown():
            continue

        left_feat = self._similarity_features(left_tag)

        for parse_index, right_parse in enumerate(right_parses):

            right_feat = right_features[parse_index]

            if left_feat != right_feat:
                continue

            left_methods = left_parse[4]
            right_methods = right_parse[4]

            new_methods_stack = ((self, left_methods, right_methods),)

            # tag
            parse = (
                '-'.join((left_parse[0], right_parse[0])),  # word
                left_tag,
                '-'.join((left_parse[2], right_parse[2])),  # normal form
                left_parse[3] * self.score_multiplier,
                new_methods_stack
            )
            result.append(parse)
            # add_parse_if_not_seen(parse, result, seen_right)

    return result
```

--------------------------------

### Get TagClass

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Provides access to the TagClass used by the dictionary, typically pymorphy2.tagset.OpencorporaTag.

```python
    @property
    def TagClass(self):
        """
        :rtype: pymorphy2.tagset.OpencorporaTag
        """
        return self.dictionary.Tag
```

--------------------------------

### Parsing as Fixed Left Part

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Assumes the left part of a hyphenated word is an immutable prefix and parses the right part. Examples: интернет-магазин, воздушно-капельный.

```python
def _parse_as_fixed_left(self, right_parses, seen, left):
    """
    Step 1: Assume that the left part is an immutable prefix.
    Examples: интернет-магазин, воздушно-капельный
    """
    result = []

    for fixed_word, tag, normal_form, score, right_methods in right_parses:

        if tag._is_unknown():
            continue

        new_methods_stack = ((self, left, right_methods),)

        parse = (
            '-'.join((left, fixed_word)),
            tag,
            '-'.join((left, normal_form)),
            score * self.score_multiplier,
            new_methods_stack
        )
        result.append(parse)
        # add_parse_if_not_seen(parse, result, seen_left)

    return result
```

--------------------------------

### Get lexeme for a form

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Retrieves the lexeme associated with a given parse form. This is useful for understanding the inflectional paradigm of a word.

```python
    def get_lexeme(self, form):
        """
        Return the lexeme this parse belongs to.
        """
        methods_stack = form[4]
        last_method = methods_stack[-1]
        result = last_method[0].get_lexeme(form)

        if self._result_type is None:
            return result
        return [self._result_type(*p) for p in result]
```

--------------------------------

### Get normal forms of a word

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Returns a list of unique normal forms for a given word. It utilizes the parse method to find all possible forms and extracts their normal forms.

```python
    def normal_forms(self, word):
        """
        Return a list of word normal forms.
        """
        seen = set()
        result = []

        for p in self.parse(word):
            normal_form = p[2]
            if normal_form not in seen:
                result.append(normal_form)
                seen.add(normal_form)
        return result
```

--------------------------------

### Build Paradigm Info

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Retrieves paradigm information as a list of (prefix, tag, suffix) tuples for a given paradigm ID.

```python
build_paradigm_info(_para_id_)

```

--------------------------------

### Initialize OpenCorpora Dictionary Wrapper

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Instantiates the Dictionary wrapper for OpenCorpora dictionaries. Requires the path to the dictionary folder.

```python
pymorphy2.opencorpora_dict.wrapper.Dictionary(_path_)

```

--------------------------------

### HyphenatedWordsAnalyzer Init Method

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Initializes the analyzer with the morph object and sets up feature grammemes and prefix checking.

```python
def init(self, morph):
    super(HyphenatedWordsAnalyzer, self).init(morph)
    Tag = morph.TagClass
    self._FEATURE_GRAMMEMES = (Tag.PARTS_OF_SPEECH | Tag.NUMBERS |
                               Tag.CASES | Tag.PERSONS | Tag.TENSES)
    self._has_skip_prefix = PrefixMatcher(self.skip_prefixes).is_prefixed
```

--------------------------------

### Extracting Word Paradigms

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Demonstrates the initial tokenization of a word ('хомяковый') into its lemma and associated grammatical tags.

```text
  СЛОВО       ТЕГ
  хомяковый   ADJF,Qual masc,sing,nomn
  хомякового  ADJF,Qual masc,sing,gent
              ...
  хомяковы    ADJS,Qual plur
  хомяковее   COMP,Qual
  хомяковей   COMP,Qual V-ej
похомяковее   COMP,Qual Cmp2
похомяковей   COMP,Qual Cmp2,V-ej
```

--------------------------------

### pymorphy2 CLI Options

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Lists the available options for the pymorphy2 command-line tool, including lemmatization, scoring, tagging, caching, and language selection.

```bash
-l --lemmatize      Include normal forms (lemmas)

```

```bash
-s --score          Include non-contextual P(tag|word) scores

```

```bash
-t --tag            Include tags

```

```bash
--thresh <NUM>      Drop all results with estimated P(tag|word) less
                    than a threshold [default: 0.0]

```

```bash
--tokenized         Assume that input text is already tokenized:
                    one token per line.

```

```bash
-c --cache <SIZE>   Cache size, in entries. Set it to 0 to disable
                    cache; use 'unlim' value for unlimited cache
                    size [default: 20000]

```

```bash
--lang <lang>       Language to use. Allowed values: ru, uk [default: ru]

```

```bash
--dict <path>       Dictionary folder path

```

```bash
-v --verbose        Be more verbose

```

```bash
-h --help           Show this help

```

--------------------------------

### pymorphy2 Command-Line Usage

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Shows the available commands and options for the pymorphy2 command-line interface. Use 'pymorphy -h' for detailed help.

```bash
pymorphy parse [options] [<input>]

```

```bash
pymorphy dict meta [--lang <lang> | --dict <path>]

```

```bash
pymorphy dict mem_usage [--lang <lang> | --dict <path>] [--verbose]

```

```bash
pymorphy -h | --help

```

```bash
pymorphy --version

```

--------------------------------

### Initialize Grammemes and Incompatibilities

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

This method initializes class attributes like GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE based on provided grammeme data. It also expands EXTRA_INCOMPATIBLE sets.

```python
        # figure out parents & children
        children = collections.defaultdict(set)
        for index, (name, parent, alias, description) in enumerate(dict_grammemes):
            if parent:
                children[parent].add(name)
            if gr.get(parent, None):  # parent's parent
                children[gr[parent]].add(name)

        # expand EXTRA_INCOMPATIBLE
        for grammeme, g_set in cls._EXTRA_INCOMPATIBLE.items():
            for g in g_set.copy():
                g_set.update(children[g])

        # fill GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE
        for index, (name, parent, alias, description) in enumerate(dict_grammemes):
            cls._GRAMMEME_INDICES[name] = index
            incompatible = cls._EXTRA_INCOMPATIBLE.get(name, set())
            incompatible = (incompatible | children[parent]) - set([name])

            cls._GRAMMEME_INCOMPATIBLE[name] = frozenset(incompatible)
```

--------------------------------

### Build Tag Info

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Returns the morphological tag as a string for a specific word index within a paradigm.

```python
build_tag_info(_para_id_ , _idx_)

```

--------------------------------

### Initialize Grammemes from Dictionary

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Class method to initialize class attributes like KNOWN_GRAMMEMES, _LAT2CYR, and _CYR2LAT from a provided dictionary of grammeme information. The input is expected to be a list of tuples.

```python
    @classmethod
    def _init_grammemes(cls, dict_grammemes):
        """
        Initialize various class attributes with grammeme
        information obtained from XML dictionary.

        ``dict_grammemes`` is a list of tuples::

            [
                (name, parent, alias, description),
                ...
            ]

        """
        cls.KNOWN_GRAMMEMES = set()
        cls._CYR2LAT = {}
        cls._LAT2CYR = {}
        for name, parent, alias, description in dict_grammemes:
            cls.add_grammemes_to_known(name, alias)

        gr = dict((name, parent) for (name, parent, alias, description) in dict_grammemes)

```

--------------------------------

### Get Largest Elements by Key

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Returns a list of elements from an iterable that have one of the top 'n' key values. Useful for retrieving elements based on a specific criterion.

```python
def largest_elements(iterable, key, n=1):
    """
    Return a list of large elements of the ``iterable``
    (according to ``key`` function).

    ``n`` is a number of top element values to consider; when n==1
    (default) only largest elements are returned; when n==2 - elements
    with one of the top-2 values, etc.

    >>> s = [-4, 3, 5, 7, 4, -7]
    >>> largest_elements(s, abs)
    [7, -7]
    >>> largest_elements(s, abs, 2)
    [5, 7, -7]
    >>> largest_elements(s, abs, 3)
    [-4, 5, 7, 4, -7]

    """
    it1, it2 = itertools.tee(iterable)
    top_keys = set(heapq.nlargest(n, set(map(key, it1))))
    return [el for el in it2 if key(el) in top_keys]
```

--------------------------------

### Split Word into Prefix and Suffix

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Returns all possible splits of a word into a prefix and suffix, respecting minimum reminder and maximum prefix length constraints.

```python
def word_splits(word, min_reminder=3, max_prefix_length=5):
    """
    Return all splits of a word (taking in account min_reminder and
    max_prefix_length).
    """
    max_split = min(max_prefix_length, len(word)-min_reminder)
    split_indexes = range(1, 1+max_split)
    return [(word[:i], word[i:]) for i in split_indexes]
```

--------------------------------

### Initialize Cyrillic Alias Map

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Initializes the _GRAMMEME_ALIAS_MAP by iterating through the dictionary of grammemes and storing the mapping from grammeme name to its Cyrillic alias.

```python
    @classmethod
    def _init_alias_map(cls, dict_grammemes):
        for name, parent, alias, description in dict_grammemes:
            cls._GRAMMEME_ALIAS_MAP[name] = alias
```

--------------------------------

### Assert Grammemes Initialized

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Internal class method to ensure that the grammeme data structures have been initialized. Raises RuntimeError if not.

```python
    @classmethod
    def _assert_grammemes_initialized(cls):
        if not cls.KNOWN_GRAMMEMES:
            msg = "The class was not properly initialized."
            raise RuntimeError(msg)
```

--------------------------------

### HyphenatedWordsAnalyzer Initialization

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_hyphen.html

Initializes the HyphenatedWordsAnalyzer with a list of prefixes to skip and an optional score multiplier.

```python
def __init__(self, skip_prefixes, score_multiplier=0.75):
    self.score_multiplier = score_multiplier
    self.skip_prefixes = skip_prefixes
```

--------------------------------

### Get largest elements by key

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Returns a list of elements from an iterable that have the top 'n' largest values according to a specified key function. Defaults to returning the single largest element.

```python
>>> s = [-4, 3, 5, 7, 4, -7]
>>> largest_elements(s, abs)
[7, -7]
```

```python
>>> largest_elements(s, abs, 2)
[5, 7, -7]
```

```python
>>> largest_elements(s, abs, 3)
[-4, 5, 7, 4, -7]
```

--------------------------------

### Write JSON to File

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Creates a file with the given object serialized to JSON. Sets default options for ensuring ASCII is false and indentation is 2.

```python
def json_write(filename, obj, **json_options):
    """ Create file ``filename`` with ``obj`` serialized to JSON """

    json_options.setdefault('ensure_ascii', False)
    json_options.setdefault('indent', 2)
    with codecs.open(filename, 'w', 'utf8') as f:
        json.dump(obj, f, **json_options)
```

--------------------------------

### PyMorphy2 Command-Line Interface

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

The PyMorphy2 command-line interface allows users to parse text, manage dictionaries, and access help information.

```APIDOC
## Command-Line Interface

Usage:
```
pymorphy parse [options] [<input>]
pymorphy dict meta [--lang <lang> | --dict <path>]
pymorphy dict mem_usage [--lang <lang> | --dict <path>] [--verbose]
pymorphy -h | --help
pymorphy --version
```

Options:
```
-l --lemmatize      Include normal forms (lemmas)
-s --score          Include non-contextual P(tag|word) scores
-t --tag            Include tags
--thresh <NUM>      Drop all results with estimated P(tag|word) less
                    than a threshold [default: 0.0]
--tokenized         Assume that input text is already tokenized:
                    one token per line.
-c --cache <SIZE>   Cache size, in entries. Set it to 0 to disable
                    cache; use 'unlim' value for unlimited cache
                    size [default: 20000]
--lang <lang>       Language to use. Allowed values: ru, uk [default: ru]
--dict <path>       Dictionary folder path
-v --verbose        Be more verbose
-h --help           Show this help
```
```

--------------------------------

### Paradigm Structure: Prefix, Stem, Suffix

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Illustrates how a word's paradigm can be broken down into prefix, stem, and suffix components, with the stem being common across all forms.

```text
ПРЕФИКС СТЕМ      ХВОСТ   ТЕГ
        хомяков   ый      ADJF,Qual masc,sing,nomn
        хомяков   ого     ADJF,Qual masc,sing,gent
                  ...
        хомяков   ы       ADJS,Qual plur
        хомяков   ее      COMP,Qual
        хомяков   ей      COMP,Qual V-ej
    по  хомяков   ее      COMP,Qual Cmp2
    по  хомяков   ей      COMP,Qual Cmp2,V-ej
```

--------------------------------

### Initialize Cyrillic Grammemes

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Initializes class attributes for Cyrillic tags, including mapping internal grammemes to their Cyrillic aliases and updating GRAMMEME_INDICES and GRAMMEME_INCOMPATIBLE accordingly.

```python
    @classmethod
    def _init_grammemes(cls, dict_grammemes):
        """
        Initialize various class attributes with grammeme
        information obtained from XML dictionary.
        """
        cls._init_alias_map(dict_grammemes)
        super(CyrillicOpencorporaTag, cls)._init_grammemes(dict_grammemes)

        GRAMMEME_INDICES = collections.defaultdict(int)
        for name, idx in cls._GRAMMEME_INDICES.items():
            GRAMMEME_INDICES[cls._from_internal_grammeme(name)] = idx
        cls._GRAMMEME_INDICES = GRAMMEME_INDICES

        GRAMMEME_INCOMPATIBLE = collections.defaultdict(set)
        for name, value in cls._GRAMMEME_INCOMPATIBLE.items():
            GRAMMEME_INCOMPATIBLE[cls._from_internal_grammeme(name)] = set([
                cls._from_internal_grammeme(gr) for gr in value
            ])
        cls._GRAMMEME_INCOMPATIBLE = GRAMMEME_INCOMPATIBLE

        cls._NON_PRODUCTIVE_GRAMMEMES = set([
            cls._from_internal_grammeme(gr) for gr in cls._NON_PRODUCTIVE_GRAMMEMES
        ])
```

--------------------------------

### Generate Combinations of All Lengths

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Returns an iterable with all possible combinations of items from an input iterable. Demonstrates generating combinations of increasing lengths.

```python
def combinations_of_all_lengths(it):
    """
    Return an iterable with all possible combinations of items from ``it``:

        >>> for comb in combinations_of_all_lengths('ABC'):
        ...     print("".join(comb))
        A
        B
        C
        AB
        AC
        BC
        ABC

    """
    return itertools.chain(
        *(itertools.combinations(it, num+1) for num in range(len(it))))
```

--------------------------------

### Check if Grammeme is Known

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Class method to verify if a given grammeme is recognized by the system. Asserts that grammemes have been initialized.

```python
    @classmethod
    def grammeme_is_known(cls, grammeme):
        cls._assert_grammemes_initialized()
        return grammeme in cls.KNOWN_GRAMMEMES
```

--------------------------------

### Building Tag Information

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Reconstructs grammatical tag information from paradigm data using paradigm ID and word index. It accesses the paradigm array and extracts the tag ID.

```python
def _build_tag_info(self, para_id, idx):

    # получаем массив с данными парадигмы
    paradigm = self._dictionary.paradigms[para_id]

    # индексы грамматической информации начинаются со второй трети
    # массива с парадигмой
    tag_info_offset = len(paradigm) // 3

    # получаем искомый индекс
    tag_id = paradigm[tag_info_offset + tag_id_index]

    # возвращаем соответствующую строку из таблицы с грамматической информацией
    return self._dictionary.gramtab[tag_id]
```

--------------------------------

### Build Stem

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Calculates the stem of a word given its paradigm, index, and fixed word form. Useful for analyzing word structure.

```python
build_stem(_paradigm_ , _idx_ , _fixed_word_)

```

--------------------------------

### Less Than Comparison for Tags

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Compares if one tag is lexicographically less than another based on their grammeme tuples.

```python
    def __lt__(self, other):
        return self._grammemes_tuple < other._grammemes_tuple
```

--------------------------------

### Format Keyword Arguments for Representation

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/utils.html

Generates a string representation of keyword arguments, optionally hiding values for specified keys. Useful for debugging and logging.

```python
def kwargs_repr(kwargs=None, dont_show_value=None):
    """
    >>> kwargs_repr(dict(foo="123", a=5, x=8))
    "a=5, foo='123', x=8"
    >>> kwargs_repr(dict(foo="123", a=5, x=8), dont_show_value=['foo'])
    'a=5, foo=<...>, x=8'
    >>> kwargs_repr()
    ''
    """
    kwargs = kwargs or {}
    dont_show_value = set(dont_show_value or [])
    return ", ".join(
        "%s=%s" % (k, repr(v) if k not in dont_show_value else "<...>")
        for k, v in sorted(kwargs.items()))
```

--------------------------------

### Paradigm Data Structure

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Stores paradigms as arrays of integers, referencing suffixes, tag IDs, and prefix IDs. This structure is memory-efficient.

```python
paradigms = [
    array.array("<H", [
        suff_id1, .., suff_idN,
        tag_id1, .., tag_idN,
        pref_id1, .., pref_idN
    ]),

    array.array("<H", [
        ...
    ]),

    ...
]

suffixes = ['suffix1', 'suffix2', ...]
prefixes = ['prefix1', 'prefix2', ...]
```

--------------------------------

### Build Normal Form

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Constructs a normal form for a given word, paradigm ID, and index. Used internally for morphological analysis.

```python
build_normal_form(_para_id_ , _idx_ , _fixed_word_)

```

--------------------------------

### Paradigm After Stem Removal

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Shows the resulting paradigm table after removing the common stem, leaving only prefixes and suffixes associated with grammatical tags.

```text
ПРЕФИКС           ХВОСТ   ТЕГ
                  ый      ADJF,Qual masc,sing,nomn
                  ого     ADJF,Qual masc,sing,gent
                  ...
                  ы       ADJS,Qual plur
                  ее      COMP,Qual
                  ей      COMP,Qual V-ej
    по            ее      COMP,Qual Cmp2
    по            ей      COMP,Qual Cmp2,V-ej
```

--------------------------------

### Dictionary Parsing Algorithm

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Parses a given word by searching the dictionary DAWG for matching entries and reconstructing the grammatical tag and normal form.

```python
result = []

# Ищем в DAWG со словами все ключи, которые начинаются
# с <СЛОВО><sep> (обходом по графу); из этих ключей (из того, что за <sep>)
# получаем список кортежей [(para_id1, index1), (para_id2, index2), ...].
#
# RecordDAWG из библиотек DAWG или DAWG-Python умеет это делать
# одной командой (с возможностью нечеткого поиска для буквы Ё):

para_data = self._dictionary.words.similar_items(word, self._ee)

# fixed_word - это слово с исправленной буквой Ё, для которого был
# проведен разбор.

for fixed_word, parse in para_data:
    for para_id, idx in parse:

        # по информации о номере парадигмы и номере слова в
        # парадигме восстанавливаем нормальную форму слова и
        # грамматическую информацию.

        tag = self._build_tag_info(para_id, idx)
        normal_form = self._build_normal_form(para_id, idx, fixed_word)

        result.append(
            (fixed_word, tag, normal_form)
        )
```

--------------------------------

### pymorphy2.utils.with_progress

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Returns an iterator that displays iteration progress using the 'tqdm' package. If 'tqdm' is unavailable, the iterable is returned unchanged.

```APIDOC
## pymorphy2.utils.with_progress

### Description
Return an iterator which prints the iteration progress using tqdm package. Return iterable intact if tqdm is not available.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Request Example
```python
# Example usage not provided in source
```

### Response
#### Success Response (200)
- type: iterator
- description: An iterator that yields progress updates.

### Response Example
```json
{
  "example": "<iterator object>"
}
```
```

--------------------------------

### OpenCorpora Dictionary Wrapper

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Utilities for working with OpenCorpora dictionaries, providing methods to build word forms, retrieve paradigm information, and check word existence.

```APIDOC
## Utilities for OpenCorpora Dictionaries

_class_`pymorphy2.opencorpora_dict.wrapper.``Dictionary`(_path_)[source]¶
    
OpenCorpora dictionary wrapper class. 

`build_normal_form`(_para_id_ , _idx_ , _fixed_word_)[source]¶
    
Build a normal form. 

`build_paradigm_info`(_para_id_)[source]¶
    
Return a list of
> (prefix, tag, suffix)
tuples representing the paradigm. 

`build_stem`(_paradigm_ , _idx_ , _fixed_word_)[source]¶
    
Return word stem (given a word, paradigm and the word index). 

`build_tag_info`(_para_id_ , _idx_)[source]¶
    
Return tag as a string. 

`iter_known_words`(_prefix=''_)[source]¶
    
Return an iterator over `(word, tag, normal_form, para_id, idx)` tuples with dictionary words that starts with a given prefix (default empty prefix means “all words”). 

`word_is_known`(_word_ , _substitutes_compiled=None_)[source]¶
    
Check if a `word` is in the dictionary.
To allow some fuzzyness pass `substitutes_compiled` argument; it should be a result of `DAWG.compile_replaces()`. This way you can e.g. handle ё letters replaced with е in the input words.
Note
Dictionary words are not always correct words; the dictionary also contains incorrect forms which are commonly used. So for spellchecking tasks this method should be used with extra care.
```

--------------------------------

### DAWG Graph Representation

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

A DOT language representation of the Deterministic Acyclic Finite State Automaton (DAWG) graph. This graph encodes the words and their parse information, demonstrating how common prefixes are shared and data is compressed.

```dot
digraph foo { rankdir=LR; size=9; node [shape = doublecircle]; 10 14; node [shape = circle]; 0 -> 2 [label=Д]; 0 -> 3 [label=Ё]; 1 -> 4 [label=О]; 2 -> 1 [label=В]; 3 -> 16 [label=Ж]; 4 -> 6 [label=Р]; 5 -> 8 [label=К]; 6 -> 7 [label=Н]; 6 -> 22 [label=sep]; 7 -> 5 [label=И]; 8 -> 9 [label=sep]; 9 -> 12 [label="103"]; 9 -> 15 [label="102"]; 12 -> 10 [label="2"]; 13 -> 14 [label="0"]; 15 -> 10 [label="2"]; 16 -> 32 [label=И]; 16 -> 54 [label=sep]; 17 -> 14 [label="2"]; 22 -> 13 [label="103"]; 32 -> 8 [label=К]; 54 -> 17 [label="104"]; }
```

--------------------------------

### Base Shape Analyzer (_ShapeAnalyzer)

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html

Abstract base class for shape-based analyzers. Subclasses must implement `check_shape` and `get_tag` methods.

```python
# -*- coding: utf-8 -*-
"""
Analyzer units that analyzes non-word tokes
-------------------------------------------
"""

from __future__ import absolute_import, unicode_literals, division

from pymorphy2.units.base import BaseAnalyzerUnit
from pymorphy2.shapes import is_latin, is_punctuation, is_roman_number


class _ShapeAnalyzer(BaseAnalyzerUnit):
    EXTRA_GRAMMEMES = []
    EXTRA_GRAMMEMES_CYR = []

    def __init__(self, score=0.9):
        self.score = score

    def init(self, morph):
        super(_ShapeAnalyzer, self).init(morph)

        for lat, cyr in zip(self.EXTRA_GRAMMEMES, self.EXTRA_GRAMMEMES_CYR):
            self.morph.TagClass.add_grammemes_to_known(lat, cyr)

    def parse(self, word, word_lower, seen_parses):
        shape = self.check_shape(word, word_lower)
        if not shape:
            return []

        methods = ((self, word),)
        return [(word_lower, self.get_tag(word, shape), word_lower, self.score, methods)]

    def tag(self, word, word_lower, seen_tags):
        shape = self.check_shape(word, word_lower)
        if not shape:
            return []
        return [self.get_tag(word, shape)]

    def get_lexeme(self, form):
        return [form]

    def normalized(self, form):
        return form

    # implement these 2 methods in a subclass:
    def check_shape(self, word, word_lower):
        raise NotImplementedError()

    def get_tag(self, word, shape):
        raise NotImplementedError()

```

--------------------------------

### Inequality Comparison for Tags

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Compares two tag objects for inequality based on their grammeme tuples.

```python
    def __ne__(self, other):
        return self._grammemes_tuple != other._grammemes_tuple
```

--------------------------------

### pymorphy2.utils.word_splits

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Generates all possible splits of a word, considering minimum reminder length and maximum prefix length.

```APIDOC
## pymorphy2.utils.word_splits

### Description
Return all splits of a word (taking in account min_reminder and max_prefix_length).

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Request Example
```python
# Example usage not provided in source
```

### Response
#### Success Response (200)
- type: list of tuples
- description: A list where each tuple represents a split of the word (prefix, suffix).

### Response Example
```json
{
  "example": [["split1_prefix", "split1_suffix"], ["split2_prefix", "split2_suffix"]]
}
```
```

--------------------------------

### Equality Comparison for Tags

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Compares two tag objects for equality based on their grammeme tuples.

```python
    def __eq__(self, other):
        return self._grammemes_tuple == other._grammemes_tuple
```

--------------------------------

### Update Grammemes with Required and Incompatible

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Returns a new set of grammemes by adding required grammemes and removing any grammemes incompatible with the required ones. Raises ValueError for unknown required grammemes.

```python
    def updated_grammemes(self, required):
        """
        Return a new set of grammemes with ``required`` grammemes added
        and incompatible grammemes removed.
        """
        new_grammemes = self.grammemes | required
        for grammeme in required:
            if not self.grammeme_is_known(grammeme):
                raise ValueError("Unknown grammeme: %s" % grammeme)
            new_grammemes -= self._GRAMMEME_INCOMPATIBLE[grammeme]
        return new_grammemes
```

--------------------------------

### Object Representation of Tag

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Returns the developer-friendly representation of the tag object.

```python
    def __repr__(self):
        return "OpencorporaTag('%s')" % self
```

--------------------------------

### Cyrillic Grammemes Property

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Returns a frozenset of Cyrillic grammemes for a given tag. Caches the result for efficiency.

```python
    @property
    def grammemes_cyr(self):
        """ A frozenset with Cyrillic grammemes for this tag. """
        if self._cyr_grammemes_cache is None:
            cyr_grammemes = [self._LAT2CYR[g] for g in self._grammemes_tuple]
            self._cyr_grammemes_cache = frozenset(cyr_grammemes)
        return self._cyr_grammemes_cache
```

--------------------------------

### pymorphy2.utils.get_mem_usage

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Retrieves the memory usage of the current process in bytes. Requires the 'psutil' Python package.

```APIDOC
## pymorphy2.utils.get_mem_usage

### Description
Return memory usage of the current process, in bytes. Requires psutil Python package.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Request Example
```python
# Example usage not provided in source
```

### Response
#### Success Response (200)
- type: integer
- description: Memory usage in bytes.

### Response Example
```json
{
  "example": 123456789
}
```
```

--------------------------------

### Length of Tag

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Returns the number of grammemes in the tag.

```python
    def __len__(self):
        return len(self._grammemes_tuple)
```

--------------------------------

### Greater Than Comparison for Tags

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Compares if one tag is lexicographically greater than another based on their grammeme tuples.

```python
    def __gt__(self, other):
        return self._grammemes_tuple > other._grammemes_tuple
```

--------------------------------

### pymorphy2.utils.kwargs_repr

Source: https://pymorphy2.readthedocs.io/en/stable/misc/api_reference.html

Generates a string representation of keyword arguments, with an option to hide values.

```APIDOC
## pymorphy2.utils.kwargs_repr

### Description
Generates a string representation of keyword arguments.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Request Example
```python
>>> pymorphy2.utils.kwargs_repr(dict(foo="123", a=5, x=8))
"a=5, foo='123', x=8"
>>> pymorphy2.utils.kwargs_repr(dict(foo="123", a=5, x=8), dont_show_value=['foo'])
'a=5, foo=<...>, x=8'
>>> pymorphy2.utils.kwargs_repr()
''
```

### Response
#### Success Response (200)
- type: string
- description: A string representation of the keyword arguments.

### Response Example
```json
{
  "example": "a=5, foo='123', x=8"
}
```
```

--------------------------------

### Pickle Reduction for Tag

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Defines how the tag object is pickled, specifying the class and arguments needed for reconstruction.

```python
    def __reduce__(self):
        return self.__class__, (self._str,), None
```

--------------------------------

### Tag a word

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Analyzes a word and returns a list of tags. This method is similar to parse but returns only the tags associated with the word.

```python
    def tag(self, word):
        res = []
        seen = set()
        word_lower = word.lower()

        for analyzer, is_terminal in self._units:
            res.extend(analyzer.tag(word, word_lower, seen))

            if is_terminal and res:
                break

        if self.prob_estimator is not not None:
            res = self.prob_estimator.apply_to_tags(word, word_lower, res)
        return res
```

--------------------------------

### Word Storage in RecordDAWG

Source: https://pymorphy2.readthedocs.io/en/stable/internals/dict.html

Words are stored in a RecordDAWG, mapping each word to its paradigm ID and index within that paradigm.

```python
dawg.RecordDAWG

    'word1': (para_id1, para_index1),
    'word1': (para_id2, para_index2),
    'word2': (para_id1, para_index1),
    ...
```

--------------------------------

### Single Shape Analyzer (_SingleShapeAnalyzer)

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/units/by_shape.html

Base class for analyzers that recognize a single, specific shape. It simplifies tag assignment.

```python
class _SingleShapeAnalyzer(_ShapeAnalyzer):
    TAG_STR = None
    TAG_STR_CYR = None

    def init(self, morph):
        assert self.TAG_STR is not None
        assert self.TAG_STR_CYR is not None
        self.EXTRA_GRAMMEMES = self.TAG_STR.split(',')
        self.EXTRA_GRAMMEMES_CYR = self.TAG_STR_CYR.split(',')
        super(_SingleShapeAnalyzer, self).init(morph)
        self._tag = self.morph.TagClass(self.TAG_STR)

    def get_tag(self, word, shape):
        return self._tag

```

--------------------------------

### Assert Grammemes are Known

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/tagset.html

Internal class method to check if all provided grammemes are known. Raises ValueError if any unknown grammemes are found.

```python
    @classmethod
    def _assert_grammemes_are_known(cls, grammemes):
        if not grammemes <= cls.KNOWN_GRAMMEMES:
            cls._assert_grammemes_initialized()
            unknown = grammemes - cls.KNOWN_GRAMMEMES
            unknown_repr = ", ".join(["'%s'" % g for g in sorted(unknown)])
            raise ValueError("Grammemes are unknown: {%s}" % unknown_repr)
```

--------------------------------

### Analyzer.tag

Source: https://pymorphy2.readthedocs.io/en/stable/_modules/pymorphy2/analyzer.html

Analyzes a word and returns a list of possible tags. This method is similar to `parse` but focuses solely on the grammatical tags associated with the word.

```APIDOC
## Analyzer.tag

### Description
Analyzes a word and returns a list of its possible grammatical tags. This method is a specialized version of `parse` that returns only the tag information.

### Method
`tag(self, word)`

### Parameters
* **word** (string) - The word to analyze.

### Returns
List of grammatical tags associated with the word.
```
Tokens:
Raw