### Install Maha using Pip Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Installs Maha using pip. This method is an alternative to Poetry but does not support editable installs, requiring re-installation after source code changes. ```shell python3 -m pip install . ``` -------------------------------- ### Install Pre-Commit Hooks Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Installs pre-commit hooks for the project, ensuring code formatting and linting (using black and isort) on each commit. Requires Poetry to be installed. ```shell poetry run pre-commit install ``` -------------------------------- ### Install Maha with Poetry Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Installs Maha and its dependencies using Poetry, a Python dependency management tool. This command also sets up and enters a virtual environment for the project. ```shell poetry install ``` -------------------------------- ### Python Example Docstring - NumPy Style (pycon) Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/docstring.md Illustrates how to include an example of function usage within a docstring using the NumPy format and the `pycon` directive for Python console examples. ```python # python code here ``` -------------------------------- ### Install Maha using pip Source: https://github.com/troboto/maha/blob/main/README.md This command installs the Maha library, which is pronounced 'maha d', using pip. Ensure you have Python and pip installed. ```bash pip install mahad # pronounced maha d ``` -------------------------------- ### Extract Emails using Maha Parser Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example shows how to extract email addresses from a text string using the `parse` function. The `emails` parameter should be set to `True`. The function returns a list of `Dimension` objects, each containing the email's body, value, start and end indices, and its type. ```python from maha.parsers.functions import parse sample_text = "الإيميل: test@example.com أو test2@example.com" parse(sample_text, emails=True) [Dimension(body=test@example.com, value=test@example.com, start=9, end=25, dimension_type=DimensionType.EMAILS), Dimension(body=test2@example.com, value=test2@example.com, start=29, end=46, dimension_type=DimensionType.EMAILS)] ``` -------------------------------- ### Python Enum Example - Basic Usage Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/templates/index.md Demonstrates the fundamental creation and usage of an enumeration class in Python using the `enum.Enum` base class. It shows how to define members, access them via attribute, value, or name, and iterate over the enumeration. ```python from enum import Enum class Color(Enum): RED = 1 BLUE = 2 GREEN = 3 # Accessing members print(Color.RED) # Output: print(Color(1)) # Output: print(Color['RED']) # Output: # Iterating over members print(len(Color)) # Output: 3 print(list(Color)) # Output: [, , ] ``` -------------------------------- ### Check for Specific Arabic Patterns in Text using Python Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/index.md This Python example demonstrates checking for specific Arabic patterns like hashtags and mentions using the `contains` function. It leverages the `arabic_hashtags` and `arabic_mentions` parameters, which rely on predefined expressions. ```python from maha.contains import contains text = "#مرحبا @user" if contains(text, arabic_hashtags=True): print("The text contains Arabic hashtags.") if contains(text, arabic_mentions=True): print("The text contains Arabic mentions.") ``` -------------------------------- ### Stream File Processing with StreamFileProcessor in Python Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md Illustrates the usage of StreamFileProcessor for processing files in a streaming manner. The example shows how to initialize the processor with a file path and apply cleaning operations like normalization and keeping only Arabic letters. It also indicates how to save the processed content to a new file. ```pycon >>> from pathlib import Path >>> import maha >>> # This is a sample file that comes with Maha. >>> resource_path = Path(maha.__file__).parents[1] / "sample_data/surah_al-ala.txt" >>> from maha.processors import StreamFileProcessor >>> proc = StreamFileProcessor(resource_path) >>> proc = proc.normalize(all=True).keep(arabic_letters=True).drop_empty_lines() >>> # To start processing the file, run the following commented code. >>> # proc.process_and_save(Path("output.txt")) ``` -------------------------------- ### Process Text with TextProcessor in Python Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md Shows how to use the TextProcessor to clean and normalize Arabic text. It demonstrates chaining methods like normalize, keep, and drop_empty_lines to achieve specific text cleaning requirements. The example also shows how to extract unique characters from the text. ```pycon >>> from maha.processors import TextProcessor >>> sample_text = """ ... بِسْمِ اللَّهِ الرَّحْمَنِ الرَّحِيمِ ... الْحَمْدُ لِلَّهِ رَبِّ الْعَالَمِينَ ... الرَّحْمَنِ الرَّحِيمِ ... مَالِكِ يَوْمِ الدِّينِ ... إِيَّاكَ نَعْبُدُ وَإِيَّاكَ نَسْتَعِينُ ... اهدِنَا الصِّرَاطَ الْمُسْتَقِيمَ ... صِرَاطَ الَّذِينَ أَنْعَمْتَ عَلَيْهِمْ غَيْرِ الْمَغْضُوبِ عَلَيْهِمْ وَلاَ الضَّالِّينَ ... """ >>> processor = TextProcessor.from_text(sample_text, sep="\n") >>> cleaned_text = ( ... processor.normalize(alef=True).keep(arabic_letters=True).drop_empty_lines().text ... ) >>> print(cleaned_text) بسم الله الرحمن الرحيم الحمد لله رب العالمين الرحمن الرحيم مالك يوم الدين اياك نعبد واياك نستعين اهدنا الصراط المستقيم صراط الذين انعمت عليهم غير المغضوب عليهم ولا الضالين >>> # To print the unique characters >>> unique_char = processor.get(unique_characters=True) >>> unique_char.sort() >>> unique_char [' ', 'ا', 'ب', 'ت', 'ح', 'د', 'ذ', 'ر', 'س', 'ص', 'ض', 'ط', 'ع', 'غ', 'ق', 'ك', 'ل', 'م', 'ن', 'ه', 'و', 'ي'] ``` -------------------------------- ### Extract Arabic Characters using Maha Parser Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example demonstrates how to extract all Arabic characters from a given text using the `parse` function from the `maha.parsers.functions` module. It requires setting the `arabic` parameter to `True` and optionally `include_space` to `True`. ```python from maha.parsers.functions import parse sample_text = "أنا وأخي في المكتبةِ نَدرسُ برمجه Python " parse(sample_text, arabic=True, include_space=True) [Dimension(body=أنا وأخي في المكتبةِ نَدرسُ برمجه , value=أنا وأخي في المكتبةِ نَدرسُ برمجه , start=0, end=34, dimension_type=DimensionType.ARABIC)] ``` -------------------------------- ### Wrap Pattern with Start and End Expressions Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/rules/index.md Adds start and end anchors to a given regex pattern, ensuring it matches the entire input string or specific boundaries. This is crucial for precise matching in text analysis. ```python def wrap_pattern(pattern): """Adds start and end expression to the pattern. * **Parameters:** **pattern** (*str*) * **Return type:** str """ # Implementation details for wrapping the pattern. ``` -------------------------------- ### Extract Arabic Hashtags and Mentions using Maha Parser Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example demonstrates extracting Arabic hashtags and user mentions from text using the `parse` function. Set `arabic_hashtags=True` and `mentions=True`. The output is a list of `Dimension` objects, specifying the extracted entity and its properties. ```python from maha.parsers.functions import parse sample_text = "#مها #maha @Maha @مها" parse(sample_text, arabic_hashtags=True, mentions=True) [Dimension(body=#مها, value=#مها, start=0, end=4, dimension_type=DimensionType.ARABIC_HASHTAGS), Dimension(body=@Maha, value=@Maha, start=11, end=16, dimension_type=DimensionType.MENTIONS), Dimension(body=@مها, value=@مها, start=17, end=21, dimension_type=DimensionType.MENTIONS)] ``` -------------------------------- ### Parse Numbers from Text After Text Conversion Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md Illustrates the integration of numbers_to_text with parse_dimension. This example shows how to first convert numbers to text and then use parse_dimension to extract the original numerical value, demonstrating a round-trip conversion and validation process. It handles various number formats and the accusative case. ```python from maha.cleaners.functions import numbers_to_text from maha.parsers.functions import parse_dimension # Parse numeral after basic text conversion text_with_numbers = "في المكتبة 152 كتاب" text_as_words = numbers_to_text(text_with_numbers) parsed_numeral = parse_dimension(text_as_words, numeral=True)[0].value print(f"Original: {text_with_numbers}, Text: {text_as_words}, Parsed Numeral: {parsed_numeral}") # Expected: Parsed Numeral: 152 # Parse numeral after accusative text conversion text_with_numbers_acc = "في المكتبة 152 كتاب" text_as_words_acc = numbers_to_text(text_with_numbers_acc, accusative=True) parsed_numeral_acc = parse_dimension(text_as_words_acc, numeral=True)[0].value print(f"Original: {text_with_numbers_acc}, Text (Acc): {text_as_words_acc}, Parsed Numeral: {parsed_numeral_acc}") # Expected: Parsed Numeral: 152 # Parse large number after text conversion large_number_text = "يقدر عدد سكان العالم حوالي 7.9 مليار خلال عام 2022" parsed_large_numeral = parse_dimension(numbers_to_text(large_number_text), numeral=True)[0].value print(f"Parsed Large Numeral: {parsed_large_numeral}") # Expected: Parsed Large Numeral: 7900000000 # Parse decimal after text conversion decimal_text = "0.5" parsed_decimal = parse_dimension(numbers_to_text(decimal_text), numeral=True)[0].value print(f"Parsed Decimal: {parsed_decimal}") # Expected: Parsed Decimal: 0.5 ``` -------------------------------- ### Check for Arabic Hashtags in Text using maha.contains Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/contains_fn/index.md This example demonstrates how to detect Arabic hashtags within a text. By setting `arabic_hashtags=True`, the `contains` function utilizes the predefined `EXPRESSION_ARABIC_HASHTAGS` pattern to identify them. The function returns a boolean. ```python from maha import contains text = "This is a sample text." result = contains(text, arabic_hashtags=True) print(f"Does the text contain Arabic hashtags? {result}") text_hashtag = "#تحليل_البيانات" result_hashtag = contains(text_hashtag, arabic_hashtags=True) print(f"Does the text contain Arabic hashtags? {result_hashtag}") ``` -------------------------------- ### Robust Punctuation Removal in Arabic Text with Maha Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md Provides an example of using Maha's `remove` function to effectively clean Arabic text containing punctuation, even when preceded by symbols like '#'. This highlights the robustness of the cleaner functions in handling real-world text data. ```python from maha.cleaners.functions import remove sample_text = "#بسم_الله_الرحمن_الرحيم" print(remove(sample_text, punctuations=True)) ``` -------------------------------- ### Extract Durations using Maha parse_dimension Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example demonstrates extracting time durations from text using `parse_dimension` with `duration=True`. It handles various units like days, months, seconds, minutes, and hours, as well as combinations and fractional values. The `value` attribute contains a `DurationValue` object, which can be normalized to a specific unit. ```python from maha.parsers.functions import parse_dimension parse_dimension("العام ثلاثمئة وستة وخمسون يوما او 12 شهرا", duration=True) [Dimension(body=ثلاثمئة وستة وخمسون يوما, value=DurationValue(values=[ValueUnit(value=356, unit=)], normalized_unit=), start=6, end=30, dimension_type=DimensionType.DURATION), Dimension(body=12 شهرا, value=DurationValue(values=[ValueUnit(value=12, unit=)], normalized_unit=), start=34, end=41, dimension_type=DimensionType.DURATION)] parse_dimension("مليون ثانية ودقيقة", duration=True)[0].value DurationValue(values=[ValueUnit(value=1, unit=), ValueUnit(value=1000000, unit=)], normalized_unit=) parse_dimension("مليون ثانية ودقيقة", duration=True)[0].value.normalized_value ValueUnit(value=1000060, unit=) parse_dimension("يوم ونصف دقيقة", duration=True)[0].value DurationValue(values=[ValueUnit(value=1, unit=), ValueUnit(value=0.5, unit=)], normalized_unit=) parse_dimension("ساعة الا ثلث", duration=True)[0].value.normalized_value ValueUnit(value=2400, unit=) parse_dimension("يومين و10 ساعات", duration=True)[0].value.values [ValueUnit(value=2, unit=), ValueUnit(value=10, unit=)] ``` -------------------------------- ### BaseProcessor: Get lines from text Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/processors/index.md The get_lines method in BaseProcessor returns a generator that yields lists of strings. Each list contains a specified number of lines, with the last list potentially being shorter. This is useful for processing large amounts of text in manageable chunks. ```python class BaseProcessor: ... def get_lines(self, n_lines=100): """ Returns a generator of list of strings with length of `n_lines` Parameters: n_lines (int): Number of lines to yield, Defaults to 100 Yields: List[str]: List of strings with length of `n_lines`. The last list maybe of length less than `n_lines`. """ pass ``` -------------------------------- ### Extract Numerals using Maha parse_dimension Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example shows how to extract numerical values from text, including those expressed in words (cardinal and ordinal numbers, fractions) and digits, using the `parse_dimension` function with `numeral=True`. It can handle large numbers and combinations of words and digits. The `value` attribute of the returned `Dimension` object holds the parsed numerical value. ```python from maha.parsers.functions import parse_dimension parse_dimension("العام ثلاثمئة وستة وخمسون يوما او 12 شهرا", numeral=True) [Dimension(body=ثلاثمئة وستة وخمسون, value=356, start=6, end=25, dimension_type=DimensionType.NUMERAL), Dimension(body=12, value=12, start=34, end=36, dimension_type=DimensionType.NUMERAL)] parse_dimension("مليون واربعمئة وستة عشر", numeral=True)[0].value 1000416 parse_dimension("خمسين فاصلة ثلاثة واربعين", numeral=True)[0].value 50.43 parse_dimension("تسعة عشر الف ومئة", numeral=True)[0].value 19100 parse_dimension("تسع وتسعين مليار وتسعة وتسعين الف وتسع مائة وتسع وتسعين ", numeral=True)[0].value 99000099999 parse_dimension("100 وخمسين", numeral=True)[0].value 150 parse_dimension("100 و1", numeral=True)[0].value 101 ``` -------------------------------- ### Parse Arabic Names Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example shows the basic usage of `parse_dimension` for extracting names from Arabic text. By setting the `names=True` argument, the function attempts to identify and return any recognized names present in the input string. This is a foundational step for any text processing that requires name recognition. ```python from maha.parsers.functions import parse_dimension # Example: Parsing "محمد" result = parse_dimension("محمد", names=True) print(result) ``` -------------------------------- ### Convert Numbers to Arabic Text with Maha Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md Demonstrates the core functionality of converting numbers within Arabic text to their textual representation using the numbers_to_text function. It shows examples with and without the 'accusative' parameter, illustrating different grammatical forms. The function handles integers, decimals, and large number formats. ```python from maha.cleaners.functions import numbers_to_text # Basic conversion print(numbers_to_text("في المكتبة 152 كتاب")) # Expected: 'في المكتبة مائة وإثنان وخمسون كتاب' # Conversion with accusative case print(numbers_to_text("في المكتبة 152 كتاب", accusative=True)) # Expected: 'في المكتبة مائة وإثنين وخمسين كتاب' # Conversion of decimals and large numbers print(numbers_to_text("يقدر عدد سكان العالم حوالي 7.9 مليار خلال عام 2022")) # Expected: 'يقدر عدد سكان العالم حوالي سبعة فاصلة تسعة مليار خلال عام ألفان وإثنان وعشرون' print(numbers_to_text("1,111,111")) # Expected: 'مليون ومائة وأحد عشر ألف ومائة وأحد عشر' print(numbers_to_text("0.5")) # Expected: 'خمسة من عشرة' print(numbers_to_text("101.00102")) # Expected: 'مائة وواحد فاصلة مائة وإثنان من مائة ألف' ``` -------------------------------- ### Build HTML Documentation Locally (Shell) Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Generates HTML versions of the project's documentation from the 'docs' directory. This allows contributors to preview formatting and check for Sphinx errors before submitting a pull request. ```shell make html ``` -------------------------------- ### Run Project Tests with Poetry and Tox (Shell) Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Executes all checks and tests defined in the project's tox configuration using Poetry as the build tool. This ensures that code changes do not break existing functionality and adhere to coding standards. ```shell poetry run tox ``` -------------------------------- ### Arabic Start Of Rub El Hizb Constant Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/constants/arabic/index.md Defines the constant for the Arabic 'Rub El Hizb' sign, marking the start of a Hizb (a section of the Quran). This is a single character string literal. ```python HIZB_START = '۞' ``` -------------------------------- ### Navigate to Project Directory Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Changes the current directory to the root of the cloned Maha project. This is a prerequisite for running subsequent project-specific commands. ```shell cd maha ``` -------------------------------- ### Create TimeValue with keyword arguments Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/rules/time/template/index.md Illustrates creating a TimeValue instance using keyword arguments, which can represent either absolute or relative time components. This is the second method for TimeValue instantiation. ```python from dateutil.relativedelta import relativedelta # Example with relative time components relativedelta(years=1, months=2, days=3) # Example with absolute time components relativedelta(year=2023, month=10, day=26) # Example combining relative and absolute components relativedelta(day=1, weekday=MO(1), hours=25) ``` -------------------------------- ### BaseProcessor: Get text statistics Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/processors/index.md The get method of BaseProcessor provides various statistics about the input text, such as unique characters, character length, and word length. It can return a single statistic or a dictionary of statistics based on the boolean flags provided. ```python class BaseProcessor: ... def get(self, unique_characters=False, character_length=False, word_length=False): """ Returns statistics about the provided text Parameters: unique_characters (bool, optional): Return all unique characters, by default False character_length (bool, optional): Return the character length of each string, by default False word_length (bool, optional): Return the word length of each string (split by space), by default False Returns: Union[Dict[str, Any], Any]: If one argument is set to True, its value is return If more than one argument is set to True, a dictionary is returned where keys are the True passed arguments with the corresponding values """ pass ``` -------------------------------- ### Get Fractions of Unit Pattern Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/rules/index.md Generates a regex pattern representing fractions of a given unit. This is useful for parsing numerical expressions involving fractional parts of measurements or quantities. ```python def get_fractions_of_unit_pattern(unit): """Returns the fractions of a unit pattern. * **Parameters:** **unit** (*str*) – The unit pattern. * **Returns:** Pattern for the fractions of the unit. * **Return type:** str """ # Implementation details for generating the pattern would go here. ``` -------------------------------- ### Clone Project Repository Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Clones the forked repository of the Maha project to your local machine. Ensure you clone your fork, not the original repository. Supports both SSH and HTTPS protocols. ```shell git clone https://github.com//maha.git ``` -------------------------------- ### Remove Hashtags from Text Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/remove_fn/index.md The `remove_hashtags` function uses a regular expression (`EXPRESSION_HASHTAGS`) to remove entire hashtags (strings starting with '#') from the text. This function is useful for cleaning social media data. ```python from maha.cleaners.functions.remove_fn import remove_hashtags text = "Check out this #awesome #project on GitHub!" cleaned_text = remove_hashtags(text) print(cleaned_text) ``` -------------------------------- ### Re-enter Poetry Virtual Environment Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Activates the virtual environment created by Poetry for the Maha project. Use this command to re-enter the environment after you have left it. ```shell poetry shell ``` -------------------------------- ### Parse Arabic Time Intervals Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This snippet demonstrates parsing time intervals from Arabic text. It covers intervals specified with start and end times, such as "من الساعة 9 الى 11 صباحا", and relative intervals like "من 13 هذا الشهر الى 20 الشهر القادم". The function returns `TimeInterval` objects, which can be used to calculate specific dates when combined with a reference `datetime` object. It also handles partial intervals, like specifying only a start date or end time. ```python from maha.parsers.functions import parse_dimension from datetime import datetime now = datetime(2021, 9, 15) # Example 1: Parsing "من الساعة 9 الى 11 صباحا" result1 = parse_dimension("من الساعة 9 الى 11 صباحا", time=True) print(result1) # Example 2: Parsing relative month interval and calculating dates interval2 = parse_dimension("من 13 هذا الشهر الى 20 الشهر القادم", time=True)[0].value print(interval2.start + now) print(interval2.end + now) # Example 3: Parsing interval with specific hours and minutes result3 = parse_dimension("الساعة ثلاثة الا ثلث للساعة 4 ونص", time=True)[0].value print(result3) # Example 4: Parsing interval with only a start date result4 = parse_dimension("من 6 اكتوبر", time=True)[0].value print(result4) # Example 5: Parsing interval with only an end time result5 = parse_dimension("حتى الرابعة وربع بعد العصر", time=True)[0].value print(result5) ``` -------------------------------- ### Utility Functions Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/index.md Provides utility functions for compiling rules, durations, and parsing specific formats. ```APIDOC ## Utility Functions ### compile_time_rules() Compiles time-related rules. ### compile_duration_rules() Compiles duration-related rules. ### get_fractions_of_unit_pattern(unit) **Description**: Returns the fractions of a unit pattern. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * None **Returns**: * **str** - Pattern for the fractions of the unit. ### wrap_pattern(pattern) **Description**: Adds start and end expressions to a given pattern. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * **pattern** (str) - The pattern to wrap. **Returns**: * **str** - The pattern with start and end expressions added. ### spaced_patterns(*patterns) **Description**: Returns a regex pattern that matches any of the given patterns, separated by spaces. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * **patterns** (*str*) - The patterns to match. **Returns**: * **str** - A regex pattern that matches any of the provided patterns separated by spaces. ### parse_duration(match) **Description**: Parses a duration match object. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * **match** - The match object to parse. ### parse_ordinal(match) **Description**: Parses an ordinal match object. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * **match** - The match object to parse. ### parse_time(match) **Description**: Parses a time match object. **Method**: N/A (Function) **Parameters**: * **Path Parameters**: * None * **Query Parameters**: * None * **Request Body**: * **match** - The match object to parse. ``` -------------------------------- ### Remove Mentions from Text Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/remove_fn/index.md The `remove_mentions` function uses a regular expression (`EXPRESSION_MENTIONS`) to identify and remove user mentions (strings starting with '@') from the text. This is commonly used in social media text preprocessing. ```python from maha.cleaners.functions.remove_fn import remove_mentions text = "Thanks for the shoutout, @user1 and @another_user!" cleaned_text = remove_mentions(text) print(cleaned_text) ``` -------------------------------- ### Create TimeValue with two datetimes Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/rules/time/template/index.md Demonstrates initializing a TimeValue instance by providing two datetime objects. This is one of the two ways to create a TimeValue. ```python from dateutil.relativedelta import relativedelta # Assuming datetime1 and datetime2 are defined datetime objects relativedelta(datetime1, datetime2) ``` -------------------------------- ### Check for English Letters in Text using maha.contains Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/contains_fn/index.md This example shows how to use the `contains` function to identify if a text includes English letters. By setting `english_letters=True`, the function checks for the presence of characters from the English alphabet. The output is a boolean. ```python from maha import contains text = "مرحبا بالعالم" result = contains(text, english_letters=True) print(f"Does the text contain English letters? {result}") text_english = "Hello World" result_english = contains(text_english, english_letters=True) print(f"Does the text contain English letters? {result_english}") ``` -------------------------------- ### Create New Feature Branch Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Creates a new branch for development, based on the latest 'upstream/main' branch. This isolates your work and facilitates easy updates. ```shell git checkout -b upstream/main ``` -------------------------------- ### Stage Changes for Commit Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/guidelines.md Stages all changes in the current directory for the next commit. Alternatively, specific files or directories can be added using `git add `. ```shell git add . ``` -------------------------------- ### Expression Class - Match Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/rexy/templates/expression/index.md Attempts to match the regex pattern from the beginning of a given text. ```APIDOC ## match(text) ### Description Match the pattern in the input `text` from the beginning. ### Method Instance Method ### Parameters #### Query Parameters * **text** (str) - Required - The text to match against. ### Returns * **Matched object** - The result of the match. ### Return Type `Match[str]` ``` -------------------------------- ### Remove Mentions from Text in Python Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/remove_fn/index.md Removes user mentions (words starting with '@') from a given text string. This function uses the EXPRESSION_MENTIONS pattern to identify and remove social media mentions. It processes a string input and returns the text with mentions removed. ```python from maha.cleaners.functions import remove_mentions text = "@test لو سمحت صديقنا تزورنا على المعرض لاستلام الجائزة" remove_mentions(text) ``` -------------------------------- ### Convert Arabic Numbers to Text Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/index.md Demonstrates the 'numbers_to_text' function, which converts numerical digits within a string to their corresponding Arabic text representation. The 'accusative' parameter allows for conversion to the accusative form. Input is a string, output is the text with numbers converted. ```python from maha.cleaners.functions import numbers_to_text text = "العدد 123" numbers_to_text(text) ``` ```python from maha.cleaners.functions import numbers_to_text text = "رأيت 5 قطط" numbers_to_text(text, accusative=True) ``` -------------------------------- ### Remove Hashtags from Text in Python Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/remove_fn/index.md Removes hashtags (words starting with '#') from a given text string. This function utilizes the EXPRESSION_HASHTAGS pattern to identify and remove these elements, commonly used in social media contexts. It accepts a string and returns the text with hashtags stripped. ```python from maha.cleaners.functions import remove_hashtags text = "ويمكن القول أن مكة المكرمة من أجمل المناطق على وجه الأرض #السعودية" remove_hashtags(text) ``` -------------------------------- ### Expression Class - Initialization Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/rexy/templates/expression/index.md Initializes an Expression object with a regex pattern. Optionally compiles the pattern for faster subsequent operations. ```APIDOC ## Expression(pattern, pickle=False) ### Description Initializes a regex pattern holder. ### Parameters * **pattern** (str) - Required - The regular expression pattern. * **pickle** (bool) - Optional - If True, the compiled pattern will be pickled to save compilation time for large patterns. ``` -------------------------------- ### Regex Utility Functions Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/rexy/index.md Provides utility functions for creating different types of regex groups and patterns. ```APIDOC ## Regex Utility Functions ### `optional_non_capturing_group(*patterns)` Returns an optional non capturing group of patterns. * **Parameters:** * `*patterns` - A variable number of `Expression` objects or strings. ### `non_capturing_group(*patterns)` Returns a non capturing group of patterns. * **Parameters:** * `*patterns` - A variable number of `Expression` objects or strings. ### `positive_lookbehind(*patterns)` Returns a positive lookbehind pattern. * **Parameters:** * `*patterns` - A variable number of `Expression` objects or strings. ### `positive_lookahead(*patterns)` Returns positive lookahead pattern. * **Parameters:** * `*patterns` - A variable number of `Expression` objects or strings. ### `named_group(name, pattern)` Returns a named pattern group. * **Parameters:** * `name` (str) - The name for the group. * `pattern` (`Expression` or str) - The pattern to be named. ### `capture_group(*patterns)` Returns a capturing group pattern. * **Parameters:** * `*patterns` - A variable number of `Expression` objects or strings. ### Usage Examples ```python from maha.rexy import optional_non_capturing_group, named_group, capture_group # Optional non-capturing group opt_group = optional_non_capturing_group("a", "b") # Named group named = named_group("my_group", r"\d+") # Capture group cap_group = capture_group("start", "middle", "end") ``` ``` -------------------------------- ### Extract Distances using Maha parse_dimension Source: https://github.com/troboto/maha/blob/main/docs/source/overview.md This example shows how to extract distance measurements from text using `parse_dimension` with `distance=True`. It supports various units like kilometers, yards, centimeters, and meters, and can normalize them to a base unit (meters). The `value` attribute returns a `DistanceValue` object. ```python from maha.parsers.functions import parse_dimension parse_dimension("قطعت ثلاثين كيلو مترا خلال سفري.", distance=True) [Dimension(body=ثلاثين كيلو مترا, value=DistanceValue(value=30, unit=DistanceUnit.KILOMETERS), start=5, end=21, dimension_type=DimensionType.DISTANCE)] parse_dimension("مئة وعشرين ياردة", distance=True)[0].value DistanceValue(value=120, unit=DistanceUnit.YARDS) parse_dimension("ثلاثة سم", distance=True)[0].value.normalized_value ValueUnit(value=0.03, unit=) ``` -------------------------------- ### TextExpression Class Initialization (Python) Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/templates/text_expression/index.md Initializes a TextExpression object, which is an expression that returns the matched text as its value. It takes a required 'pattern' string and an optional 'pickle' boolean parameter. This class inherits from maha.rexy.Expression. ```python from maha.parsers.templates.text_expression import TextExpression # Example usage: text_expr = TextExpression(pattern="some_pattern") ``` -------------------------------- ### Expression Class - Full Match Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/rexy/templates/expression/index.md Attempts to match the regex pattern against the entire given text. ```APIDOC ## fullmatch(text) ### Description Match the pattern against the entire input `text`. ### Method Instance Method ### Parameters #### Query Parameters * **text** (str) - Required - The text to match against. ### Returns * **Matched object** - The result of the full match. ### Return Type `Match[str]` ``` -------------------------------- ### Arabic Patterns Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/index.md Predefined regex patterns for various Arabic linguistic elements. ```APIDOC ## Arabic Patterns ### THIRD Pattern that matches the pronunciation of 'third' in Arabic. ### QUARTER Pattern that matches the pronunciation of 'quarter' in Arabic. ### HALF Pattern that matches the pronunciation of 'half' in Arabic. ### THREE_QUARTERS Pattern that matches the pronunciation of 'three quarters' in Arabic. ### WAW_CONNECTOR Pattern that matches 'WAW' as a connector between two words. ### WORD_SEPARATOR Pattern that matches the word separator between numerals in Arabic. ### ALL_ALEF Pattern that matches all possible forms of 'ALEF' in Arabic. ### TWO_SUFFIX Pattern that matches the two-suffix of words in Arabic. ### SUM_SUFFIX Pattern that matches the sum-suffix of words in Arabic. ### EXPRESSION_START Pattern that matches the start of a rule expression in Arabic. ### EXPRESSION_END Pattern that matches the end of a rule expression in Arabic. ### FRACTIONS Placeholder for fractions pattern. ### TEH_OPTIONAL_SUFFIX Pattern: `'[ةه]?'` - Matches an optional 'Teh' suffix. ### AFTER Pattern related to 'after'. ### BEFORE Pattern related to 'before'. ### PREVIOUS Pattern related to 'previous'. ### NEXT Pattern related to 'next'. ### AFTER_NEXT Pattern related to 'after next'. ### BEFORE_PREVIOUS Pattern related to 'before previous'. ### IN_FROM_AT Pattern related to 'in', 'from', 'at'. ### FROM Pattern related to 'from'. ### TO Pattern related to 'to'. ### RULE_DISTANCE_KILOMETERS Pattern for distance in kilometers. ### RULE_DISTANCE_CENTIMETERS Pattern for distance in centimeters. ### RULE_DISTANCE_MILLIMETERS Pattern for distance in millimeters. ### RULE_DISTANCE_DECIMETERS Pattern for distance in decimeters. ### RULE_DISTANCE_METERS Pattern for distance in meters. ### RULE_DISTANCE_FEET Pattern for distance in feet. ### RULE_DISTANCE_INCHES Pattern for distance in inches. ### RULE_DISTANCE_YARDS Pattern for distance in yards. ### RULE_DISTANCE_MILES Pattern for distance in miles. ### RULE_DISTANCE General pattern for distance. ### RULE_DURATION_SECONDS Pattern for duration in seconds. ### RULE_DURATION_MINUTES Pattern for duration in minutes. ### RULE_DURATION_HOURS Pattern for duration in hours. ### RULE_DURATION_DAYS Pattern for duration in days. ### RULE_DURATION_WEEKS Pattern for duration in weeks. ### RULE_DURATION_MONTHS Pattern for duration in months. ### RULE_DURATION_YEARS Pattern for duration in years. ### RULE_DURATION General pattern for duration. ### RULE_NAME Pattern for rule names. ### RULE_NUMERAL_ONES Pattern for numeral ones. ### RULE_NUMERAL_TENS Pattern for numeral tens. ### RULE_NUMERAL_HUNDREDS Pattern for numeral hundreds. ### RULE_NUMERAL_THOUSANDS Pattern for numeral thousands. ### RULE_NUMERAL_MILLIONS Pattern for numeral millions. ### RULE_NUMERAL_BILLIONS Pattern for numeral billions. ### RULE_NUMERAL_TRILLIONS Pattern for numeral trillions. ### RULE_NUMERAL_INTEGERS Pattern for integer numerals. ### RULE_NUMERAL General pattern for numerals. ### RULE_ORDINAL_ONES Pattern for ordinal ones. ### RULE_ORDINAL_TENS Pattern for ordinal tens. ### RULE_ORDINAL_HUNDREDS Pattern for ordinal hundreds. ### RULE_ORDINAL_THOUSANDS Pattern for ordinal thousands. ### RULE_ORDINAL_MILLIONS Pattern for ordinal millions. ### RULE_ORDINAL_BILLIONS Pattern for ordinal billions. ### RULE_ORDINAL_TRILLIONS Pattern for ordinal trillions. ### RULE_ORDINAL General pattern for ordinals. ### RULE_TIME_YEARS Pattern for time in years. ### RULE_TIME_MONTHS Pattern for time in months. ### RULE_TIME_WEEKS Pattern for time in weeks. ### RULE_TIME_DAYS Pattern for time in days. ### RULE_TIME_HOURS Pattern for time in hours. ### RULE_TIME_MINUTES Pattern for time in minutes. ### RULE_TIME_AM_PM Pattern for AM/PM indicator. ### RULE_TIME_NOW Pattern for 'now'. ### RULE_TIME General pattern for time. ``` -------------------------------- ### Python Docstring Formatting - Correct vs. Incorrect Source: https://github.com/troboto/maha/blob/main/docs/source/contributing/docstring.md Demonstrates the correct and incorrect ways to format docstrings in Python. The correct method places the docstring on the same line as the opening quotes, while the incorrect method places it on a new line. ```python def do_this(): """This is correct. (...) """ def dont_do_this(): """ This is incorrect. (...) """ ``` -------------------------------- ### Check for Strings Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/contains_fn/index.md Determines if one or more specified strings are present within the given text. ```APIDOC ## POST /check/strings ### Description Checks if the provided text contains any of the specified strings. Can check for a single string or a list of strings. ### Method POST ### Endpoint /check/strings ### Parameters #### Request Body - **text** (string) - Required - The text to search within. - **strings** (string or list of strings) - Required - The string or list of strings to look for in the text. ### Request Example ```json { "text": "الله أكبر، الحمد لله رب العالمين", "strings": "الله" } ``` ### Response #### Success Response (200) - **strings_found** (boolean) - True if any of the provided strings are found in the text, False otherwise. #### Response Example ```json { "strings_found": true } ``` #### Error Response (400) - **error** (string) - Indicates that no strings were provided for the check. #### Error Example ```json { "error": "No strings provided to check." } ``` ``` -------------------------------- ### Replace Pairs Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/processors/base_processor/index.md Applies a replacement of specified keys with corresponding values to each line of text. ```APIDOC ## POST /replace_pairs ### Description Applies a replacement of specified keys with corresponding values to each line of text. ### Method POST ### Endpoint /replace_pairs ### Parameters #### Request Body - **keys** (list[str]) - Required - A list of strings to be replaced. - **values** (list[str]) - Required - A list of strings to replace the keys with. ### Request Example ```json { "keys": ["old_string1", "old_string2"], "values": ["new_string1", "new_string2"] } ``` ### Response #### Success Response (200) - **processed_text** (str) - The text after applying the replacements. #### Response Example ```json { "processed_text": "This is the text with replacements." } ``` ``` -------------------------------- ### Expression Class - Load from Cache Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/rexy/templates/expression/index.md Loads a compiled Expression object from a cache file. ```APIDOC ## from_cache(cache) ### Description Load an expression from cache. ### Method Class Method ### Parameters #### Query Parameters * **cache** (str) - Required - Name of the cache file. ### Returns * **Expression** - The loaded Expression object. ### Return Type [`Expression`](#maha.rexy.templates.expression.Expression) ``` -------------------------------- ### Drop Empty Lines Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/processors/base_processor/index.md Removes all lines that are empty. ```APIDOC ## POST /drop_empty_lines ### Description Removes all lines from the text that are empty. ### Method POST ### Endpoint /drop_empty_lines ### Parameters There are no parameters for this endpoint. ### Request Example ```json { "text": "Line 1\n\nLine 3" } ``` ### Response #### Success Response (200) - **processed_text** (str) - The text with empty lines removed. #### Response Example ```json { "processed_text": "Line 1\nLine 3" } ``` ``` -------------------------------- ### Applying TimeValue to a datetime Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/parsers/rules/time/template/index.md Shows how to add a TimeValue (representing a relative or absolute time difference) to an existing datetime object. The order of application (absolute then relative) and specific rules for arguments like 'weekday' are demonstrated. ```python from datetime import datetime from dateutil.relativedelta import relativedelta, MO dt = datetime(2018, 4, 9, 13, 37, 0) delta = relativedelta(hours=25, day=1, weekday=MO(1)) result = dt + delta print(result) ``` -------------------------------- ### Replace Pairs of Strings in Text using Python Source: https://github.com/troboto/maha/blob/main/docs/source/autoapi/maha/cleaners/functions/index.md Replaces multiple key-value pairs within the input text. It takes lists of keys (strings to be replaced) and values (strings to replace with) and performs the substitutions accordingly. Raises ValueError if the lengths of keys and values lists do not match. ```python from maha.cleaners.functions import replace_pairs text = "This is a test string with old_word and another_word." keys = ["old_word", "another_word"] values = ["new_word", "yet_another_word"] replace_pairs(text, keys, values) ```