### Node.js Example: PyCantonese with Pyodide Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/quickstart.md This JavaScript snippet demonstrates how to set up Pyodide in a Node.js environment, install PyCantonese and Rustling, and execute PyCantonese functions like characters_to_jyutping. ```javascript const { loadPyodide } = require("pyodide"); async function main() { const pyodide = await loadPyodide(); await pyodide.loadPackage("micropip"); const micropip = pyodide.pyimport("micropip"); // Update these version numbers as needed. const RUSTLING_VERSION = "0.8.0"; const PYCANTONESE_VERSION = "5.0.0"; const EMSCRIPTEN_TAG = "cp310-abi3-emscripten_4_0_9_wasm32"; await micropip.install( `https://github.com/jacksonllee/rustling/releases/download/v${RUSTLING_VERSION}/rustling-${RUSTLING_VERSION}-${EMSCRIPTEN_TAG}.whl` ); await micropip.install( `https://github.com/jacksonllee/pycantonese/releases/download/v${PYCANTONESE_VERSION}/pycantonese-${PYCANTONESE_VERSION}-${EMSCRIPTEN_TAG}.whl` ); const jyutping = pyodide.runPython( ` import pycantonese pycantonese.characters_to_jyutping("香港人講廣東話") ` ); console.log(jyutping.toJs()); // [['香港人', 'hoeng1 gong2 jan4'], ['講', 'gong2'], ['廣東話', 'gwong2 dung1 waa2']] } main(); ``` -------------------------------- ### Install Baseline Version Source: https://github.com/jacksonllee/pycantonese/blob/main/benchmarks/README.md Installs the v3.4.0 baseline for comparison into a local directory. Delete the directory and re-run to reinstall. ```bash bash benchmarks/setup_baseline.sh ``` -------------------------------- ### Install PyCantonese Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Installs the PyCantonese library, pinning the version to v3.3.1 for reproducibility. ```python !pip install pycantonese==3.3.1 ``` -------------------------------- ### Install nskipgrams Package Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Install the `nskipgrams` package using pip, which is useful for creating n-grams from sequences. ```python !pip install nskipgrams==0.3.0 ``` -------------------------------- ### Install PyCantonese using pip Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/index.md Install the PyCantonese library using pip. This is the standard method for installing Python packages. ```bash pip install pycantonese ``` -------------------------------- ### Install PyCantonese using uv Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/index.md Install the PyCantonese library using the uv package manager. uv is a fast Python package installer. ```bash uv add pycantonese ``` -------------------------------- ### Browser Example: PyCantonese with Pyodide Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/quickstart.md This HTML snippet shows how to load Pyodide, install PyCantonese and Rustling wheels, and use PyCantonese's characters_to_jyutping function directly in a web page. ```html ``` -------------------------------- ### Set Up Development Environment Source: https://github.com/jacksonllee/pycantonese/blob/main/CONTRIBUTING.md Clone the repository, create a virtual environment, install maturin, and develop the package with development dependencies. ```bash git clone https://github.com//pycantonese.git cd pycantonese uv venv uv pip install maturin uv run maturin develop -E dev ``` -------------------------------- ### Install PyCantonese using pip Source: https://github.com/jacksonllee/pycantonese/blob/main/README.rst Upgrade PyCantonese to the latest version using pip. ```bash pip install --upgrade pycantonese ``` -------------------------------- ### HKCanCor CHAT File Example Source: https://github.com/jacksonllee/pycantonese/blob/main/src/pycantonese/data/hkcancor/README.md An example of the beginning of a HKCanCor CHAT data file, showing headers and the first three utterances with their transcriptions and morphological annotations. ```CHAT @UTF8 @Begin @Languages: yue , eng @Participants: XXA A Adult , XXB B Adult @ID: yue , eng|HKCanCor|XXA|34;|female|||Adult||origin:HK| @ID: yue , eng|HKCanCor|XXB|37;|female|||Adult||origin:HK| @Date: 30-APR-1997 @Tape Number: 001 *XXA: 喂遲啲去唔去旅行啊 ? %mor: e|wai3 a|ci4 u|di1 v|heoi3 d|m4 v|heoi3 vn|leoi5hang4 y|aa3 ? *XXA: 你老公有冇平機票啊 ? %mor: r|nei5 n|lou5gung1 v1|jau5mou5 a|peng4 n|gei1piu3 y|aa3 ? *XXB: 平機票要淡季先有得平𡃉喎 . %mor: a|peng4 n|gei1piu3 vu|jiu3 an|daam6gwai3 d|sin1 vu|jau5dak1 a|peng4 y|gaa3 y|wo3 . ``` -------------------------------- ### Create New Benchmark Script Source: https://github.com/jacksonllee/pycantonese/blob/main/benchmarks/README.md Example structure for a new benchmark script, including importing utilities, defining a benchmark function, and implementing the main logic for comparison. ```python from _utils import bench, print_comparison, run_baseline def benchmark(): import pycantonese results = {} results["my_benchmark"] = bench(lambda: pycantonese.some_func(), n=10) return results def main(): if "--json" in sys.argv: print(json.dumps(benchmark())) return print("My benchmark") current = benchmark() baseline = run_baseline(__file__) print_comparison(current, baseline) ``` -------------------------------- ### Display Words Starting with 'a' Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Prints the list of words that were identified as starting with the letter 'a'. ```python start_with_a ``` -------------------------------- ### Install PyCantonese using conda Source: https://github.com/jacksonllee/pycantonese/blob/main/README.rst Install PyCantonese using conda from the conda-forge channel. ```bash conda install -c conda-forge pycantonese ``` -------------------------------- ### Display CHAT File Header Example Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Shows a typical CHAT file header format, including language, participant, and date information. This format is parsed by PyCantonese. ```default @UTF8 @Begin @Languages: yue , eng @Participants: XXA A Adult , XXB B Adult @ID: yue , eng|HKCanCor|XXA|34;|female|||Adult|| @ID: yue , eng|HKCanCor|XXB|37;|female|||Adult|| @Date: 30-APR-1997 @Tape Number: 001 ``` -------------------------------- ### Import PyCantonese Package Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/quickstart.md Import the pycantonese package to begin using its functionalities. Ensure the package is installed before importing. ```python import pycantonese ``` -------------------------------- ### Install PyCantonese Dependencies Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Installs seaborn, matplotlib, and pandas, which are commonly used for data visualization and manipulation with PyCantonese. ```shell !pip install seaborn==0.11.1 matplotlib pandas ``` -------------------------------- ### Access List Elements and Slice Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Demonstrates accessing individual elements by index and slicing lists to get sub-lists. ```python result[0] result[-1] result[:2] ``` -------------------------------- ### Example CHAT File Content Source: https://github.com/jacksonllee/pycantonese/blob/main/src/pycantonese/data/cantomap/README.md Illustrates the structure of extracted CHAT files, showing utterances with main and morphology tiers, including part-of-speech tags. ```text *XXE: 好咁樣 . %mor: a|hou2 d|gam2joeng2 . *XXG: 起點喺引依湖 . %mor: n|hei2dim2 v|hai2 n|jan5ji1wu4 . ``` -------------------------------- ### Search by Part-of-Speech Tag (Regex) Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/searches.md Use the `pos` parameter with a regular expression to find tokens matching a pattern of part-of-speech tags. This example finds all verbs. ```python all_verbs = corpus.search(pos='^v') len(all_verbs) ## number of all verbs -- more than just "v" alone above # 29663 all_verbs[:20] ## printing the first 20 results # [Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='旅行', pos='vn', jyutping='leoi5hang4', mor=None, gloss=None, gra=None), # Token(word='有冇', pos='v1', jyutping='jau5mou5', mor=None, gloss=None, gra=None), # Token(word='要', pos='vu', jyutping='jiu3', mor=None, gloss=None, gra=None), # Token(word='有得', pos='vu', jyutping='jau5dak1', mor=None, gloss=None, gra=None), # Token(word='冇得', pos='vu', jyutping='mou5dak1', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None), # Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None), # Token(word='聽', pos='v', jyutping='teng1', mor=None, gloss=None, gra=None), # Token(word='講', pos='v', jyutping='gong2', mor=None, gloss=None, gra=None), # Token(word='話', pos='v', jyutping='waa6', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='玩', pos='v', jyutping='waan2', mor=None, gloss=None, gra=None), # Token(word='可以', pos='vu', jyutping='ho2ji5', mor=None, gloss=None, gra=None), # Token(word='住', pos='v', jyutping='zyu6', mor=None, gloss=None, gra=None), # Token(word='話', pos='v', jyutping='waa6', mor=None, gloss=None, gra=None), # Token(word='跟', pos='v', jyutping='gan1', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None)] ``` -------------------------------- ### Verifying Segmented Words with Offsets Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/word_segmentation.md This example demonstrates how to use the character offsets returned by `segment(offsets=True)` to verify that slicing the original text with these offsets correctly recovers the segmented words. ```python text = "廣東話容唔容易學？" for word, (start, end) in pycantonese.segment(text, offsets=True): assert text[start:end] == word ``` -------------------------------- ### Get Corpus Information Summary Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Obtain a summary of the corpus, including the total number of files and utterances. Useful for a quick overview of the dataset size. ```python corpus.info() ## 58 file(s), 16162 utterance(s) ``` -------------------------------- ### Search by Word Range Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/searches.md Include neighboring words in search results using the `word_range` parameter. This example searches for '過' and includes one word before and two words after the match within the same utterance. ```python gwo3 = corpus.search(character='過', word_range=(1, 2)) len(gwo3) # 705 gwo3[:5] # [[Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='過', pos='u', jyutping='gwo3', mor=None, gloss=None, gra=None), # Token(word='喇', pos='y', jyutping='laa1', mor=None, gloss=None, gra=None), # Token(word='.', pos='', jyutping=None, mor=None, gloss=None, gra=None)], # [Token(word='不過', pos='c', jyutping='bat1gwo3', mor=None, gloss=None, gra=None), # Token(word='幾', pos='m', jyutping='gei2', mor=None, gloss=None, gra=None), # Token(word='日', pos='q', jyutping='jat6', mor=None, gloss=None, gra=None)], # [Token(word='去', pos='vk', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='過', pos='v', jyutping='gwo3', mor=None, gloss=None, gra=None), # Token(word='嗰邊', pos='r', jyutping='go2bin1', mor=None, gloss=None, gra=None), # Token(word='瞓覺', pos='v', jyutping='fan3gaau3', mor=None, gloss=None, gra=None)], # [Token(word='不過', pos='c', jyutping='bat1gwo3', mor=None, gloss=None, gra=None), # Token(word='都', pos='d', jyutping='dou1', mor=None, gloss=None, gra=None), # Token(word=',', pos='', jyutping=None, mor=None, gloss=None, gra=None)], # [Token(word=',', pos='', jyutping=None, mor=None, gloss=None, gra=None), # Token(word='不過', pos='c', jyutping='bat1gwo3', mor=None, gloss=None, gra=None), # Token(word='真係', pos='d', jyutping='zan1hai6', mor=None, gloss=None, gra=None), # Token(word='好', pos='d', jyutping='hou2', mor=None, gloss=None, gra=None)]] ``` -------------------------------- ### Search HKCanCor Corpus for Verbs Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/quickstart.md Access and search the HKCanCor corpus to find words tagged as verbs. This example demonstrates how to retrieve corpus data and filter it by part-of-speech. ```python corpus = pycantonese.hkcancor() # get HKCanCor all_verbs = corpus.search(pos='^[Vv]') len(all_verbs) # number of all verbs # 29726 all_verbs[:10] ## print 10 results # [Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='旅行', pos='vn', jyutping='leoi5hang4', mor=None, gloss=None, gra=None), # Token(word='有冇', pos='v1', jyutping='jau5mou5', mor=None, gloss=None, gra=None), # Token(word='要', pos='vu', jyutping='jiu3', mor=None, gloss=None, gra=None), # Token(word='有得', pos='vu', jyutping='jau5dak1', mor=None, gloss=None, gra=None), # Token(word='冇得', pos='vu', jyutping='mou5dak1', mor=None, gloss=None, gra=None), # Token(word='去', pos='v', jyutping='heoi3', mor=None, gloss=None, gra=None), # Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None), # Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None)] ``` -------------------------------- ### Search by Utterance Range Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/searches.md Include neighboring utterances in search results using the `utterance_range` parameter. This example searches for 'laa1' and includes one utterance before and one utterance after the match. ```python laa1 = corpus.search(jyutping='laa1', utterance_range=(1, 1)) len(laa1) # 1681 laa1[0] ## print the 1st result # [[Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None), # Token(word='唔係', pos='v', jyutping='m4hai6', mor=None, gloss=None, gra=None), # Token(word='啊', pos='y', jyutping='aa3', mor=None, gloss=None, gra=None), # Token(word='?', pos='', jyutping=None, mor=None, gloss=None, gra=None)], # [Token(word='你', pos='r', jyutping='nei5', mor=None, gloss=None, gra=None), # Token(word='係', pos='v', jyutping='hai6', mor=None, gloss=None, gra=None), # Token(word='唔係', pos='v', jyutping='m4hai6', mor=None, gloss=None, gra=None), # Token(word='啊', pos='y', jyutping='aa3', mor=None, gloss=None, gra=None), # Token(word='?', pos='', jyutping=None, mor=None, gloss=None, gra=None)]] ``` -------------------------------- ### Read Cantonese CHAT Corpus from URL Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Reads a Cantonese corpus in CHAT format from a provided ZIP file URL. This example uses the Lee-Wong-Leung corpus from TalkBank. ```python corpus = pycantonese.read_chat("https://childes.talkbank.org/data/Chinese/Cantonese/LeeWongLeung.zip") ``` -------------------------------- ### Get Most Common Items Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Retrieves the top N most frequent items from a Counter object. This example gets the top 4. ```python word_counter.most_common(4) # top 4 most common items and their counts ``` -------------------------------- ### Word Segmentation with Character Offsets Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/word_segmentation.md Pass `offsets=True` to the `segment()` function to get each word paired with its start and end character offsets in the original string. This is useful for mapping segmented words back to their positions. ```python pycantonese.segment("廣東話容唔容易學？", offsets=True) # [('廣東話', (0, 3)), ('容', (3, 4)), ('唔', (4, 5)), # ('容易', (5, 7)), ('學', (7, 8)), ('？', (8, 9))] ``` -------------------------------- ### Create a Dictionary Mapping Strings to Strings Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Initializes a dictionary where keys and values are strings, representing animal sounds. ```python animals = { "dog": "woof", "cat": "meow", "pig": "oink", } ``` -------------------------------- ### Basic POS Tagging with Universal Dependencies Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/pos_tagging.md Demonstrates how to perform part-of-speech tagging on a segmented sentence using the default Universal Dependencies tagset. Requires the `pycantonese` library and prior word segmentation. ```python import pycantonese unsegmented = '我噚日買對鞋。' # I bought a pair of shoes yesterday. segmented = pycantonese.segment(unsegmented) segmented # ['我', '噚日', '買', '對鞋', '。'] pycantonese.pos_tag(segmented) # [('我', 'PRON'), ('噚日', 'ADV'), ('買', 'VERB'), ('對鞋', 'NOUN'), ('。', 'PUNCT')] ``` -------------------------------- ### Display Words Not Starting with 'a' Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Prints the list of words that were identified as not starting with the letter 'a'. ```python not_start_with_a ``` -------------------------------- ### Create a List of Strings Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Initializes a list containing string elements. ```python words = ["cats", "dogs"] ``` -------------------------------- ### head Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/api.md Returns the first n utterances with a formatted display. ```APIDOC ## head ### Description Returns the first n utterances with a formatted display. ### Method Instance Method ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters * **n** (*int*) – Default is 5. ### Returns Formatted utterances. ``` -------------------------------- ### Get Dictionary Type Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Confirms that a variable holds a dictionary data type. ```python type(animals) ``` -------------------------------- ### Run Benchmarks Source: https://github.com/jacksonllee/pycantonese/blob/main/benchmarks/README.md Executes the benchmark scripts for corpus loading, POS tagging, and word segmentation. If the baseline is not set up, only current version results are shown. ```bash uv run python benchmarks/run_corpus.py uv run python benchmarks/run_pos_tagging.py uv run python benchmarks/run_word_segmentation.py ``` -------------------------------- ### Get List Type Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Confirms that a variable holds a list data type. ```python type(words) ``` -------------------------------- ### Get String Type Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Determines the data type of a variable, confirming it is a string. ```python type(x) ``` -------------------------------- ### Run Benchmarks with JSON Output Source: https://github.com/jacksonllee/pycantonese/blob/main/benchmarks/README.md Runs benchmark scripts to produce machine-readable JSON output to standard output. ```bash uv run python benchmarks/run_corpus.py --json ``` -------------------------------- ### Display Sample Transcriptions Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Show a preview of the transcriptions from the corpus, including speaker, words, and morphological information. This method provides a glimpse into the raw data. ```python corpus.head() # *XXA: 喂遲啲去唔去旅行啊 ? # %mor: e|wai3 a|ci4 u|di1 v|heoi3 d|m4 v|heoi3 vn|leoi5hang4 y|aa3 ? # # *XXA: 你老公有冇平機票啊 ? # %mor: r|nei5 n|lou5gung1 v1|jau5mou5 a|peng4 n|gei1piu3 y|aa3 ? # # *XXB: 平機票要淡季先有得平𡃉喎 . # %mor: a|peng4 n|gei1piu3 vu|jiu3 an|daam6gwai3 d|sin1 vu|jau5dak1 a|peng4 y|gaa3 y|wo3 . # # *XXB: 而家旺 - . # %mor: t|ji4gaa1 a|wong6 - . # # *XXA: 冇得去嗱 . # %mor: vu|mou5dak1 v|heoi3 y|laa4 . ``` -------------------------------- ### Slice a String Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Extracts a portion of a string from a starting index up to (but not including) an ending index. ```python x[5:10] ``` -------------------------------- ### Run Code Style and Linting Checks Source: https://github.com/jacksonllee/pycantonese/blob/main/CONTRIBUTING.md Execute flake8 for linting and black for code formatting checks on the source and test directories. ```bash uvx flake8 src tests uvx black --check src tests ``` -------------------------------- ### Get List Length Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Determines the number of elements in a list using the len() function. ```python len(words) ``` -------------------------------- ### Get String Length Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Calculates the number of characters in a string using the len() function. ```python len(x) ``` -------------------------------- ### Initialize CSV File for Annotation Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-cantonese-childes-2022-05-27.ipynb This Python code initializes the process for creating a CSV file to be used for manual annotation. It sets up the file in write mode and defines the header row with the necessary columns for annotation. ```python import csv from typing import List from pycantonese.corpus import CHATReader from pylangacq.objects import Utterance def create_annotation_csv(corpus: CHATReader, child_in_csv_filename: str) -> List[Utterance]: # `child` (case-sensitive) from the file paths: Tim, Sophie, Alicia, etc. # Start an empty list. We'll keep track of the utterances to annotate in this list. utterances_to_annotate = [] with open(f"annotations_{child_in_csv_filename}.csv", "w") as csv_file: # Start writing the CSV on disk (set the column names, write the header). ``` -------------------------------- ### Initialize a Counter Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Sets up an empty Counter object from the collections module for tallying items. ```python from collections import Counter word_counter = Counter() # Initialize an empty counter ``` -------------------------------- ### Get All Words from Corpus Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Retrieves all words from the HKCanCor corpus as a flat list. Note that this removes utterance-level structure. ```python words = corpus.words() ``` -------------------------------- ### Display First 3 Custom Search Results Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Show the first few utterances that were found by a custom search. This helps in verifying the search results. ```python result[:3] ``` -------------------------------- ### Display First 10 Words Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Shows the first 10 words from the HKCanCor corpus. The output may be truncated. ```python words[:10] ``` -------------------------------- ### Simple Conditional Statement Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Executes a print statement if a condition is true. This example checks if 3 is greater than 2. ```python if 3 > 2: print("hi") ``` -------------------------------- ### Display First Two Utterances Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Shows the first two utterances from the HKCanCor corpus, where each utterance is a list of words. ```python words_by_utterances[:2] ``` -------------------------------- ### head() Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Displays the beginning of the transcriptions and annotations for the corpus data, showing utterances with associated linguistic information. ```APIDOC ## head() ### Description Displays the beginning of the transcriptions and annotations for the corpus data, showing utterances with associated linguistic information. ### Method `corpus.head()` ### Parameters None ### Response - A string representation of the initial utterances and their annotations. ### Request Example ```python corpus.head() ``` ### Response Example ``` *XXA: 喂遲啲去唔去旅行啊 ? %mor: e|wai3 a|ci4 u|di1 v|heoi3 d|m4 v|heoi3 vn|leoi5hang4 y|aa3 ? *XXA: 你老公有冇平機票啊 ? %mor: r|nei5 n|lou5gung1 v1|jau5mou5 a|peng4 n|gei1piu3 y|aa3 ? *XXB: 平機票要淡季先有得平𡃉喎 . %mor: a|peng4 n|gei1piu3 vu|jiu3 an|daam6gwai3 d|sin1 vu|jau5dak1 a|peng4 y|gaa3 y|wo3 . *XXB: 而家旺 - . %mor: t|ji4gaa1 a|wong6 - . *XXA: 冇得去嗱 . %mor: vu|mou5dak1 v|heoi3 y|laa4 . ``` ``` -------------------------------- ### Get Words by Utterances Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Retrieves words from the HKCanCor corpus, preserving utterance structure. Each utterance is represented as a list of words. ```python words_by_utterances = corpus.words(by_utterances=True) ``` -------------------------------- ### Conditional Logic for Word Sorting Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Splits a text into words, converts them to lowercase, and categorizes them into two lists based on whether they start with 'a'. ```python text = "Among the languages that are spoken today, only few are even tolerably well known to science. Of many we have inadequate information, of others none at all." # The very first sentence of the chapter "The Languages of the World" in Bloomfield's (1933) _Language_ start_with_a = [] not_start_with_a = [] for word in text.strip().lower().split(): if word.startswith("a"): start_with_a.append(word) else: not_start_with_a.append(word) ``` -------------------------------- ### G2P Conversion with Pre-segmented Input Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/g2p.md If you prefer to supply your own word segmentation, pass a list of words to the `g2p()` function instead of a raw string. This gives you explicit control over how the input is divided. ```python import pycantonese pycantonese.g2p(['廣東', '話']) # Cantonese # [('廣東', 'kʷɔŋ25 tʊŋ55'), ('話', 'waː22')] ``` -------------------------------- ### Initialize HKCanCor Corpus Reader Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Create a corpus reader instance for the HKCanCor dataset. This is the first step to access its data. ```python import pycantonese corpus = pycantonese.hkcancor() ``` -------------------------------- ### Extract First 10 Words Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/reader.md Get a list of the first 10 words from the corpus. This method extracts only the word strings, without any annotations. ```python corpus.words()[:10] # ['喂', '遲', '啲', '去', '唔', '去', '旅行', '啊', '?', '你'] ``` -------------------------------- ### Get Default Stop Words Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/stop_words.md Retrieves the default set of Cantonese stop words. The number of stop words is typically around 100. ```python import pycantonese stop_words = pycantonese.stop_words() len(stop_words) # 104 stop_words # doctest: +SKIP ## {'一啲', '一定', '不如', '不過', ...} ``` -------------------------------- ### Get Tokens by Utterances Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Retrieves tokens from the corpus, with each utterance represented as a list of Token objects. This allows access to word, part-of-speech, and Jyutping information. ```python tokens_by_utterances = corpus.tokens(by_utterances=True) ``` -------------------------------- ### Import Libraries for Cantonese Data Analysis Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-cantonese-childes-2022-05-27.ipynb Imports the pycantonese library along with pandas and seaborn for data manipulation and visualization. Assumes these packages are already installed. ```python # Assumption: your Python environment has pycantonese, pandas, and seaborn available. import pycantonese # We'll use these other packages later. import pandas as pd import seaborn as sns sns.set(color_codes=True) %matplotlib inline ``` -------------------------------- ### Print Word and POS Tag for First Utterance Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-pycantonese-2021-05-16.ipynb Iterates through the tokens of the first utterance and prints each word along with its corresponding part-of-speech tag. ```python for token in tokens_by_utterances[0]: print(f"{token.word} {token.pos}") ``` -------------------------------- ### Handle Multiple Pronunciations with Characters-to-Jyutping Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/jyutping.md Demonstrates how characters_to_jyutping() handles multiple pronunciations of a Cantonese character based on context, such as '蛋' (egg) versus '蛋糕' (cake). ```python import pycantonese ## The correct pronunciation of 蛋 is with tone 2 (high-rising) as a standalone word. pycantonese.characters_to_jyutping('蛋') # egg # [('蛋', 'daan2')] ## The correct pronunciation of 蛋 is with tone 6 (low-level) in 蛋糕. pycantonese.characters_to_jyutping('蛋糕') # cake # [('蛋糕', 'daan6 gou1')] ``` -------------------------------- ### Regenerate CantoMap Data Source: https://github.com/jacksonllee/pycantonese/blob/main/src/pycantonese/data/cantomap/README.md Command to re-download source .eaf files and regenerate .cha files using the download script. Requires Git LFS to be installed. ```bash python src/pycantonese/data/cantomap/download.py ``` -------------------------------- ### Conditional Statement with Else Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Executes one block of code if a condition is true, and another block if the condition is false. This example checks if 3 is less than 2. ```python if 3 < 2: print("hi") else: print("bye") ``` -------------------------------- ### Create a String Variable Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Assigns a text sequence to a variable. ```python x = "this is a string" ``` -------------------------------- ### Download File from Google Colab Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-python-2021-april.ipynb Provides functionality to download a specified file from the Google Colab environment to the user's local computer. Requires the 'files' module from 'google.colab'. ```python from google.colab import files files.download("results.csv") ``` -------------------------------- ### Displaying Utterances with Audio Playback Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/tutorials/lee-cantonese-childes-2022-05-27.ipynb Iterates through a list of utterances and displays each one with an HTML representation that includes an audio playback button. This facilitates manual annotation by allowing researchers to listen to each utterance. This feature is experimental and may differ in future releases. ```python from IPython.display import display, HTML # Showing just the first 10 utterances in this demo notebook for u in utterances_to_annotate[:10]: display(HTML(u._repr_html_())) ``` -------------------------------- ### POS Tagging with HKCanCor Tagset Source: https://github.com/jacksonllee/pycantonese/blob/main/docs/pos_tagging.md Shows how to obtain part-of-speech tags using the original HKCanCor tagset by specifying the `tagset='hkcancor'` argument. This is useful if you need to work with the raw HKCanCor tags. ```python pycantonese.pos_tag(segmented, tagset="hkcancor") # [('我', 'r'), ('噚日', 't'), ('買', 'v'), ('對鞋', 'n'), ('。', '。')] ```