### jieba Command Line - Basic Segmentation

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Example of using jieba from the command line for basic word segmentation. It takes an input filename and outputs the segmented words to a file, using '/' as a delimiter by default.

```Shell
python -m jieba news.txt > cut_result.txt
```

--------------------------------

### Jieba: Loading Custom Dictionary

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Illustrates how to load a custom dictionary in Jieba to improve segmentation accuracy for specific terms. The example shows the format of a custom dictionary file with words, their frequencies, and POS tags, and how it affects segmentation.

```Python
# Example custom dictionary format:
# 创新办 3 i
# 云计算 5
# 凱特琳 nz
# 台中

# Example of changing tmp_dir and cache_file for restricted file systems:
# 云计算 5
# 李小福 2
# 创新办 3

# [Before]: 李小福 / 是 / 创新 / 办 / 主任 / 也 / 是 / 云 / 计算 / 方面 / 的 / 专家 /
# [After]:　李小福 / 是 / 创新办 / 主任 / 也 / 是 / 云计算 / 方面 / 的 / 专家 /
```

--------------------------------

### Jieba: Handling Unknown Words

Source: https://github.com/fxsjy/jieba/blob/master/README.md

This snippet demonstrates how Jieba handles words not present in its dictionary by using the Viterbi algorithm for new word discovery. The example shows segmentation of a sentence containing an unknown word.

```Python
jieba.cut('丰田太省了', HMM=False)
jieba.cut('我们中出了一个叛徒', HMM=False)
```

--------------------------------

### Tokenize Chinese Text (Default Mode)

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates the default tokenization mode of Jieba using `jieba.tokenize`, which returns words along with their start and end positions in the original text. The input must be Unicode.

```Python
import jieba

# Example of default tokenization
result = jieba.tokenize(u'永和服装饰品有限公司')
for tk in result:
    print("word %s\t\t start: %d \t\t end:%d" % (tk[0],tk[1],tk[2]))
```

--------------------------------

### jieba Tokenization - Search Mode

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates jieba's tokenize function in search mode, which provides more granular segmentation suitable for search engines. It also returns the start and end positions of each token.

```Python
result = jieba.tokenize(u'永和服装饰品有限公司', mode='search')
for tk in result:
    print("word %s\t\t start: %d \t\t end:%d" % (tk[0],tk[1],tk[2]))
```

--------------------------------

### jieba Tokenization - Default Mode

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to use jieba's tokenize function in default mode to get words along with their start and end positions in the original text. The input must be a unicode string.

```Python
result = jieba.tokenize(u'永和服装饰品有限公司')
for tk in result:
    print("word %s\t\t start: %d \t\t end:%d" % (tk[0],tk[1],tk[2]))
```

--------------------------------

### Adjusting jieba Word Frequency (Python)

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to dynamically adjust word frequencies in jieba using `suggest_freq`. This can be used to ensure specific words are segmented correctly or to prevent them from being split. The example demonstrates changing the segmentation of '台中'.

```Python
>>> print('/'.join(jieba.cut('如果放到post中将出错。', HMM=False)))
如果/放到/post/中将/出错/。
>>> jieba.suggest_freq(('中', '将'), True)
494
>>> print('/'.join(jieba.cut('如果放到post中将出错。', HMM=False)))
如果/放到/post/中/将/出错/。
>>> print('/'.join(jieba.cut('「台中」正确应该不会被切开', HMM=False)))
「/台/中/」/正确/应该/不会/被/切开
>>> jieba.suggest_freq('台中', True)
69
>>> print('/'.join(jieba.cut('「台中」正确应该不会被切开', HMM=False)))
「/台中/」/正确/应该/不会/被/切开
```

--------------------------------

### Initialize Jieba

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to manually initialize the Jieba library. Jieba uses lazy loading, so explicit initialization is optional but can be done using `jieba.initialize()`.

```Python
import jieba
jieba.initialize()  # Manual initialization (optional)
```

--------------------------------

### Initialize Jieba Manually

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to manually initialize the Jieba library in Python. This is an optional step that can pre-load dictionaries for faster subsequent operations.

```Python
import jieba
jieba.initialize()
```

--------------------------------

### jieba Command Line - Full Mode

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates enabling full mode (all possible words) segmentation via the command line using the '-a' or '--cut-all' option. Note that this mode does not support POS tagging.

```Shell
python -m jieba -a news.txt > cut_result.txt
```

--------------------------------

### jieba Segmentation Modes (Python)

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates the different segmentation modes in jieba: Paddle, Full, Default (Precise), and Search Engine. It shows how to use `jieba.cut` and `jieba.cut_for_search` with various parameters and how to enable the Paddle mode.

```Python
# encoding=utf-8
import jieba

jieba.enable_paddle()# 启动paddle模式。 0.40版之后开始支持，早期版本不支持
strs=["我来到北京清华大学","乒乓球拍卖完了","中国科学技术大学"]
for str in strs:
    seg_list = jieba.cut(str,use_paddle=True) # 使用paddle模式
    print("Paddle Mode: " + '/'.join(list(seg_list)))

seg_list = jieba.cut("我来到北京清华大学", cut_all=True)
print("Full Mode: " + "/ ".join(seg_list))  # 全模式

seg_list = jieba.cut("我来到北京清华大学", cut_all=False)
print("Default Mode: " + "/ ".join(seg_list))  # 精确模式

seg_list = jieba.cut("他来到了网易杭研大厦")  # 默认是精确模式
print(", ".join(seg_list))

seg_list = jieba.cut_for_search("小明硕士毕业于中国科学院计算所，后在日本京都大学深造")  # 搜索引擎模式
print(", ".join(seg_list))
```

--------------------------------

### Part of Speech Tagging with Jieba

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Illustrates how to perform Part of Speech (POS) tagging on Chinese text using `jieba.posseg`. It shows how to iterate through segmented words and their corresponding POS tags.

```Python
import jieba.posseg as pseg

# Example of POS tagging
words = pseg.cut("我爱北京天安门")
for w in words:
    print('%s %s' % (w.word, w.flag))
```

--------------------------------

### Jieba Segmentation Modes

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates the different segmentation modes in Jieba: Full Mode (cut_all=True) for all possible words, Default Mode (cut_all=False) for accurate segmentation, and Search Engine Mode for shorter words suitable for search engines. It also shows how Jieba handles unknown words using the Viterbi algorithm.

```Python
#encoding=utf-8
import jieba

seg_list = jieba.cut("我来到北京清华大学", cut_all=True)
print("Full Mode: " + "/ ".join(seg_list))  # 全模式

seg_list = jieba.cut("我来到北京清华大学", cut_all=False)
print("Default Mode: " + "/ ".join(seg_list))  # 默认模式

seg_list = jieba.cut("他来到了网易杭研大厦")
print(", ".join(seg_list))

seg_list = jieba.cut_for_search("小明硕士毕业于中国科学院计算所，后在日本京都大学深造")  # 搜索引擎模式
print(", ".join(seg_list))
```

--------------------------------

### jieba Command Line - Help

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Displays the help message for the jieba command-line interface, showing available options and arguments for segmentation and customization.

```Shell
python -m jieba --help
```

--------------------------------

### Jieba Command Line Help

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Displays the help message for the Jieba command-line interface, outlining available arguments for text segmentation.

```Shell
python -m jieba --help
```

--------------------------------

### Extract Keywords using TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Explains how to extract keywords from a sentence using the TF-IDF algorithm with `jieba.analyse.extract_tags`. It covers parameters like `topK`, `withWeight`, and `allowPOS` for filtering and weighting keywords.

```Python
import jieba.analyse

# Example of keyword extraction
# sentence = "..."
# keywords = jieba.analyse.extract_tags(sentence, topK=20, withWeight=False, allowPOS=())
# print(keywords)
```

--------------------------------

### jieba POS Tagging with Paddle Mode

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to perform part-of-speech tagging using jieba's paddle mode. It requires enabling paddle mode and then using the pseg.cut function with the use_paddle=True argument. The output shows words and their corresponding part-of-speech tags.

```Python
import jieba
import jieba.posseg as pseg

# jieba default mode
words = pseg.cut("我爱北京天安门") 

# enable paddle mode
jieba.enable_paddle() 

# paddle mode
words = pseg.cut("我爱北京天安门", use_paddle=True)
for word, flag in words:
   print('%s %s' % (word, flag))
```

--------------------------------

### jieba Command Line - POS Tagging

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to enable part-of-speech tagging when using jieba from the command line. The '-p' option can be used with an optional delimiter to separate words and their tags.

```Shell
python -m jieba -p news.txt > cut_result.txt
```

--------------------------------

### Extract Keywords using TextRank

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Details the usage of the TextRank algorithm for keyword extraction via `jieba.analyse.textrank`. It allows filtering keywords based on their Part-of-Speech tags.

```Python
import jieba.analyse

# Example of TextRank keyword extraction with POS filtering
# sentence = "..."
# keywords = jieba.analyse.textrank(sentence, topK=20, withWeight=False, allowPOS=('ns', 'n', 'vn', 'v'))
# print(keywords)
```

--------------------------------

### Suggest Word Frequency in Jieba

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to adjust the frequency of words or phrases in Jieba to influence segmentation. `jieba.suggest_freq()` can be used to increase or decrease a word's probability of being recognized.

```Python
jieba.suggest_freq('台中', True)
jieba.suggest_freq(('今天', '天气'), True)
```

--------------------------------

### Change Jieba Dictionary Path

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to change the path to the main dictionary file. This is useful for using custom dictionaries or different versions, like `dict.txt.big` for better Traditional Chinese segmentation.

```Python
jieba.set_dictionary('data/dict.txt.big')
```

--------------------------------

### Customize IDF Corpus for TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to specify a custom IDF corpus file for Jieba's TF-IDF keyword extraction using `jieba.analyse.set_idf_path`. This allows for domain-specific keyword relevance.

```Python
import jieba.analyse

# Example of setting a custom IDF path
# jieba.analyse.set_idf_path('path/to/your/idf.txt')
```

--------------------------------

### ChineseAnalyzer for Whoosh Integration

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Provides information on using `jieba.analyse.ChineseAnalyzer` for integrating Jieba's text analysis capabilities with the Whoosh search engine library.

```Python
from jieba.analyse import ChineseAnalyzer

# Usage with Whoosh would involve configuring an analyzer:
# from whoosh.fields import Schema, TEXT, ID
# from whoosh.index import create_in
# schema = Schema(content=TEXT(analyzer=ChineseAnalyzer()))
# ix = create_in("indexdir", schema)
```

--------------------------------

### Tokenize Chinese Text (Search Mode)

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to use Jieba's `jieba.tokenize` in 'search' mode, which provides different segmentation results suitable for search engine indexing, including overlapping words.

```Python
import jieba

# Example of tokenization in search mode
result = jieba.tokenize(u'永和服装饰品有限公司',mode='search')
for tk in result:
    print("word %s\t\t start: %d \t\t end:%d" % (tk[0],tk[1],tk[2]))
```

--------------------------------

### Set Jieba Dictionary

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Shows how to specify a custom dictionary file for Jieba to use, overriding the default dictionary. This is useful for tailoring segmentation to specific domains or languages.

```Python
jieba.set_dictionary('data/dict.txt.big')
```

--------------------------------

### Customize IDF Path for TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Allows customization of the Inverse Document Frequency (IDF) corpus path for the TF-IDF keyword extraction algorithm. This enables the use of custom corpora for more tailored keyword analysis.

```Python
import jieba.analyse

# Set custom IDF path
custom_idf_path = "/path/to/your/idf.txt"
jieba.analyse.set_idf_path(custom_idf_path)

# Example usage with custom IDF
sentence = "基于 TF-IDF 算法的关键词抽取"
tags = jieba.analyse.extract_tags(sentence)
print(tags)
```

--------------------------------

### Extract Tags with TextRank

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Extracts keywords from a given sentence using the TextRank algorithm. This method builds a graph based on word co-occurrence within a sliding window and calculates PageRank for keyword scoring. It allows specifying the number of top keywords, whether to return weights, and filtering by part-of-speech.

```Python
import jieba.analyse

# Example usage:
sentence = "基于 TextRank 算法的关键词抽取"
tags = jieba.analyse.textrank(sentence, topK=5, withWeight=True, allowPOS=('ns', 'n', 'vn', 'v'))
print(tags)
```

--------------------------------

### Extract Tags with TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Extracts keywords from a given sentence using the TF-IDF algorithm. Supports specifying the number of top keywords, whether to return weights, and filtering by part-of-speech. The IDF corpus path can also be customized.

```Python
import jieba.analyse

# Example usage:
sentence = "基于 TF-IDF 算法的关键词抽取"
tags = jieba.analyse.extract_tags(sentence, topK=5, withWeight=True)
print(tags)
```

--------------------------------

### Enable and Disable Parallel Processing

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Explains how to enable and disable parallel processing in Jieba for faster word segmentation. It mentions that parallel processing is based on Python's `multiprocessing` module and supports specific tokenizers.

```Python
import jieba

# Enable parallel processing with 4 processes
# jieba.enable_parallel(4)

# Disable parallel processing
# jieba.disable_parallel()
```

--------------------------------

### Customize Stop Words for TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Enables customization of the stop words corpus path for the TF-IDF keyword extraction algorithm. This allows for the exclusion of specific words that do not contribute to meaningful keyword analysis.

```Python
import jieba.analyse

# Set custom stop words path
custom_stop_words_path = "/path/to/your/stop_words.txt"
jieba.analyse.set_stop_words(custom_stop_words_path)

# Example usage with custom stop words
sentence = "基于 TF-IDF 算法的关键词抽取"
tags = jieba.analyse.extract_tags(sentence)
print(tags)
```

--------------------------------

### Modify Jieba Dictionary Dynamically

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to dynamically add or remove words from the Jieba dictionary and adjust word frequencies using `add_word`, `del_word`, and `suggest_freq`. It also shows how HMM affects segmentation results.

```Python
import jieba

# Example of dynamic dictionary modification and frequency suggestion
print('/'.join(jieba.cut('如果放到post中将出错。', HMM=False)))
jieba.suggest_freq(('中', '将'), True)
print('/'.join(jieba.cut('如果放到post中将出错。', HMM=False)))
jieba.suggest_freq('台中', True)
print('/'.join(jieba.cut('「台中」正确应该不会被切开', HMM=False)))
```

--------------------------------

### Customize Stop Words Corpus for TF-IDF

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Demonstrates how to set a custom stop words corpus for Jieba's TF-IDF keyword extraction using `jieba.analyse.set_stop_words`. This helps in filtering out common or irrelevant words.

```Python
import jieba.analyse

# Example of setting custom stop words
# jieba.analyse.set_stop_words('path/to/your/stop_words.txt')
```

--------------------------------

### Add Custom Words to Jieba

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Explains how to add custom words to Jieba's dictionary to improve segmentation accuracy for specific terms, such as '台中'. This can be done using `jieba.add_word()`.

```Python
jieba.add_word('台中')
```

--------------------------------

### Delete Words from Jieba

Source: https://github.com/fxsjy/jieba/blob/master/README.md

Illustrates how to remove specific words or phrases from Jieba's dictionary to prevent incorrect segmentation. `jieba.del_word()` can be used for this purpose.

```Python
jieba.del_word('今天天气')
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.