CppJieba

https://github.com/yanyiwu/cppjieba
Admin
CppJieba is a high-performance C++ implementation of Chinese word segmentation with support for...
Tokens:11,991
Snippets:79
Trust Score:9.8
License:MIT
Update:6 months ago
Show doc for...
Tokens:
Raw
### Basic CppJieba Usage in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

A simple C++ example demonstrating basic word segmentation using the Jieba class. This snippet is intended for use with the CMake integration example.

```cpp
// main.cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

int main() {
    cppjieba::Jieba jieba;
    std::vector<std::string> words;

    jieba.Cut("中文分词测试", words);

    for (const auto& word : words) {
        std::cout << word << " ";
    }
    std::cout << std::endl;

    return 0;
}

```

--------------------------------

### Build and Install CppJieba

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Standard shell commands to clone the repository, initialize submodules, and build the project using CMake.

```sh
git clone https://github.com/yanyiwu/cppjieba.git
cd cppjieba
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make

make test
```

--------------------------------

### Demo Output Examples

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Sample outputs demonstrating various segmentation modes, user word insertion, tagging, and keyword extraction.

```text
[demo] Cut With HMM
他/来到/了/网易/杭研/大厦
[demo] Cut Without HMM
他/来到/了/网易/杭/研/大厦
我来到北京清华大学
[demo] CutAll
我/来到/北京/清华/清华大学/华大/大学
小明硕士毕业于中国科学院计算所，后在日本京都大学深造
[demo] CutForSearch
小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所/，/后/在/日本/京都/大学/日本京都大学/深造
[demo] Insert User Word
男默/女泪
男默女泪
[demo] CutForSearch Word With Offset
[{"word": "小明", "offset": 0}, {"word": "硕士", "offset": 6}, {"word": "毕业", "offset": 12}, {"word": "于", "offset": 18}, {"word": "中国", "offset": 21}, {"word": "科学", "offset": 27}, {"word": "学院", "offset": 30}, {"word": "科学院", "offset": 27}, {"word": "中国科学院", "offset": 21}, {"word": "计算", "offset": 36}, {"word": "计算所", "offset": 36}, {"word": "，", "offset": 45}, {"word": "后", "offset": 48}, {"word": "在", "offset": 51}, {"word": "日本", "offset": 54}, {"word": "京都", "offset": 60}, {"word": "大学", "offset": 66}, {"word": "日本京都大学", "offset": 54}, {"word": "深造", "offset": 72}]
[demo] Tagging
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
[我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x, 不用:v, 多久:m, ，:x, 我:r, 就:d, 会:v, 升职:v, 加薪:nr, ，:x, 当上:t, CEO:eng, ，:x, 走上:v, 人生:n, 巅峰:n, 。:x]
[demo] Keyword Extraction
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
[{"word": "CEO", "offset": [93], "weight": 11.7392}, {"word": "升职", "offset": [72], "weight": 10.8562}, {"word": "加薪", "offset": [78], "weight": 10.6426}, {"word": "手扶拖拉机", "offset": [21], "weight": 10.0089}, {"word": "巅峰", "offset": [111], "weight": 9.49396}]
```

--------------------------------

### Custom User Dictionary Example

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Demonstrates the effect of using a custom user dictionary on word segmentation.

```text
没有使用自定义用户词典时的结果:
令狐冲/是/云/计算/行业/的/专家

使用自定义用户词典时的结果:
令狐冲/是/云计算/行业/的/专家
```

--------------------------------

### Keyword Extraction Example

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Extracts keywords and their weights from a sample sentence.

```text
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
["CEO:11.7392", "升职:10.8562", "加薪:10.6426", "手扶拖拉机:10.0089", "巅峰:9.49396"]
```

--------------------------------

### Part-of-Speech Tagging Example

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Performs part-of-speech tagging on a sample sentence, including custom dictionary tagging.

```text
我是蓝翔技工拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上总经理，出任CEO，迎娶白富美，走上人生巅峰。
["我:r", "是:v", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", "，:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", "，:x", "当上:t", "CEO:eng", "，:x", "走上:v", "人生:n", "巅峰:n", "。:x"]

蓝翔 nz

["我:r", "是:v", "蓝翔:nz", "技工:n", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", "，:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", "，:x", "当:t", "上:f", "总经理:n", "，:x", "出任:v", "CEO:eng", "，:x", "迎娶:v", "白富美:x", "，:x", "走上:v", "人生:n", "巅峰:n", "。:x"]
```

--------------------------------

### Run Demo

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Execute the compiled demo binary.

```sh
./demo
```

--------------------------------

### Jieba Initialization

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

The Jieba class is the main entry point for the library. It requires dictionary files for initialization, including the main dictionary, HMM model, user dictionary, IDF dictionary, and stop words list.

```APIDOC
## Jieba Initialization

### Description
Initializes the Jieba segmenter with required dictionary files.

### Parameters
- **dict_path** (string) - Required - Path to main dictionary
- **hmm_path** (string) - Required - Path to HMM model
- **user_dict_path** (string) - Required - Path to user dictionary
- **idf_path** (string) - Required - Path to IDF dictionary
- **stop_words_path** (string) - Required - Path to stop words list
```

--------------------------------

### Initialize Jieba Segmenter

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

The Jieba class is the main entry point. It can be initialized with default paths or by specifying custom dictionary and model files.

```cpp
#include "cppjieba/Jieba.hpp"

using namespace cppjieba;

int main() {
    // 方式一：使用默认词典路径（自动从库目录加载词典）
    Jieba jieba;

    // 方式二：指定完整词典路径
    Jieba jieba(
        "dict/jieba.dict.utf8",      // 主词典
        "dict/hmm_model.utf8",       // HMM 模型
        "dict/user.dict.utf8",       // 用户自定义词典
        "dict/idf.utf8",             // IDF 词典（用于关键词抽取）
        "dict/stop_words.utf8"       // 停用词表
    );

    return 0;
}
```

--------------------------------

### Load User Dictionaries in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Demonstrates loading custom dictionaries from files, vectors, or sets to influence segmentation results.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>
#include <vector>
#include <set>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // 方式一：从文件加载（支持多文件，用 | 或 ; 分隔）
    jieba.LoadUserDict("dict/user.dict.utf8");
    jieba.LoadUserDict("dict1.txt|dict2.txt;dict3.txt");

    // 方式二：从 vector 加载
    vector<string> dictBuf = {
        "云计算",              // 仅词语
        "蓝翔 nz",             // 词语 + 词性
        "区块链 10 nz"         // 词语 + 词频 + 词性
    };
    jieba.LoadUserDict(dictBuf);

    // 方式三：从 set 加载
    set<string> dictSet = {"人工智能", "机器学习", "深度学习"};
    jieba.LoadUserDict(dictSet);

    // 测试分词效果
    jieba.Cut("令狐冲是云计算行业的专家", words);
    // 使用用户词典后: ["令狐冲", "是", "云计算", "行业", "的", "专家"]
    // 未使用时: ["令狐冲", "是", "云", "计算", "行业", "的", "专家"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;

    return 0;
}

// 用户词典格式（UTF-8 编码）:
// 云计算               # 仅词语，使用默认词频和词性
// 韩玉鉴赏             # 仅词语
// 蓝翔 nz              # 词语 + 词性
// 区块链 10 nz         # 词语 + 词频 + 词性
```

--------------------------------

### CMake Integration for CppJieba

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Integrate CppJieba into your CMake project using FetchContent or as a subdirectory. Ensure C++11 standard is set and link the library to your executable.

```cmake
# CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(MyProject)

set(CMAKE_CXX_STANDARD 11)

# Method 1: Using FetchContent
include(FetchContent)
FetchContent_Declare(
    cppjieba
    GIT_REPOSITORY https://github.com/yanyiwu/cppjieba.git
    GIT_TAG v5.5.0
)
FetchContent_MakeAvailable(cppjieba)

# Method 2: Add as a subdirectory
# add_subdirectory(cppjieba)

# Link cppjieba
add_executable(my_app main.cpp)
target_link_libraries(my_app PRIVATE cppjieba)

```

--------------------------------

### Manage User Dictionary Dynamically in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Shows how to insert or delete custom words at runtime without reloading the entire dictionary.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // 分词前 - "男默女泪" 被切分
    jieba.Cut("男默女泪", words);
    // 输出: ["男默", "女泪"]

    // 动态添加新词
    jieba.InsertUserWord("男默女泪");

    // 分词后 - "男默女泪" 作为整词
    jieba.Cut("男默女泪", words);
    // 输出: ["男默女泪"]

    // 添加带词频的新词
    jieba.InsertUserWord("区块链", 1000, "nz");

    // 添加带词性的新词
    jieba.InsertUserWord("蓝翔", "nz");

    // 查询词语是否存在
    bool exists = jieba.Find("男默女泪");
    std::cout << "词语存在: " << (exists ? "是" : "否") << std::endl;

    // 删除用户词语
    jieba.DeleteUserWord("男默女泪");

    return 0;
}
```

--------------------------------

### Extract Keywords with TF-IDF in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Demonstrates extracting keywords using the TF-IDF algorithm, supporting output of just words, words with weights, or full word metadata.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;

    string text = "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。";
    size_t topN = 5;

    // 方式一：只获取关键词
    vector<string> keywords;
    jieba.extractor.Extract(text, keywords, topN);
    // 输出: ["CEO", "升职", "加薪", "手扶拖拉机", "巅峰"]

    // 方式二：获取关键词和权重
    vector<pair<string, double>> keywordsWithWeight;
    jieba.extractor.Extract(text, keywordsWithWeight, topN);
    // 输出: [CEO:11.7392, 升职:10.8562, 加薪:10.6426, 手扶拖拉机:10.0089, 巅峰:9.49396]

    for (const auto& kw : keywordsWithWeight) {
        std::cout << kw.first << ":" << kw.second << " ";
    }
    std::cout << std::endl;

    // 方式三：获取完整信息（词语、位置、权重）
    vector<KeywordExtractor::Word> wordweights;
    jieba.extractor.Extract(text, wordweights, topN);
    // 输出: [{"word": "CEO", "offset": [93], "weight": 11.7392}, ...]

    return 0;
}
```

--------------------------------

### LoadUserDict - Load User Dictionary

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Supports loading user-defined dictionaries from files or memory. Dictionary format supports various forms: word only, word + part-of-speech, word + frequency + part-of-speech. Multiple dictionary files can be separated by '|' or ';'.

```APIDOC
## LoadUserDict - Load User Dictionary

### Description
Supports loading user-defined dictionaries from files or memory. Dictionary format supports various forms: word only, word + part-of-speech, word + frequency + part-of-speech. Multiple dictionary files can be separated by '|' or ';'.

### Method
```cpp
// Example usage within a C++ program
```

### Endpoint
N/A (This is a library function, not a web endpoint)

### Parameters
N/A (Function parameters are defined within the C++ code)

### Request Example
```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>
#include <vector>
#include <set>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // Method 1: Load from file (supports multiple files, separated by | or ;)
    jieba.LoadUserDict("dict/user.dict.utf8");
    jieba.LoadUserDict("dict1.txt|dict2.txt;dict3.txt");

    // Method 2: Load from vector
    vector<string> dictBuf = {
        "云计算",              // Word only
        "蓝翔 nz",             // Word + part-of-speech
        "区块链 10 nz"         // Word + frequency + part-of-speech
    };
    jieba.LoadUserDict(dictBuf);

    // Method 3: Load from set
    set<string> dictSet = {"人工智能", "机器学习", "深度学习"};
    jieba.LoadUserDict(dictSet);

    // Test segmentation effect
    jieba.Cut("令狐冲是云计算行业的专家", words);
    // After using user dictionary: ["令狐冲", "是", "云计算", "行业", "的", "专家"]
    // Without using: ["令狐冲", "是", "云", "计算", "行业", "的", "专家"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;

    return 0;
}

// User dictionary format (UTF-8 encoded):
// 云计算               # Word only, uses default frequency and part-of-speech
// 韩玉鉴赏             # Word only
// 蓝翔 nz              # Word + part-of-speech
// 区块链 10 nz         # Word + frequency + part-of-speech
```

### Response
N/A (Function modifies internal dictionary state)
```

--------------------------------

### Segmentation Algorithm Outputs

Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md

Comparison of outputs from different segmentation algorithms: MPSegment, HMMSegment, MixSegment, FullSegment, and QuerySegment.

```text
**MPSegment**

Output:
我来到北京清华大学
我/来到/北京/清华大学

他来到了网易杭研大厦
他/来到/了/网易/杭/研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小/明/硕士/毕业/于/中国科学院/计算所/，/后/在/日本京都大学/深造

**HMMSegment**

我来到北京清华大学
我来/到/北京/清华大学

他来到了网易杭研大厦
他来/到/了/网易/杭/研大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业于/中国/科学院/计算所/，/后/在/日/本/京/都/大/学/深/造

**MixSegment**

我来到北京清华大学
我/来到/北京/清华大学

他来到了网易杭研大厦
他/来到/了/网易/杭研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业/于/中国科学院/计算所/，/后/在/日本京都大学/深造

**FullSegment**

我来到北京清华大学
我/来到/北京/清华/清华大学/华大/大学

他来到了网易杭研大厦
他/来到/了/网易/杭/研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小/明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算/计算所/，/后/在/日本/日本京都大学/京都/京都大学/大学/深造

**QuerySegment**

我来到北京清华大学
我/来到/北京/清华/清华大学/华大/大学

他来到了网易杭研大厦
他/来到/了/网易/杭研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算所/，/后/在/中国/中国科学院/科学/科学院/学院/日本/日本京都大学/京都/京都大学/大学/深造
```

--------------------------------

### Extract Keywords with TextRank in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Uses the TextRank algorithm to extract keywords based on word co-occurrence graphs.

```cpp
#include "cppjieba/TextRankExtractor.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    // 初始化 TextRank 抽取器
    TextRankExtractor extractor(
        "dict/jieba.dict.utf8",      // 主词典
        "dict/hmm_model.utf8",       // HMM 模型
        "dict/stop_words.utf8"       // 停用词表
    );

    string text = "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。";
    size_t topN = 5;

    // 获取关键词和权重
    vector<pair<string, double>> keywords;
    extractor.Extract(text, keywords, topN);

    for (const auto& kw : keywords) {
        std::cout << kw.first << ":" << kw.second << " ";
    }
    std::cout << std::endl;
    // 输出: 当上:1 不用:0.989848 多久:0.985126 加薪:0.983046 升职:0.980278

    // 获取完整信息
    vector<TextRankExtractor::Word> wordweights;
    extractor.Extract(text, wordweights, topN);
    // 输出: [{"word": "当上", "offset": [87], "weight": 1}, ...]

    return 0;
}
```

--------------------------------

### MixSegment - Direct Usage for Mixed Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Use MixSegment for precise segmentation and part-of-speech tagging. It requires dictionary and HMM model paths for initialization.

```cpp
#include "cppjieba/MixSegment.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    // Initialize the mixed segmenter
    MixSegment segment(
        "dict/jieba.dict.utf8",      // Main dictionary
        "dict/hmm_model.utf8",       // HMM model
        "dict/user.dict.utf8"        // User dictionary (optional)
    );

    vector<string> words;

    // Word segmentation
    segment.Cut("我来到北京清华大学", words);
    // Output: ["我", "来到", "北京", "清华大学"]

    // Part-of-speech tagging
    vector<pair<string, string>> tagResult;
    segment.Tag("iPhone6手机的最大特点是很容易弯曲。", tagResult);
    // Output: [iPhone6:eng, 手机:n, 的:uj, 最大:a, 特点:n, 是:v, 很:zg, 容易:a, 弯曲:v, 。:x]

    for (const auto& item : tagResult) {
        std::cout << item.first << ":" << item.second << " ";
    }
    std::cout << std::endl;

    return 0;
}

```

--------------------------------

### InsertUserWord / DeleteUserWord - Dynamic Dictionary Management

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Allows dynamic addition or deletion of user-defined words at runtime without reloading the dictionary. Supports specifying word frequency and part-of-speech.

```APIDOC
## InsertUserWord / DeleteUserWord - Dynamic Dictionary Management

### Description
Allows dynamic addition or deletion of user-defined words at runtime without reloading the dictionary. Supports specifying word frequency and part-of-speech.

### Method
```cpp
// Example usage within a C++ program
```

### Endpoint
N/A (This is a library function, not a web endpoint)

### Parameters
N/A (Function parameters are defined within the C++ code)

### Request Example
```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // Before segmentation - "男默女泪" is split
    jieba.Cut("男默女泪", words);
    // Output: ["男默", "女泪"]

    // Dynamically add a new word
    jieba.InsertUserWord("男默女泪");

    // After segmentation - "男默女泪" is treated as a whole word
    jieba.Cut("男默女泪", words);
    // Output: ["男默女泪"]

    // Add a new word with frequency
    jieba.InsertUserWord("区块链", 1000, "nz");

    // Add a new word with part-of-speech
    jieba.InsertUserWord("蓝翔", "nz");

    // Check if a word exists
    bool exists = jieba.Find("男默女泪");
    std::cout << "Word exists: " << (exists ? "Yes" : "No") << std::endl;

    // Delete a user word
    jieba.DeleteUserWord("男默女泪");

    return 0;
}
```

### Response
N/A (Function modifies internal dictionary state)
```

--------------------------------

### Perform Full Mode Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Full mode extracts all possible words from the dictionary, which is useful for indexing in search engines.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    jieba.CutAll("我来到北京清华大学", words);
    // 输出: ["我", "来到", "北京", "清华", "清华大学", "华大", "大学"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;
    // 输出: 我/来到/北京/清华/清华大学/华大/大学/

    // 带位置信息版本
    vector<Word> wordWithOffset;
    jieba.CutAll("我来到北京邮电大学", wordWithOffset);
    // 输出: [{"word": "我", "offset": 0}, {"word": "来自", "offset": 3},
    //        {"word": "北京", "offset": 9}, {"word": "北京邮电", "offset": 9}, ...]

    return 0;
}
```

--------------------------------

### Perform Part-of-Speech Tagging

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Assigns part-of-speech tags to segmented words. Custom tags can be defined via the user dictionary.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<pair<string, string>> tagResult;

    jieba.Tag("我是拖拉机学院手扶拖拉机专业的。", tagResult);
    // 输出: [我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x]

    for (const auto& item : tagResult) {
        std::cout << item.first << ":" << item.second << " ";
    }
    std::cout << std::endl;
    // 输出: 我:r 是:v 拖拉机:n 学院:n 手扶拖拉机:n 专业:n 的:uj 。:x

    // 查询单个词的词性
    string tag = jieba.LookupTag("拖拉机");
    std::cout << "拖拉机的词性: " << tag << std::endl;
    // 输出: 拖拉机的词性: n

    return 0;
}

// 常见词性标签:
// n-名词, v-动词, a-形容词, r-代词, d-副词, m-数词
// eng-英文, x-标点符号, uj-助词, nr-人名, ns-地名, nz-其他专名
```

--------------------------------

### CutAll - Full Mode Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Segments all possible words found in the dictionary, useful for search engine indexing.

```APIDOC
## CutAll (Full Mode)

### Description
Segments the sentence into all possible words found in the dictionary.

### Parameters
- **sentence** (string) - Required - The input text to segment
- **words** (vector<string>) - Required - Output container for segmented words
```

--------------------------------

### Perform Exact Mode Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

The default mode combines maximum probability and HMM to identify dictionary words and unknown words. HMM can be toggled to control new word discovery.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // 启用 HMM（默认）- 可识别未登录词
    jieba.Cut("他来到了网易杭研大厦", words);
    // 输出: ["他", "来到", "了", "网易", "杭研", "大厦"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;
    // 输出: 他/来到/了/网易/杭研/大厦/

    // 禁用 HMM - 仅使用词典分词
    jieba.Cut("他来到了网易杭研大厦", words, false);
    // 输出: ["他", "来到", "了", "网易", "杭", "研", "大厦"]

    // 获取带位置信息的分词结果
    vector<Word> wordWithOffset;
    jieba.Cut("他来到了网易杭研大厦", wordWithOffset);
    // 输出: [{"word": "他", "offset": 0}, {"word": "来到", "offset": 3}, ...]

    return 0;
}
```

--------------------------------

### Perform HMM-only Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

This mode relies solely on the Hidden Markov Model and does not use a dictionary, making it suitable for processing text with many unknown words.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    jieba.CutHMM("我来自北京邮电大学。。。学号123456", words);
    // 输出: ["我来", "自北京", "邮电大学", "。", "。", "。", "学号", "123456"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;
    // 输出: 我来/自北京/邮电大学/。/。/。/学号/123456/

    return 0;
}
```

--------------------------------

### Perform Search Mode Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Search mode performs exact segmentation followed by full segmentation on long words to improve recall.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    jieba.CutForSearch("小明硕士毕业于中国科学院计算所，后在日本京都大学深造", words);
    // 输出: ["小明", "硕士", "毕业", "于", "中国", "科学", "学院", "科学院",
    //        "中国科学院", "计算", "计算所", "，", "后", "在", "日本",
    //        "京都", "大学", "日本京都大学", "深造"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;

    // 带位置信息版本
    vector<Word> wordWithOffset;
    jieba.CutForSearch("小明硕士毕业于中国科学院计算所", wordWithOffset);
    // 每个词包含 word 和 offset 字段

    return 0;
}
```

--------------------------------

### Customize Segmentation Separators in C++

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Configures the set of characters used as separators during segmentation, allowing punctuation to be treated as part of words.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // 先添加包含标点的词
    jieba.InsertUserWord("同一个世界，同一个梦想");

    // 默认分隔符 - 逗号会分隔词语
    jieba.Cut("同一个世界，同一个梦想", words);
    // 输出: ["同一个", "世界", "，", "同一个", "梦想"]

    // 清空分隔符 - 标点不再分隔
    jieba.ResetSeparators("");

    jieba.Cut("同一个世界，同一个梦想", words);
    // 输出: ["同一个世界，同一个梦想"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;

    return 0;
}
```

--------------------------------

### Tag - Part-of-Speech Tagging

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Provides POS tagging for segmented words, returning pairs of word and tag.

```APIDOC
## Tag (POS Tagging)

### Description
Performs word segmentation and assigns a part-of-speech tag to each word.

### Parameters
- **sentence** (string) - Required - The input text to tag
- **tagResult** (vector<pair<string, string>>) - Required - Output container for word-tag pairs
```

--------------------------------

### Perform Maximum Probability Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

This mode allows for fine-grained control by specifying a maximum word length.

```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // 限制最大词长为 3
    jieba.CutSmall("南京市长江大桥", words, 3);
    // 输出: ["南京市", "长江", "大桥"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;
    // 输出: 南京市/长江/大桥/

    return 0;
}
```

--------------------------------

### KeywordExtractor - TF-IDF Keyword Extraction

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Extracts keywords with the highest weights from text using the TF-IDF algorithm. Supports returning words, weights, and position information.

```APIDOC
## KeywordExtractor - TF-IDF Keyword Extraction

### Description
Extracts keywords with the highest weights from text using the TF-IDF algorithm. Supports returning words, weights, and position information.

### Method
```cpp
// Example usage within a C++ program
```

### Endpoint
N/A (This is a library function, not a web endpoint)

### Parameters
N/A (Function parameters are defined within the C++ code)

### Request Example
```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    string text = "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。";
    size_t topN = 5;

    // Method 1: Get only keywords
    vector<string> keywords;
    jieba.extractor.Extract(text, keywords, topN);
    // Output: ["CEO", "升职", "加薪", "手扶拖拉机", "巅峰"]

    // Method 2: Get keywords and weights
    vector<pair<string, double>> keywordsWithWeight;
    jieba.extractor.Extract(text, keywordsWithWeight, topN);
    // Output: [CEO:11.7392, 升职:10.8562, 加薪:10.6426, 手扶拖拉机:10.0089, 巅峰:9.49396]

    for (const auto& kw : keywordsWithWeight) {
        std::cout << kw.first << ":" << kw.second << " ";
    }
    std::cout << std::endl;

    // Method 3: Get full information (word, offset, weight)
    vector<KeywordExtractor::Word> wordweights;
    jieba.extractor.Extract(text, wordweights, topN);
    // Output: [{"word": "CEO", "offset": [93], "weight": 11.7392}, ...]

    return 0;
}
```

### Response
N/A (Function returns data directly to C++ variables)
```

--------------------------------

### ResetSeparators - Custom Separators

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Allows customization of the separator set used during word segmentation. By default, punctuation marks are treated as separators and split out individually.

```APIDOC
## ResetSeparators - Custom Separators

### Description
Allows customization of the separator set used during word segmentation. By default, punctuation marks are treated as separators and split out individually.

### Method
```cpp
// Example usage within a C++ program
```

### Endpoint
N/A (This is a library function, not a web endpoint)

### Parameters
N/A (Function parameters are defined within the C++ code)

### Request Example
```cpp
#include "cppjieba/Jieba.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    Jieba jieba;
    vector<string> words;

    // First, add a word containing punctuation
    jieba.InsertUserWord("同一个世界，同一个梦想");

    // Default separators - comma separates words
    jieba.Cut("同一个世界，同一个梦想", words);
    // Output: ["同一个", "世界", "，", "同一个", "梦想"]

    // Clear separators - punctuation no longer separates words
    jieba.ResetSeparators("");

    jieba.Cut("同一个世界，同一个梦想", words);
    // Output: ["同一个世界，同一个梦想"]

    for (const auto& word : words) {
        std::cout << word << "/";
    }
    std::cout << std::endl;

    return 0;
}
```

### Response
N/A (Function modifies internal separator settings)
```

--------------------------------

### Cut - Exact Mode Segmentation

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

The default segmentation mode combining maximum probability and HMM to identify known words and unknown words.

```APIDOC
## Cut (Exact Mode)

### Description
Performs exact mode segmentation using a mix of MP and HMM algorithms.

### Parameters
- **sentence** (string) - Required - The input text to segment
- **words** (vector<string>) - Required - Output container for segmented words
- **hmm** (bool) - Optional - Enable/Disable HMM (default: true)
```

--------------------------------

### TextRankExtractor - TextRank Keyword Extraction

Source: https://context7.com/yanyiwu/cppjieba/llms.txt

Extracts keywords using the TextRank graph ranking algorithm by building and ranking a graph based on word co-occurrence.

```APIDOC
## TextRankExtractor - TextRank Keyword Extraction

### Description
Extracts keywords using the TextRank graph ranking algorithm by building and ranking a graph based on word co-occurrence.

### Method
```cpp
// Example usage within a C++ program
```

### Endpoint
N/A (This is a library function, not a web endpoint)

### Parameters
N/A (Function parameters are defined within the C++ code)

### Request Example
```cpp
#include "cppjieba/TextRankExtractor.hpp"
#include <iostream>

using namespace cppjieba;

int main() {
    // Initialize TextRank extractor
    TextRankExtractor extractor(
        "dict/jieba.dict.utf8",      // Main dictionary
        "dict/hmm_model.utf8",       // HMM model
        "dict/stop_words.utf8"       // Stop words list
    );

    string text = "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。";
    size_t topN = 5;

    // Get keywords and weights
    vector<pair<string, double>> keywords;
    extractor.Extract(text, keywords, topN);

    for (const auto& kw : keywords) {
        std::cout << kw.first << ":" << kw.second << " ";
    }
    std::cout << std::endl;
    // Output: 当上:1 不用:0.989848 多久:0.985126 加薪:0.983046 升职:0.980278

    // Get full information
    vector<TextRankExtractor::Word> wordweights;
    extractor.Extract(text, wordweights, topN);
    // Output: [{"word": "当上", "offset": [87], "weight": 1}, ...]

    return 0;
}
```

### Response
N/A (Function returns data directly to C++ variables)
```
Tokens:
Raw