### Basic CppJieba Usage in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt A simple C++ example demonstrating basic word segmentation using the Jieba class. This snippet is intended for use with the CMake integration example. ```cpp // main.cpp #include "cppjieba/Jieba.hpp" #include int main() { cppjieba::Jieba jieba; std::vector words; jieba.Cut("中文分词测试", words); for (const auto& word : words) { std::cout << word << " "; } std::cout << std::endl; return 0; } ``` -------------------------------- ### Build and Install CppJieba Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Standard shell commands to clone the repository, initialize submodules, and build the project using CMake. ```sh git clone https://github.com/yanyiwu/cppjieba.git cd cppjieba git submodule init git submodule update mkdir build cd build cmake .. make make test ``` -------------------------------- ### Demo Output Examples Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Sample outputs demonstrating various segmentation modes, user word insertion, tagging, and keyword extraction. ```text [demo] Cut With HMM 他/来到/了/网易/杭研/大厦 [demo] Cut Without HMM 他/来到/了/网易/杭/研/大厦 我来到北京清华大学 [demo] CutAll 我/来到/北京/清华/清华大学/华大/大学 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 [demo] CutForSearch 小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所/,/后/在/日本/京都/大学/日本京都大学/深造 [demo] Insert User Word 男默/女泪 男默女泪 [demo] CutForSearch Word With Offset [{"word": "小明", "offset": 0}, {"word": "硕士", "offset": 6}, {"word": "毕业", "offset": 12}, {"word": "于", "offset": 18}, {"word": "中国", "offset": 21}, {"word": "科学", "offset": 27}, {"word": "学院", "offset": 30}, {"word": "科学院", "offset": 27}, {"word": "中国科学院", "offset": 21}, {"word": "计算", "offset": 36}, {"word": "计算所", "offset": 36}, {"word": ",", "offset": 45}, {"word": "后", "offset": 48}, {"word": "在", "offset": 51}, {"word": "日本", "offset": 54}, {"word": "京都", "offset": 60}, {"word": "大学", "offset": 66}, {"word": "日本京都大学", "offset": 54}, {"word": "深造", "offset": 72}] [demo] Tagging 我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。 [我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x, 不用:v, 多久:m, ,:x, 我:r, 就:d, 会:v, 升职:v, 加薪:nr, ,:x, 当上:t, CEO:eng, ,:x, 走上:v, 人生:n, 巅峰:n, 。:x] [demo] Keyword Extraction 我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。 [{"word": "CEO", "offset": [93], "weight": 11.7392}, {"word": "升职", "offset": [72], "weight": 10.8562}, {"word": "加薪", "offset": [78], "weight": 10.6426}, {"word": "手扶拖拉机", "offset": [21], "weight": 10.0089}, {"word": "巅峰", "offset": [111], "weight": 9.49396}] ``` -------------------------------- ### Custom User Dictionary Example Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Demonstrates the effect of using a custom user dictionary on word segmentation. ```text 没有使用自定义用户词典时的结果: 令狐冲/是/云/计算/行业/的/专家 使用自定义用户词典时的结果: 令狐冲/是/云计算/行业/的/专家 ``` -------------------------------- ### Keyword Extraction Example Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Extracts keywords and their weights from a sample sentence. ```text 我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。 ["CEO:11.7392", "升职:10.8562", "加薪:10.6426", "手扶拖拉机:10.0089", "巅峰:9.49396"] ``` -------------------------------- ### Part-of-Speech Tagging Example Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Performs part-of-speech tagging on a sample sentence, including custom dictionary tagging. ```text 我是蓝翔技工拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上总经理,出任CEO,迎娶白富美,走上人生巅峰。 ["我:r", "是:v", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", ",:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", ",:x", "当上:t", "CEO:eng", ",:x", "走上:v", "人生:n", "巅峰:n", "。:x"] 蓝翔 nz ["我:r", "是:v", "蓝翔:nz", "技工:n", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", ",:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", ",:x", "当:t", "上:f", "总经理:n", ",:x", "出任:v", "CEO:eng", ",:x", "迎娶:v", "白富美:x", ",:x", "走上:v", "人生:n", "巅峰:n", "。:x"] ``` -------------------------------- ### Run Demo Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Execute the compiled demo binary. ```sh ./demo ``` -------------------------------- ### Jieba Initialization Source: https://context7.com/yanyiwu/cppjieba/llms.txt The Jieba class is the main entry point for the library. It requires dictionary files for initialization, including the main dictionary, HMM model, user dictionary, IDF dictionary, and stop words list. ```APIDOC ## Jieba Initialization ### Description Initializes the Jieba segmenter with required dictionary files. ### Parameters - **dict_path** (string) - Required - Path to main dictionary - **hmm_path** (string) - Required - Path to HMM model - **user_dict_path** (string) - Required - Path to user dictionary - **idf_path** (string) - Required - Path to IDF dictionary - **stop_words_path** (string) - Required - Path to stop words list ``` -------------------------------- ### Initialize Jieba Segmenter Source: https://context7.com/yanyiwu/cppjieba/llms.txt The Jieba class is the main entry point. It can be initialized with default paths or by specifying custom dictionary and model files. ```cpp #include "cppjieba/Jieba.hpp" using namespace cppjieba; int main() { // 方式一:使用默认词典路径(自动从库目录加载词典) Jieba jieba; // 方式二:指定完整词典路径 Jieba jieba( "dict/jieba.dict.utf8", // 主词典 "dict/hmm_model.utf8", // HMM 模型 "dict/user.dict.utf8", // 用户自定义词典 "dict/idf.utf8", // IDF 词典(用于关键词抽取) "dict/stop_words.utf8" // 停用词表 ); return 0; } ``` -------------------------------- ### Load User Dictionaries in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt Demonstrates loading custom dictionaries from files, vectors, or sets to influence segmentation results. ```cpp #include "cppjieba/Jieba.hpp" #include #include #include using namespace cppjieba; int main() { Jieba jieba; vector words; // 方式一:从文件加载(支持多文件,用 | 或 ; 分隔) jieba.LoadUserDict("dict/user.dict.utf8"); jieba.LoadUserDict("dict1.txt|dict2.txt;dict3.txt"); // 方式二:从 vector 加载 vector dictBuf = { "云计算", // 仅词语 "蓝翔 nz", // 词语 + 词性 "区块链 10 nz" // 词语 + 词频 + 词性 }; jieba.LoadUserDict(dictBuf); // 方式三:从 set 加载 set dictSet = {"人工智能", "机器学习", "深度学习"}; jieba.LoadUserDict(dictSet); // 测试分词效果 jieba.Cut("令狐冲是云计算行业的专家", words); // 使用用户词典后: ["令狐冲", "是", "云计算", "行业", "的", "专家"] // 未使用时: ["令狐冲", "是", "云", "计算", "行业", "的", "专家"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; return 0; } // 用户词典格式(UTF-8 编码): // 云计算 # 仅词语,使用默认词频和词性 // 韩玉鉴赏 # 仅词语 // 蓝翔 nz # 词语 + 词性 // 区块链 10 nz # 词语 + 词频 + 词性 ``` -------------------------------- ### CMake Integration for CppJieba Source: https://context7.com/yanyiwu/cppjieba/llms.txt Integrate CppJieba into your CMake project using FetchContent or as a subdirectory. Ensure C++11 standard is set and link the library to your executable. ```cmake # CMakeLists.txt cmake_minimum_required(VERSION 3.10) project(MyProject) set(CMAKE_CXX_STANDARD 11) # Method 1: Using FetchContent include(FetchContent) FetchContent_Declare( cppjieba GIT_REPOSITORY https://github.com/yanyiwu/cppjieba.git GIT_TAG v5.5.0 ) FetchContent_MakeAvailable(cppjieba) # Method 2: Add as a subdirectory # add_subdirectory(cppjieba) # Link cppjieba add_executable(my_app main.cpp) target_link_libraries(my_app PRIVATE cppjieba) ``` -------------------------------- ### Manage User Dictionary Dynamically in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt Shows how to insert or delete custom words at runtime without reloading the entire dictionary. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // 分词前 - "男默女泪" 被切分 jieba.Cut("男默女泪", words); // 输出: ["男默", "女泪"] // 动态添加新词 jieba.InsertUserWord("男默女泪"); // 分词后 - "男默女泪" 作为整词 jieba.Cut("男默女泪", words); // 输出: ["男默女泪"] // 添加带词频的新词 jieba.InsertUserWord("区块链", 1000, "nz"); // 添加带词性的新词 jieba.InsertUserWord("蓝翔", "nz"); // 查询词语是否存在 bool exists = jieba.Find("男默女泪"); std::cout << "词语存在: " << (exists ? "是" : "否") << std::endl; // 删除用户词语 jieba.DeleteUserWord("男默女泪"); return 0; } ``` -------------------------------- ### Extract Keywords with TF-IDF in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt Demonstrates extracting keywords using the TF-IDF algorithm, supporting output of just words, words with weights, or full word metadata. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; string text = "我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。"; size_t topN = 5; // 方式一:只获取关键词 vector keywords; jieba.extractor.Extract(text, keywords, topN); // 输出: ["CEO", "升职", "加薪", "手扶拖拉机", "巅峰"] // 方式二:获取关键词和权重 vector> keywordsWithWeight; jieba.extractor.Extract(text, keywordsWithWeight, topN); // 输出: [CEO:11.7392, 升职:10.8562, 加薪:10.6426, 手扶拖拉机:10.0089, 巅峰:9.49396] for (const auto& kw : keywordsWithWeight) { std::cout << kw.first << ":" << kw.second << " "; } std::cout << std::endl; // 方式三:获取完整信息(词语、位置、权重) vector wordweights; jieba.extractor.Extract(text, wordweights, topN); // 输出: [{"word": "CEO", "offset": [93], "weight": 11.7392}, ...] return 0; } ``` -------------------------------- ### LoadUserDict - Load User Dictionary Source: https://context7.com/yanyiwu/cppjieba/llms.txt Supports loading user-defined dictionaries from files or memory. Dictionary format supports various forms: word only, word + part-of-speech, word + frequency + part-of-speech. Multiple dictionary files can be separated by '|' or ';'. ```APIDOC ## LoadUserDict - Load User Dictionary ### Description Supports loading user-defined dictionaries from files or memory. Dictionary format supports various forms: word only, word + part-of-speech, word + frequency + part-of-speech. Multiple dictionary files can be separated by '|' or ';'. ### Method ```cpp // Example usage within a C++ program ``` ### Endpoint N/A (This is a library function, not a web endpoint) ### Parameters N/A (Function parameters are defined within the C++ code) ### Request Example ```cpp #include "cppjieba/Jieba.hpp" #include #include #include using namespace cppjieba; int main() { Jieba jieba; vector words; // Method 1: Load from file (supports multiple files, separated by | or ;) jieba.LoadUserDict("dict/user.dict.utf8"); jieba.LoadUserDict("dict1.txt|dict2.txt;dict3.txt"); // Method 2: Load from vector vector dictBuf = { "云计算", // Word only "蓝翔 nz", // Word + part-of-speech "区块链 10 nz" // Word + frequency + part-of-speech }; jieba.LoadUserDict(dictBuf); // Method 3: Load from set set dictSet = {"人工智能", "机器学习", "深度学习"}; jieba.LoadUserDict(dictSet); // Test segmentation effect jieba.Cut("令狐冲是云计算行业的专家", words); // After using user dictionary: ["令狐冲", "是", "云计算", "行业", "的", "专家"] // Without using: ["令狐冲", "是", "云", "计算", "行业", "的", "专家"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; return 0; } // User dictionary format (UTF-8 encoded): // 云计算 # Word only, uses default frequency and part-of-speech // 韩玉鉴赏 # Word only // 蓝翔 nz # Word + part-of-speech // 区块链 10 nz # Word + frequency + part-of-speech ``` ### Response N/A (Function modifies internal dictionary state) ``` -------------------------------- ### Segmentation Algorithm Outputs Source: https://github.com/yanyiwu/cppjieba/blob/master/README.md Comparison of outputs from different segmentation algorithms: MPSegment, HMMSegment, MixSegment, FullSegment, and QuerySegment. ```text **MPSegment** Output: 我来到北京清华大学 我/来到/北京/清华大学 他来到了网易杭研大厦 他/来到/了/网易/杭/研/大厦 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 小/明/硕士/毕业/于/中国科学院/计算所/,/后/在/日本京都大学/深造 **HMMSegment** 我来到北京清华大学 我来/到/北京/清华大学 他来到了网易杭研大厦 他来/到/了/网易/杭/研大厦 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 小明/硕士/毕业于/中国/科学院/计算所/,/后/在/日/本/京/都/大/学/深/造 **MixSegment** 我来到北京清华大学 我/来到/北京/清华大学 他来到了网易杭研大厦 他/来到/了/网易/杭研/大厦 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 小明/硕士/毕业/于/中国科学院/计算所/,/后/在/日本京都大学/深造 **FullSegment** 我来到北京清华大学 我/来到/北京/清华/清华大学/华大/大学 他来到了网易杭研大厦 他/来到/了/网易/杭/研/大厦 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 小/明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算/计算所/,/后/在/日本/日本京都大学/京都/京都大学/大学/深造 **QuerySegment** 我来到北京清华大学 我/来到/北京/清华/清华大学/华大/大学 他来到了网易杭研大厦 他/来到/了/网易/杭研/大厦 小明硕士毕业于中国科学院计算所,后在日本京都大学深造 小明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算所/,/后/在/中国/中国科学院/科学/科学院/学院/日本/日本京都大学/京都/京都大学/大学/深造 ``` -------------------------------- ### Extract Keywords with TextRank in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt Uses the TextRank algorithm to extract keywords based on word co-occurrence graphs. ```cpp #include "cppjieba/TextRankExtractor.hpp" #include using namespace cppjieba; int main() { // 初始化 TextRank 抽取器 TextRankExtractor extractor( "dict/jieba.dict.utf8", // 主词典 "dict/hmm_model.utf8", // HMM 模型 "dict/stop_words.utf8" // 停用词表 ); string text = "我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。"; size_t topN = 5; // 获取关键词和权重 vector> keywords; extractor.Extract(text, keywords, topN); for (const auto& kw : keywords) { std::cout << kw.first << ":" << kw.second << " "; } std::cout << std::endl; // 输出: 当上:1 不用:0.989848 多久:0.985126 加薪:0.983046 升职:0.980278 // 获取完整信息 vector wordweights; extractor.Extract(text, wordweights, topN); // 输出: [{"word": "当上", "offset": [87], "weight": 1}, ...] return 0; } ``` -------------------------------- ### MixSegment - Direct Usage for Mixed Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt Use MixSegment for precise segmentation and part-of-speech tagging. It requires dictionary and HMM model paths for initialization. ```cpp #include "cppjieba/MixSegment.hpp" #include using namespace cppjieba; int main() { // Initialize the mixed segmenter MixSegment segment( "dict/jieba.dict.utf8", // Main dictionary "dict/hmm_model.utf8", // HMM model "dict/user.dict.utf8" // User dictionary (optional) ); vector words; // Word segmentation segment.Cut("我来到北京清华大学", words); // Output: ["我", "来到", "北京", "清华大学"] // Part-of-speech tagging vector> tagResult; segment.Tag("iPhone6手机的最大特点是很容易弯曲。", tagResult); // Output: [iPhone6:eng, 手机:n, 的:uj, 最大:a, 特点:n, 是:v, 很:zg, 容易:a, 弯曲:v, 。:x] for (const auto& item : tagResult) { std::cout << item.first << ":" << item.second << " "; } std::cout << std::endl; return 0; } ``` -------------------------------- ### InsertUserWord / DeleteUserWord - Dynamic Dictionary Management Source: https://context7.com/yanyiwu/cppjieba/llms.txt Allows dynamic addition or deletion of user-defined words at runtime without reloading the dictionary. Supports specifying word frequency and part-of-speech. ```APIDOC ## InsertUserWord / DeleteUserWord - Dynamic Dictionary Management ### Description Allows dynamic addition or deletion of user-defined words at runtime without reloading the dictionary. Supports specifying word frequency and part-of-speech. ### Method ```cpp // Example usage within a C++ program ``` ### Endpoint N/A (This is a library function, not a web endpoint) ### Parameters N/A (Function parameters are defined within the C++ code) ### Request Example ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // Before segmentation - "男默女泪" is split jieba.Cut("男默女泪", words); // Output: ["男默", "女泪"] // Dynamically add a new word jieba.InsertUserWord("男默女泪"); // After segmentation - "男默女泪" is treated as a whole word jieba.Cut("男默女泪", words); // Output: ["男默女泪"] // Add a new word with frequency jieba.InsertUserWord("区块链", 1000, "nz"); // Add a new word with part-of-speech jieba.InsertUserWord("蓝翔", "nz"); // Check if a word exists bool exists = jieba.Find("男默女泪"); std::cout << "Word exists: " << (exists ? "Yes" : "No") << std::endl; // Delete a user word jieba.DeleteUserWord("男默女泪"); return 0; } ``` ### Response N/A (Function modifies internal dictionary state) ``` -------------------------------- ### Perform Full Mode Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt Full mode extracts all possible words from the dictionary, which is useful for indexing in search engines. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; jieba.CutAll("我来到北京清华大学", words); // 输出: ["我", "来到", "北京", "清华", "清华大学", "华大", "大学"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; // 输出: 我/来到/北京/清华/清华大学/华大/大学/ // 带位置信息版本 vector wordWithOffset; jieba.CutAll("我来到北京邮电大学", wordWithOffset); // 输出: [{"word": "我", "offset": 0}, {"word": "来自", "offset": 3}, // {"word": "北京", "offset": 9}, {"word": "北京邮电", "offset": 9}, ...] return 0; } ``` -------------------------------- ### Perform Part-of-Speech Tagging Source: https://context7.com/yanyiwu/cppjieba/llms.txt Assigns part-of-speech tags to segmented words. Custom tags can be defined via the user dictionary. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector> tagResult; jieba.Tag("我是拖拉机学院手扶拖拉机专业的。", tagResult); // 输出: [我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x] for (const auto& item : tagResult) { std::cout << item.first << ":" << item.second << " "; } std::cout << std::endl; // 输出: 我:r 是:v 拖拉机:n 学院:n 手扶拖拉机:n 专业:n 的:uj 。:x // 查询单个词的词性 string tag = jieba.LookupTag("拖拉机"); std::cout << "拖拉机的词性: " << tag << std::endl; // 输出: 拖拉机的词性: n return 0; } // 常见词性标签: // n-名词, v-动词, a-形容词, r-代词, d-副词, m-数词 // eng-英文, x-标点符号, uj-助词, nr-人名, ns-地名, nz-其他专名 ``` -------------------------------- ### CutAll - Full Mode Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt Segments all possible words found in the dictionary, useful for search engine indexing. ```APIDOC ## CutAll (Full Mode) ### Description Segments the sentence into all possible words found in the dictionary. ### Parameters - **sentence** (string) - Required - The input text to segment - **words** (vector) - Required - Output container for segmented words ``` -------------------------------- ### Perform Exact Mode Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt The default mode combines maximum probability and HMM to identify dictionary words and unknown words. HMM can be toggled to control new word discovery. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // 启用 HMM(默认)- 可识别未登录词 jieba.Cut("他来到了网易杭研大厦", words); // 输出: ["他", "来到", "了", "网易", "杭研", "大厦"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; // 输出: 他/来到/了/网易/杭研/大厦/ // 禁用 HMM - 仅使用词典分词 jieba.Cut("他来到了网易杭研大厦", words, false); // 输出: ["他", "来到", "了", "网易", "杭", "研", "大厦"] // 获取带位置信息的分词结果 vector wordWithOffset; jieba.Cut("他来到了网易杭研大厦", wordWithOffset); // 输出: [{"word": "他", "offset": 0}, {"word": "来到", "offset": 3}, ...] return 0; } ``` -------------------------------- ### Perform HMM-only Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt This mode relies solely on the Hidden Markov Model and does not use a dictionary, making it suitable for processing text with many unknown words. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; jieba.CutHMM("我来自北京邮电大学。。。学号123456", words); // 输出: ["我来", "自北京", "邮电大学", "。", "。", "。", "学号", "123456"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; // 输出: 我来/自北京/邮电大学/。/。/。/学号/123456/ return 0; } ``` -------------------------------- ### Perform Search Mode Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt Search mode performs exact segmentation followed by full segmentation on long words to improve recall. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; jieba.CutForSearch("小明硕士毕业于中国科学院计算所,后在日本京都大学深造", words); // 输出: ["小明", "硕士", "毕业", "于", "中国", "科学", "学院", "科学院", // "中国科学院", "计算", "计算所", ",", "后", "在", "日本", // "京都", "大学", "日本京都大学", "深造"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; // 带位置信息版本 vector wordWithOffset; jieba.CutForSearch("小明硕士毕业于中国科学院计算所", wordWithOffset); // 每个词包含 word 和 offset 字段 return 0; } ``` -------------------------------- ### Customize Segmentation Separators in C++ Source: https://context7.com/yanyiwu/cppjieba/llms.txt Configures the set of characters used as separators during segmentation, allowing punctuation to be treated as part of words. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // 先添加包含标点的词 jieba.InsertUserWord("同一个世界,同一个梦想"); // 默认分隔符 - 逗号会分隔词语 jieba.Cut("同一个世界,同一个梦想", words); // 输出: ["同一个", "世界", ",", "同一个", "梦想"] // 清空分隔符 - 标点不再分隔 jieba.ResetSeparators(""); jieba.Cut("同一个世界,同一个梦想", words); // 输出: ["同一个世界,同一个梦想"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; return 0; } ``` -------------------------------- ### Tag - Part-of-Speech Tagging Source: https://context7.com/yanyiwu/cppjieba/llms.txt Provides POS tagging for segmented words, returning pairs of word and tag. ```APIDOC ## Tag (POS Tagging) ### Description Performs word segmentation and assigns a part-of-speech tag to each word. ### Parameters - **sentence** (string) - Required - The input text to tag - **tagResult** (vector>) - Required - Output container for word-tag pairs ``` -------------------------------- ### Perform Maximum Probability Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt This mode allows for fine-grained control by specifying a maximum word length. ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // 限制最大词长为 3 jieba.CutSmall("南京市长江大桥", words, 3); // 输出: ["南京市", "长江", "大桥"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; // 输出: 南京市/长江/大桥/ return 0; } ``` -------------------------------- ### KeywordExtractor - TF-IDF Keyword Extraction Source: https://context7.com/yanyiwu/cppjieba/llms.txt Extracts keywords with the highest weights from text using the TF-IDF algorithm. Supports returning words, weights, and position information. ```APIDOC ## KeywordExtractor - TF-IDF Keyword Extraction ### Description Extracts keywords with the highest weights from text using the TF-IDF algorithm. Supports returning words, weights, and position information. ### Method ```cpp // Example usage within a C++ program ``` ### Endpoint N/A (This is a library function, not a web endpoint) ### Parameters N/A (Function parameters are defined within the C++ code) ### Request Example ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; string text = "我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。"; size_t topN = 5; // Method 1: Get only keywords vector keywords; jieba.extractor.Extract(text, keywords, topN); // Output: ["CEO", "升职", "加薪", "手扶拖拉机", "巅峰"] // Method 2: Get keywords and weights vector> keywordsWithWeight; jieba.extractor.Extract(text, keywordsWithWeight, topN); // Output: [CEO:11.7392, 升职:10.8562, 加薪:10.6426, 手扶拖拉机:10.0089, 巅峰:9.49396] for (const auto& kw : keywordsWithWeight) { std::cout << kw.first << ":" << kw.second << " "; } std::cout << std::endl; // Method 3: Get full information (word, offset, weight) vector wordweights; jieba.extractor.Extract(text, wordweights, topN); // Output: [{"word": "CEO", "offset": [93], "weight": 11.7392}, ...] return 0; } ``` ### Response N/A (Function returns data directly to C++ variables) ``` -------------------------------- ### ResetSeparators - Custom Separators Source: https://context7.com/yanyiwu/cppjieba/llms.txt Allows customization of the separator set used during word segmentation. By default, punctuation marks are treated as separators and split out individually. ```APIDOC ## ResetSeparators - Custom Separators ### Description Allows customization of the separator set used during word segmentation. By default, punctuation marks are treated as separators and split out individually. ### Method ```cpp // Example usage within a C++ program ``` ### Endpoint N/A (This is a library function, not a web endpoint) ### Parameters N/A (Function parameters are defined within the C++ code) ### Request Example ```cpp #include "cppjieba/Jieba.hpp" #include using namespace cppjieba; int main() { Jieba jieba; vector words; // First, add a word containing punctuation jieba.InsertUserWord("同一个世界,同一个梦想"); // Default separators - comma separates words jieba.Cut("同一个世界,同一个梦想", words); // Output: ["同一个", "世界", ",", "同一个", "梦想"] // Clear separators - punctuation no longer separates words jieba.ResetSeparators(""); jieba.Cut("同一个世界,同一个梦想", words); // Output: ["同一个世界,同一个梦想"] for (const auto& word : words) { std::cout << word << "/"; } std::cout << std::endl; return 0; } ``` ### Response N/A (Function modifies internal separator settings) ``` -------------------------------- ### Cut - Exact Mode Segmentation Source: https://context7.com/yanyiwu/cppjieba/llms.txt The default segmentation mode combining maximum probability and HMM to identify known words and unknown words. ```APIDOC ## Cut (Exact Mode) ### Description Performs exact mode segmentation using a mix of MP and HMM algorithms. ### Parameters - **sentence** (string) - Required - The input text to segment - **words** (vector) - Required - Output container for segmented words - **hmm** (bool) - Optional - Enable/Disable HMM (default: true) ``` -------------------------------- ### TextRankExtractor - TextRank Keyword Extraction Source: https://context7.com/yanyiwu/cppjieba/llms.txt Extracts keywords using the TextRank graph ranking algorithm by building and ranking a graph based on word co-occurrence. ```APIDOC ## TextRankExtractor - TextRank Keyword Extraction ### Description Extracts keywords using the TextRank graph ranking algorithm by building and ranking a graph based on word co-occurrence. ### Method ```cpp // Example usage within a C++ program ``` ### Endpoint N/A (This is a library function, not a web endpoint) ### Parameters N/A (Function parameters are defined within the C++ code) ### Request Example ```cpp #include "cppjieba/TextRankExtractor.hpp" #include using namespace cppjieba; int main() { // Initialize TextRank extractor TextRankExtractor extractor( "dict/jieba.dict.utf8", // Main dictionary "dict/hmm_model.utf8", // HMM model "dict/stop_words.utf8" // Stop words list ); string text = "我是拖拉机学院手扶拖拉机专业的。不用多久,我就会升职加薪,当上CEO,走上人生巅峰。"; size_t topN = 5; // Get keywords and weights vector> keywords; extractor.Extract(text, keywords, topN); for (const auto& kw : keywords) { std::cout << kw.first << ":" << kw.second << " "; } std::cout << std::endl; // Output: 当上:1 不用:0.989848 多久:0.985126 加薪:0.983046 升职:0.980278 // Get full information vector wordweights; extractor.Extract(text, wordweights, topN); // Output: [{"word": "当上", "offset": [87], "weight": 1}, ...] return 0; } ``` ### Response N/A (Function returns data directly to C++ variables) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.