GoJieba

https://github.com/yanyiwu/gojieba
Admin
GoJieba is a Go language implementation of Jieba Chinese word segmentation that supports multiple...
Tokens:11,421
Snippets:67
Trust Score:9.8
License:MIT
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install GoJieba

Source: https://github.com/yanyiwu/gojieba/blob/master/README.md

Use the standard go get command to add the library to your project.

```bash
go get github.com/yanyiwu/gojieba
```

--------------------------------

### Comprehensive Example

Source: https://context7.com/yanyiwu/gojieba/llms.txt

A comprehensive example demonstrating various gojieba functionalities including different segmentation modes, part-of-speech tagging, keyword extraction, dynamic dictionary management, and tokenization with position information.

```APIDOC
## Comprehensive Example - Integrated Application

This example demonstrates various functionalities of the gojieba library, including different segmentation modes, part-of-speech tagging, keyword extraction, dynamic dictionary management, and tokenization.

### Functionalities Demonstrated

1.  **Segmentation Modes Comparison**: Shows the output of `CutAll`, `Cut` (precise mode), and `CutForSearch`.
2.  **Part-of-Speech Tagging**: Uses the `Tag` method to assign parts of speech to words.
3.  **Keyword Extraction**: Uses the `Extract` method to find important keywords in a text.
4.  **Dynamic Dictionary Management**: Demonstrates adding a word using `AddWord` and observing its effect on keyword extraction.
5.  **Tokenization**: Uses the `Tokenize` method to get words along with their start and end positions in the text.

### Request Example

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    // 1. Comparison of various segmentation modes
    s := "我来到北京清华大学"
    fmt.Println("Original:", s)
    fmt.Println("Full Mode:", strings.Join(x.CutAll(s), "/"))
    fmt.Println("Precise Mode:", strings.Join(x.Cut(s, true), "/"))
    fmt.Println("Search Engine Mode:", strings.Join(x.CutForSearch(s, true), "/"))
    fmt.Println()

    // 2. Part-of-speech tagging
    s2 := "长春市长春药店"
    fmt.Println("Original:", s2)
    fmt.Println("POS Tagging:", strings.Join(x.Tag(s2), ", "))
    fmt.Println()

    // 3. Keyword extraction
    s3 := "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。"
    fmt.Println("Original:", s3)
    fmt.Println("Keywords:", strings.Join(x.Extract(s3, 5), ", "))
    fmt.Println()

    // 4. Dynamic dictionary management
    x.AddWord("人生巅峰")
    fmt.Println("After adding '人生巅峰':", strings.Join(x.Extract(s3, 5), ", "))
    // Output: After adding '人生巅峰': CEO, 人生巅峰, 升职, 加薪, 手扶拖拉机

    // 5. Tokenize to get position information
    s4 := "长江大桥"
    tokens := x.Tokenize(s4, gojieba.SearchMode, false)
    fmt.Println("\nTokenize Results:")
    for _, t := range tokens {
        fmt.Printf("  Word='%s', Position=[%d:%d]\n", t.Str, t.Start, t.End)
    }
}
```

### Response

The output of the `main` function will be printed to the console, demonstrating the results of each operation.
```

--------------------------------

### Comprehensive gojieba Example

Source: https://context7.com/yanyiwu/gojieba/llms.txt

A comprehensive example demonstrating various gojieba functionalities including different segmentation modes (full, precise, search engine), part-of-speech tagging, keyword extraction, dynamic dictionary management, and tokenization with position information.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    // 1. 各种分词模式对比
    s := "我来到北京清华大学"
    fmt.Println("原文:", s)
    fmt.Println("全模式:", strings.Join(x.CutAll(s), "/"))
    fmt.Println("精确模式:", strings.Join(x.Cut(s, true), "/"))
    fmt.Println("搜索引擎模式:", strings.Join(x.CutForSearch(s, true), "/"))
    fmt.Println()
    // 输出:
    // 原文: 我来到北京清华大学
    // 全模式: 我/来到/北京/清华/清华大学/华大/大学
    // 精确模式: 我/来到/北京/清华大学
    // 搜索引擎模式: 我/来到/北京/清华/清华大学/华大/大学

    // 2. 词性标注
    s2 := "长春市长春药店"
    fmt.Println("原文:", s2)
    fmt.Println("词性标注:", strings.Join(x.Tag(s2), ", "))
    fmt.Println()
    // 输出:
    // 原文: 长春市长春药店
    // 词性标注: 长春市/ns, 长春/ns, 药店/n

    // 3. 关键词提取
    s3 := "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。"
    fmt.Println("原文:", s3)
    fmt.Println("关键词:", strings.Join(x.Extract(s3, 5), ", "))
    fmt.Println()
    // 输出:
    // 原文: 我是拖拉机学院手扶拖拉机专业的。...
    // 关键词: CEO, 升职, 加薪, 手扶拖拉机, 巅峰

    // 4. 动态词典管理
    x.AddWord("人生巅峰")
    fmt.Println("添加'人生巅峰'后:", strings.Join(x.Extract(s3, 5), ", "))
    // 输出: 添加'人生巅峰'后: CEO, 人生巅峰, 升职, 加薪, 手扶拖拉机

    // 5. Tokenize 获取位置信息
    s4 := "长江大桥"
    tokens := x.Tokenize(s4, gojieba.SearchMode, false)
    fmt.Println("\nTokenize结果:")
    for _, t := range tokens {
        fmt.Printf("  词语='%s', 位置=[%d:%d]\n", t.Str, t.Start, t.End)
    }
    // 输出:
    // Tokenize结果:
    //   词语='长江', 位置=[0:6]
    //   词语='大桥', 位置=[6:12]
    //   词语='长江大桥', 位置=[0:12]
}
```

--------------------------------

### Demo Output Examples

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Sample outputs demonstrating various segmentation, tagging, and keyword extraction features.

```text
[demo] Cut With HMM
他/来到/了/网易/杭研/大厦
[demo] Cut Without HMM
他/来到/了/网易/杭/研/大厦
我来到北京清华大学
[demo] CutAll
我/来到/北京/清华/清华大学/华大/大学
小明硕士毕业于中国科学院计算所，后在日本京都大学深造
[demo] CutForSearch
小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所/，/后/在/日本/京都/大学/日本京都大学/深造
[demo] Insert User Word
男默/女泪
男默女泪
[demo] CutForSearch Word With Offset
[{"word": "小明", "offset": 0}, {"word": "硕士", "offset": 6}, {"word": "毕业", "offset": 12}, {"word": "于", "offset": 18}, {"word": "中国", "offset": 21}, {"word": "科学", "offset": 27}, {"word": "学院", "offset": 30}, {"word": "科学院", "offset": 27}, {"word": "中国科学院", "offset": 21}, {"word": "计算", "offset": 36}, {"word": "计算所", "offset": 36}, {"word": "，", "offset": 45}, {"word": "后", "offset": 48}, {"word": "在", "offset": 51}, {"word": "日本", "offset": 54}, {"word": "京都", "offset": 60}, {"word": "大学", "offset": 66}, {"word": "日本京都大学", "offset": 54}, {"word": "深造", "offset": 72}]
[demo] Tagging
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
[我:r, 是:v, 拖拉机:n, 学院:n, 手扶拖拉机:n, 专业:n, 的:uj, 。:x, 不用:v, 多久:m, ，:x, 我:r, 就:d, 会:v, 升职:v, 加薪:nr, ，:x, 当上:t, CEO:eng, ，:x, 走上:v, 人生:n, 巅峰:n, 。:x]
[demo] Keyword Extraction
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
[{"word": "CEO", "offset": [93], "weight": 11.7392}, {"word": "升职", "offset": [72], "weight": 10.8562}, {"word": "加薪", "offset": [78], "weight": 10.6426}, {"word": "手扶拖拉机", "offset": [21], "weight": 10.0089}, {"word": "巅峰", "offset": [111], "weight": 9.49396}]
```

--------------------------------

### Keyword Extraction Example

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Output format for keyword extraction.

```text
我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。
["CEO:11.7392", "升职:10.8562", "加薪:10.6426", "手扶拖拉机:10.0089", "巅峰:9.49396"]
```

--------------------------------

### Part-of-Speech Tagging Example

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Output format for part-of-speech tagging, including custom tag support.

```text
我是蓝翔技工拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上总经理，出任CEO，迎娶白富美，走上人生巅峰。
["我:r", "是:v", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", "，:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", "，:x", "当上:t", "CEO:eng", "，:x", "走上:v", "人生:n", "巅峰:n", "。:x"]

支持自定义词性。
比如在(`dict/user.dict.utf8`)增加一行
蓝翔 nz

结果如下：
["我:r", "是:v", "蓝翔:nz", "技工:n", "拖拉机:n", "学院:n", "手扶拖拉机:n", "专业:n", "的:uj", "。:x", "不用:v", "多久:m", "，:x", "我:r", "就:d", "会:v", "升职:v", "加薪:nr", "，:x", "当:t", "上:f", "总经理:n", "，:x", "出任:v", "CEO:eng", "，:x", "迎娶:v", "白富美:x", "，:x", "走上:v", "人生:n", "巅峰:n", "。:x"]
```

--------------------------------

### User Dictionary Customization

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Examples showing the impact of user dictionaries on segmentation results.

```text
没有使用自定义用户词典时的结果:
令狐冲/是/云/计算/行业/的/专家

使用自定义用户词典时的结果:
令狐冲/是/云计算/行业/的/专家
```

--------------------------------

### Tokenize - Position Information

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segments text and returns the start and end byte offsets for each word.

```APIDOC
## Tokenize

### Description
Segments text and returns word content along with start and end byte offsets.

### Parameters
#### Arguments
- **sentence** (string) - Required - The text to tokenize
- **mode** (int) - Required - Segmentation mode (DefaultMode or SearchMode)
- **hmm** (bool) - Required - Whether to enable HMM

### Response
- **tokens** (array of objects) - Contains Str, Start, and End fields
```

--------------------------------

### Run Demo

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Command to execute the compiled demo application.

```sh
./demo
```

--------------------------------

### Download and Build CppJieba

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Commands to clone the repository, initialize submodules, and compile the project using CMake.

```sh
git clone https://github.com/yanyiwu/cppjieba.git
cd cppjieba
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make
```

--------------------------------

### Create Jieba Instance

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Instantiate a Jieba segmenter. Use default dictionaries or specify custom paths. Remember to call Free() to release resources, preferably with defer.

```go
package main

import (
    "fmt"
    "github.com/yanyiwu/gojieba"
)

func main() {
    // 方式一：使用默认词典（推荐）
    x := gojieba.NewJieba()
    defer x.Free()

    // 方式二：自定义词典路径
    // 参数顺序：主词典、HMM模型、用户词典、IDF词典、停用词表
    // 空字符串表示使用默认路径
    customJieba := gojieba.NewJieba(
        "",  // 使用默认主词典
        "",  // 使用默认HMM模型
        "./my_user_dict.txt",  // 自定义用户词典
        "",  // 使用默认IDF词典
        "",  // 使用默认停用词表
    )
    defer customJieba.Free()

    words := x.Cut("我来到北京清华大学", true)
    fmt.Println(words)
    // 输出: [我 来到 北京 清华大学]
}
```

--------------------------------

### Run Tests and Benchmarks

Source: https://github.com/yanyiwu/gojieba/blob/master/README.md

Commands to execute unit tests and performance benchmarks.

```bash
go test ./...
```

```bash
go test -bench "Jieba" -test.benchtime 10s
go test -bench "Extractor" -test.benchtime 10s
```

--------------------------------

### Run Tests

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Optional command to run the project's test suite.

```sh
make test
```

--------------------------------

### Cross-compile GoJieba

Source: https://github.com/yanyiwu/gojieba/blob/master/README.md

When cross-compiling, ensure CGO_ENABLED is set to 1 and provide the appropriate C/C++ toolchain.

```bash
CGO_ENABLED=1 \
CC=x86_64-linux-gnu-gcc \
CXX=x86_64-linux-gnu-g++ \
GOOS=linux \
GOARCH=amd64 \
go build
```

--------------------------------

### NewJieba - Initialize Segmenter

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Creates a new Jieba segmenter instance. Supports optional custom dictionary paths; if empty, default dictionaries are used.

```APIDOC
## NewJieba

### Description
Initializes a new Jieba segmenter instance. It is recommended to use the default dictionary path by passing empty strings. Resources must be freed using the Free() method.

### Parameters
#### Path Parameters
- **dictPath** (string) - Optional - Path to main dictionary
- **hmmPath** (string) - Optional - Path to HMM model
- **userDictPath** (string) - Optional - Path to user dictionary
- **idfPath** (string) - Optional - Path to IDF dictionary
- **stopWordsPath** (string) - Optional - Path to stop words list
```

--------------------------------

### Perform Chinese Word Segmentation

Source: https://github.com/yanyiwu/gojieba/blob/master/README.md

Demonstrates various segmentation modes, dictionary management, and keyword extraction.

```golang
package main

import (
	"fmt"
	"strings"

	"github.com/yanyiwu/gojieba"
)

func main() {
	var s string
	var words []string
	use_hmm := true
	x := gojieba.NewJieba()
	defer x.Free()

	s = "我来到北京清华大学"
	words = x.CutAll(s)
	fmt.Println(s)
	fmt.Println("全模式:", strings.Join(words, "/"))

	words = x.Cut(s, use_hmm)
	fmt.Println(s)
	fmt.Println("精确模式:", strings.Join(words, "/"))
	s = "比特币"
	words = x.Cut(s, use_hmm)
	fmt.Println(s)
	fmt.Println("精确模式:", strings.Join(words, "/"))

	x.AddWord("比特币")
	// `AddWordEx` 支持指定词语的权重，作为 `AddWord` 权重太低加词失败的补充。
	// `tag` 参数可以为空字符串，也可以指定词性。
	// x.AddWordEx("比特币", 100000, "")
	s = "比特币"
	words = x.Cut(s, use_hmm)
	fmt.Println(s)
	fmt.Println("添加词典后,精确模式:", strings.Join(words, "/"))

	s = "他来到了网易杭研大厦"
	words = x.Cut(s, use_hmm)
	fmt.Println(s)
	fmt.Println("新词识别:", strings.Join(words, "/"))

	s = "小明硕士毕业于中国科学院计算所，后在日本京都大学深造"
	words = x.CutForSearch(s, use_hmm)
	fmt.Println(s)
	fmt.Println("搜索引擎模式:", strings.Join(words, "/"))

	s = "长春市长春药店"
	words = x.Tag(s)
	fmt.Println(s)
	fmt.Println("词性标注:", strings.Join(words, ","))

	s = "区块链"
	words = x.Tag(s)
	fmt.Println(s)
	fmt.Println("词性标注:", strings.Join(words, ","))

	s = "长江大桥"
	words = x.CutForSearch(s, !use_hmm)
	fmt.Println(s)
	fmt.Println("搜索引擎模式:", strings.Join(words, "/"))

	wordinfos := x.Tokenize(s, gojieba.SearchMode, !use_hmm)
	fmt.Println(s)
	fmt.Println("Tokenize:(搜索引擎模式)", wordinfos)

	wordinfos = x.Tokenize(s, gojieba.DefaultMode, !use_hmm)
	fmt.Println(s)
	fmt.Println("Tokenize:(默认模式)", wordinfos)

	keywords := x.ExtractWithWeight(s, 5)
	fmt.Println("Extract:", keywords)
}
```

--------------------------------

### AddWord and AddWordEx - Dynamic Word Addition

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Dynamically add words to the dictionary at runtime. AddWord uses default weight, while AddWordEx allows specifying word weight and part of speech for scenarios requiring high weight to ensure correct segmentation.

```APIDOC
## AddWord / AddWordEx - Dynamic Word Addition

Dynamically add words to the dictionary at runtime. `AddWord` uses default weight, `AddWordEx` can specify word weight and part of speech, suitable for scenarios requiring high weight to ensure correct segmentation.

### Method

- `AddWord(word string)`
- `AddWordEx(word string, weight int, pos string)`

### Parameters

#### AddWord

- **word** (string) - Required - The word to add to the dictionary.

#### AddWordEx

- **word** (string) - Required - The word to add to the dictionary.
- **weight** (int) - Required - The weight of the word (higher value means higher priority).
- **pos** (string) - Optional - The part of speech for the word (can be empty).

### Request Example

```go
// Using AddWord
x.AddWord("比特币")

// Using AddWordEx
x.AddWordEx("区块链技术", 100000, "n")
```

### Response

This method does not return a value. It modifies the internal dictionary of the Jieba instance.
```

--------------------------------

### Segmentation Algorithm Results

Source: https://github.com/yanyiwu/gojieba/blob/master/deps/cppjieba/README.md

Comparison of different segmentation algorithms: MPSegment, HMMSegment, MixSegment, FullSegment, and QuerySegment.

```text
**MPSegment**

Output:
我来到北京清华大学
我/来到/北京/清华大学

他来到了网易杭研大厦
他/来到/了/网易/杭/研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小/明/硕士/毕业/于/中国科学院/计算所/，/后/在/日本京都大学/深造

**HMMSegment**

我来到北京清华大学
我来/到/北京/清华大学

他来到了网易杭研大厦
他来/到/了/网易/杭/研大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业于/中国/科学院/计算所/，/后/在/日/本/京/都/大/学/深/造

**MixSegment**

我来到北京清华大学
我/来到/北京/清华大学

他来到了网易杭研大厦
他/来到/了/网易/杭研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业/于/中国科学院/计算所/，/后/在/日本京都大学/深造

**FullSegment**

我来到北京清华大学
我/来到/北京/清华/清华大学/华大/大学

他来到了网易杭研大厦
他/来到/了/网易/杭/研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小/明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算/计算所/，/后/在/日本/日本京都大学/京都/京都大学/大学/深造

**QuerySegment**

我来到北京清华大学
我/来到/北京/清华/清华大学/华大/大学

他来到了网易杭研大厦
他/来到/了/网易/杭研/大厦

小明硕士毕业于中国科学院计算所，后在日本京都大学深造
小明/硕士/毕业/于/中国/中国科学院/科学/科学院/学院/计算所/，/后/在/中国/中国科学院/科学/科学院/学院/日本/日本京都大学/京都/京都大学/大学/深造
```

--------------------------------

### Full Mode Segmentation (CutAll)

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segment text by finding all possible words. Fast but may include ambiguous results. Suitable for retrieving all potential word combinations.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "我来到北京清华大学"
    words := x.CutAll(s)
    fmt.Println("全模式:", strings.Join(words, "/"))
    // 输出: 全模式: 我/来到/北京/清华/清华大学/华大/大学

    s2 := "长江大桥"
    words2 := x.CutAll(s2)
    fmt.Println("全模式:", strings.Join(words2, "/"))
    // 输出: 全模式: 长江/长江大桥/大桥
}
```

--------------------------------

### Part-of-Speech Tagging (Tag)

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segment text and tag each word with its part of speech (e.g., n for noun, v for verb). Useful for linguistic analysis.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "长春市长春药店"
    words := x.Tag(s)
    fmt.Println("词性标注:", strings.Join(words, ", "))
    // 输出: 词性标注: 长春市/ns, 长春/ns, 药店/n

    s2 := "区块链"
    words2 := x.Tag(s2)
    fmt.Println("词性标注:", strings.Join(words2, ", "))
    // 输出: 词性标注: 区块链/nz

    s3 := "我爱北京天安门"
    words3 := x.Tag(s3)
    fmt.Println("词性标注:", strings.Join(words3, ", "))
    // 输出: 词性标注: 我/r, 爱/v, 北京/ns, 天安门/ns
}
```

--------------------------------

### Add Word to Dictionary Dynamically

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Dynamically add words to the dictionary using AddWord with default weight or AddWordEx with custom weight and part-of-speech. Useful for ensuring specific terms are correctly segmented.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "比特币"

    // 添加前：分词不正确
    words := x.Cut(s, true)
    fmt.Println("添加前:", strings.Join(words, "/"))
    // 输出: 添加前: 比特/币

    // 使用 AddWord 添加词语
    x.AddWord("比特币")
    words = x.Cut(s, true)
    fmt.Println("AddWord后:", strings.Join(words, "/"))
    // 输出: AddWord后: 比特币

    // 使用 AddWordEx 添加词语（指定权重和词性）
    // 参数: 词语, 权重(越大优先级越高), 词性(可为空)
    x.AddWordEx("区块链技术", 100000, "n")
    s2 := "区块链技术是未来趋势"
    words2 := x.Cut(s2, true)
    fmt.Println("AddWordEx后:", strings.Join(words2, "/"))
    // 输出: AddWordEx后: 区块链技术/是/未来/趋势
}
```

--------------------------------

### ExtractWithWeight - Keyword Extraction with Weights

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Extract keywords from text using the TF-IDF algorithm and return each keyword's weight. Higher weight indicates greater importance in the text.

```APIDOC
## ExtractWithWeight - Keyword Extraction (with Weights)

Extract keywords from text using the TF-IDF algorithm, and return each keyword's weight value. Higher weight values indicate that the word is more important in the text.

### Method

- `ExtractWithWeight(text string, number int) []WordWeight`

### Parameters

#### ExtractWithWeight

- **text** (string) - Required - The input text from which to extract keywords.
- **number** (int) - Required - The maximum number of keywords to return.

### Response

#### Success Response (200)

- **[]WordWeight** - A slice of `WordWeight` structs, where each struct contains:
  - **Word** (string) - The extracted keyword.
  - **Weight** (float64) - The calculated weight of the keyword.

### Request Example

```go
wordWeights := x.ExtractWithWeight(s, 5)
```

### Response Example

```json
{
  "wordWeights": [
    { "Word": "CEO", "Weight": 11.7392 },
    { "Word": "升职", "Weight": 10.8562 },
    { "Word": "加薪", "Weight": 10.6426 },
    { "Word": "手扶拖拉机", "Weight": 10.0089 },
    { "Word": "巅峰", "Weight": 9.4940 }
  ]
}
```
```

--------------------------------

### Release Jieba Resources

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Release the memory resources occupied by the Jieba segmenter using Free(). It is recommended to use `defer x.Free()` to ensure proper resource deallocation. Multiple calls to Free() are safe and will only execute once.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()  // 推荐：使用 defer 确保资源释放

    words := x.Cut("我来到北京清华大学", true)
    fmt.Println(strings.Join(words, "/"))
    // 输出: 我/来到/北京/清华大学

    // Free() 会自动调用 Trim() 释放C++内存
    // 多次调用 Free() 是安全的，只会执行一次
}
```

--------------------------------

### Tokenize with Position Information

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Tokenize text and return word start/end positions (byte offsets). Supports DefaultMode and SearchMode.

```go
package main

import (
    "fmt"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "长江大桥"

    // 搜索引擎模式
    wordinfos := x.Tokenize(s, gojieba.SearchMode, false)
    fmt.Println("Tokenize(搜索引擎模式):", wordinfos)
    // 输出: Tokenize(搜索引擎模式): [{长江 0 6} {大桥 6 12} {长江大桥 0 12}]

    // 默认模式
    wordinfos2 := x.Tokenize(s, gojieba.DefaultMode, false)
    fmt.Println("Tokenize(默认模式):", wordinfos2)
    // 输出: Tokenize(默认模式): [{长江大桥 0 12}]

    // Word 结构体包含: Str(词语), Start(起始位置), End(结束位置)
    for _, word := range wordinfos {
        fmt.Printf("词语: %s, 起始: %d, 结束: %d\n", word.Str, word.Start, word.End)
    }
    // 输出:
    // 词语: 长江, 起始: 0, 结束: 6
    // 词语: 大桥, 起始: 6, 结束: 12
    // 词语: 长江大桥, 起始: 0, 结束: 12
}
```

--------------------------------

### Extract Keywords with Weights using TF-IDF

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Extract keywords from text using TF-IDF and return their corresponding weights. Higher weights indicate greater importance. The WordWeight struct contains the Word and its Weight.

```go
package main

import (
    "fmt"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。"

    // 提取前5个关键词及其权重
    wordWeights := x.ExtractWithWeight(s, 5)
    fmt.Println("关键词及权重:")
    for _, ww := range wordWeights {
        fmt.Printf("  %s: %.4f\n", ww.Word, ww.Weight)
    }
    // 输出:
    // 关键词及权重:
    //   CEO: 11.7392
    //   升职: 10.8562
    //   加薪: 10.6426
    //   手扶拖拉机: 10.0089
    //   巅峰: 9.4940

    // WordWeight 结构体包含: Word(词语), Weight(权重)
    fmt.Println("完整结构:", wordWeights)
    // 输出: 完整结构: [{CEO 11.739204307083542} {升职 10.8561552143} {加薪 10.642581114} {手扶拖拉机 10.0088573539} {巅峰 9.49395840471}]
}
```

--------------------------------

### CutAll - Full Mode Segmentation

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Scans all possible words in the sentence. Very fast but does not resolve ambiguity.

```APIDOC
## CutAll

### Description
Segments text using full mode, identifying all possible word combinations.

### Parameters
#### Arguments
- **sentence** (string) - Required - The text to segment

### Response
- **words** (array of strings) - All possible segmented words
```

--------------------------------

### Extract Keywords using TF-IDF

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Extract keywords from text using the TF-IDF algorithm. Returns a list of keywords sorted by importance. Specify the number of keywords to extract.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "我是拖拉机学院手扶拖拉机专业的。不用多久，我就会升职加薪，当上CEO，走上人生巅峰。"

    // 提取前5个关键词
    keywords := x.Extract(s, 5)
    fmt.Println("关键词抽取:", strings.Join(keywords, "/"))
    // 输出: 关键词抽取: CEO/升职/加薪/手扶拖拉机/巅峰

    // 提取前3个关键词
    keywords3 := x.Extract(s, 3)
    fmt.Println("Top3关键词:", strings.Join(keywords3, "/"))
    // 输出: Top3关键词: CEO/升职/加薪
}
```

--------------------------------

### Free - Release Resources

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Release the memory resources occupied by the Jieba segmenter. It is recommended to use `defer x.Free()` to ensure resources are correctly released. This method handles repeated calls internally, making multiple calls safe.

```APIDOC
## Free - Release Resources

Release the memory resources occupied by the Jieba segmenter. It is recommended to use `defer x.Free()` to ensure resources are correctly released. This method handles repeated calls internally, making multiple calls safe.

### Method

- `Free()`

### Parameters

This method does not take any parameters.

### Request Example

```go
x := gojieba.NewJieba()
deffer x.Free() // Recommended: Use defer for resource release
```

### Response

This method does not return a value. It releases internal resources.
```

--------------------------------

### Search Engine Mode Segmentation (CutForSearch)

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segment text for search engines by splitting longer words further. Enhances recall for search indexing.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "小明硕士毕业于中国科学院计算所，后在日本京都大学深造"
    words := x.CutForSearch(s, true)
    fmt.Println("搜索引擎模式:", strings.Join(words, "/"))
    // 输出: 搜索引擎模式: 小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所/，/后/在/日本/京都/大学/日本京都大学/深造

    s2 := "长江大桥"
    words2 := x.CutForSearch(s2, false)
    fmt.Println("搜索引擎模式:", strings.Join(words2, "/"))
    // 输出: 搜索引擎模式: 长江/大桥/长江大桥
}
```

--------------------------------

### Precise Mode Segmentation (Cut)

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segment text precisely, optionally enabling HMM for new word discovery. Useful for text analysis.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "我来到北京清华大学"

    // 启用HMM新词发现
    words := x.Cut(s, true)
    fmt.Println("精确模式(HMM):", strings.Join(words, "/"))
    // 输出: 精确模式(HMM): 我/来到/北京/清华大学

    // 不启用HMM
    words = x.Cut(s, false)
    fmt.Println("精确模式(无HMM):", strings.Join(words, "/"))
    // 输出: 精确模式(无HMM): 我/来到/北京/清华大学

    // HMM对新词识别的作用
    s2 := "他来到了网易杭研大厦"
    fmt.Println("启用HMM:", strings.Join(x.Cut(s2, true), "/"))
    // 输出: 启用HMM: 他/来到/了/网易/杭研/大厦
    fmt.Println("关闭HMM:", strings.Join(x.Cut(s2, false), "/"))
    // 输出: 关闭HMM: 他/来到/了/网易/杭/研/大厦
}
```

--------------------------------

### Extract - Keyword Extraction (TF-IDF)

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Extract keywords from text using the TF-IDF algorithm. Returns a list of keywords sorted by importance, up to a specified number.

```APIDOC
## Extract - Keyword Extraction

Extract keywords from text using the TF-IDF algorithm. Returns a specified number of keywords, sorted by importance.

### Method

- `Extract(text string, number int) []string`

### Parameters

#### Extract

- **text** (string) - Required - The input text from which to extract keywords.
- **number** (int) - Required - The maximum number of keywords to return.

### Response

#### Success Response (200)

- **[]string** - A slice of strings, where each string is a keyword extracted from the text, sorted by importance.

### Request Example

```go
keywords := x.Extract(s, 5)
```

### Response Example

```json
{
  "keywords": [
    "CEO",
    "升职",
    "加薪",
    "手扶拖拉机",
    "巅峰"
  ]
}
```
```

--------------------------------

### CutForSearch - Search Engine Mode

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segments text for search engine indexing by further splitting long words to improve recall.

```APIDOC
## CutForSearch

### Description
Segments text for search engine indexing, splitting long words to increase recall.

### Parameters
#### Arguments
- **sentence** (string) - Required - The text to segment
- **hmm** (bool) - Required - Whether to enable HMM new word discovery

### Response
- **words** (array of strings) - Segmented words optimized for search
```

--------------------------------

### Tag - Part-of-Speech Tagging

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Segments text and labels each word with its part-of-speech (e.g., noun, verb).

```APIDOC
## Tag

### Description
Performs segmentation and POS tagging for each word.

### Parameters
#### Arguments
- **sentence** (string) - Required - The text to tag

### Response
- **taggedWords** (array of strings) - Format: "word/tag"
```

--------------------------------

### Cut - Exact Mode Segmentation

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Performs exact mode segmentation, which is ideal for text analysis. Includes an option to enable HMM-based new word discovery.

```APIDOC
## Cut

### Description
Segments text using the exact mode. The HMM parameter allows for the identification of new words not present in the dictionary.

### Parameters
#### Arguments
- **sentence** (string) - Required - The text to segment
- **hmm** (bool) - Required - Whether to enable HMM new word discovery

### Response
- **words** (array of strings) - The segmented words
```

--------------------------------

### Remove Word from Dictionary Dynamically

Source: https://context7.com/yanyiwu/gojieba/llms.txt

Dynamically remove words from the dictionary using RemoveWord. After removal, the word will no longer be recognized as a single unit and will be re-segmented according to other rules.

```go
package main

import (
    "fmt"
    "strings"
    "github.com/yanyiwu/gojieba"
)

func main() {
    x := gojieba.NewJieba()
    defer x.Free()

    s := "这是一个很长的关键字"

    // 添加自定义词语
    x.AddWord("这是一个很长的关键字")
    words := x.Extract(s, 3)
    fmt.Println("添加后:", strings.Join(words, "/"))
    // 输出: 添加后: 这是一个很长的关键字

    // 删除词语
    x.RemoveWord("这是一个很长的关键字")
    words = x.Extract(s, 3)
    fmt.Println("删除后:", strings.Join(words, "/"))
    // 输出: 删除后: 关键字/很长/这是
}
```
Tokens:
Raw