### Install gse using go get Source: https://github.com/go-ego/gse/blob/master/README.md If you are not using Go modules or are on an older version of Go, you can install the gse package using the `go get` command. ```bash go get -u github.com/go-ego/gse ``` -------------------------------- ### Install gse with Go Modules Source: https://github.com/go-ego/gse/blob/master/README.md To install the gse package using Go modules, import it directly into your project. This is the recommended method for Go 1.11 and later. ```go import "github.com/go-ego/gse" ``` -------------------------------- ### Advanced Text Segmentation with HMM and POS Tagging in Go Source: https://github.com/go-ego/gse/blob/master/README.md Illustrates advanced text segmentation using Hidden Markov Models (HMM) and Part-of-Speech (POS) tagging. Includes examples of loading default and custom dictionaries, using CutSearch, CutAll, CutDAG with regular expressions, and analyzing/trimming results. ```go package main import ( "fmt" "regexp" "github.com/go-ego/gse" "github.com/go-ego/gse/hmm/pos" ) var ( text = "Hello world, Helloworld. Winter is coming! こんにちは世界, 你好世界." new, _ = gse.New("zh,testdata/test_en_dict3.txt", "alpha") seg gse.Segmenter posSeg pos.Segmenter ) func main() { // Loading the default dictionary seg.LoadDict() // Loading the default dictionary with embed // seg.LoadDictEmbed() // // Loading the Simplified Chinese dictionary // seg.LoadDict("zh_s") // seg.LoadDictEmbed("zh_s") // // Loading the Traditional Chinese dictionary // seg.LoadDict("zh_t") // // Loading the Japanese dictionary // seg.LoadDict("jp") // // Load the dictionary // seg.LoadDict("your gopath"+"/src/github.com/go-ego/gse/data/dict/dictionary.txt") cut() segCut() } func cut() { hmm := new.Cut(text, true) fmt.Println("cut use hmm: ", hmm) hmm = new.CutSearch(text, true) fmt.Println("cut search use hmm: ", hmm) fmt.Println("analyze: ", new.Analyze(hmm, text)) hmm = new.CutAll(text) fmt.Println("cut all: ", hmm) reg := regexp.MustCompile(`(\d+年|\d+月|\d+日|[\p{Latin}]+|[\p{Hangul}]+|\d+\.\d+|[a-zA-Z0-9]+)`) text1 := ``할로월드 할로 서울, 2021年09月10日, 3.14` hmm = seg.CutDAG(text1, reg) fmt.Println("Cut with hmm and regexp: ", hmm, hmm[0], hmm[6]) } func analyzeAndTrim(cut []string) { a := seg.Analyze(cut, "") fmt.Println("analyze the segment: ", a) cut = seg.Trim(cut) fmt.Println("cut all: ", cut) fmt.Println(seg.String(text, true)) fmt.Println(seg.Slice(text, true)) } func cutPos() { po := seg.Pos(text, true) fmt.Println("pos: ", po) po = seg.TrimPos(po) fmt.Println("trim pos: ", po) pos.WithGse(seg) po = posSeg.Cut(text, true) fmt.Println("pos: ", po) po = posSeg.TrimWithPos(po, "zg") fmt.Println("trim pos: ", po) } func segCut() { // Text Segmentation tb := []byte(text) fmt.Println(seg.String(text, true)) segments := seg.Segment(tb) // Handle word segmentation results, search mode fmt.Println(gse.ToString(segments, true)) } ``` -------------------------------- ### Standard Go-Ego Project Copyright Header Source: https://github.com/go-ego/gse/blob/master/CONTRIBUTING.md Use this standard copyright header for all contributed code. If the copyright author changes, paste the new header below the old one. ```go // Copyright 2016 The go-ego Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // https://github.com/go-ego/gse/blob/master/LICENSE // // Licensed under the Apache License, Version 2.0 // // This file may not be copied, modified, or distributed // except according to those terms. ``` -------------------------------- ### Build Commands for gse Source: https://github.com/go-ego/gse/blob/master/AGENTS.md Common commands for building, testing, and linting the gse library. These commands cover various testing scenarios, coverage analysis, benchmarking, and formatting. ```bash go build -v . ``` ```bash go test -v . ``` ```bash go test -v ./... ``` ```bash go test -v -run TestHMM . ``` ```bash go test -v ./hmm/... ``` ```bash go test -v -covermode=count -coverprofile=coverage.out ``` ```bash go test -v -bench=. -benchmem . ``` ```bash go run ./tools/benchmark/benchmark.go ``` ```bash ./tools/benchmark/goroutines/goroutines.go ``` ```bash go run ./tools/server/server.go ``` ```bash go build -tags ne . ``` ```bash gofmt -s -w . ``` ```bash go vet ./... ```