Vaporetto

https://docs.rs/vaporetto/0.6.5/vaporetto/
Admin
Vaporetto is a fast and lightweight pointwise prediction based tokenizer.
Tokens:27,997
Snippets:324
Trust Score:9.1
Update:3 months ago
Show doc for...
Tokens:
Raw
### Training a Model

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Example showing how to initialize a trainer, add examples, and train a model.

```rust
use std::fs::File;
use std::io::{prelude::*, BufReader, BufWriter};

use vaporetto::{Sentence, SolverType, Trainer};

let mut train_sents = vec![];
let f = BufReader::new(File::open("dataset-train.txt").unwrap());
for (i, line) in f.lines().enumerate() {
    train_sents.push(Sentence::from_tokenized(&line.unwrap()).unwrap());
}

let dict: Vec<String> = vec![];
let mut trainer = Trainer::new(3, 3, 3, 3, dict, 0, &[]).unwrap();
for (i, s) in train_sents.iter().enumerate() {
    trainer.add_example(&s);
}

let model = trainer.train(0.01, 1., SolverType::L1RegularizedL2LossSVC).unwrap();
let mut f = BufWriter::new(File::create("model.bin").unwrap());
model.write(&mut f).unwrap();
```

--------------------------------

### Predictor usage examples

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Predictor.html

Demonstrates how to initialize a Predictor and perform tokenization with or without tag prediction.

```rust
use std::fs::File;

use vaporetto::{Model, Predictor, Sentence};

let f = File::open("../resources/model.bin").unwrap();
let model = Model::read(f).unwrap();
let predictor = Predictor::new(model, false).unwrap();

let mut s = Sentence::from_raw("まぁ社長は火星猫だ").unwrap();
predictor.predict(&mut s);
// s.fill_tags(); will panic!

let mut buf = String::new();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ 社長 は 火星 猫 だ",
    buf,
);
```

```rust
use std::fs::File;

use vaporetto::{Model, Predictor, Sentence};

let mut f = File::open("../resources/model.bin").unwrap();
let model = Model::read(f).unwrap();
let predictor = Predictor::new(model, true).unwrap();

let mut s = Sentence::from_raw("まぁ社長は火星猫だ").unwrap();
predictor.predict(&mut s);
s.fill_tags();

let mut buf = String::new();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ 火星/名詞/カセー 猫/名詞/ネコ だ/助動詞/ダ",
    buf,
);
```

--------------------------------

### Trainer::add_example

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html

Adds a sentence to the trainer to be used as a training example.

```APIDOC
## Trainer::add_example

### Description
Adds a sentence to the trainer instance.

### Signature
`pub fn add_example<'b>(&mut self, sentence: &'a Sentence<'a, 'b>)`

### Parameters
- **sentence** (&'a Sentence<'a, 'b>) - The sentence to add to the training set.
```

--------------------------------

### Trainer::add_example

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=

Adds a sentence to the trainer to be used as a training example.

```APIDOC
## Trainer::add_example

### Description
Adds a sentence to the trainer for model training.

### Parameters
- **sentence** (&Sentence) - The sentence to add to the training set.
```

--------------------------------

### Token::start

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Token.html

Returns the start character position of the token.

```APIDOC
## pub const fn start(&self) -> usize

### Description
Returns the start position of this token in characters.
```

--------------------------------

### Trainer::train

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Trains the model using the added examples.

```APIDOC
## Trainer::train

### Description
Trains word boundaries and tags based on the added examples.

### Parameters
- **epsilon** (f64) - The tolerance of the termination criterion.
- **cost** (f64) - The parameter C.
- **solver** (SolverType) - The solver type to use for training.
```

--------------------------------

### Trainer::train

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html

Executes the training process to generate a model based on the added examples.

```APIDOC
## Trainer::train

### Description
Trains word boundaries and tags using the provided parameters.

### Signature
`pub fn train(self, epsilon: f64, cost: f64, solver: SolverType) -> Result<Model>`

### Parameters
- **epsilon** (f64) - The tolerance of the termination criterion.
- **cost** (f64) - The parameter C.
- **solver** (SolverType) - The solver type to use for training.
```

--------------------------------

### Get Raw Text from Sentence

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Retrieves the raw text of the sentence without any annotations.

```rust
use vaporetto::Sentence;

let s = Sentence::from_tokenized("まぁ/副詞 良い/形容詞 だろう/助動詞").unwrap();
assert_eq!("まぁ良いだろう", s.as_raw_text());
```

--------------------------------

### Get Character Types

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Returns a slice of character types for the sentence.

```rust
use vaporetto::{CharacterType, Sentence};

let s = Sentence::from_tokenized("火星 に 行き まし た").unwrap();
assert_eq!(&[
    CharacterType::Kanji as u8,
    CharacterType::Kanji as u8,
    CharacterType::Hiragana as u8,
    CharacterType::Kanji as u8,
    CharacterType::Hiragana as u8,
    CharacterType::Hiragana as u8,
    CharacterType::Hiragana as u8,
    CharacterType::Hiragana as u8,
], s.char_types());
```

--------------------------------

### Get Maximum Tag Count

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Returns the maximum number of tags allowed per character.

```rust
use vaporetto::Sentence;

let s = Sentence::from_tokenized("火星/名詞/カセー に 行き/動詞 まし/助動詞/マシ た").unwrap();
assert_eq!(2, s.n_tags());
```

--------------------------------

### Termination::report

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Is called to get the representation of the value as status code. This status code is returned to the operating system.

```APIDOC
### fn report(self) -> ExitCode

Is called to get the representation of the value as status code. This status code is returned to the operating system.
```

--------------------------------

### Trainer::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html

Creates a new instance of the Trainer with specified window sizes, n-gram lengths, and dictionary configurations.

```APIDOC
## Trainer::new

### Description
Creates a new trainer instance with the provided configuration parameters.

### Signature
`pub fn new(char_window_size: u8, char_ngram_size: u8, type_window_size: u8, type_ngram_size: u8, dict_words: Vec<String>, dict_word_max_len: u8, tag_dictionary: &'a [Sentence<'a, '_>]) -> Result<Self>`

### Parameters
- **char_window_size** (u8) - The character window size.
- **char_ngram_size** (u8) - The character n-gram length.
- **type_window_size** (u8) - The character type window size.
- **type_ngram_size** (u8) - The character type n-gram length.
- **dict_words** (Vec<String>) - A word dictionary.
- **dict_word_max_len** (u8) - Dictionary words longer than this value will be grouped together.
- **tag_dictionary** (&'a [Sentence<'a, '_>]) - A tag dictionary for annotating words not included in the corpus.
```

--------------------------------

### Trainer::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=

Creates a new instance of the Trainer with specified configuration parameters.

```APIDOC
## Trainer::new

### Description
Creates a new trainer instance with the provided window sizes, n-gram lengths, and dictionary configurations.

### Parameters
- **char_window_size** (u8) - The character window size.
- **char_ngram_size** (u8) - The character n-gram length.
- **type_window_size** (u8) - The character type window size.
- **type_ngram_size** (u8) - The character type n-gram length.
- **dict_words** (Vec<String>) - A word dictionary.
- **dict_word_max_len** (u8) - Dictionary words longer than this value will be grouped together.
- **tag_dictionary** (&[Sentence]) - A tag dictionary for annotating words not in the corpus.

### Returns
- **Result<Self>** - Returns the new Trainer instance or an error if parameters are invalid.
```

--------------------------------

### Read and use a KyteaModel

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.KyteaModel.html?search=

Demonstrates reading a KyteaModel from a file and using it to initialize a Predictor for tokenization.

```rust
use std::fs::File;
use std::io::BufReader;

use vaporetto::{KyteaModel, Model, Predictor, Sentence};

let f = BufReader::new(File::open("../resources/kytea-model.bin").unwrap());
let model = KyteaModel::read(f).unwrap();
let model = Model::try_from(model).unwrap();

// Vaporetto does not support tagging using KyTea's models.
let predictor = Predictor::new(model, false).unwrap();

let mut buf = String::new();

let mut s = Sentence::default();

s.update_raw("まぁ社長は火星猫だ").unwrap();
predictor.predict(&mut s);
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ 社長 は 火星 猫 だ", buf);
```

--------------------------------

### Trainer::add_example

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Adds a sentence to the trainer for model training.

```APIDOC
## Trainer::add_example

### Description
Adds a sentence to the trainer instance.

### Parameters
- **sentence** (&'a Sentence<'a, 'b>) - The sentence to add to the training set.
```

--------------------------------

### Predictor Usage

Source: https://docs.rs/vaporetto

Demonstrates how to load a model, initialize a predictor, and perform tokenization and tag prediction on a sentence.

```APIDOC
## Rust API: vaporetto::Predictor

### Description
The `Predictor` is the primary interface for performing tokenization and tag prediction on input text using a loaded `Model`.

### Methods
- **Predictor::new(model: Model, tag_prediction: bool)**: Creates a new predictor instance. Requires the `tag-prediction` feature if `tag_prediction` is set to true.
- **predictor.predict(&mut sentence)**: Performs tokenization on the provided `Sentence` object.

### Example
```rust
let model = Model::read(f)?;
let predictor = Predictor::new(model, true)?;
let mut s = Sentence::default();
s.update_raw("まぁ社長は火星猫だ")?;
predictor.predict(&mut s);
s.fill_tags();
```
```

--------------------------------

### Trainer::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Creates a new instance of the Trainer.

```APIDOC
## Trainer::new

### Description
Creates a new trainer instance with specified window sizes, n-gram lengths, dictionary settings, and tag dictionary.

### Parameters
- **char_window_size** (u8) - The character window size.
- **char_ngram_size** (u8) - The character n-gram length.
- **type_window_size** (u8) - The character type window size.
- **type_ngram_size** (u8) - The character type n-gram length.
- **dict_words** (Vec<String>) - A word dictionary.
- **dict_word_max_len** (u8) - Dictionary words longer than this value will be grouped together.
- **tag_dictionary** (&'a [Sentence<'a, '_>]) - A tag dictionary for annotating words not in the corpus.
```

--------------------------------

### Get Character Type

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/enum.CharacterType.html

Usage of the get_type method to identify the category of a specific character.

```rust
use vaporetto::CharacterType;

let t = CharacterType::get_type('A');
assert_eq!(CharacterType::Roman, t);
```

--------------------------------

### Initialize Predictor with tag prediction

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Predictor.html?search=std%3A%3Avec

Demonstrates creating a Predictor instance with tag prediction enabled. Requires the 'tag-prediction' crate feature.

```rust
use std::fs::File;

use vaporetto::{Model, Predictor, Sentence};

let mut f = File::open("../resources/model.bin").unwrap();
let model = Model::read(f).unwrap();
let predictor = Predictor::new(model, true).unwrap();

let mut s = Sentence::from_raw("まぁ社長は火星猫だ").unwrap();
predictor.predict(&mut s);
s.fill_tags();

let mut buf = String::new();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ 火星/名詞/カセー 猫/名詞/ネコ だ/助動詞/ダ",
    buf,
);
```

--------------------------------

### WordWeightRecord::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.WordWeightRecord.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Creates a new instance of a WordWeightRecord.

```APIDOC
## pub fn new(word: String, weights: Vec<i32>, comment: String) -> Result<Self>

### Description
Creates a new word weight record. The number of weights must match the number of characters in the word plus one.

### Parameters
- **word** (String) - A word.
- **weights** (Vec<i32>) - Weights of each character boundary.
- **comment** (String) - A comment that does not affect the behaviour.

### Errors
Returns an error if `weights.len() != word.chars().count() + 1`.
```

--------------------------------

### WordWeightRecord::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.WordWeightRecord.html?search=u32+-%3E+bool

Creates a new instance of a WordWeightRecord.

```APIDOC
## pub fn new(word: String, weights: Vec<i32>, comment: String) -> Result<Self>

### Description
Creates a new word weight record. The weights vector length must match the character count of the word plus one.

### Parameters
- **word** (String) - A word.
- **weights** (Vec<i32>) - Weights of each character boundary.
- **comment** (String) - A comment that does not affect the behaviour.

### Errors
Returns an error if `weights.len() != word.chars().count() + 1`.
```

--------------------------------

### Initialize from Partial Annotation

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Creates a Sentence using pipe symbols for token boundaries and dashes for non-boundaries. Whitespace is used for unannotated boundaries.

```rust
use vaporetto::Sentence;

let mut buf = String::new();

let s = Sentence::from_partial_annotation(
    "ま-ぁ|良-い|だ-ろ-う"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ 良い だろう", buf);

let s = Sentence::from_partial_annotation(
    "ま-ぁ/名詞/マー|社-長/名詞/シャチョー|は/助詞/ワ|火-星 猫|だ/助動詞/ダ"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ だ/助動詞/ダ", buf);

let s = Sentence::from_partial_annotation(
    "ま-ぁ/名詞/マー|社-長/名詞/シャチョー|は/助詞/ワ|火/名詞/ヒ-星|猫|だ/助動詞/ダ"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ 火星 猫 だ/助動詞/ダ", buf);
```

--------------------------------

### Tokenizing text with Vaporetto

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/index.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Demonstrates loading a model from a file, initializing a predictor, and tokenizing input strings with tag prediction.

```rust
use std::fs::File;

use vaporetto::{Model, Predictor, Sentence};

let f = File::open("../resources/model.bin")?;
let model = Model::read(f)?;
let predictor = Predictor::new(model, true)?;

let mut buf = String::new();

let mut s = Sentence::default();

s.update_raw("まぁ社長は火星猫だ")?;
predictor.predict(&mut s);
s.fill_tags();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ 火星/名詞/カセー 猫/名詞/ネコ だ/助動詞/ダ",
    buf,
);

s.update_raw("まぁ良いだろう")?;
predictor.predict(&mut s);
s.fill_tags();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ/副詞/マー 良い/形容詞/ヨイ だろう/助動詞/ダロー",
    buf,
);
```

--------------------------------

### Calculate Product of Result Items

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Demonstrates using the product method on an iterator of Results, which short-circuits on the first Err encountered.

```rust
let nums = vec!["5", "10", "1", "2"];
let total: Result<usize, _> = nums.iter().map(|w| w.parse::<usize>()).product();
assert_eq!(total, Ok(100));
let nums = vec!["5", "10", "one", "2"];
let total: Result<usize, _> = nums.iter().map(|w| w.parse::<usize>()).product();
assert!(total.is_err());
```

--------------------------------

### Access the VERSION constant

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/constant.VERSION.html

Retrieves the current version string of the library.

```rust
pub const VERSION: &str = "0.6.5";
```

--------------------------------

### into_ok

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Returns the contained Ok value without panicking.

```APIDOC
## pub const fn into_ok(self) -> T

### Description
Returns the contained `Ok` value, but never panics. This is a nightly-only experimental API.

### Signature
`pub const fn into_ok(self) -> T` where `E: Into<!>`
```

--------------------------------

### Create a default Sentence

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Initializes a new Sentence instance containing a single space character.

```rust
use vaporetto::Sentence;

let s = Sentence::default();

assert_eq!(" ", s.as_raw_text());
assert_eq!(0, s.n_tags());
```

--------------------------------

### Create Sentence from raw text

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=std%3A%3Avec

Initializes a new Sentence from raw text. Returns an error if the input text is empty.

```rust
use vaporetto::Sentence;

let s = Sentence::from_raw("まぁ良いだろう").unwrap();
let mut buf = String::new();
s.write_partial_annotation_text(&mut buf);
assert_eq!("ま ぁ 良 い だ ろ う", buf);

let s = Sentence::from_raw("");
assert!(s.is_err());
```

--------------------------------

### WordWeightRecord::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.WordWeightRecord.html?search=std%3A%3Avec

Creates a new instance of WordWeightRecord with the specified word, weights, and comment.

```APIDOC
## fn new(word: String, weights: Vec<i32>, comment: String) -> Result<Self>

### Description
Creates a new word weight record. The number of weights must match the number of characters in the word plus one.

### Parameters
- **word** (String) - A word.
- **weights** (Vec<i32>) - Weights of each character boundary.
- **comment** (String) - A comment that does not affect the behaviour.

### Errors
Returns an error if `weights.len() != word.chars().count() + 1`.
```

--------------------------------

### KyteaModel::read<R>

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.KyteaModel.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Creates a new KyteaModel instance from a provided reader.

```APIDOC
## pub fn read<R>(rdr: R) -> Result<Self>

### Description
Creates a model from a reader. This method is available when the `kytea` feature is enabled.

### Parameters
- **rdr** (R: BufRead) - A data source to read the model from.

### Errors
Returns an error if the reader generates an error during the read process.
```

--------------------------------

### Use or to provide fallback Results

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Returns the current Result if it is Ok, otherwise returns the provided fallback Result.

```rust
let x: Result<u32, &str> = Ok(2);
let y: Result<u32, &str> = Err("late error");
assert_eq!(x.or(y), Ok(2));

let x: Result<u32, &str> = Err("early error");
let y: Result<u32, &str> = Ok(2);
assert_eq!(x.or(y), Ok(2));

let x: Result<u32, &str> = Err("not a 2");
let y: Result<u32, &str> = Err("late error");
assert_eq!(x.or(y), Err("late error"));

let x: Result<u32, &str> = Ok(2);
let y: Result<u32, &str> = Ok(100);
assert_eq!(x.or(y), Ok(2));
```

--------------------------------

### pub const fn n_tags(&self) -> usize

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html

Returns the maximum number of tags.

```APIDOC
## pub const fn n_tags(&self) -> usize

### Description
Returns the maximum number of tags supported by the sentence structure.
```

--------------------------------

### Initialize Predictor without tag prediction

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Predictor.html?search=std%3A%3Avec

Demonstrates creating a Predictor instance and performing tokenization without tag prediction enabled.

```rust
use std::fs::File;

use vaporetto::{Model, Predictor, Sentence};

let f = File::open("../resources/model.bin").unwrap();
let model = Model::read(f).unwrap();
let predictor = Predictor::new(model, false).unwrap();

let mut s = Sentence::from_raw("まぁ社長は火星猫だ").unwrap();
predictor.predict(&mut s);
// s.fill_tags(); will panic!

let mut buf = String::new();
s.write_tokenized_text(&mut buf);
assert_eq!(
    "まぁ 社長 は 火星 猫 だ",
    buf,
);
```

--------------------------------

### Initialize and Update Raw Text

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Methods to create or update a Sentence from unannotated text. An error is returned if the input text is empty.

```rust
use vaporetto::Sentence;

let s = Sentence::from_raw("まぁ良いだろう").unwrap();
let mut buf = String::new();
s.write_partial_annotation_text(&mut buf);
assert_eq!("ま ぁ 良 い だ ろ う", buf);

let s = Sentence::from_raw("");
assert!(s.is_err());
```

```rust
use vaporetto::Sentence;

let mut s = Sentence::from_raw("まぁ良いだろう").unwrap();
s.update_raw("まぁ社長は火星猫だ").unwrap();
assert_eq!("まぁ社長は火星猫だ", s.as_raw_text());
```

--------------------------------

### Initialize and Update Tokenized Text

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Methods to handle text where token boundaries are marked by spaces. Input must not be empty, start/end with spaces, or contain consecutive spaces.

```rust
use vaporetto::Sentence;

let s = Sentence::from_tokenized("まぁ 社長 は 火星 猫 だ");
assert_eq!("まぁ社長は火星猫だ", s.unwrap().as_raw_text());

let s = Sentence::from_tokenized("まぁ/名詞 社長/名詞 は/助詞 火星/名詞 猫/名詞 だ/助動詞");
assert_eq!("まぁ社長は火星猫だ", s.unwrap().as_raw_text());

let s = Sentence::from_tokenized("まぁ 社長  は 火星 猫 だ");
assert!(s.is_err());
```

```rust
use vaporetto::Sentence;

let mut s = Sentence::default();

s.update_tokenized("まぁ 良い だろう").unwrap();
assert_eq!("まぁ良いだろう", s.as_raw_text());

s.update_tokenized("まぁ/副詞/マー 良い/形容詞/ヨイ だろう/助動詞/ダロー").unwrap();
assert_eq!("まぁ良いだろう", s.as_raw_text());
```

--------------------------------

### Character equivalence methods

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/index.html?search=u32+-%3E+bool

Method signatures for checking equivalence in character types and boundaries.

```rust
method
vaporetto::CharacterType::equivalent
&, &**K** -> **bool**
where
u32 matches **K**
```

```rust
method
vaporetto::CharacterBoundary::equivalent
&, &**K** -> **bool**
where
u32 matches **K**
```

--------------------------------

### Summing Results with Iterator

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Demonstrates how to sum an iterator of Results, which short-circuits and returns the first encountered error.

```rust
let f = |&x: &i32| if x < 0 { Err("Negative element found") } else { Ok(x) };
let v = vec![1, 2];
let res: Result<i32, _> = v.iter().map(f).sum();
assert_eq!(res, Ok(3));
let v = vec![1, -2];
let res: Result<i32, _> = v.iter().map(f).sum();
assert_eq!(res, Err("Negative element found"));
```

--------------------------------

### pub fn tags(&self) -> &[Option<Cow<'b, str>>]

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html

Returns a reference to the internal representation of tags.

```APIDOC
## pub fn tags(&self) -> &[Option<Cow<'b, str>>]

### Description
Returns a reference to the internal representation of tags. Tags are stored in an array where the j-th tag of the i-th character is stored in the i*k+j-th element, where k is the maximum number of tags.
```

--------------------------------

### KyteaModel::read

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.KyteaModel.html?search=std%3A%3Avec

Creates a new KyteaModel instance by reading data from a provided reader.

```APIDOC
## pub fn read<R>(rdr: R) -> Result<Self>

### Description
Creates a model from a reader. This method is available when the `kytea` crate feature is enabled.

### Parameters
- **rdr** (R: BufRead) - Required - A data source to read the model from.

### Errors
Returns an error if the reader generates an error during the reading process.
```

--------------------------------

### Copying Result Contents

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Methods to copy or clone the contents of an Ok variant from a reference.

```rust
let val = 12;
let x: Result<&i32, i32> = Ok(&val);
assert_eq!(x, Ok(&12));
let copied = x.copied();
assert_eq!(copied, Ok(12));
```

```rust
let val = 12;
let x: Result<&i32, i32> = Ok(&val);
assert_eq!(x, Ok(&12));
let cloned = x.cloned();
assert_eq!(cloned, Ok(12));
```

```rust
let mut val = 12;
let x: Result<&mut i32, i32> = Ok(&mut val);
assert_eq!(x, Ok(&mut 12));
let copied = x.copied();
assert_eq!(copied, Ok(12));
```

```rust
let mut val = 12;
let x: Result<&mut i32, i32> = Ok(&mut val);
assert_eq!(x, Ok(&mut 12));
let cloned = x.cloned();
assert_eq!(cloned, Ok(12));
```

--------------------------------

### VERSION

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/constant.VERSION.html?search=u32+-%3E+bool

The version number of the Vaporetto library.

```APIDOC
## Constant: VERSION

### Description
Returns the current version string of the Vaporetto library.

### Value
`"0.6.5"`
```

--------------------------------

### Copying Result Contents

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Copies the contents of an Ok variant from a reference.

```rust
let val = 12;
let x: Result<&i32, i32> = Ok(&val);
assert_eq!(x, Ok(&12));
let copied = x.copied();
assert_eq!(copied, Ok(12));
```

```rust
let mut val = 12;
let x: Result<&mut i32, i32> = Ok(&mut val);
assert_eq!(x, Ok(&mut 12));
let copied = x.copied();
assert_eq!(copied, Ok(12));
```

--------------------------------

### Provide default values with unwrap_or

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Returns the contained Ok value or a provided default value.

```rust
let default = 2;
let x: Result<u32, &str> = Ok(9);
assert_eq!(x.unwrap_or(default), 9);

let x: Result<u32, &str> = Err("error");
assert_eq!(x.unwrap_or(default), default);
```

--------------------------------

### Chain Results with and

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Returns the provided result if the current result is Ok, otherwise returns the current Err.

```rust
let x: Result<u32, &str> = Ok(2);
let y: Result<&str, &str> = Err("late error");
assert_eq!(x.and(y), Err("late error"));

let x: Result<u32, &str> = Err("early error");
let y: Result<&str, &str> = Ok("foo");
assert_eq!(x.and(y), Err("early error"));

let x: Result<u32, &str> = Err("not a 2");
let y: Result<&str, &str> = Err("late error");
assert_eq!(x.and(y), Err("not a 2"));

let x: Result<u32, &str> = Ok(2);
let y: Result<&str, &str> = Ok("different result type");
assert_eq!(x.and(y), Ok("different result type"));
```

--------------------------------

### Token Accessor Methods

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Token.html

Methods to retrieve token surface, tags, and character positions.

```rust
pub fn surface(&self) -> &'a str
```

```rust
pub fn tags(&self) -> &'a [Option<Cow<'b, str>>]
```

```rust
pub fn tag_candidates(&self) -> Vec<Vec<(&'b str, i32)>>
```

```rust
pub const fn start(&self) -> usize
```

```rust
pub const fn end(&self) -> usize
```

--------------------------------

### unwrap

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Returns the contained Ok value, consuming the self value. Panics if the value is an Err.

```APIDOC
## pub fn unwrap(self) -> T

### Description
Returns the contained `Ok` value, consuming the `self` value. Panics if the value is an `Err`.

### Signature
`pub fn unwrap(self) -> T` where `E: Debug`
```

--------------------------------

### Access Token properties

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Token.html?search=u32+-%3E+bool

Methods to retrieve the surface text, tags, and character positions of a token.

```rust
pub fn surface(&self) -> &'a str
```

```rust
pub fn tags(&self) -> &'a [Option<Cow<'b, str>>]
```

```rust
pub const fn start(&self) -> usize
```

```rust
pub const fn end(&self) -> usize
```

--------------------------------

### Reference Result with as_ref and as_mut

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Creates a new Result containing references to the original values.

```rust
let x: Result<u32, &str> = Ok(2);
assert_eq!(x.as_ref(), Ok(&2));

let x: Result<u32, &str> = Err("Error");
assert_eq!(x.as_ref(), Err(&"Error"));
```

```rust
fn mutate(r: &mut Result<i32, i32>) {
    match r.as_mut() {
        Ok(v) => *v = 42,
        Err(e) => *e = 0,
    }
}

let mut x: Result<i32, i32> = Ok(2);
mutate(&mut x);
assert_eq!(x.unwrap(), 42);

let mut x: Result<i32, i32> = Err(13);
mutate(&mut x);
assert_eq!(x.unwrap_err(), 0);
```

--------------------------------

### Collect Results into a Container

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Demonstrates using FromIterator to collect a sequence of Results into a single Result containing a collection, stopping at the first error.

```rust
let v = vec![1, 2];
let res: Result<Vec<u32>, &'static str> = v.iter().map(|x: &u32|
    x.checked_add(1).ok_or("Overflow!")
).collect();
assert_eq!(res, Ok(vec![2, 3]));
```

```rust
let v = vec![1, 2, 0];
let res: Result<Vec<u32>, &'static str> = v.iter().map(|x: &u32|
    x.checked_sub(1).ok_or("Underflow!")
).collect();
assert_eq!(res, Err("Underflow!"));
```

```rust
let v = vec![3, 2, 1, 10];
let mut shared = 0;
let res: Result<Vec<u32>, &'static str> = v.iter().map(|x: &u32| {
    shared += x;
    x.checked_sub(2).ok_or("Underflow!")
}).collect();
assert_eq!(res, Err("Underflow!"));
assert_eq!(shared, 6);
```

--------------------------------

### Sentence::default()

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Creates a new Sentence instance consisting of a space.

```APIDOC
## fn default() -> Self

### Description
Creates a new `Sentence` consisting of a space.

### Signature
`pub fn default() -> Self`

### Example
```rust
use vaporetto::Sentence;

let s = Sentence::default();

assert_eq!(" ", s.as_raw_text());
assert_eq!(0, s.n_tags());
```
```

--------------------------------

### unwrap_or_default

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Returns the contained Ok value or a default value if the result is Err.

```APIDOC
## pub fn unwrap_or_default(self) -> T

### Description
Consumes the `self` argument then, if `Ok`, returns the contained value, otherwise if `Err`, returns the default value for that type.

### Signature
`pub fn unwrap_or_default(self) -> T` where `T: Default`
```

--------------------------------

### Trainer::train

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=

Executes the training process to generate a model.

```APIDOC
## Trainer::train

### Description
Trains word boundaries and tags based on the added examples.

### Parameters
- **epsilon** (f64) - The tolerance of the termination criterion.
- **cost** (f64) - The parameter C.
- **solver** (SolverType) - The solver type to use.

### Returns
- **Result<Model>** - Returns the trained Model or an error if the solver fails.
```

--------------------------------

### Cloning Result Contents

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Clones the contents of an Ok variant from a reference.

```rust
let val = 12;
let x: Result<&i32, i32> = Ok(&val);
assert_eq!(x, Ok(&12));
let cloned = x.cloned();
assert_eq!(cloned, Ok(12));
```

```rust
let mut val = 12;
let x: Result<&mut i32, i32> = Ok(&mut val);
assert_eq!(x, Ok(&mut 12));
let cloned = x.cloned();
assert_eq!(cloned, Ok(12));
```

--------------------------------

### Create Sentence from tokenized text

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=std%3A%3Avec

Creates a Sentence from a tokenized string where boundaries are marked by spaces and tags are prefixed with slashes.

```rust
use vaporetto::Sentence;

let s = Sentence::from_tokenized("まぁ 社長 は 火星 猫 だ");
assert_eq!("まぁ社長は火星猫だ", s.unwrap().as_raw_text());

let s = Sentence::from_tokenized("まぁ/名詞 社長/名詞 は/助詞 火星/名詞 猫/名詞 だ/助動詞");
assert_eq!("まぁ社長は火星猫だ", s.unwrap().as_raw_text());

let s = Sentence::from_tokenized("まぁ 社長  は 火星 猫 だ");
assert!(s.is_err());
```

--------------------------------

### pub fn boundaries(&self) -> &[CharacterBoundary]

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html

Returns a slice of boundary types for the sentence.

```APIDOC
## pub fn boundaries(&self) -> &[CharacterBoundary]

### Description
Returns a slice of boundary types for the sentence.
```

--------------------------------

### ResultExt::compat

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Wraps the error in Compat to make it compatible with older error handling APIs that expect std::error::Error.

```APIDOC
### fn compat(self) -> Result<T, Compat<E>>

Wraps the error in `Compat` to make it compatible with older error handling APIs that expect `std::error::Error`.
```

--------------------------------

### Trainer Definition

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

The basic structure definition for the Trainer.

```rust
pub struct Trainer<'a> { /* private fields */ }
```

--------------------------------

### Token::surface

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Token.html

Returns the surface text of the token.

```APIDOC
## pub fn surface(&self) -> &'a str

### Description
Returns the surface of this token.
```

--------------------------------

### Predictor::new

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Predictor.html

Creates a new predictor instance from a provided model.

```APIDOC
## pub fn new(model: Model, predict_tags: bool) -> Result<Self>

### Description
Creates a new predictor from the model. If you want to predict tags, set the predict_tags argument to true.

### Parameters
- **model** (Model) - Required - A model data.
- **predict_tags** (bool) - Required - If you want to predict tags, set to true.

### Errors
Returns an error variant when the model is invalid.
```

--------------------------------

### Write Partial Annotation Text to Buffer

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Writes the sentence as a string with partial annotations to a provided buffer.

```rust
use vaporetto::Sentence;

let mut buf = String::new();

let s = Sentence::from_tokenized("まぁ 良い だろう").unwrap();
s.write_partial_annotation_text(&mut buf);
assert_eq!("ま-ぁ|良-い|だ-ろ-う", buf);

let s = Sentence::from_tokenized(
    "まぁ/副詞/マー 良い/形容詞/ヨイ だろう/助動詞/ダロー"
).unwrap();
s.write_partial_annotation_text(&mut buf);
assert_eq!("ま-ぁ/副詞/マー|良-い/形容詞/ヨイ|だ-ろ-う/助動詞/ダロー", buf);
```

--------------------------------

### Into Ok Value

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=std%3A%3Avec

Extracts the Ok value from a Result that is guaranteed to be Ok.

```rust
fn only_good_news() -> Result<String, !> {
    Ok("this is fine".into())
}

let s: String = only_good_news().into_ok();
println!("{s}");
```

--------------------------------

### Expect Result Value

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html

Returns the Ok value or panics with a provided message if the Result is Err.

```rust
let x: Result<u32, &str> = Err("emergency failure");
x.expect("Testing expect"); // panics with `Testing expect: emergency failure`
```

```rust
let path = std::env::var("IMPORTANT_PATH")
    .expect("env variable `IMPORTANT_PATH` should be set by `wrapper_script.sh`");
```

--------------------------------

### Sentence::from_tokenized

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Creates a new Sentence from a tokenized text.

```APIDOC
## pub fn from_tokenized(tokenized_text: &str) -> Result<Self>

### Description
Creates a new `Sentence` from a tokenized text. A tokenized text must be annotated by the following rules: a whitespace is inserted to each token boundary, multiple tags following each slash can be added to each token, and each character following a back slash is escaped.

### Errors
This function will return an error variant when the given text is empty, starts/ends with a whitespace, or contains consecutive whitespaces.
```

--------------------------------

### Define Model struct

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Model.html

The primary structure for holding model data.

```rust
pub struct Model(/* private fields */);
```

--------------------------------

### pub fn fill_tags(&mut self)

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html

Updates the tag information using the predictor.

```APIDOC
## pub fn fill_tags(&mut self)

### Description
Update the tag information. This function is available only when the 'tag-prediction' crate feature is enabled. It should be called after Predictor::predict() and once word boundaries are fixed.
```

--------------------------------

### pub fn boundary_scores(&self) -> &[i32]

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html

Returns a slice of boundary scores for the sentence.

```APIDOC
## pub fn boundary_scores(&self) -> &[i32]

### Description
Returns a slice of boundary scores for the sentence.
```

--------------------------------

### CharacterType::equivalent

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/constant.VERSION.html?search=u32+-%3E+bool

Checks for equivalence between a character type and a generic key.

```APIDOC
## Method: vaporetto::CharacterType::equivalent

### Description
Determines if the character type is equivalent to the provided key.

### Signature
`&self, &K -> bool`

### Constraints
`u32` matches `K`
```

--------------------------------

### WordWeightRecord Getters

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.WordWeightRecord.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Methods to retrieve data from a WordWeightRecord instance.

```APIDOC
## pub fn get_word(&self) -> &str
Gets a reference to the word.

## pub fn get_weights(&self) -> &[i32]
Gets weights.

## pub fn get_comment(&self) -> &str
Gets a reference to the comment.
```

--------------------------------

### Token::tags

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Token.html

Returns the tags associated with the token.

```APIDOC
## pub fn tags(&self) -> &'a [Option<Cow<'b, str>>]

### Description
Returns tags of this token.
```

--------------------------------

### and_then

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Calls op if the result is Ok, otherwise returns the Err value of self.

```APIDOC
## pub fn and_then<U, F>(self, op: F) -> Result<U, E>

### Description
Calls `op` if the result is `Ok`, otherwise returns the `Err` value of `self`.

### Signature
`pub fn and_then<U, F>(self, op: F) -> Result<U, E>` where `F: FnOnce(T) -> Result<U, E>`
```

--------------------------------

### Update Sentence with Partial Annotations

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Updates a sentence using a string with partial annotations where '|' denotes token boundaries and '-' denotes non-token boundaries.

```rust
use vaporetto::Sentence;

let mut buf = String::new();
let mut s = Sentence::default();

s.update_partial_annotation(
    "ま-ぁ|良-い|だ-ろ-う"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ 良い だろう", buf);

s.update_partial_annotation(
    "ま-ぁ/名詞/マー|社-長/名詞/シャチョー|は/助詞/ワ|火-星 猫|だ/助動詞/ダ"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ だ/助動詞/ダ", buf);

s.update_partial_annotation(
    "ま-ぁ/名詞/マー|社-長/名詞/シャチョー|は/助詞/ワ|火/名詞/ヒ-星|猫|だ/助動詞/ダ"
).unwrap();
s.write_tokenized_text(&mut buf);
assert_eq!("まぁ/名詞/マー 社長/名詞/シャチョー は/助詞/ワ 火星 猫 だ/助動詞/ダ", buf);
```

--------------------------------

### CharacterBoundary::equivalent

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/constant.VERSION.html?search=u32+-%3E+bool

Checks for equivalence between a character boundary and a generic key.

```APIDOC
## Method: vaporetto::CharacterBoundary::equivalent

### Description
Determines if the character boundary is equivalent to the provided key.

### Signature
`&self, &K -> bool`

### Constraints
`u32` matches `K`
```

--------------------------------

### Trainer::n_features

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Trainer.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Returns the number of boundary features.

```APIDOC
## Trainer::n_features

### Description
Returns the number of boundary features currently configured in the trainer.
```

--------------------------------

### Reset Sentence Tags

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/struct.Sentence.html?search=

Removes existing tag information and updates the number of tags associated with the sentence.

```rust
use vaporetto::Sentence;

let mut s = Sentence::from_tokenized("火星/名詞/カセー に 行き/動詞 まし/助動詞/マシ た").unwrap();
let mut buf = String::new();
assert_eq!(2, s.n_tags());
assert_eq!(16, s.tags().len());
s.write_tokenized_text(&mut buf);
assert_eq!("火星/名詞/カセー に 行き まし た", buf);

s.reset_tags(1);
assert_eq!(1, s.n_tags());
assert_eq!(8, s.tags().len());
s.write_tokenized_text(&mut buf);
assert_eq!("火星 に 行き まし た", buf);
```

--------------------------------

### Compute default values with unwrap_or_else

Source: https://docs.rs/vaporetto/0.6.5/vaporetto/errors/type.Result.html?search=u32+-%3E+bool

Returns the contained Ok value or computes a default value using a closure.

```rust
fn count(x: &str) -> usize { x.len() }

assert_eq!(Ok(2).unwrap_or_else(count), 2);
assert_eq!(Err("foo").unwrap_or_else(count), 3);
```
Tokens:
Raw