Bayespam

https://docs.rs/bayespam
Admin
Bayespam is a simple Bayesian spam classifier library that uses Naive Bayes techniques to classify...
Tokens:5,769
Snippets:38
Trust Score:9.1
Update:6 months ago
Show doc for...
Tokens:
Raw
### Train a custom classifier model

Source: https://docs.rs/bayespam/latest/src/bayespam/lib.rs.html

Create a new classifier instance and train it with specific spam and ham examples.

```rust
use bayespam::classifier::Classifier;

fn main() -> Result<(), std::io::Error> {
    // Create a new classifier with an empty model
    let mut classifier = Classifier::new();

    // Train the classifier with a new spam example
    let spam = "Don't forget our special promotion: -30% on men shoes, only today!";
    classifier.train_spam(spam);

    // Train the classifier with a new ham example
    let ham = "Hi Bob, don't forget our meeting today at 4pm.";
    classifier.train_ham(ham);

    // Identify a typical spam message
    let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
    let is_spam = classifier.identify(spam);
    assert!(is_spam);

    // Identify a typical ham message
    let ham = "Hi Bob, can you send me your machine learning homework?";
    let is_spam = classifier.identify(ham);
    assert!(!is_spam);

    Ok(())
}
```

--------------------------------

### Train a custom spam classifier model

Source: https://docs.rs/bayespam/latest/bayespam

Train a new classifier model from scratch by providing spam and ham examples. This allows for custom classification based on your data.

```rust
use bayespam::classifier::Classifier;

fn main() -> Result<(), std::io::Error> {
    // Create a new classifier with an empty model
    let mut classifier = Classifier::new();

    // Train the classifier with a new spam example
    let spam = "Don't forget our special promotion: -30% on men shoes, only today!";
    classifier.train_spam(spam);

    // Train the classifier with a new ham example
    let ham = "Hi Bob, don't forget our meeting today at 4pm.";
    classifier.train_ham(ham);

    // Identify a typical spam message
    let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
    let is_spam = classifier.identify(spam);
    assert!(is_spam);

    // Identify a typical ham message
    let ham = "Hi Bob, can you send me your machine learning homework?";
    let is_spam = classifier.identify(ham);
    assert!(!is_spam);

    Ok(())
}

```

--------------------------------

### Classifier Initialization and Loading

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Provides methods for creating a new classifier or loading a pre-trained model from a file.

```APIDOC
## Classifier Initialization and Loading

### Description
Methods for initializing a new Bayesian spam classifier or loading a pre-trained model from a file.

### Methods

#### `new()`
- **Description**: Builds a new classifier with an empty model.
- **Returns**: A new `Classifier` instance.

#### `new_from_pre_trained(file: &mut File)`
- **Description**: Builds a new classifier with a pre-trained model loaded from the specified file.
- **Parameters**:
  - **file** (*File*) - Required - A mutable reference to a `File` object containing the pre-trained model.
- **Returns**: `Result<Classifier, io::Error>` - Ok containing the loaded `Classifier` or an `io::Error` if loading fails.
```

--------------------------------

### Add dependency to Cargo.toml

Source: https://docs.rs/bayespam/latest/src/bayespam/lib.rs.html

Include the bayespam crate in your project manifest.

```ini
[dependencies]
bayespam = "1.2.0"
```

--------------------------------

### Add bayespam to Cargo.toml

Source: https://docs.rs/bayespam/latest/bayespam

Add this to your Cargo.toml manifest to include the bayespam crate in your project.

```toml
[dependencies]
bayespam = "1.2.0"

```

--------------------------------

### Classifier Unit Tests

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Test suite covering new classifier training, Unicode support, and pre-trained model loading.

```rust
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_new() {
        // Create a new classifier with an empty model
        let mut classifier = Classifier::new();

        // Train the classifier with a new spam example
        let spam = "Don't forget our special promotion: -30% on men shoes, only today!";
        classifier.train_spam(spam);

        // Train the classifier with a new ham example
        let ham = "Hi Bob, don't forget our meeting today at 4pm.";
        classifier.train_ham(ham);

        // Identify a typical spam message
        let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
        let is_spam = classifier.identify(spam);
        assert!(is_spam);

        // Identify a typical ham message
        let ham = "Hi Bob, can you send me your machine learning homework?";
        let is_spam = classifier.identify(ham);
        assert!(!is_spam);
    }

    #[test]
    fn test_new_unicode() {
        // Create a new classifier with an empty model
        let mut classifier = Classifier::new();

        // Train the classifier with a new spam example
        let spam = "Bon plan pour Nöel: profitez de -50% sur le 2ème article.";
        classifier.train_spam(spam);

        // Train the classifier with a new ham example
        let ham = "Vous êtes tous cordialement invités à notre repas de Noël.";
        classifier.train_ham(ham);

        // Identify a typical spam message
        let spam = "Préparez les fêtes de Nöel: 1 article offert!";
        let is_spam = classifier.identify(spam);
        assert!(is_spam);

        // Identify a typical ham message
        let ham = "Pourras-tu être des nôtres pour le repas de Noël?";
        let is_spam = classifier.identify(ham);
        assert!(!is_spam);
    }

    #[test]
    fn test_new_from_pre_trained() -> Result<(), io::Error> {
        // Identify a typical spam message
        let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
        let is_spam = identify(spam)?;
        assert!(is_spam);

        // Identify a typical ham message
        let ham = "Hi Bob, can you send me your machine learning homework?";
        let is_spam = identify(ham)?;
        assert!(!is_spam);

        Ok(())
    }
}
```

--------------------------------

### Classifier Methods

Source: https://docs.rs/bayespam/latest/bayespam/classifier/struct.Classifier.html

Methods for initializing, training, and utilizing the Bayesian spam classifier.

```APIDOC
## Classifier Methods

### Description
Methods to manage the lifecycle and classification logic of the Classifier.

### Methods
- **new()**: Build a new classifier with an empty model.
- **new_from_pre_trained(file: &mut File)**: Build a new classifier with a pre-trained model loaded from a file.
- **save(&self, file: &mut File, pretty: bool)**: Save the classifier to a file as JSON.
- **train_spam(&mut self, msg: &str)**: Train the classifier with a spam message.
- **train_ham(&mut self, msg: &str)**: Train the classifier with a ham message.
- **score(&self, msg: &str) -> f32**: Compute the spam score of a message.
- **identify(&self, msg: &str) -> bool**: Identify whether a message is spam or not.
```

--------------------------------

### Identify messages using a pre-trained model

Source: https://docs.rs/bayespam/latest/src/bayespam/lib.rs.html

Use a model.json file in the package root to classify messages as spam or ham.

```rust
use bayespam::classifier;

fn main() -> Result<(), std::io::Error> {
    // Identify a typical spam message
    let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
    let is_spam = classifier::identify(spam)?;
    assert!(is_spam);

    // Identify a typical ham message
    let ham = "Hi Bob, can you send me your machine learning homework?";
    let is_spam = classifier::identify(ham)?;
    assert!(!is_spam);

    Ok(())
}
```

--------------------------------

### Identify messages using a pre-trained model

Source: https://docs.rs/bayespam/latest/bayespam

Use a pre-trained model (model.json) to score and identify messages. Ensure the model.json file is in your package root.

```rust
use bayespam::classifier;

fn main() -> Result<(), std::io::Error> {
    // Identify a typical spam message
    let spam = "Lose up to 19% weight. Special promotion on our new weightloss.";
    let is_spam = classifier::identify(spam)?;
    assert!(is_spam);

    // Identify a typical ham message
    let ham = "Hi Bob, can you send me your machine learning homework?";
    let is_spam = classifier::identify(ham)?;
    assert!(!is_spam);

    Ok(())
}

```

--------------------------------

### Pre-trained Model Scoring and Identification

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Utility functions for scoring and identifying spam using a pre-trained model loaded from a file.

```rust
/// Compute the spam score of `msg`, based on a pre-trained model.
/// The higher the score, the stronger the liklihood that `msg` is a spam is.
pub fn score(msg: &str) -> Result<f32, io::Error> {
    let mut file = File::open(DEFAULT_FILE_PATH)?;
    Classifier::new_from_pre_trained(&mut file).map(|classifier| classifier.score(msg))
}

/// Identify whether `msg` is a spam or not, based on a pre-trained model.
pub fn identify(msg: &str) -> Result<bool, io::Error> {
    let score = score(msg)?;
    let is_spam = score > SPAM_PROB_THRESHOLD;
    Ok(is_spam)
}
```

--------------------------------

### Bayesian Spam Classifier Implementation

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

The core implementation of the Classifier struct, including serialization logic, training methods, and probability calculation.

```rust
1use std::collections::HashMap;
2use std::fs::File;
3use std::io;
4
5use serde::{Deserialize, Serialize};
6use serde_json::{from_reader, to_writer, to_writer_pretty};
7use unicode_segmentation::UnicodeSegmentation;
8
9const DEFAULT_FILE_PATH: &str = "model.json";
10const INITIAL_RATING: f32 = 0.5;
11const SPAM_PROB_THRESHOLD: f32 = 0.8;
12
13#[derive(Debug, Default, Serialize, Deserialize)]
14struct Counter {
15    ham: u32,
16    spam: u32,
17}
18
19/// A bayesian spam classifier.
20#[derive(Default, Debug, Deserialize, Serialize)]
21#[serde(from = "ClassifierSerialized")]
22pub struct Classifier {
23    token_table: HashMap<String, Counter>,
24    #[serde(skip)]
25    spam_total_count: u32,
26    #[serde(skip)]
27    ham_total_count: u32,
28}
29
30/// The classifier model as it is serialized to disk.
31///
32/// Does not include the `spam_total_count` and `ham_total_count` fields which
33/// can be recomputed from `token_table`.
34#[derive(Deserialize, Serialize)]
35struct ClassifierSerialized {
36    token_table: HashMap<String, Counter>,
37}
38
39impl std::convert::From<ClassifierSerialized> for Classifier {
40    fn from(c: ClassifierSerialized) -> Self {
41        let spam_total_count = c.token_table.values().map(|x| x.spam).sum();
42        let ham_total_count = c.token_table.values().map(|x| x.ham).sum();
43        Self {
44            token_table: c.token_table,
45            spam_total_count,
46            ham_total_count,
47        }
48    }
49}
50
51impl Classifier {
52    /// Build a new classifier with an empty model.
53    pub fn new() -> Self {
54        Default::default()
55    }
56
57    /// Build a new classifier with a pre-trained model loaded from `file`.
58    pub fn new_from_pre_trained(file: &mut File) -> Result<Self, io::Error> {
59        let pre_trained_model = from_reader(file)?;
60        Ok(pre_trained_model)
61    }
62
63    /// Save the classifier to `file` as JSON.
64    /// The JSON will be pretty printed if `pretty` is `true`.
65    pub fn save(&self, file: &mut File, pretty: bool) -> Result<(), io::Error> {
66        if pretty {
67            to_writer_pretty(file, &self)?;
68        } else {
69            to_writer(file, &self)?;
70        }
71        Ok(())
72    }
73
74    /// Split `msg` into a list of words.
75    fn load_word_list(msg: &str) -> Vec<String> {
76        let word_list = msg.unicode_words().collect::<Vec<&str>>();
77        word_list.iter().map(|word| word.to_string()).collect()
78    }
79
80    /// Train the classifier with a spam `msg`.
81    pub fn train_spam(&mut self, msg: &str) {
82        for word in Self::load_word_list(msg) {
83            let counter = self.token_table.entry(word).or_default();
84            counter.spam += 1;
85            self.spam_total_count += 1;
86        }
87    }
88
89    /// Train the classifier with a ham `msg`.
90    pub fn train_ham(&mut self, msg: &str) {
91        for word in Self::load_word_list(msg) {
92            let counter = self.token_table.entry(word).or_default();
93            counter.ham += 1;
94            self.ham_total_count += 1;
95        }
96    }
97
98    /// Return the total number of spam in token table.
99    fn spam_total_count(&self) -> u32 {
100        self.spam_total_count
101    }
102
103    /// Return the total number of ham in token table.
104    fn ham_total_count(&self) -> u32 {
105        self.ham_total_count
106    }
107
108    /// Compute the probability of each word of `msg` to be part of a spam.
109    fn rate_words(&self, msg: &str) -> Vec<f32> {
110        Self::load_word_list(msg)
111            .into_iter()
112            .map(|word| {
113                // If word was previously added in the model
114                if let Some(counter) = self.token_table.get(&word) {
115                    // If the word has only been part of spam messages,
116                    // assign it a probability of 0.99 to be part of a spam
117                    if counter.spam > 0 && counter.ham == 0 {
118                        return 0.99;
119                    // If the word has only been part of ham messages,
120                    // assign it a probability of 0.01 to be part of a spam
121                    } else if counter.spam == 0 && counter.ham > 0 {
122                        return 0.01;
123                    // If the word has been part of both spam and ham messages,
124                    // calculate the probability to be part of a spam
125                    } else if self.spam_total_count() > 0 && self.ham_total_count() > 0 {
126                        let ham_prob = (counter.ham as f32) / (self.ham_total_count() as f32);
127                        let spam_prob = (counter.spam as f32) / (self.spam_total_count() as f32);
128                        return (spam_prob / (ham_prob + spam_prob)).max(0.01);
129                    }
130                }
131                // If word was never added to the model,
```

--------------------------------

### Classifier Training and Identification

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

This section covers the core functionalities of the Classifier struct, including training with spam and ham messages, and identifying messages based on the trained model.

```APIDOC
## Classifier Methods

### `new()`

Creates a new classifier with an empty model.

### `train_spam(msg: &str)`

Trains the classifier with a given spam message.

### `train_ham(msg: &str)`

Trains the classifier with a given ham (non-spam) message.

### `score(msg: &str) -> f32`

Computes the spam score of a message. A higher score indicates a stronger likelihood of the message being spam.

### `identify(msg: &str) -> bool`

Identifies whether a message is spam or not based on its score. Returns `true` if the message is identified as spam, `false` otherwise.

### Example Usage:

```rust
let mut classifier = Classifier::new();
classifier.train_spam("Buy now and get 50% off!");
classifier.train_ham("Meeting scheduled for tomorrow.");
let is_spam = classifier.identify("Special offer: limited time discount!");
assert!(is_spam);
```
```

--------------------------------

### Pre-trained Model Functions

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

These functions allow you to use a pre-trained spam classification model without needing to train it yourself. They load the model from a default file path.

```APIDOC
## Pre-trained Model Functions

### `score(msg: &str) -> Result<f32, io::Error>`

Computes the spam score of a message using a pre-trained model. The model is loaded from a default file path.

- **Returns**: A `Result` containing the spam score (`f32`) on success, or an `io::Error` if the model file cannot be opened or processed.

### `identify(msg: &str) -> Result<bool, io::Error>`

Identifies whether a message is spam or not using a pre-trained model. This function first calculates the spam score and then compares it against a predefined spam probability threshold.

- **Returns**: A `Result` containing a boolean (`true` if spam, `false` if ham) on success, or an `io::Error` if the score calculation fails.

### Example Usage:

```rust
match identify("Claim your prize now!") {
    Ok(is_spam) => {
        if is_spam {
            println!("Message is likely spam.");
        } else {
            println!("Message is likely ham.");
        }
    }
    Err(e) => {
        eprintln!("Error identifying message: {}", e);
    }
}
```
```

--------------------------------

### Classifier Training

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Methods for training the classifier with spam and ham messages.

```APIDOC
## Classifier Training

### Description
Methods for training the Bayesian spam classifier with examples of spam and ham messages.

### Methods

#### `train_spam(msg: &str)`
- **Description**: Trains the classifier with a given message, marking it as spam.
- **Parameters**:
  - **msg** (*&str*) - Required - The message content to train as spam.

#### `train_ham(msg: &str)`
- **Description**: Trains the classifier with a given message, marking it as ham.
- **Parameters**:
  - **msg** (*&str*) - Required - The message content to train as ham.
```

--------------------------------

### Classifier Saving

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Method for saving the current state of the classifier to a file.

```APIDOC
## Classifier Saving

### Description
Saves the current state of the Bayesian spam classifier to a specified file.

### Method

#### `save(file: &mut File, pretty: bool)`
- **Description**: Saves the classifier model to the given file.
- **Parameters**:
  - **file** (*File*) - Required - A mutable reference to a `File` object where the model will be saved.
  - **pretty** (*bool*) - Required - If `true`, the JSON output will be pretty-printed; otherwise, it will be compact.
- **Returns**: `Result<(), io::Error>` - Ok if saving is successful, or an `io::Error` if an error occurs during saving.
```

--------------------------------

### identify

Source: https://docs.rs/bayespam/latest/bayespam/classifier/fn.identify.html

Classifies a given message string as spam or not spam.

```APIDOC
## identify

### Description
Identify whether a message is spam or not, based on a pre-trained model.

### Method
Function Call

### Parameters
#### Arguments
- **msg** (&str) - Required - The message content to be classified.

### Response
- **Result<bool, Error>** - Returns Ok(true) if the message is identified as spam, Ok(false) if not, or an Error if classification fails.
```

--------------------------------

### Classifier Struct

Source: https://docs.rs/bayespam/latest/bayespam/classifier/index.html

The core structure used for Bayesian spam classification.

```APIDOC
## Struct: Classifier

### Description
A bayesian spam classifier used to process messages against a pre-trained model.
```

--------------------------------

### Compute spam score with bayespam

Source: https://docs.rs/bayespam/latest/bayespam/classifier/fn.score.html

Calculates the spam likelihood of a message string. Returns a Result containing an f32 score or an Error.

```rust
pub fn score(msg: &str) -> Result<f32, Error>
```

--------------------------------

### Compute Spam Score and Identify Spam

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Methods for calculating the probability of a message being spam and determining its classification status.

```rust
    /// Compute the spam score of `msg`.
    /// The higher the score, the stronger the liklihood that `msg` is a spam is.
    pub fn score(&self, msg: &str) -> f32 {
        // Compute the probability of each word to be part of a spam
        let ratings = self.rate_words(msg);

        let ratings = match ratings.len() {
            // If there are no ratings, return a score of 0
            0 => return 0.0,
            // If there are more than 20 ratings, keep only the 10 first
            // and 10 last ratings to calculate a score
            x if x > 20 => {
                let length = ratings.len();
                let mut ratings = ratings;
                ratings.sort_unstable_by(|a, b| a.partial_cmp(b).unwrap());
                [&ratings[..10], &ratings[length - 10..]].concat()
            }
            // In all other cases, keep ratings to calculate a score
            _ => ratings,
        };

        // Combine individual ratings
        let product: f32 = ratings.iter().product();
        let alt_product: f32 = ratings.iter().map(|x| 1.0 - x).product();
        product / (product + alt_product)
    }

    /// Identify whether `msg` is a spam or not.
    pub fn identify(&self, msg: &str) -> bool {
        self.score(msg) > SPAM_PROB_THRESHOLD
    }
}
```

--------------------------------

### Classifier struct definition

Source: https://docs.rs/bayespam/latest/bayespam/classifier/struct.Classifier.html

The primary structure used for Bayesian spam classification.

```rust
pub struct Classifier { /* private fields */ }
```

--------------------------------

### Message Classification

Source: https://docs.rs/bayespam/latest/src/bayespam/classifier.rs.html

Methods for classifying messages and calculating spam probabilities.

```APIDOC
## Message Classification

### Description
Provides functionality to classify messages and determine the probability of them being spam.

### Methods

#### `rate_message(msg: &str)`
- **Description**: Calculates the probability of a given message being spam.
- **Parameters**:
  - **msg** (*&str*) - Required - The message content to classify.
- **Returns**: `f32` - The probability (between 0.0 and 1.0) that the message is spam.
```

--------------------------------

### identify Function

Source: https://docs.rs/bayespam/latest/bayespam/classifier/index.html

Function to determine if a message is spam.

```APIDOC
## Function: identify

### Description
Identify whether `msg` is a spam or not, based on a pre-trained model.
```

--------------------------------

### score Function

Source: https://docs.rs/bayespam/latest/bayespam/classifier/index.html

Function to calculate the spam probability score.

```APIDOC
## Function: score

### Description
Compute the spam score of `msg`, based on a pre-trained model. The higher the score, the stronger the likelihood that `msg` is a spam.
```

--------------------------------

### score function

Source: https://docs.rs/bayespam/latest/bayespam/classifier/fn.score.html

Computes the spam score of a message based on a pre-trained model.

```APIDOC
## Function: score

### Description
Compute the spam score of a message, based on a pre-trained model. The higher the score, the stronger the likelihood that the message is spam.

### Signature
`pub fn score(msg: &str) -> Result<f32, Error>`

### Parameters
- **msg** (str) - Required - The message content to be evaluated.

### Response
- **Result<f32, Error>** - Returns a floating-point score representing the spam probability or an Error if the evaluation fails.
```
Tokens:
Raw