Happy Transformer

https://happytransformer.com
Admin
Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.
Tokens:13,089
Snippets:128
Trust Score:6.3
Update:4 months ago
Show doc for...
Tokens:
Raw
### Install Deepspeed

Source: https://happytransformer.com/deepspeed

Clone the DeepSpeed repository, navigate to the directory, and install it with build utilities enabled.

```bash
git clone https://github.com/microsoft/DeepSpeed.git

cd DeepSpeed

DS_BUILD_UTILS=1 pip install .
```

--------------------------------

### Log in to WandB

Source: https://happytransformer.com/wandb

Use this command to log in to your WandB account before starting a training run.

```bash
wandb login

```

--------------------------------

### Initialize HappyGeneration

Source: https://happytransformer.com/text-generation

Instantiate HappyGeneration with a model type and name. Examples show default, larger, and private model configurations.

```python
from happytransformer import HappyGeneration
# --------------------------------------#
happy_gen = HappyGeneration("GPT2", "gpt2")  # default
# happy_gen = HappyGeneration("GPT2", "gpt2-large")  # Good for Google Colab
# happy_gen = HappyGeneration("GPT2", "gpt2-xl")  # Best performance 
# happy_gen_private = HappyGeneration("GPT2", "ericfillion/gpt2", use_auth_token="123abc")
```

--------------------------------

### Initialize HappyQuestionAnswering with Different Models

Source: https://happytransformer.com/question-answering

Demonstrates how to initialize HappyQuestionAnswering with various model types like DISTILBERT, ALBERT, BERT, and ROBERTA. Includes an example of loading a private model using an authentication token.

```Python
from happytransformer import HappyQuestionAnswering
# --------------------------------------#
happy_qa_distilbert = HappyQuestionAnswering("DISTILBERT", "distilbert-base-cased-distilled-squad")  # default
happy_qa_albert = HappyQuestionAnswering("ALBERT", "mfeb/albert-xxlarge-v2-squad2")
# good model when using with limited hardware 
happy_qa_bert = HappyQuestionAnswering("BERT", "mrm8488/bert-tiny-5-finetuned-squadv2")
happy_qa_roberta = HappyQuestionAnswering("ROBERTA", "deepset/roberta-base-squad2")
happy_qa__private_roberta = HappyQuestionAnswering("ROBERTA", "user-repo/roberta-base-squad2", use_auth_token="123abc")
```

--------------------------------

### Initialize and Classify Tokens

Source: https://happytransformer.com/token-classification/usage

Initializes a HappyTokenClassification object with a specified model and then classifies tokens in a given sentence. This example demonstrates how to instantiate the class and interpret the results, showing the type of the result and accessing individual classification details like word, entity, score, and indices.

```python
from happytransformer import HappyTokenClassification
# --------------------------------------#
happy_toc = HappyTokenClassification(model_type="BERT", model_name="dslim/bert-base-NER")
result = happy_toc.classify_token("My name is Geoffrey and I live in Toronto")
print(type(result))  # <class 'list'>
print(result[0].word)  # Geoffrey
print(result[0].entity)  # B-PER
print(result[0].score)  # 0.9988969564437866
print(result[0].index)  # 4
print(result[0].start) # 11
print(result[0].end)  # 19
print(result[1].word)  # Toronto
print(result[1].entity)  # B-LOC
```

--------------------------------

### Initialize HappyWordPrediction Models

Source: https://happytransformer.com/word-prediction

Instantiate HappyWordPrediction with different model types and names. Includes an example for loading a private model using an authentication token.

```python
from happytransformer import HappyWordPrediction
# --------------------------------------#
happy_wp_distilbert = HappyWordPrediction("DISTILBERT", "distilbert-base-uncased")  # default
happy_wp_albert = HappyWordPrediction("ALBERT", "albert-base-v2")
happy_wp_bert = HappyWordPrediction("BERT", "bert-base-uncased")
happy_wp_roberta = HappyWordPrediction("ROBERTA", "roberta-base")
happy_wp_private_roberta = HappyWordPrediction("ROBERTA", "user-repo/roberta-base", use_auth_token="123abc")
```

--------------------------------

### Initialize and Answer Question

Source: https://happytransformer.com/question-answering/usage

Demonstrates how to initialize `HappyQuestionAnswering` and use the `answer_question` method with default `top_k` value. Shows how to inspect the returned list and its elements.

```python
from happytransformer import HappyQuestionAnswering
# --------------------------------------#
happy_qa = HappyQuestionAnswering()
result = happy_qa.answer_question("Today's date is January 10th, 2021", "What is the date?")
print(type(result))  # <class 'list'>
print(result)  # [QuestionAnsweringResult(answer='January 10th, 2021', score=0.9711642265319824, start=16, end=34)]
print(type(result[0]))  # <class 'happytransformer.happy_question_answering.QuestionAnsweringResult'>
print(result[0])  # QuestionAnsweringResult(answer='January 10th, 2021', score=0.9711642265319824, start=16, end=34)
print(result[0].answer)  # January 10th, 2021

```

--------------------------------

### Generate Text with HappyGeneration

Source: https://happytransformer.com/text-generation/usage

Demonstrates how to initialize HappyGeneration, set generation parameters using GENSettings, and generate text based on a prompt. Shows how to access the generated text from the result object.

```python
from happytransformer import HappyGeneration, GENSettings
#--------------------------------------#
happy_gen = HappyGeneration()  # default uses gpt2
args = GENSettings(max_length=15)
result = happy_gen.generate_text("artificial intelligence is ", args=args)    
print(result)  # GenerationResult(text='\xa0a new field of research that has been gaining momentum in recent years.')
print(result.text)  # a new field of research that has been gaining momentum in recent years.
```

--------------------------------

### Configure Text Generation Settings

Source: https://happytransformer.com/text-to-text/settings

Demonstrates various text generation strategies including greedy, beam search, and different sampling methods (generic, top-k, top-p). Each snippet shows how to instantiate TTSettings with specific parameters and use them with HappyTextToText.generate_text().

```python
from happytransformer import HappyTextToText, TTSettings

#---------------------------------------------------
happy_tt = HappyTextToText("T5", "t5-small")

greedy_settings = TTSettings(no_repeat_ngram_size=2, max_length=20)
output_greedy = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=greedy_settings)

beam_settings = TTSettings(num_beams=5, max_length=20)
output_beam_search = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=beam_settings)

generic_sampling_settings = TTSettings(do_sample=True, top_k=0, temperature=0.7, max_length=20)
output_generic_sampling = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=generic_sampling_settings)

top_k_sampling_settings = TTSettings(do_sample=True, top_k=50, temperature=0.7, max_length=20)
output_top_k_sampling = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=top_k_sampling_settings)

top_p_sampling_settings = TTSettings(do_sample=True, top_k=0, top_p=0.8, temperature=0.7, max_length=20)
output_top_p_sampling = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=top_p_sampling_settings)

print("Greedy:", output_greedy.text)  # Greedy: nlp est un domaine de l'intelligence artificielle
print("Beam:", output_beam_search.text)  # Beam: nlp est un domaine de l'intelligence artificielle
print("Generic Sampling:", output_generic_sampling.text)  # Generic Sampling: nlp est un champ d'intelligence artificielle
print("Top-k Sampling:", output_top_k_sampling.text)  # Top-k Sampling: nlp est un domaine de l’intelligence artificielle
print("Top-p Sampling:", output_top_p_sampling.text)  # Top-p Sampling: nlp est un domaine de l'intelligence artificielle
```

--------------------------------

### Configure Text Generation Settings

Source: https://happytransformer.com/text-generation/settings

Demonstrates various settings for text generation including greedy decoding, beam search, generic sampling, top-k sampling, top-p sampling, and using bad words.

```python
from happytransformer import HappyGeneration, GENSettings

#---------------------------------------------------
happy_gen = HappyGeneration()

greedy_settings = GENSettings(no_repeat_ngram_size=2,  max_length=10)
output_greedy = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=greedy_settings)

beam_settings = GENSettings(num_beams=5,  max_length=10)
output_beam_search = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=beam_settings)

generic_sampling_settings = GENSettings(do_sample=True, top_k=0, temperature=0.7,  max_length=10)
output_generic_sampling = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=generic_sampling_settings)

top_k_sampling_settings = GENSettings(do_sample=True, top_k=50, temperature=0.7,  max_length=10)
output_top_k_sampling = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=top_k_sampling_settings)

top_p_sampling_settings = GENSettings(do_sample=True, top_k=0, top_p=0.8, temperature=0.7,  max_length=10)
output_top_p_sampling = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=top_p_sampling_settings)

bad_words_settings = GENSettings(bad_words = ["new form", "social"])
output_bad_words = happy_gen.generate_text(
    "Artificial intelligence is ",
    args=bad_words_settings)

print("Greedy:", output_greedy.text)  # a new field of research that has been gaining
print("Beam:", output_beam_search.text) # one of the most promising areas of research in
print("Generic Sampling:", output_generic_sampling.text)  # 
an area of highly promising research, and a
print("Top-k Sampling:", output_top_k_sampling.text)  # a new form of social engineering. In this
print("Top-p Sampling:", output_top_p_sampling.text)  # a new form of social engineering. In this
print("Bad Words:", output_bad_words.text) # a technology that enables us to help people deal

```

--------------------------------

### Login to Hugging Face CLI

Source: https://happytransformer.com/push

Authenticate your local environment with the Hugging Face CLI to enable pushing models. This command initiates the login process.

```bash
huggingface-cli login

```

--------------------------------

### Import GENSettings

Source: https://happytransformer.com/text-generation/settings

Import the GENSettings class for controlling text generation parameters.

```python
from happytransformer import GENSettings
```

--------------------------------

### Save and Load Data for Word Prediction Training

Source: https://happytransformer.com/save-load-data

Demonstrates saving preprocessed data using `WPTrainArgs(save_path='data/')` and then loading it for subsequent training by setting `WPTrainArgs(load_path='data/')`. Ensure the `save_path` directory exists.

```python
from happytransformer import HappyWordPrediction, WPTrainArgs
# ---------------------------------------------------------
happy_wp = HappyWordPrediction()
train_args_1 = WPTrainArgs(save_path="data/")
happy_wp.train("data/wp/train-eval.txt", args=train_args_1)


train_args_2 = WPTrainArgs( load_path="data/")
# if you're loading data, then you can set input_filepath to anything 
happy_wp.train(input_filepath="", args=train_args_2)
```

--------------------------------

### Run Training Script with Deepspeed

Source: https://happytransformer.com/deepspeed

Execute your training script using the 'deepspeed' command instead of 'python3' to leverage distributed training capabilities.

```bash
deepspeed train.py
```

--------------------------------

### Initialize HappyTextToText with Default and Private Models

Source: https://happytransformer.com/text-to-text

Demonstrates initializing HappyTextToText with a default T5 model and a private T5 model requiring an authentication token.

```python
from happytransformer import HappyTextToText
# --------------------------------------#
happy_tt = HappyTextToText("T5", "t5-small")  # default
happy_tt_private = HappyTextToText("T5", "ericfillion/t5-small", use_auth_token="123abc")  # default
```

--------------------------------

### Generate Text with Top-P Sampling

Source: https://happytransformer.com/text-to-text/usage

Demonstrates how to use the generate_text() method with custom TTSettings for top-p sampling. This is useful for controlling the creativity and coherence of generated text.

```Python
from happytransformer import HappyTextToText, TTSettings
#--------------------------------------#
happy_tt = HappyTextToText()  # default uses t5-small
top_p_sampling_settings = TTSettings(do_sample=True, top_k=0, top_p=0.8, temperature=0.7,  min_length=20, max_length=20, early_stopping=True)
result = happy_tt.generate_text("translate English to French: nlp is a field of artificial intelligence", args=top_p_sampling_settings)
print(result)  # TextToTextResult(text="nlp est un domaine de l'intelligence artificielle...")
print(result.text)  # nlp est un domaine de l’intelligence artificielle. n

```

--------------------------------

### Import TTSettings

Source: https://happytransformer.com/text-to-text/settings

Import the TTSettings class for configuring text generation parameters.

```python
from happytransformer import TTSettings
```

--------------------------------

### Compare Model Performance Before and After Finetuning

Source: https://happytransformer.com/text-classification/finetuning

Compares the evaluation loss of a text classification model before and after finetuning on the same dataset. Demonstrates the improvement in model performance after training. Note: It's generally recommended to use a separate dataset for evaluation.

```python
from happytransformer import HappyTextClassification
# --------------------------------------#
happy_tc = HappyTextClassification(model_type="DISTILBERT",
                                   model_name="distilbert-base-uncased-finetuned-sst-2-english",
                                   num_labels=2)  # Don't forget to set num_labels!
before_loss = happy_tc.eval("../../data/tc/train-eval.csv").loss
happy_tc.train("../../data/tc/train-eval.csv")
after_loss = happy_tc.eval("../../data/tc/train-eval.csv").loss
print("Before loss: ", before_loss)  # 0.007262040860950947
print("After loss: ", after_loss)  # 0.000162081079906784
# Since after_loss < before_loss, the model learned!
# Note: typically you evaluate with a separate dataset
# but for simplicity we used the same one

```

--------------------------------

### Initialize HappyTextClassification Models

Source: https://happytransformer.com/text-classification

Demonstrates initializing HappyTextClassification with various model types and configurations, including default settings, custom model names, and private models.

```python
from happytransformer import HappyTextClassification
# --------------------------------------#
happy_tc_distilbert = HappyTextClassification("DISTILBERT", "distilbert-base-uncased", num_labels=2)  # default 
happy_tc_albert = HappyTextClassification(model_type="ALBERT", model_name="albert-base-v2")
happy_tc_bert = HappyTextClassification("BERT", "bert-base-uncased")
happy_tc_roberta = HappyTextClassification("ROBERTA", "roberta-base")
happy_tc_private_roberta = HappyTextClassification("ROBERTA", "user-repo/roberta-base", use_auth_token="123abc")
```

--------------------------------

### Load a HappyGeneration Model

Source: https://happytransformer.com/save-load-model

Initialize a HappyGeneration object by providing the path to the saved model directory in the model_name parameter. This allows you to resume training or use the pre-trained model.

```python
from happytransformer import HappyGeneration
# ---------------------------------------------------------
happy_gen = HappyGeneration(model_type="GPT-NEO", model_name="model/")
```

--------------------------------

### Answer Question with Top K Results

Source: https://happytransformer.com/question-answering/usage

Shows how to use the `answer_question` method with a specified `top_k` value to retrieve multiple potential answers. Demonstrates accessing the answer from the second result.

```python
from happytransformer import HappyQuestionAnswering
# --------------------------------------#
happy_qa = HappyQuestionAnswering()
result = happy_qa.answer_question("Today's date is January 10th, 2021", "What is the date?", top_k=2)
print(type(result))  # <class 'list'>
print(result)  # [QuestionAnsweringResult(answer='January 10th, 2021', score=0.9711642265319824, start=16, end=34), QuestionAnsweringResult(answer='January 10th', score=0.017306014895439148, start=16, end=28)]
print(result[1].answer)  # January 10th

```

--------------------------------

### Push a HappyGeneration Model

Source: https://happytransformer.com/push

Upload a trained HappyGeneration model to the Hugging Face Model Hub. Ensure you have logged in via the CLI first. The `repo_name` should follow the format 'username/repository_name'.

```python
from happytransformer import HappyGeneration
# ---------------------------------------------------------
happy_gen_1 = HappyGeneration(model_type="GPT-NEO", model_name="EleutherAI/gpt-neo-125M")
repo_name = "EricFillion/example"
happy_gen_1.push(repo_name, private=True)

# Be sure to set use_auth_token to True
happy_gen_2 = HappyGeneration("GPT-2", "EricFillion/example", use_auth_token=True)

```

--------------------------------

### train()

Source: https://happytransformer.com/text-generation/finetuning

Fine-tunes the model to better understand a body of text. It accepts a file path for training data and optional arguments for training configuration.

```APIDOC
## train()

### Description
Fine-tunes the model to better understand a body of text. It accepts a file path for training data and optional arguments for training configuration.

### Method
`train()`

### Parameters
#### Path Parameters
* **input_filepath** (string) - Required - A path to a text file containing text to train the model, or a CSV file with a single column named 'text'.
* **eval_filepath** (string) - Optional - A filepath to a text or CSV file for standalone evaluating data. If not provided, an evaluating dataset will be generated from the training data.

#### Request Body
* **args** (GENTrainArgs) - Required - A dataclass containing training arguments such as learning rate, batch size, and output directory.

**GENTrainArgs Fields:**
* **learning_rate** (float) - Optional - Default: 5e-5
* **num_train_epochs** (int) - Optional - Default: 1
* **batch_size** (int) - Optional - Default: 1
* **weight_decay** (float) - Optional - Default: 0
* **save_path** (string) - Optional - Default: ""
* **load_path** (string) - Optional - Default: ""
* **fp16** (bool) - Optional - Default: False
* **eval_ratio** (float) - Optional - Default: 0.1
* **save_steps** (float) - Optional - Default: 0.0
* **eval_steps** (float) - Optional - Default: 0.1
* **logging_steps** (float) - Optional - Default: 0.1
* **output_dir** (string) - Optional - Default: "happy_transformer"
* **max_length** (None) - Optional - Default: None

### Request Example
```python
from happytransformer import HappyGeneration, GENTrainArgs

happy_gen = HappyGeneration()
args = GENTrainArgs(num_train_epochs=1)
happy_gen.train("../../data/gen/train-eval.txt", args=args)
```
```

--------------------------------

### Save and Load Data for Text Classification Training

Source: https://happytransformer.com/save-load-data

Illustrates saving preprocessed data for text classification using `TCEvalArgs(save_path='data/')` and loading it for evaluation with `TCEvalArgs(load_path='data/')`. The `input_filepath` can be set to any value when loading.

```python
from happytransformer import HappyTextClassification, TCEvalArgs
# ---------------------------------------------------------
happy_tc = HappyTextClassification()
eval_args_1 = TCEvalArgs(save_path="data/")
result_1 = happy_tc.eval("data/tc/train-eval.txt", args=eval_args_1)

eval_args_2 = TCEvalArgs(load_path="data/")
# if you're loading data, then you can set input_filepath to anything 
result_2 = happy_tc.eval(input_filepath="", args=eval_args_2)
```

--------------------------------

### Initialize HappyTokenClassification

Source: https://happytransformer.com/token-classification

Instantiate HappyTokenClassification with default settings, a specific model type and name, or with authentication for private models.

```python
from happytransformer import HappyTokenClassification
# --------------------------------------#
happy_toc = HappyTokenClassification("BERT", "dslim/bert-base-NER")  # default 
happy_toc_large = HappyTokenClassification("XLM-ROBERTA", "xlm-roberta-large-finetuned-conll03-english") 
happy_toc_private = HappyTokenClassification("BERT", "user-repo/bert-base-NER", use_auth_token="123abc")
```

--------------------------------

### Configure WandB Reporting in GENTrainArgs

Source: https://happytransformer.com/wandb

Configure WandB tracking by specifying the report_to, project_name, and run_name parameters in GENTrainArgs. Ensure 'wandb' is included in the report_to tuple.

```python
from happytransformer import GENTrainArgs

args = GENTrainArgs(report_to=tuple(["wandb"])
                    )

```

--------------------------------

### test()

Source: https://happytransformer.com/question-answering/finetuning

Run the model on an unlabeled dataset to produce predictions. It requires a file path to the test dataset and optional test arguments.

```APIDOC
## test()

### Description
Run the model on an unlabeled dataset to produce predictions.

### Method
`test()`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
*   **input_filepath** (string) - Required - A path to a CSV file containing test data. The CSV should have columns: 'context' and 'question' (as described in Table 3.3).
*   **args** (QATestArgs) - Optional - A dataclass containing test parameters. (See Table x for details).

### Request Example
```python
from happytransformer import HappyQuestionAnswering, QATestArgs

happy_qa = HappyQuestionAnswering()
result = happy_qa.test("../../data/qa/test.csv")
print(result[0].answer)
```

### Response
#### Success Response
Output: A list of named tuples with keys: "answer", "score", "start", and "end".

#### Response Example
```
[QuestionAnsweringResult(answer='October 31st', score=0.9939756989479065, start=0, end=12), QuestionAnsweringResult(answer='November 23rd', score=0.967872679233551, start=12, end=25)]
```
```

--------------------------------

### Finetune Text-to-Text Model

Source: https://happytransformer.com/text-to-text/finetuning

Fine-tune a text-to-text model for tasks like grammar correction. Requires a CSV file for training data and accepts custom training arguments.

```python
from happytransformer import HappyTextToText, TTTrainArgs
# --------------------------------------#
happy_tt = HappyTextToText()
args = TTTrainArgs(num_train_epochs=1)
happy_tt.train("../../data/tt/train-eval-grammar.csv", args=args)
```

--------------------------------

### Initialize HappyNextSentence

Source: https://happytransformer.com/next-sentence

Instantiate HappyNextSentence with default BERT model, a larger BERT model, or a private BERT model using an authentication token.

```python
from happytransformer import HappyNextSentence
# --------------------------------------#
happy_ns = HappyNextSentence("BERT", "bert-base-uncased")  # default 
happy_ns_large = HappyNextSentence("BERT", "bert-large-uncased") 
happy_ns_private = HappyNextSentence("BERT", "ericfillion/bert-base-uncased", use_auth_token="123abc")
```

--------------------------------

### train()

Source: https://happytransformer.com/question-answering/finetuning

Fine-tune a question answering model to improve its performance on a specific task. It accepts a training data file, training arguments, and an optional evaluation data file.

```APIDOC
## train()

### Description
Fine-tune a question answering model to become better at a certain task.

### Method
`train()`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
*   **input_filepath** (string) - Required - A path to a CSV file containing training data. The CSV should have columns: 'context', 'question', 'answer_text', and 'answer_start'.
*   **args** (QATrainArgs) - Required - A dataclass containing training parameters such as learning rate, number of epochs, batch size, etc. (See Table 3.1 for details).
*   **eval_filepath** (string) - Optional - A filepath to a CSV file containing standalone evaluating data. If not provided, an evaluating dataset will be generated from the training data.

### Request Example
```python
from happytransformer import HappyQuestionAnswering, QATrainArgs

happy_qa = HappyQuestionAnswering()
args = QATrainArgs(num_train_epochs=1)
happy_qa.train("../../data/qa/train-eval.csv", args=args)
```

### Response
#### Success Response
Output: None

#### Response Example
None
```

--------------------------------

### Evaluate Text Generation Model

Source: https://happytransformer.com/text-generation/finetuning

Use this snippet to evaluate the performance of a text generation model. Customize preprocessing with GENEvalArgs. The output includes the loss.

```python
from happytransformer import HappyGeneration, GENEvalArgs
# --------------------------------------#
happy_gen = HappyGeneration()
args = GENEvalArgs(preprocessing_processes=2)
result = happy_gen.eval("../../data/gen/train-eval.txt", args=args)
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(loss=3.3437771797180176)
print(result.loss)  # 3.3437771797180176
```

--------------------------------

### Finetune Text Generation Model

Source: https://happytransformer.com/text-generation/finetuning

Use this snippet to fine-tune a text generation model with custom text data. Specify the training epochs using GENTrainArgs.

```python
from happytransformer import HappyGeneration, GENTrainArgs
# --------------------------------------#

happy_gen = HappyGeneration()
args = GENTrainArgs(num_train_epochs=1)
happy_gen.train("../../data/gen/train-eval.txt", args=args)
```

--------------------------------

### Save a HappyGeneration Model

Source: https://happytransformer.com/save-load-model

Use the save() method to persist your trained HappyGeneration model to a specified directory. Ensure the directory exists or will be created. Any existing files with the same names will be overwritten.

```python
from happytransformer import HappyGeneration
# ---------------------------------------------------------
happy_gen = HappyGeneration(model_type="GPT-NEO", model_name="EleutherAI/gpt-neo-125M")
happy_gen.save("model/")
```

--------------------------------

### Initialize and Classify Text

Source: https://happytransformer.com/text-classification/usage

Initializes the HappyTextClassification model and classifies a given text. Shows how to access the label and score from the result.

```python
from happytransformer import HappyTextClassification
# --------------------------------------#
happy_tc = HappyTextClassification(model_type="DISTILBERT",  model_name="distilbert-base-uncased-finetuned-sst-2-english")
result = happy_tc.classify_text("Great movie! 5/5")
print(type(result))  # <class 'happytransformer.happy_text_classification.TextClassificationResult'>
print(result)  # TextClassificationResult(label='POSITIVE', score=0.9998761415481567)
print(result.label)  # LABEL_1
```

--------------------------------

### Basic Masked Word Prediction

Source: https://happytransformer.com/word-prediction/usage

Demonstrates the basic usage of predict_mask() to predict a single masked token in a sentence. Shows how to access the predicted token and its score.

```python
from happytransformer import HappyWordPrediction
#--------------------------------------#
happy_wp = HappyWordPrediction()  # default uses distilbert-base-uncased
result = happy_wp.predict_mask("I think therefore I [MASK]")
print(type(result))  # <class 'list'>
print(result)  # [WordPredictionResult(token='am', score=0.10172799974679947)]
print(type(result[0]))  # <class 'happytransformer.happy_word_prediction.WordPredictionResult'>
print(result[0])  # [WordPredictionResult(token='am', score=0.10172799974679947)]
print(result[0].token)  # am
print(result[0].score)  # 0.10172799974679947
```

--------------------------------

### Test a Question Answering Model

Source: https://happytransformer.com/question-answering/finetuning

Use the `test()` method to run a question answering model on an unlabeled dataset and generate predictions. The output is a list of named tuples containing the answer, score, and start/end indices.

```python
from happytransformer import HappyQuestionAnswering, QATestArgs
# --------------------------------------#
happy_qa = HappyQuestionAnswering()
result = happy_qa.test("../../data/qa/test.csv")
print(type(result))
print(result)  # [QuestionAnsweringResult(answer='October 31st', score=0.9939756989479065, start=0, end=12), QuestionAnsweringResult(answer='November 23rd', score=0.967872679233551, start=12, end=25)]
print(result[0])  # QuestionAnsweringResult(answer='October 31st', score=0.9939756989479065, start=0, end=12)
print(result[0].answer)  # October 31st

```

--------------------------------

### Fine-tune a Question Answering Model

Source: https://happytransformer.com/question-answering/finetuning

Use the `train()` method to fine-tune a question answering model on a custom dataset. Provide the path to your training data and optional training arguments.

```python
from happytransformer import HappyQuestionAnswering, QATrainArgs
# --------------------------------------#
happy_qa = HappyQuestionAnswering()
args = QATrainArgs(num_train_epochs=1)
happy_qa.train("../../data/qa/train-eval.csv", args=args)

```

--------------------------------

### train()

Source: https://happytransformer.com/word-prediction/finetuning

Fine-tunes the word prediction model to better understand a given body of text. It accepts a training data file, training arguments, and an optional evaluation file.

```APIDOC
## train()

### Description
Fine-tunes the word prediction model to better understand a given body of text. It accepts a training data file, training arguments, and an optional evaluation file.

### Method
`train(input_filepath: str, args: WPTrainArgs, eval_filepath: Optional[str] = None)`

### Parameters
#### Path Parameters
- **input_filepath** (string) - Required - A path to a text file containing the training data.
- **eval_filepath** (string) - Optional - A path to a text or CSV file for standalone evaluation data. If not provided, an evaluation dataset will be generated from the training data.

#### Request Body
- **args** (WPTrainArgs) - Required - A dataclass containing training configuration parameters such as learning rate, number of epochs, batch size, etc. (See Table 4.1 for details).

### Request Example
```python
from happytransformer import HappyWordPrediction, WPTrainArgs

happy_wp = HappyWordPrediction()
args = WPTrainArgs(num_train_epochs=1)
happy_wp.train("../../data/wp/train-eval.txt", args=args)
```

### Response
This method does not return a value directly, but updates the model state and saves checkpoints based on the `args`.
```

--------------------------------

### Masked Word Prediction with Top K Results

Source: https://happytransformer.com/word-prediction/usage

Shows how to retrieve the top K most likely predictions for a masked token using the top_k argument. Includes accessing specific results from the returned list.

```python
from happytransformer import HappyWordPrediction
#--------------------------------------#
happy_wp = HappyWordPrediction()
result = happy_wp.predict_mask("To better the world I would invest in [MASK] and education.", top_k=2)
print(result)  # [WordPredictionResult(token='health', score=0.1280556619167328), WordPredictionResult(token='science', score=0.07976455241441727)]
print(result[1]) # WordPredictionResult(token='science', score=0.07976455241441727)
print(result[1].token) # science
```

--------------------------------

### eval()

Source: https://happytransformer.com/question-answering/finetuning

Determine how well the model performs on a labeled dataset. It takes a file path to an evaluation dataset and evaluation arguments.

```APIDOC
## eval()

### Description
Determine how well the model performs on a labeled dataset.

### Method
`eval()`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
*   **input_filepath** (string) - Required - A path to a CSV file containing evaluation data. The CSV should have columns: 'context', 'question', 'answer_text', and 'answer_start' (as described in Table 3.0).
*   **args** (QAEvalArgs) - Optional - A dataclass containing evaluation parameters. Defaults are used if not provided. (See Table 3.2 for details).

### Request Example
```python
from happytransformer import HappyQuestionAnswering, QAEvalArgs

happy_qa = HappyQuestionAnswering()
args = QAEvalArgs() # Using default settings
result = happy_qa.eval("../../data/qa/train-eval.csv")
print(result.eval_loss)
```

### Response
#### Success Response
Output: A dataclass with the variable "loss".

#### Response Example
```
EvalResult(eval_loss=0.11738169193267822)
```
```

--------------------------------

### Configure Deepspeed for Training and Evaluation

Source: https://happytransformer.com/deepspeed

Set the 'deepspeed' parameter in GENTrainArgs or GENEvalArgs to 'ZERO-3' for distributed training. Ensure ZERO-2 is not used for evaluation.

```python
from happytransformer import GENTrainArgs, GENEvalArgs

train_args = GENTrainArgs(deepspeed="ZERO-3")

eval_args = GENEvalArgs(deepspeed="ZERO-3")
```

--------------------------------

### Masked Word Prediction with Specific Targets

Source: https://happytransformer.com/word-prediction/usage

Demonstrates how to provide a list of target words to predict. When targets are specified, top_k is ignored, and scores are returned only for the provided targets.

```python
from happytransformer import HappyWordPrediction
#--------------------------------------#
happy_wp = HappyWordPrediction()
targets = ["technology", "healthcare"]
result = happy_wp.predict_mask("To better the world I would invest in [MASK] and education.", targets=targets, top_k=2)
print(result)  # [WordPredictionResult(token='healthcare', score=0.07380751520395279), WordPredictionResult(token='technology', score=0.009395276196300983)]
print(result[1])  # WordPredictionResult(token='technology', score=0.009395276196300983)
print(result[1].token)  # technology
```

--------------------------------

### eval()

Source: https://happytransformer.com/text-generation/finetuning

Determines how well the model performs on a given dataset. It accepts a file path for evaluation data and optional arguments for evaluation configuration.

```APIDOC
## eval()

### Description
Determines how well the model performs on a given dataset. It accepts a file path for evaluation data and optional arguments for evaluation configuration.

### Method
`eval()`

### Parameters
#### Path Parameters
* **input_filepath** (string) - Required - A path to a text file containing text to evaluate.

#### Request Body
* **args** (WPEvalArgs) - Required - A dataclass containing evaluation arguments.

**WPEvalArgs Fields:**
* **save_path** (string) - Optional - Default: ""
* **load_path** (string) - Optional - Default: ""
* **max_length** (None) - Optional - Default: None

### Output
An object with the field "loss".

#### Response Example
```python
from happytransformer import HappyGeneration, GENEvalArgs

happy_gen = HappyGeneration()
args = GENEvalArgs(preprocessing_processes=2)
result = happy_gen.eval("../../data/gen/train-eval.txt", args=args)
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(loss=3.3437771797180176)
print(result.loss)  # 3.3437771797180176
```
```

--------------------------------

### Evaluate HappyWordPrediction Model

Source: https://happytransformer.com/word-prediction/finetuning

Evaluate the performance of the HappyWordPrediction model using a specified text file and evaluation arguments. The output includes the evaluation loss.

```python
from happytransformer import HappyWordPrediction, WPEvalArgs
# --------------------------------------#
happy_wp = HappyWordPrediction()
args = WPEvalArgs(preprocessing_processes=2)
result = happy_wp.eval("../../data/wp/train-eval.txt", args=args)
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(eval_loss=0.459536075592041)
print(result.loss)  # 0.459536075592041
```

--------------------------------

### Train HappyWordPrediction Model

Source: https://happytransformer.com/word-prediction/finetuning

Fine-tune the HappyWordPrediction model with custom training data. Requires specifying the input file path and training arguments.

```python
from happytransformer import HappyWordPrediction, WPTrainArgs
# --------------------------------------#
happy_wp = HappyWordPrediction()
args = WPTrainArgs(num_train_epochs=1)
happy_wp.train("../../data/wp/train-eval.txt", args=args)
```

--------------------------------

### Evaluate Text-to-Text Model

Source: https://happytransformer.com/text-to-text/finetuning

Evaluate the performance of a finetuned text-to-text model. Accepts a CSV file for evaluation data and custom evaluation arguments. Returns an EvalResult object containing the loss.

```python
from happytransformer import HappyTextToText, TTEvalArgs
# --------------------------------------#
happy_tt = HappyTextToText()
args = TTEvalArgs(preprocessing_processes=1)
result = happy_tt.eval("../../data/tt/train-eval-grammar.csv", args=args)
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(loss=3.2277376651763916)
print(result.loss)  # 3.2277376651763916
```

--------------------------------

### Finetune Text Classification Model

Source: https://happytransformer.com/text-classification/finetuning

Fine-tune a DistilBERT model for text classification. Requires specifying the model type, name, number of labels, and training arguments. Use a CSV file with 'text' and 'label' columns for training data.

```python
from happytransformer import HappyTextClassification, TCTrainArgs
# --------------------------------------#
happy_tc = HappyTextClassification(model_type="DISTILBERT",
                                   model_name="distilbert-base-uncased-finetuned-sst-2-english",
                                   num_labels=2)  # Don't forget to set num_labels! 
args = TCTrainArgs(num_train_epochs=1)
happy_tc.train("../../data/tc/train-eval.csv", args=args)

```

--------------------------------

### Predict Next Sentence - Likely

Source: https://happytransformer.com/next-sentence/usage

Demonstrates a likely sentence following the input sentence. Initializes HappyNextSentence and calls predict_next_sentence.

```python
from happytransformer import HappyNextSentence
# --------------------------------------#
happy_ns = HappyNextSentence()
result = happy_ns.predict_next_sentence(
    "How old are you?",
    "I am 21 years old."
)
print(type(result))  # <class 'float'>
print(result)  # 0.9999918937683105
```

--------------------------------

### Evaluate Text Classification Model Performance

Source: https://happytransformer.com/text-classification/finetuning

Evaluate the performance of a DistilBERT text classification model. Requires the model to be initialized and an evaluation CSV file. The output is an EvalResult object containing the loss.

```python
from happytransformer import HappyTextClassification, TCEvalArgs
# --------------------------------------#
happy_tc = HappyTextClassification(model_type="DISTILBERT",
                                   model_name="distilbert-base-uncased-finetuned-sst-2-english",
                                   num_labels=2)  # Don't forget to set num_labels!
result = happy_tc.eval("../../data/tc/train-eval.csv")
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(eval_loss=0.007262040860950947)
print(result.loss)  # 0.007262040860950947

```

--------------------------------

### Predict Next Sentence - Unlikely

Source: https://happytransformer.com/next-sentence/usage

Demonstrates an unlikely sentence following the input sentence. Initializes HappyNextSentence and calls predict_next_sentence.

```python
from happytransformer import HappyNextSentence
# --------------------------------------#
happy_ns = HappyNextSentence()
result = happy_ns.predict_next_sentence(
    "How old are you?",
    "Queen's University is in Kingston Ontario Canada"
)
print(type(result))  # <class 'float'>
print(result)  # 0.00018497584096621722
```

--------------------------------

### Evaluate a Question Answering Model

Source: https://happytransformer.com/question-answering/finetuning

Use the `eval()` method to assess the performance of a question answering model on a labeled dataset. It returns an evaluation result object containing the loss.

```python
from happytransformer import HappyQuestionAnswering, QAEvalArgs
# --------------------------------------#
happy_qa = HappyQuestionAnswering()
args = QAEvalArgs() #  The default settings as an example
result = happy_qa.eval("../../data/qa/train-eval.csv")
print(type(result))  # <class 'happytransformer.happy_trainer.EvalResult'>
print(result)  # EvalResult(eval_loss=0.11738169193267822)
print(result.loss)  # 0.1173816919326782

```

--------------------------------

### Test Text Classification Model Predictions

Source: https://happytransformer.com/text-classification/finetuning

Generate predictions on an unlabeled dataset using a DistilBERT text classification model. Input is a CSV file with a 'text' column. The output is a list of TextClassificationResult objects, each containing a predicted label and score.

```python
from happytransformer import HappyTextClassification
# --------------------------------------#
happy_tc = HappyTextClassification(model_type="DISTILBERT",
                                   model_name="distilbert-base-uncased-finetuned-sst-2-english",
                                   num_labels=2)  # Don't forget to set num_labels!
result = happy_tc.test("../../data/tc/test.csv")
print(type(result))  # <class 'list'>
print(result)  # [TextClassificationResult(label='POSITIVE', score=0.9998401999473572), TextClassificationResult(label='LABEL_0', score=0.9772131443023682)...
print(type(result[0]))  # <class 'happytransformer.happy_text_classification.TextClassificationResult'>
print(result[0])  # TextClassificationResult(label='POSITIVE', score=0.9998401999473572)
print(result[0].label)  # POSITIVE

```
Tokens:
Raw