### Install Flair

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/intro.md

Install the Flair library using pip. Requires Python 3.9+.

```default
pip install flair
```

--------------------------------

### Start Model Training

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Initiates the training process by calling the train() method on the initialized ModelTrainer. This starts a standard training run.

```python
# 7. train
trainer.train()
```

--------------------------------

### Install googledrivedownloader

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR_CORPORA.md

Install the googledrivedownloader package to download corpora hosted on Google Drive.

```bash
pip install googledrivedownloader
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/flairnlp/flair/blob/master/CONTRIBUTING.md

Installs the necessary development requirements and the Flair library in editable mode. Ensure you are using Python 3.9 or higher for development.

```bash
pip install -r requirements-dev.txt
pip install -e .
```

--------------------------------

### Install SciSpaCy and Download Model

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/tagging.md

Installs the SciSpaCy library and downloads a small English scientific language model, required for using the SciSpacyTokenizer.

```bash
pip install scispacy==0.5.1
pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.5.1/en_core_sci_sm-0.5.1.tar.gz
```

--------------------------------

### NER Configuration File Example

Source: https://github.com/flairnlp/flair/blob/master/examples/ner/README.md

A JSON configuration file storing parameters for fine-tuning the Masakhane NER dataset.

```json
{
    "dataset_name": "NER_MASAKHANE",
    "dataset_arguments": "{\"languages\": \"luo\"}",
    "model_name_or_path": "xlm-roberta-base",
    "batch_size": 32,
    "learning_rate": 5e-05,
    "num_epochs": 50,
    "output_dir": "masakhane-luo-ner-base-2"
}
```

--------------------------------

### NER Model Fine-Tuning Output Example

Source: https://github.com/flairnlp/flair/blob/master/examples/ner/README.md

Example of the final output from the NER model fine-tuning script, including micro/macro F-scores and per-class performance metrics.

```bash
Results:
- F-score (micro) 0.7538
- F-score (macro) 0.7246
- Accuracy 0.6162

By class:
              precision    recall  f1-score   support

         PER     0.8626    0.7958    0.8278       142
         LOC     0.7727    0.7969    0.7846       128
        DATE     0.7458    0.6377    0.6875        69
         ORG     0.6230    0.5758    0.5984        66

   micro avg     0.7755    0.7333    0.7538       405
   macro avg     0.7510    0.7015    0.7246       405
weighted avg     0.7752    0.7333    0.7529       405
 samples avg     0.6162    0.6162    0.6162       405
```

--------------------------------

### Example Commit Message

Source: https://github.com/flairnlp/flair/blob/master/docs/contributing/making_a_pull_request.md

Follow this format for commit messages when addressing an existing GitHub issue.

```git
GH-42: Added new type of embeddings: DocumentEmbedding.
```

--------------------------------

### Initialize and Use ELMo Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/embeddings/ELMO_EMBEDDINGS.md

Demonstrates how to initialize ELMo embeddings, create a Flair Sentence object, and embed words within the sentence. Ensure 'allennlp==0.9.0' is installed.

```python
from flair.embeddings import ELMoEmbeddings

# init embedding
embedding = ELMoEmbeddings()

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
embedding.embed(sentence)
```

--------------------------------

### Run NER Prediction Example

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/how-predictions-work.md

Loads a pre-trained NER tagger, makes a sentence, predicts NER tags, and prints the sentence with tags. This demonstrates the basic prediction workflow.

```python
from flair.nn import Classifier
from flair.data import Sentence

# load the model
tagger = Classifier.load('ner')

# make a sentence
sentence = Sentence('George Washington went to Washington.')

# predict NER tags
tagger.predict(sentence)

# print the sentence with the tags
print(sentence)
```

--------------------------------

### Initialize and Train Flair Model Trainer

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Initializes a ModelTrainer with a tagger and corpus, then starts the training process for a specified number of epochs.

```python
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.train('resources/taggers/example-ner',
              train_with_dev=True,
              max_epochs=150)
```

--------------------------------

### NER Model Fine-Tuning Results

Source: https://github.com/flairnlp/flair/blob/master/examples/ner/README.md

Example of the final output on the test set after fine-tuning an NER model, showing performance metrics.

```text
Results:
- F-score (micro) 0.7793
- F-score (macro) 0.7418
- Accuracy 0.657

By class:
              precision    recall  f1-score   support

         PER     0.9058    0.8803    0.8929       142
         LOC     0.7698    0.8359    0.8015       128
        DATE     0.7302    0.6667    0.6970        69
         ORG     0.5758    0.5758    0.5758        66

   micro avg     0.7783    0.7802    0.7793       405
   macro avg     0.7454    0.7397    0.7418       405
weighted avg     0.7791    0.7802    0.7789       405
 samples avg     0.6570    0.6570    0.6570       405
```

--------------------------------

### Train a TARS Model with a Dataset

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_10_TRAINING_ZERO_SHOT_MODEL.md

Demonstrates the initial setup for training a TARS classifier using a dataset like TREC_6. This involves loading the corpus, the TARSClassifier, and the ModelTrainer.

```python
from flair.data import Corpus
from flair.datasets import TREC_6
from flair.models import TARSClassifier
from flair.trainers import ModelTrainer
```

--------------------------------

### Initialize GloVe Embeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Initializes GloVe word embeddings. While this example uses GloVe for speed, transformer-based embeddings are generally recommended for better performance.

```python
# 4. initialize embeddings
embeddings = WordEmbeddings('glove')
```

--------------------------------

### Training Progress Output

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Example output during model training, showing epoch number, iteration, loss, time, samples per second, and learning rate. This helps monitor the training progress and identify potential issues.

```text
2023-02-27 17:07:38,014 ----------------------------------------------------------------------------------------------------
2023-02-27 17:07:38,016 Model training base path: "resources/taggers/example-upos"
2023-02-27 17:07:38,017 ----------------------------------------------------------------------------------------------------
2023-02-27 17:07:38,020 Device: cuda:0
2023-02-27 17:07:38,022 ----------------------------------------------------------------------------------------------------
2023-02-27 17:07:38,023 Embeddings storage mode: cpu
2023-02-27 17:07:38,025 ----------------------------------------------------------------------------------------------------
2023-02-27 17:07:39,128 epoch 1 - iter 4/40 - loss 3.28409882 - time (sec): 1.10 - samples/sec: 2611.84 - lr: 0.100000
2023-02-27 17:07:39,474 epoch 1 - iter 8/40 - loss 3.13510367 - time (sec): 1.45 - samples/sec: 3143.21 - lr: 0.100000
2023-02-27 17:07:39,910 epoch 1 - iter 12/40 - loss 3.02619775 - time (sec): 1.88 - samples/sec: 3434.39 - lr: 0.100000
2023-02-27 17:07:40,167 epoch 1 - iter 16/40 - loss 2.95288554 - time (sec): 2.14 - samples/sec: 3783.76 - lr: 0.100000
2023-02-27 17:07:40,504 epoch 1 - iter 20/40 - loss 2.86820018 - time (sec): 2.48 - samples/sec: 4171.22 - lr: 0.100000
2023-02-27 17:07:40,843 epoch 1 - iter 24/40 - loss 2.80507526 - time (sec): 2.82 - samples/sec: 4557.72 - lr: 0.100000
2023-02-27 17:07:41,118 epoch 1 - iter 28/40 - loss 2.74217397 - time (sec): 3.09 - samples/sec: 4878.00 - lr: 0.100000
2023-02-27 17:07:41,420 epoch 1 - iter 32/40 - loss 2.69161746 - time (sec): 3.39 - samples/sec: 5072.93 - lr: 0.100000
2023-02-27 17:07:41,705 epoch 1 - iter 36/40 - loss 2.63837577 - time (sec): 3.68 - samples/sec: 5260.02 - lr: 0.100000
2023-02-27 17:07:41,972 epoch 1 - iter 40/40 - loss 2.58915523 - time (sec): 3.95 - samples/sec: 5394.33 - lr: 0.100000
2023-02-27 17:07:41,975 ----------------------------------------------------------------------------------------------------
2023-02-27 17:07:41,977 EPOCH 1 done: loss 2.5892 - lr 0.100000
2023-02-27 17:07:42,567 DEV : loss 2.009714126586914 - f1-score (micro avg)  0.41
2023-02-27 17:07:42,579 BAD EPOCHS (no improvement): 0
```

--------------------------------

### Tokenizing Japanese Sentences with JapaneseTokenizer

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_BASICS_TOKENIZATION.md

Illustrates how to tokenize Japanese text using the `JapaneseTokenizer` from Flair, which requires the `konoha` library to be installed.

```python
from flair.data import Sentence
from flair.tokenization import JapaneseTokenizer

tokenizer = JapaneseTokenizer("janome")

japanese_sentence = Sentence("私はベルリンが好き", use_tokenizer=tokenizer)

print(japanese_sentence)
```

--------------------------------

### AG_NEWS Data Format Example

Source: https://github.com/flairnlp/flair/blob/master/tests/resources/tasks/ag_news/README.md

Illustrates the expected format for AG_NEWS data entries, which includes a label and the corresponding text.

```text
__label__<class_name> <text>
```

--------------------------------

### Initiate Model Training

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Starts the training process for a part-of-speech tagger using a specified corpus. This involves loading data, creating a label dictionary, initializing embeddings, and running the trainer for a set number of epochs with specified learning rate and batch size.

```python
trainer.train('resources/taggers/example-upos',
              learning_rate=0.1,
              mini_batch_size=32,
              max_epochs=10)
```

--------------------------------

### Example Linked Entity Output

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/linking.md

This is an example of the output format when inspecting linked entities, showing the entity span, its mapped ontology identifier and name, and the similarity score.

```text
Span[4:5]: "ABCD1" → 215/name=ABCD1 (210.89810180664062)
Span[7:9]: "X-linked adrenoleukodystrophy" → MESH:D000326/name=Adrenoleukodystrophy (195.30780029296875)
Span[11:13]: "neurodegenerative disease" → MESH:D019636/name=Neurodegenerative Diseases (201.1804962158203)
Span[23:24]: "mercury" → MESH:D008628/name=Mercury (220.39199829101562)
Span[25:26]: "mouse" → 10090/name=Mus musculus (213.6201934814453)
```

--------------------------------

### Initialize and Train Flair Model Trainer

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Initializes a ModelTrainer with a tagger and corpus, then starts training. The `embeddings_storage_mode` can be set to 'none' for large datasets to manage memory.

```python
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.train('resources/taggers/example-ner',
              learning_rate=0.1,
              train_with_dev=True,
              # it's a big dataset so maybe set embeddings_storage_mode to 'none' (embeddings are not kept in memory)
              embeddings_storage_mode='none')
```

--------------------------------

### Print Entity Linking Outputs to Console

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/entity-mention-linking.md

Demonstrates how to load a NER tagger and an Entity Mention Linker, predict on a sentence, and print the resulting entity and mention linking tags to the console. This example uses the 'disease-linker-no-ab3p' model for simplicity.

```python
from flair.models import EntityMentionLinker
from flair.nn import Classifier
from flair.tokenization import SciSpacyTokenizer
from flair.data import Sentence

sentence = Sentence(
    "The mutation in the ABCD1 gene causes X-linked adrenoleukodystrophy, "
    "a neurodegenerative disease, which is exacerbated by exposure to high "
    "levels of mercury in dolphin populations.",
    use_tokenizer=SciSpacyTokenizer()
)

ner_tagger = Classifier.load("hunflair2")
ner_tagger.predict(sentence)

nen_tagger = EntityMentionLinker.load("disease-linker")
nen_tagger.predict(sentence)

for tag in sentence.get_labels():
    print(tag)
```

--------------------------------

### Quantize ONNX Model for CPU

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_PRODUCTION_FASTER_TRANSFORMERS.md

Applies quantization to the ONNX model to significantly speed up predictions on CPU. Requires `onnx` to be installed. Note that quantization is not supported for CUDAExecutionProvider. `use_external_data_format=True` is used for large models.

```python
model.embeddings.quantize_model(
            "flert-quantized-embeddings.onnx", extra_options={"DisableShapeInference": True}, use_external_data_format=True
        )
```

--------------------------------

### Combine Flair and Word Embeddings with StackedEmbeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/flair-embeddings.md

Instantiate StackedEmbeddings to combine GloVe, forward Flair, and backward Flair embeddings for enhanced performance on English tasks. This setup is recommended for most NLP tasks.

```python
from flair.embeddings import WordEmbeddings, FlairEmbeddings, StackedEmbeddings

# create a StackedEmbedding object that combines glove and forward/backward flair embeddings
stacked_embeddings = StackedEmbeddings([
                                        WordEmbeddings('glove'),
                                        FlairEmbeddings('news-forward'),
                                        FlairEmbeddings('news-backward'),
                                       ])
```

--------------------------------

### Train a Part-of-Speech Tagger in Flair

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

This code snippet demonstrates how to train a simple Part-of-Speech tagger using Flair. It involves loading a corpus, defining the label type, initializing embeddings, setting up the sequence tagger model, and configuring the trainer. The example downsamples the training data and uses GloVe embeddings for faster execution.

```python
from flair.datasets import UD_ENGLISH
from flair.embeddings import WordEmbeddings
from flair.models import SequenceTagger
from flair.trainers import ModelTrainer

# 1. load the corpus
corpus = UD_ENGLISH().downsample(0.1)
print(corpus)

# 2. what label do we want to predict?
label_type = 'upos'

# 3. make the label dictionary from the corpus
label_dict = corpus.make_label_dictionary(label_type=label_type)
print(label_dict)

# 4. initialize embeddings
embeddings = WordEmbeddings('glove')

# 5. initialize sequence tagger
model = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=label_dict,
                        tag_type=label_type)

# 6. initialize trainer
trainer = ModelTrainer(model, corpus)
```

--------------------------------

### Train HunFlair Model for Cell Line Entities

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR_TUTORIAL_2_TRAINING.md

This example shows the setup for training a HunFlair model specifically for cell line entities. It involves loading the relevant corpus (HUNER_CELL_LINE) and initializing embeddings and the sequence tagger, similar to training from scratch but with a specific corpus.

```python
from flair.datasets import HUNER_CELL_LINE

# 1. get all corpora for a specific entity type
from flair.models import SequenceTagger
corpus = HUNER_CELL_LINE()

# 2. initialize embeddings
from flair.embeddings import WordEmbeddings, FlairEmbeddings, StackedEmbeddings
embedding_types = [
    WordEmbeddings("pubmed"),
    FlairEmbeddings("pubmed-forward"),
    FlairEmbeddings("pubmed-backward"),

]

embeddings = StackedEmbeddings(embeddings=embedding_types)

# 3. initialize sequence tagger
tag_DICTIONARY = corpus.make_label_dictionary(label_type="ner", add_unk=False)

tagGER = SequenceTagger(
    hidden_size=256,
    embeddings=embeddings,
    tag_dictionary=tag_dictionary,
    tag_type="ner",
    use_crf=True,
    locked_dropout=0.5
)
```

--------------------------------

### Initialize Custom BytePairEmbeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/other-embeddings.md

Initializes custom BytePairEmbeddings by providing paths to a SentencePiece model file and an embedding file (e.g., Word2Vec).

```python
# init custom embedding
embedding = BytePairEmbeddings(model_file_path='your/path/m.model', embedding_file_path='your/path/w2v.txt')
```

--------------------------------

### Format Code and Imports

Source: https://github.com/flairnlp/flair/blob/master/CONTRIBUTING.md

Manually formats code using black and standardizes imports using ruff. This command should be run in the root folder of the Flair project.

```bash
black . && ruff --fix .
```

--------------------------------

### Embed Words with Transformer Embeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/embeddings.md

Instantiate TransformerWordEmbeddings and call embed() over a Sentence to get embeddings for each word. Requires the flair library.

```python
from flair.embeddings import TransformerWordEmbeddings
from flair.data import Sentence

# init embedding
embedding = TransformerWordEmbeddings('bert-base-uncased')

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
embedding.embed(sentence)
```

```python
# now check out the embedded tokens.
for token in sentence:
    print(token)
    print(token.embedding)
```

--------------------------------

### Example Linked Entity Output

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR2_TUTORIAL_2_LINKING.md

This output format shows the span of the entity in the text, its linked identifier and name from the ontology, and the similarity score.

```text
Span[4:5]: "ABCD1" → 215/name=ABCD1 (210.89810180664062)
Span[7:9]: "X-linked adrenoleukodystrophy" → MESH:D008628/name=Adrenoleukodystrophy (195.30780029296875)
Span[11:13]: "neurodegenerative disease" → MESH:D019636/name=Neurodegenerative Diseases (201.1804962158203)
Span[23:24]: "mercury" → MESH:D008628/name=Mercury (220.39199829101562)
Span[25:26]: "mouse" → 10090/name=Mus musculus (213.6201934814453)
```

--------------------------------

### Initialize Multilingual BytePairEmbeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/other-embeddings.md

Initializes a multilingual BytePairEmbeddings model capable of embedding words in any language by leveraging subword information.

```python
# init embedding
embedding = BytePairEmbeddings('multi')
```

--------------------------------

### Tagging Entities in Arabic Text

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/tagging-entities.md

Perform NER on Arabic text using the 'ar-ner' model. This example shows processing a right-to-left language.

```python
# load model
tagGER = Classifier.load('ar-ner')

# make Arabic sentence
sentence = Sentence("احب برلين")

# predict NER tags
tagGER.predict(sentence)

# print sentence with predicted tags
print(sentence)
```

--------------------------------

### Initialize ModelTrainer and Fine-tune NER Tagger

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR2_TUTORIAL_3_TRAINING_NER.md

Initializes a ModelTrainer with the configured tagger and corpus, then fine-tunes the model. Parameters for training include base path, maximum epochs, learning rate, mini-batch size, and shuffling.

```python
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.fine_tune(
    base_path="taggers/cdr_nlm_chem",
    train_with_dev=False,
    max_epochs=16,
    learning_rate=2.0e-5,
    mini_batch_size=16,
    shuffle=False,
)
```

--------------------------------

### Initialize BytePairEmbeddings for English

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/other-embeddings.md

Initializes BytePairEmbeddings for English, which are precomputed subword embeddings. They offer a balance between accuracy and model size.

```python
from flair.embeddings import BytePairEmbeddings

# init embedding
embedding = BytePairEmbeddings('en')

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
embedding.embed(sentence)
```

--------------------------------

### Train a Sequence Tagger with Flair Embeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-to-train-sequence-tagger.md

Initializes GloVe and Flair embeddings, stacks them, and then initializes a sequence tagger. Finally, it trains the tagger using the `ModelTrainer`.

```python
# 3. make the label dictionary from the corpus
label_dict = corpus.make_label_dictionary(label_type=label_type)
print(label_dict)

# 4. initialize embeddings
embedding_types = [
    WordEmbeddings('glove'),
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
tagGER = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=label_dict,
                        tag_type=label_type,
                        use_crf=True)

# 6. initialize trainer
trainer = ModelTrainer(tagger, corpus)

# 7. start training
trainer.train('resources/taggers/example-upos',
              learning_rate=0.1,
              mini_batch_size=32)
```

--------------------------------

### Fine-tuned Transformer Embedding Tensor

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/transformer-embeddings.md

Example output of a fine-tuned transformer embedding tensor, showing its dimensionality and gradient function, indicating it's ready for training.

```text
tensor([-0.0323, -0.3904, -1.1946,  ...,  0.1305, -0.1365, -0.4323],
       device='cuda:0', grad_fn=<CatBackward>)
```

--------------------------------

### Configure and Train English NER Model with Stacked Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Sets up a corpus, tag dictionary, and stacked embeddings (WordEmbeddings, FlairEmbeddings) for English NER. It then initializes and runs the ModelTrainer.

```python
from flair.data import Corpus
from flair.datasets import WNUT_17
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings, FlairEmbeddings
from typing import List

# 1. get the corpus
corpus: Corpus = WNUT_17()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# initialize embeddings
embedding_types: list[TokenEmbeddings] = [
    WordEmbeddings('crawl'),
    WordEmbeddings('twitter'),
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

# initialize sequence tagger
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type)

# initialize trainer
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.train('resources/taggers/example-ner',
              train_with_dev=True,
              max_epochs=150)
```

--------------------------------

### Initialize Stacked Embeddings with Word and BytePair Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/embeddings/FASTTEXT_EMBEDDINGS.md

This snippet demonstrates an alternative approach using stacked embeddings. It combines standard English WordEmbeddings with BytePairEmbeddings to provide a smaller alternative to FastText embeddings, while still offering OOV functionality.

```python
from flair.embeddings import WordEmbeddings, BytePairEmbeddings, StackedEmbeddings

# init embedding
embedding = StackedEmbeddings(
    [
        # standard FastText word embeddings for English
        WordEmbeddings('en'),
        # Byte pair embeddings for English
        BytePairEmbeddings('en'),
    ]
)

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
embedding.embed(sentence)
```

--------------------------------

### Get Structured Entity Information

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/tagging.md

Retrieves detailed information about predicted entities in a sentence, including text, start/end positions, and label confidence, using the to_dict() method.

```python
print(sentence.to_dict())
```

--------------------------------

### Initialize EntityMentionLinker with Custom Dictionary

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR2_TUTORIAL_4_CUSTOMIZE_LINKING.md

Initializes a new EntityMentionLinker model using a pre-trained model and a custom dictionary. If no dictionary is provided, the default one for the entity type is loaded.

```python
from flair.models import EntityMentionLinker

pretrained_model="cambridgeltl/SapBERT-from-PubMedBERT-fulltext"
linker = EntityMentionLinker.build(
                pretrained_model,
                dictionary=dictionary,
                hybrid_search=False, 
                entity_type="disease",
            )
```

--------------------------------

### Get Disease Entity Dictionary

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR_TUTORIAL_1_TAGGING.md

Obtain a dictionary representation of disease entities within a sentence. This provides the text, entity type, and confidence scores for each predicted disease span.

```python
print(sentence.to_dict("hunflair-disease"))
```

--------------------------------

### Configure and Train English NER Model with FastText Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Sets up a corpus, tag dictionary, and stacked embeddings (WordEmbeddings, FlairEmbeddings) for English NER using FastText. It then initializes and runs the ModelTrainer.

```python
from flair.data import Corpus
from flair.datasets import ONTONOTES
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings, FlairEmbeddings
from typing import List

# 1. get the corpus
corpus: Corpus = ONTONOTES()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# initialize embeddings
embedding_types: list[TokenEmbeddings] = [
    WordEmbeddings('crawl'),
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

# initialize sequence tagger
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type)
```

--------------------------------

### Use SciSpaCy Tokenizer for Biomedical Text

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR2_TUTORIAL_1_TAGGING.md

Instantiate a Flair Sentence object using the SciSpacyTokenizer to improve tokenization accuracy for biomedical texts. Ensure SciSpaCy and its models are installed first.

```python
from flair.tokenization import SciSpacyTokenizer

sentence = Sentence("Behavioral abnormalities in the Fmr1 KO2 Mouse Model of Fragile X Syndrome",
                    use_tokenizer=SciSpacyTokenizer())
```

--------------------------------

### Initialize EntityMentionLinker with Custom Dictionary

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/customize-linking.md

Initializes a new EntityMentionLinker model using a pre-trained model and a custom dictionary. Set `hybrid_search=False` to disable hybrid search.

```python
from flair.models import EntityMentionLinker
pretrained_model="cambridgeltl/SapBERT-from-PubMedBERT-fulltext"
linker = EntityMentionLinker.build(
                pretrained_model,
                dictionary=dictionary,
                hybrid_search=False, 
                entity_type="disease",
            )
```

--------------------------------

### Train NER with Flair Embeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-to-train-sequence-tagger.md

Train a classic NER model using a combination of Flair and GloVe embeddings with an LSTM-CRF. This setup requires less GPU memory than transformer fine-tuning.

```python
from flair.datasets import CONLL_03
from flair.embeddings import WordEmbeddings, FlairEmbeddings, StackedEmbeddings
from flair.models import SequenceTagger
from flair.trainers import ModelTrainer

# 1. get the corpus
corpus = CONLL_03()
print(corpus)

# 2. what label do we want to predict?
label_type = 'ner'

# 3. make the label dictionary from the corpus
label_dict = corpus.make_label_dictionary(label_type=label_type, add_unk=False)
print(label_dict)

# 4. initialize embedding stack with Flair and GloVe
embedding_types = [
    WordEmbeddings('glove'),
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
tagGER = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=label_dict,
                        tag_type=label_type)

# 6. initialize trainer
trainer = ModelTrainer(tagger, corpus)

# 7. start training
trainer.train('resources/taggers/sota-ner-flair',
              learning_rate=0.1,
              mini_batch_size=32,
              max_epochs=150)
```

--------------------------------

### Optimize ONNX Model

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_PRODUCTION_FASTER_TRANSFORMERS.md

Applies hardware-specific optimizations to the exported ONNX model. This creates an optimized ONNX file and potentially additional files if `use_external_data_format` is true. Requires `onnx` and `coloredlogs` to be installed.

```python
model.embeddings.optimize_model(
            "flert-optimized-embeddings.onnx", opt_level=2, use_gpu=True, only_onnxruntime=True, use_external_data_format=True,
        )
```

--------------------------------

### Configure and Train Dutch NER Model with Transformer Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Sets up a corpus, tag dictionary, TransformerWordEmbeddings, and SequenceTagger for Dutch NER. It then initializes and runs the ModelTrainer.

```python
from flair.data import Corpus
from flair.datasets import CONLL_03_DUTCH
from flair.embeddings import TransformerWordEmbeddings
from flair.models import SequenceTagger
from flair.trainers import ModelTrainer

# 1. get the corpus
corpus: Corpus = CONLL_03_DUTCH()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# initialize embeddings
embeddings = TransformerWordEmbeddings('wietsedv/bert-base-dutch-cased', allow_long_sentences=True)

# initialize sequence tagger
tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type)

# initialize trainer
trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.train('resources/taggers/example-ner',
              train_with_dev=True,
              max_epochs=150)
```

--------------------------------

### Get Default Entity Label Types

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/customize-linking.md

Retrieves the default entity label types that a linker model is configured to process. This shows which NER annotations the linker will consider for entity linking by default.

```python
print(gene_linker.entity_label_types)
print(disease_linker.entity_label_types)
```

--------------------------------

### Build Custom Entity Mention Linker

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR_TUTORIAL_3_ENTITY_LINKING.md

Demonstrates how to build a custom entity mention linker using a specified model and dictionary, or a custom model with a pre-defined dictionary.

```python
from flair.models import EntityMentionLinker

nen_tagger = EntityMentionLinker.build("name_or_path_to_your_model",
                                       dictionary_names_or_path="name_or_path_to_your_dictionary")
nen_tagger = EntityMentionLinker.build("path_to_custom_disease_model", dictionary_names_or_path="disease")
```

--------------------------------

### Run Integration Tests

Source: https://github.com/flairnlp/flair/blob/master/CONTRIBUTING.md

Executes integration tests for the Flair library, which may involve training small models and take longer. It's recommended to ensure basic tests pass first.

```bash
pytest --runintegration
```

--------------------------------

### Initialize and Use PooledFlairEmbeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/flair-embeddings.md

Instantiate and use PooledFlairEmbeddings, which manage a global representation of words by pooling past occurrences. Be aware of their high memory requirements.

```python
from flair.embeddings import PooledFlairEmbeddings

# init embedding
flair_embedding_forward = PooledFlairEmbeddings('news-forward')

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
flair_embedding_forward.embed(sentence)
```

--------------------------------

### Load and Print Corpus

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Loads the English Universal Dependencies corpus and downsamples it to 10% of its original size. Prints the corpus details to show the number of sentences in each split.

```python
from flair.datasets import UD_ENGLISH

# 1. load the corpus
corpus = UD_ENGLISH().downsample(0.1)
print(corpus)
```

--------------------------------

### Tweet Data Format for Joy Intensity

Source: https://github.com/flairnlp/flair/blob/master/tests/resources/tasks/regression/README.md

This is the expected format for tweet data when used for joy intensity regression tasks. Each line should start with a label indicating the joy intensity, followed by the tweet text.

```text
__label__<joy_intensity> <text>
```

--------------------------------

### Creating a Sentence from Pretokenized Words

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_BASICS_TOKENIZATION.md

Demonstrates how to initialize a Sentence object directly from a list of pretokenized words, bypassing the need for automatic tokenization.

```python
from flair.data import Sentence
sentence = Sentence(['The', 'grass', 'is', 'green', '.'])
print(sentence)
```

--------------------------------

### Console Output of Entity Linking

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/entity-mention-linking.md

Example console output demonstrating the results of entity mention linking, showing identified spans, their predicted entity types, and associated scores or knowledge base identifiers.

```console
Span[4:5]: "ABCD1" → Gene (0.9509)
Span[7:11]: "X-linked adrenoleukodystrophy" → Disease (0.9872)
Span[7:11]: "X-linked adrenoleukodystrophy" → MESH:D000326/name=Adrenoleukodystrophy (195.30780029296875)
Span[13:15]: "neurodegenerative disease" → Disease (0.8988)
Span[13:15]: "neurodegenerative disease" → MESH:D019636/name=Neurodegenerative Diseases (201.1804962158203)
Span[29:30]: "mercury" → Chemical (0.9484)
Span[31:32]: "dolphin" → Species (0.8071)
```

--------------------------------

### Predict Tags with a Trained Flair Model

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-model-training-works.md

Load a trained model using `SequenceTagger.load()` and then use the `.predict()` method on a `Sentence` object to get tag predictions. The predicted tags can be viewed using `sentence.to_tagged_string()`.

```python
# load the model you trained
model = SequenceTagger.load('resources/taggers/example-upos/final-model.pt')

# create example sentence
sentence = Sentence('I love Berlin')

# predict tags and print
model.predict(sentence)

print(sentence.to_tagged_string())
```

--------------------------------

### Create and Apply TorchScript Embeddings

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_PRODUCTION_FASTER_TRANSFORMERS.md

This snippet demonstrates how to create a TorchScript traced module from a `JitWrapper`, and then replace the original embeddings with `TransformerJitWordEmbeddings` for optimized inference. The optimized model is then saved.

```python
from flair.embeddings import TransformerJitWordEmbeddings

# create a JitWrapper
wrapper = JitWrapper(model.embeddings)

# create the parameters that will be passed to the jit model in the right order.
parameter_names, parameter_list = TransformerJitWordEmbeddings.parameter_to_list(model.embeddings, wrapper, sentences)

# create the script module
script_module = torch.jit.trace(wrapper, parameter_list)

# replace the embeddings with jit embeddings.
model.embeddings = TransformerJitWordEmbeddings.create_from_embedding(script_module, model.embeddings, parameter_names)

model.save("flert_jit.pt")
```

--------------------------------

### Biomedical Named Entity Normalization (NEN)

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/HUNFLAIR2.md

Perform Named Entity Normalization by linking recognized entities to standardized ontologies using specialized linker models. This example demonstrates linking genes, diseases, and species.

```python
from flair.data import Sentence
from flair.models import EntityMentionLinker
from flair.nn import Classifier

# make a sentence
sentence = Sentence("Behavioral abnormalities in the Fmr1 KO2 Mouse Model of Fragile X Syndrome")

# load biomedical NER tagger + predict entities
tagger = Classifier.load("hunflair2")
tagger.predict(sentence)

# load gene linker and perform normalization
gene_linker = EntityMentionLinker.load("gene-linker")
gene_linker.predict(sentence)

# load disease linker and perform normalization
disease_linker = EntityMentionLinker.load("disease-linker")
disease_linker.predict(sentence)

# load species linker and perform normalization
species_linker = EntityMentionLinker.load("species-linker")
species_linker.predict(sentence)
```

```python
for entity in sentence.get_labels("link"):
    print(entity)
```

--------------------------------

### Export Transformer Embeddings to ONNX

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/TUTORIAL_PRODUCTION_FASTER_TRANSFORMERS.md

Exports the model's transformer embeddings to an ONNX file. This replaces the original embeddings with TransformerOnnxEmbeddings, enabling faster predictions using ONNX Runtime. Ensure onnxruntime and an execution provider are installed.

```python
model.embeddings = model.embeddings.export_onnx("flert-embeddings.onnx", sentences, providers=["CUDAExecutionProvider", "CPUExecutionProvider"], session_options={})
```

--------------------------------

### Initialize MultiCorpus with Multiple Universal Dependency Treebanks

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-training/how-to-load-prepared-dataset.md

Combine English, German, and Dutch Universal Dependency Treebanks into a single MultiCorpus object for multi-task training. This allows training models on a diverse set of treebanks simultaneously.

```python
from flair.datasets import UD_ENGLISH, UD_GERMAN, UD_DUTCH
english_corpus = UD_ENGLISH()
german_corpus = UD_GERMAN()
dutch_corpus = UD_DUTCH()

# make a multi corpus consisting of three UDs
from flair.data import MultiCorpus
multi_corpus = MultiCorpus([english_corpus, german_corpus, dutch_corpus])
```

--------------------------------

### Initialize RoBERTa Document Embeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/transformer-embeddings.md

Loads a RoBERTa transformer model for document embeddings. Requires the flair.embeddings and flair.data modules.

```python
from flair.embeddings import TransformerDocumentEmbeddings

# init embedding
embedding = TransformerDocumentEmbeddings('roberta-base')

# create a sentence
sentence = Sentence('The grass is green .')

# embed words in sentence
embedding.embed(sentence)
```

--------------------------------

### Configure and Train POS Tagger with Universal Dependencies Corpus

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/EXPERIMENTS.md

Sets up a Flair sequence tagger for Part-of-Speech tagging using the Universal Dependencies corpus. It configures embeddings and trains the model for a specified number of epochs.

```python
from flair.data import Corpus
from flair.datasets import UniversalDependenciesCorpus
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings, FlairEmbeddings
from typing import List

# 1. get the corpus
corpus: Corpus = UniversalDependenciesCorpus(base_path='/path/to/penn')

# 2. what tag do we want to predict?
tag_type = 'pos'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# initialize embeddings
embedding_types: list[TokenEmbeddings] = [
    WordEmbeddings('extvec'),
    FlairEmbeddings('news-forward'),
    FlairEmbeddings('news-backward'),
]

embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

# initialize sequence tagger
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type)
# initialize trainer
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

trainer.train('resources/taggers/example-pos',
              train_with_dev=True,
              max_epochs=150)
```

--------------------------------

### Initialize FastText Embeddings with Remote URL

Source: https://github.com/flairnlp/flair/blob/master/resources/docs/embeddings/FASTTEXT_EMBEDDINGS.md

Initialize FastText embeddings by specifying a remote downloadable URL. The `use_local=False` parameter indicates that the model should be downloaded and used from the provided URL.

```python
embedding = FastTextEmbeddings('/path/to/remote/downloadable/custom_fasttext_embeddings.bin', use_local=False)
```

--------------------------------

### Initialize Prefixed Sequence Tagger

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-hunflair2/training-ner-models.md

Sets up a PrefixedSequenceTagger for NER tasks. Requires embeddings, tag dictionary, and tag format. Options include hidden size, CRF, RNN, reproject embeddings, and augmentation strategy.

```python
# 4. initialize sequence tagger
from flair.models.prefixed_tagger import PrefixedSequenceTagger

tagger: SequenceTagger = PrefixedSequenceTagger(
    hidden_size=256,
    embeddings=embeddings,
    tag_dictionary=tag_dictionary,
    tag_format="BIOES",
    tag_type="ner",
    use_crf=False,
    use_rnn=False,
    reproject_embeddings=False,
    augmentation_strategy=augmentation_strategy,
)
```

--------------------------------

### Create a Flair Sentence Object

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/basic-types.md

Instantiate a Sentence object by passing a string. This object holds the text and is automatically tokenized.

```python
from flair.data import Sentence

sentence = Sentence('The grass is green.')

print(sentence)
```

--------------------------------

### Combine Embeddings using StackedEmbeddings

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-embeddings/embeddings.md

Instantiate StackedEmbeddings by passing a list of desired embeddings (e.g., GloVe, Flair forward/backward) to combine them. Requires the flair library.

```python
from flair.embeddings import WordEmbeddings, FlairEmbeddings

# init standard GloVe embedding
glove_embedding = WordEmbeddings('glove')

# init Flair forward and backwards embeddings
flair_embedding_forward = FlairEmbeddings('news-forward')
flair_embedding_backward = FlairEmbeddings('news-backward')
```

```python
from flair.embeddings import StackedEmbeddings

# create a StackedEmbedding object that combines glove and forward/backward flair embeddings
stacked_embeddings = StackedEmbeddings([
                                        glove_embedding,
                                        flair_embedding_forward,
                                        flair_embedding_backward,
                                       ])
```

```python
sentence = Sentence('The grass is green .')

# just embed a sentence using the StackedEmbedding as you would with any single embedding.
stacked_embeddings.embed(sentence)

# now check out the embedded tokens.
for token in sentence:
    print(token)
    print(token.embedding)
```

--------------------------------

### Load and Use Relation Extraction Model with Entity Tagger

Source: https://github.com/flairnlp/flair/blob/master/docs/tutorial/tutorial-basics/other-models.md

Demonstrates loading an entity tagger ('ner-fast') and a relation extractor ('relations') to identify relationships between entities in a sentence. Requires both models to be loaded sequentially.

```python
from flair.data import Sentence
from flair.nn import Classifier

# 1. make example sentence
sentence = Sentence("George was born in Washington")

# 2. load entity tagger and predict entities
tagGER = Classifier.load('ner-fast')
TAGGER.predict(sentence)

# check which named entities have been found in the sentence
entities = sentence.get_labels('ner')
for entity in entities:
    print(entity)

# 3. load relation extractor
EXTRACTOR = Classifier.load('relations')

# predict relations
EXTRACTOR.predict(sentence)

# check which relations have been found
relations = sentence.get_labels('relation')
for relation in relations:
    print(relation)
print("")

# Use the `get_labels()` method with parameter 'relation' to iterate over all relation predictions. 
for label in sentence.get_labels('relation'):
    print(label)
```