SpanMarker

https://tomaarsen.github.io/SpanMarkerNER/
Admin
SpanMarker is a framework for training powerful Named Entity Recognition models using familiar...
Tokens:123,941
Snippets:977
Trust Score:9.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Install span_marker

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Installs the span_marker Python module. The commented-out line shows how to install it with Weights and Biases support.

```python
%pip install span_marker
# %pip install span_marker[wandb]
```

--------------------------------

### Training Output Example

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

Example of the output from the training process, showing loss, learning rate, and epoch.

```python
TrainOutput(global_step=1227, training_loss=0.056268237500824186, metrics={'train_runtime': 450.609, 'train_samples_per_second': 21.788, 'train_steps_per_second': 2.723, 'train_loss': 0.056268237500824186, 'epoch': 1.0})
```

--------------------------------

### Training Log Output Example

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

This snippet provides an example of the log output during the training process, showing loss, learning rate, and epoch information, as well as evaluation metrics.

```json
{"loss": 0.6974, "learning_rate": 1.991869918699187e-05, "epoch": 0.04}
{"loss": 0.0896, "learning_rate": 4.0243902439024395e-05, "epoch": 0.08}
{"loss": 0.0584, "learning_rate": 4.8822463768115946e-05, "epoch": 0.12}
{"loss": 0.0382, "learning_rate": 4.655797101449276e-05, "epoch": 0.16}
{"eval_loss": 0.03181104362010956, "eval_overall_precision": 0.6967930029154519, "eval_overall_recall": 0.5989974937343359, "eval_overall_f1": 0.6442048517520216, "eval_overall_accuracy": 0.8993717106605198, "eval_runtime": 29.16, "eval_samples_per_second": 83.985, "eval_steps_per_second": 21.022, "epoch": 0.16}
{"loss": 0.0333, "learning_rate": 4.429347826086957e-05, "epoch": 0.2}
{"loss": 0.0303, "learning_rate": 4.202898550724638e-05, "epoch": 0.24}
{"loss": 0.032, "learning_rate": 3.976449275362319e-05, "epoch": 0.29}
{"loss": 0.0304, "learning_rate": 3.7500000000000003e-05, "epoch": 0.33}
{"eval_loss": 0.02394717186689377, "eval_overall_precision": 0.7350157728706624, "eval_overall_recall": 0.7187198766146135, "eval_overall_f1": 0.7267764889365436, "eval_overall_accuracy": 0.9227489698502713, "eval_runtime": 29.481, "eval_samples_per_second": 83.07, "eval_steps_per_second": 20.793, "epoch": 0.33}
{"loss": 0.0265, "learning_rate": 3.5235507246376816e-05, "epoch": 0.37}
{"loss": 0.0254, "learning_rate": 3.297101449275363e-05, "epoch": 0.41}
{"loss": 0.0249, "learning_rate": 3.0706521739130435e-05, "epoch": 0.45}
{"loss": 0.0242, "learning_rate": 2.8442028985507245e-05, "epoch": 0.49}
{"eval_loss": 0.02163967303931713, "eval_overall_precision": 0.762808736476832, "eval_overall_recall": 0.7204549836128783, "eval_overall_f1": 0.7410271663692247, "eval_overall_accuracy": 0.9293582473175309, "eval_runtime": 29.0261, "eval_samples_per_second": 84.372, "eval_steps_per_second": 21.119, "epoch": 0.49}
{"loss": 0.0224, "learning_rate": 2.6177536231884058e-05, "epoch": 0.53}
{"loss": 0.0242, "learning_rate": 2.391304347826087e-05, "epoch": 0.57}
{"loss": 0.0226, "learning_rate": 2.164855072463768e-05, "epoch": 0.61}
{"loss": 0.0245, "learning_rate": 1.9384057971014493e-05, "epoch": 0.65}
{"eval_loss": 0.020556513220071793, "eval_overall_precision": 0.7680876026593665, "eval_overall_recall": 0.7572778099093889, "eval_overall_f1": 0.7626444034559751, "eval_overall_accuracy": 0.9338052303047611, "eval_runtime": 29.7545, "eval_samples_per_second": 82.307, "eval_steps_per_second": 20.602, "epoch": 0.65}
{"loss": 0.0231, "learning_rate": 1.7119565217391306e-05, "epoch": 0.69}
{"loss": 0.0209, "learning_rate": 1.4855072463768116e-05, "epoch": 0.73}
{"loss": 0.0202, "learning_rate": 1.2590579710144929e-05, "epoch": 0.77}
{"loss": 0.0212, "learning_rate": 1.032608695652174e-05, "epoch": 0.81}
{"eval_loss": 0.01960749179124832, "eval_overall_precision": 0.7743021183923976, "eval_overall_recall": 0.7540003855793329, "eval_overall_f1": 0.7640164094549716, "eval_overall_accuracy": 0.9358247317530904, "eval_runtime": 29.6794, "eval_samples_per_second": 82.515, "eval_steps_per_second": 20.654, "epoch": 0.81}
{"loss": 0.0202, "learning_rate": 8.061594202898551e-06, "epoch": 0.86}
{"loss": 0.0196, "learning_rate": 5.797101449275362e-06, "epoch": 0.9}
{"loss": 0.0232, "learning_rate": 3.5326086956521736e-06, "epoch": 0.94}
```

--------------------------------

### Loading a model from a local checkpoint

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Demonstrates how to load a previously saved SpanMarker model from a local checkpoint.

```python
# model = SpanMarkerModel.from_pretrained("models/span-marker-bert-base-fewnerd-coarse-super/checkpoint-final")
```

--------------------------------

### Loading a Model from a Local Checkpoint

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

This code snippet illustrates how to load a SpanMarkerNER model from a local checkpoint.

```python
# This is how it would be done using a local checkpoint.

```

--------------------------------

### Initialize SpanMarker Trainer

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Initializes the SpanMarker Trainer with the configured model, training arguments, and subsets of the training and evaluation datasets.

```python
from span_marker import Trainer

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=dataset["train"].select(range(8000)),
    eval_dataset=dataset["validation"].select(range(2000)),
)
```

--------------------------------

### Install span_marker

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/model_loading.html

Installs the span_marker Python module using pip.

```bash
%pip install span_marker

```

--------------------------------

### Full training script

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

A complete Python script for training a SpanMarker model on the few-NERD dataset, including data loading, model configuration, training arguments, and trainer setup.

```python
from datasets import load_dataset
from span_marker import SpanMarkerModel, SpanMarkerModelCardData, Trainer
from transformers import TrainingArguments

def main():
    dataset_id = "DFKI-SLT/few-nerd"
    dataset = load_dataset(dataset_id, "supervised")
    labels = dataset["train"].features["ner_tags"].feature.names

    encoder_id = "bert-base-cased"
    model = SpanMarkerModel.from_pretrained(
        # Required arguments
        encoder_id,
        labels=labels,
        # Optional arguments
        model_max_length=256,
        entity_max_length=8,
        # To improve the generated model card
        model_card_data=SpanMarkerModelCardData(
            language=["en"],
            license="cc-by-sa-4.0",
            encoder_id=encoder_id,
            dataset_id=dataset_id,
        )
    )

    args = TrainingArguments(
        output_dir="models/span-marker-bert-base-fewnerd-coarse-super",
        learning_rate=5e-5,
        gradient_accumulation_steps=2,
        per_device_train_batch_size=4,
        per_device_eval_batch_size=4,
        num_train_epochs=1,
        eval_strategy="steps",  # Or `evaluation_strategy` if your `transformers<4.54.1`
        save_strategy="steps",
        eval_steps=200,
        push_to_hub=False,
        logging_steps=50,
        fp16=True,
        warmup_ratio=0.1,
        dataloader_num_workers=2,
    )

    trainer = Trainer(
        model=model,
        args=args,
        train_dataset=dataset["train"].select(range(8000)),
        eval_dataset=dataset["validation"].select(range(2000)),
    )
    trainer.train()

    metrics = trainer.evaluate()
    print(metrics)

    trainer.save_model("models/span-marker-bert-base-fewnerd-coarse-super/checkpoint-final")

if __name__ == "__main__":
    main()

```

--------------------------------

### Install span_marker

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/model_loading.ipynb.txt

Installs the span_marker Python module using pip.

```python
%pip install span_marker
```

--------------------------------

### Training the model

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

This code snippet shows the output of training a SpanMarkerNER model, including loss, learning rate, and evaluation metrics per epoch.

```python
trainer.train()
```

--------------------------------

### Install necessary libraries

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/spacy_integration.ipynb.txt

Installs the `span_marker` and `spacy` Python modules, and downloads a spaCy English model.

```bash
%pip install span_marker spacy
!spacy download en_core_web_sm
```

--------------------------------

### Define Training Arguments

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Sets up training arguments for the SpanMarker model using transformers.TrainingArguments, including output directory, learning rate, batch sizes, number of epochs, evaluation and saving strategies, logging steps, mixed precision, and warmup ratio.

```python
from transformers import TrainingArguments

args = TrainingArguments(
    output_dir="models/span-marker-bert-base-fewnerd-coarse-super",
    learning_rate=5e-5,
    gradient_accumulation_steps=2,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    num_train_epochs=1,
    eval_strategy="steps",  # Or `evaluation_strategy` if your `transformers<4.54.1`="steps",
    save_strategy="steps",
    eval_steps=200,
    push_to_hub=False,
    logging_steps=50,
    fp16=True,
    warmup_ratio=0.1,
    dataloader_num_workers=2,
)
```

--------------------------------

### Example Selection Logic

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/span_marker/model_card.html

This code snippet selects example data points from a dataset. It first takes a random subset of a specified size, then maps a function to count unique entities, sorts the subset by the entity count, and finally selects a specified number of examples. It also detokenizes the selected examples and identifies the shortest one for prediction.

```python
# Out of `sample_subset_size=100` random samples, select `example_count=5` good examples
# based on the number of unique entity classes.
# The shortest example is used in the inference example
sample_subset_size = 100
example_count = 5
if len(dataset) > sample_subset_size:
    example_dataset = dataset.select(random.sample(range(len(dataset)), k=sample_subset_size))
else:
    example_dataset = dataset

def count_entities(sample: Dict[str, Any]) -> Dict[str, int]:
    unique_count = {reduced_label_id for reduced_label_id, _, _ in sample["ner_tags"]}
    return {"unique_entity_count": len(unique_count)}

example_dataset = (
    example_dataset.map(count_entities)
    .sort(("unique_entity_count", "entity_count"), reverse=True)
    .select(range(min(len(example_dataset), example_count)))
    .map(map_detokenize, input_columns="tokens")
)
self.widget = [{"text": sample["text"]} for sample in example_dataset]

shortest_example = example_dataset.sort("word_count")[0]["text"]
self.predict_example = shortest_example
```

--------------------------------

### Label Examples Generation

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/span_marker/model_card.html

This method generates example strings for each named entity recognition label. It iterates through the dataset, extracts entities, and collects a specified number of unique examples for each label. The output is a list of dictionaries, where each dictionary contains a 'Label' and a comma-separated string of 'Examples'.

```python
num_examples_per_label = 3
examples = {label: set() for label_id, label in id2label.items() if label_id != outside_id}
unfinished_entity_ids = set(id2label.keys()) - {outside_id}
for sample in dataset:
    for entity_id, start, end in sample["ner_tags"]:
        if entity_id in unfinished_entity_ids:
            entity = id2label[entity_id]
            example = " ".join(sample["tokens"][start:end])
            examples[entity].add(f'"{example}"')
            if len(examples[entity]) >= num_examples_per_label:
                unfinished_entity_ids.remove(entity_id)
    if not unfinished_entity_ids:
        break
self.label_example_list = [
    {"Label": label, "Examples": ", ".join(example_set)} for label, example_set in examples.items()
]
```

--------------------------------

### Inspect dataset samples

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Iterates through the first three samples of the training set and prints them to show the data structure.

```python
for sample in dataset["train"].select(range(3)):
    print(sample)
```

--------------------------------

### EntityTracker Example

Source: https://tomaarsen.github.io/SpanMarkerNER/api/span_marker.tokenizer.html

An example demonstrating how the EntityTracker class provides warnings about skipped entities due to length constraints.

```text
This SpanMarker model won't be able to predict 5.930931% of all annotated entities in the evaluation dataset.
This is caused by the SpanMarkerModel maximum entity length of 6 words and the maximum model input length of 64 tokens.
These are the frequencies of the missed entities due to maximum entity length out of 1332 total entities:
- 7 missed entities with 7 words (0.525526%)
- 2 missed entities with 8 words (0.150150%)
- 2 missed entities with 9 words (0.150150%)
- 2 missed entities with 13 words (0.150150%)
Additionally, a total of 66 (4.954955%) entities were missed due to the maximum input length.

```

--------------------------------

### SpanMarkerModel Example

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/span_marker/model_card.html

Example of creating a SpanMarkerModel with various configurations and model card data.

```python
>>> model = SpanMarkerModel.from_pretrained(
...     "bert-base-uncased",
...     labels=["O", "B-DNA", "I-DNA", "B-protein", ...],
...     # SpanMarker hyperparameters:
...     model_max_length=256,
...     marker_max_length=128,
...     entity_max_length=8,
...     # Model card variables
...     model_card_data=SpanMarkerModelCardData(
...         model_id="tomaarsen/span-marker-bbu-bionlp",
...         encoder_id="bert-base-uncased",
...         dataset_name="BioNLP2004,
...         dataset_id="tner/bionlp2004",
...         license="apache-2.0",
...         language="en",
...     ),
... )
    
```

--------------------------------

### SpanMarkerModel.from_pretrained example

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/span_marker/modeling.html

Example of how to provide a `labels` list to `SpanMarkerModel.from_pretrained()`.

```python
SpanMarkerModel.from_pretrained(
    "{pretrained_model_name_or_path}",
    labels=["O", "B-PER", "I-PER", "B-ORG", "I-ORG", ...]
)
or
SpanMarkerModel.from_pretrained(
    "{pretrained_model_name_or_path}",
    labels=["O", "PER", "ORG", "LOC", "MISC"]
)
```

--------------------------------

### SpanMarkerModel Configuration Example

Source: https://tomaarsen.github.io/SpanMarkerNER/api/span_marker.configuration.html

Example demonstrating how to provide configuration settings via kwargs to `SpanMarkerModel.from_pretrained`.

```python
# These configuration settings are provided via kwargs to `SpanMarkerModel.from_pretrained`:
model = SpanMarkerModel.from_pretrained(
    "bert-base-cased",
    labels=labels,
    model_max_length=256,
    marker_max_length=128,
    entity_max_length=8,
)

```

--------------------------------

### SpanMarkerPipeline Example

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/span_marker/pipeline_component.html

This example demonstrates how to use the SpanMarkerPipeline for named entity recognition.

```python
from span_marker import pipeline
pipe = pipeline(task="span-marker", model="tomaarsen/span-marker-mbert-base-multinerd", device_map="auto")
pipe("Amelia Earhart flew her single engine Lockheed Vega 5B across the Atlantic to Paris.")
```

--------------------------------

### Install SpanMarker via pip

Source: https://tomaarsen.github.io/SpanMarkerNER/install.html

The command to install the span_marker Python module using pip.

```bash
pip install span_marker

```

--------------------------------

### How to Train

Source: https://tomaarsen.github.io/SpanMarkerNER/index.html

Example code demonstrating how to train a SpanMarker model using the Hugging Face datasets and transformers libraries.

```python
from datasets import load_dataset
from span_marker import SpanMarkerModel, Trainer
from transformers import TrainingArguments

# The dataset labels can have a tagging schemed (IOB, IOB2, BIOES),
# but that is not necessary. This dataset has no tagging scheme:
dataset = load_dataset("DFKI-SLT/few-nerd", "supervised")
labels = ["O", "art", "building", "event", "location", "organization", "other", "person", "product"]

# Initialize a SpanMarkerModel using an encoder, e.g. BERT, and the labels:
encoder_id = "bert-base-cased"
model = SpanMarkerModel.from_pretrained(encoder_id, labels=labels)

# See the 🤗 TrainingArguments documentation for details here
args = TrainingArguments(
    output_dir="my_span_marker_model",
    learning_rate=5e-5,
    gradient_accumulation_steps=2,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    num_train_epochs=1,
    save_strategy="steps",
    eval_steps=200,
    logging_steps=50,
    warmup_ratio=0.1,
)

# Our Trainer subclasses the 🤗 Trainer, and the usage is very similar
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=dataset["train"].select(range(8000)),
    eval_dataset=dataset["validation"].select(range(2000)),
)

# Training is really simple using our Trainer!
trainer.train()

# ... and so is evaluating!
metrics = trainer.evaluate()
print(metrics)

# Save the model locally or on the Hugging Face Hub
trainer.save_model("my_span_marker_model/checkpoint-final")
trainer.push_to_hub("my_span_marker_model/checkpoint-final")

```

--------------------------------

### Evaluation Metrics

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

Example of the evaluation metrics for the model on the validation set.

```json
{
 'eval_loss': 0.019375888630747795,
 'eval_art': {'precision': 0.7661290322580645, 'recall': 0.7723577235772358, 'f1': 0.7692307692307692, 'number': 246},
 'eval_building': {'precision': 0.5842293906810035, 'recall': 0.6127819548872181, 'f1': 0.5981651376146789, 'number': 266},
 'eval_event': {'precision': 0.5497382198952879, 'recall': 0.5965909090909091, 'f1': 0.5722070844686648, 'number': 176},
 'eval_location': {'precision': 0.8036732108929703, 'recall': 0.8409542743538767, 'f1': 0.8218911917098446, 'number': 1509},
 'eval_organization': {'precision': 0.7474226804123711, 'recall': 0.6998069498069498, 'f1': 0.7228315054835494, 'number': 1036},
 'eval_other': {'precision': 0.6775818639798489, 'recall': 0.5604166666666667, 'f1': 0.61345496009122, 'number': 480},
 'eval_person': {'precision': 0.8636363636363636, 'recall': 0.9063313096270599, 'f1': 0.8844688954718578, 'number': 1153},
 'eval_product': {'precision': 0.7366666666666667, 'recall': 0.6884735202492211, 'f1': 0.7117552334943639, 'number': 321},
 'eval_overall_precision': 0.7705836876691148,
 'eval_overall_recall': 0.7686524002313476,
 'eval_overall_f1': 0.7696168323520897,
 'eval_overall_accuracy': 0.9381502182693484,
 'eval_runtime': 28.5583,
 'eval_samples_per_second': 85.754,
 'eval_steps_per_second': 21.465,
 'epoch': 1.0
}
```

--------------------------------

### Test Set Evaluation Metrics

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

Example of the evaluation metrics for the model on the test set.

```json
{
 'test_loss': 0.01918497122824192,
 'test_art': {'precision': 0.7419354838709677, 'recall': 0.7488372093023256, 'f1': 0.7453703703703703, 'number': 215},
 'test_building': {'precision': 0.6236559139784946, 'recall': 0.710204081632653, 'f1': 0.6641221374045801, 'number': 245},
 'test_event': {'precision': 0.6153846153846154, 'recall': 0.5529953917050692, 'f1': 0.5825242718446603, 'number': 217},
 'test_location': {'precision': 0.812192118226601, 'recall': 0.8515171078114913, 'f1': 0.8313898518751971, 'number': 1549},
 'test_organization': {'precision': 0.7320754716981132, 'recall': 0.6897777777777778, 'f1': 0.7102974828375286, 'number': 1125},
 'test_other': {'precision': 0.7375886524822695, 'recall': 0.6328600405679513, 'f1': 0.6812227074235807, 'number': 493},
 'test_person': {'precision': 0.8805309734513275, 'recall': 0.9061930783242259, 'f1': 0.8931777378815081, 'number': 1098},
 'test_product': {'precision': 0.6641221374045801, 'recall': 0.5898305084745763, 'f1': 0.6247755834829445, 'number': 295},
 'test_overall_precision': 0.7766859344894027,
 'test_overall_recall': 0.7697154859652473,
 'test_overall_f1': 0.7731850004795243,
 'test_overall_accuracy': 0.938954021816699,
 'test_runtime': 29.8808,
 'test_samples_per_second': 81.658,
 'test_steps_per_second': 20.414,
 'epoch': 1.0
}
```

--------------------------------

### How to predict

Source: https://tomaarsen.github.io/SpanMarkerNER/index.html

Example code demonstrating how to load a finetuned SpanMarkerModel for prediction.

```python
from span_marker import SpanMarkerModel

# Load a finetuned SpanMarkerModel from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-fewnerd-fine-super")

```

--------------------------------

### Model Prediction Method

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

Explanation of the `model.predict` method and its accepted input types.

```python
model.predict
```

--------------------------------

### Save the trained model locally

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Saves the trained SpanMarkerNER model to a local directory.

```python
trainer.save_model("models/span-marker-bert-base-fewnerd-coarse-super/checkpoint-final")
```

--------------------------------

### Optimizer setup and SageMaker MP integration

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/transformers/trainer.html

This snippet shows how the optimizer is set up, including potential overrides for bitsandbytes and integration with SageMaker's DistributedOptimizer if enabled.

```python
                        skipped += sum({p.data_ptr(): p.numel() for p in module.parameters()}.values())
                        logger.info(f"skipped {module}: {skipped / 2**20}M params")
                        manager.register_module_override(module, "weight", {"optim_bits": 32})
                        logger.debug(f"bitsandbytes: will optimize {module} in fp32")
                logger.info(f"skipped: {skipped / 2**20}M params")

        if is_sagemaker_mp_enabled():
            self.optimizer = smp.DistributedOptimizer(self.optimizer)

        return self.optimizer
```

--------------------------------

### Training Arguments

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/model_training.html

Defines the training configuration, including output directory, learning rate, batch size, epochs, evaluation strategy, and logging steps. It's recommended to lower the batch size if out-of-memory errors occur.

```python
from transformers import TrainingArguments

args = TrainingArguments(
    output_dir="models/span-marker-roberta-base-conll03",
    learning_rate=1e-5,
    gradient_accumulation_steps=2,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    num_train_epochs=1,
    eval_strategy="steps",  # Or `evaluation_strategy` if your `transformers<4.54.1`
    save_strategy="steps",
    eval_steps=500,
    push_to_hub=False,
    logging_steps=50,
    fp16=True,
    warmup_ratio=0.1,
)
```

--------------------------------

### Evaluate the trained model

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Evaluates the trained SpanMarkerNER model on a test dataset and prints evaluation metrics.

```python
trainer.evaluate(dataset["test"].select(range(2000)), metric_key_prefix="test")
```

--------------------------------

### Print Fine NER Tags

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

This code snippet prints the available fine-grained NER tags from the dataset.

```python
fine_labels = dataset["train"].features["fine_ner_tags"].feature.names
print(fine_labels)
```

--------------------------------

### Trainer Initialization and Setup

Source: https://tomaarsen.github.io/SpanMarkerNER/_modules/transformers/trainer.html

This snippet shows the initialization of training control variables, including dataloader adjustments for FSDP/XLA, calculation of training epochs, steps, and batch sizes, and handling of token counts per second.

```python
if self.is_fsdp_xla_v2_enabled:
            train_dataloader = tpu_spmd_dataloader(train_dataloader)

        # Setting up training control variables:
        # number of training epochs: num_train_epochs
        # number of training steps per epoch: num_update_steps_per_epoch
        # total number of training steps to execute: max_steps
        total_train_batch_size = self.get_total_train_batch_size(args)

        (
            num_train_epochs,
            num_update_steps_per_epoch,
            num_examples,
            num_train_samples,
            epoch_based,
            len_dataloader,
            max_steps,
        ) = self.set_initial_training_values(args, train_dataloader, total_train_batch_size)

        num_train_tokens = None
        if self.args.include_tokens_per_second:
            num_train_tokens = self.num_tokens(train_dataloader, None if epoch_based else max_steps)
            # If going by epochs, multiply tokens linearly
            if len_dataloader is not None and epoch_based:
                num_train_tokens *= args.num_train_epochs
            # Otherwise since its steps, we just multiply by grad accum
            else:
                num_train_tokens *= args.gradient_accumulation_steps
```

--------------------------------

### Print NER tags

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Retrieves and prints the names of the NER tags from the 'ner_tags' feature of the training dataset.

```python
labels = dataset["train"].features["ner_tags"].feature.names
print(labels)
```

--------------------------------

### Saving a Trained Model Locally

Source: https://tomaarsen.github.io/SpanMarkerNER/notebooks/getting_started.html

This code snippet shows how to save a trained SpanMarkerNER model to a local directory.

```python
trainer.save_model("models/span-marker-bert-base-fewnerd-coarse-super/checkpoint-final")

```

--------------------------------

### SpanMarker NER Example with spaCy

Source: https://tomaarsen.github.io/SpanMarkerNER/api/span_marker.spacy_integration.html

A complete example demonstrating loading a spaCy model, adding SpanMarker, processing text, and inspecting the extracted entities.

```python
import spacy
import span_marker
nlp = spacy.load("en_core_web_sm", exclude=["ner"])
nlp.add_pipe("span_marker", config={"model": "tomaarsen/span-marker-roberta-large-ontonotes5"})
text = '''Cleopatra VII, also known as Cleopatra the Great, was the last active ruler of the
... Ptolemaic Kingdom of Egypt. She was born in 69 BCE and ruled Egypt from 51 BCE until her
... death in 30 BCE.'''
doc = nlp(text)
print(doc.ents)
for span in doc.ents:
    print((span, span.label_))
```

--------------------------------

### Load FewNERD dataset

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Loads the 'supervised' version of the FewNERD dataset from the Hugging Face hub using the datasets library.

```python
from datasets import load_dataset

dataset_id = "DFKI-SLT/few-nerd"
dataset = load_dataset(dataset_id, "supervised")
dataset
```

--------------------------------

### Push the trained model to the Hugging Face Hub

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Pushes the trained SpanMarkerNER model to the Hugging Face Hub for sharing and collaboration.

```python
trainer.push_to_hub(repo_id="span-marker-bert-base-fewnerd-coarse-super")
```

--------------------------------

### Predict entities in sentences

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Uses the trained SpanMarkerNER model to predict entities in a list of sentences and prints the predicted spans and their labels.

```python
sentences = [
    "The Ninth suffered a serious defeat at the Battle of Camulodunum under Quintus Petillius Cerialis in the rebellion of Boudica (61), when most of the foot-soldiers were killed in a disastrous attempt to relieve the besieged city of Camulodunum (Colchester).",
    "He was born in Wellingborough, Northamptonshire, where he attended Victoria Junior School, Westfield Boys School and Sir Christopher Hatton School.",
    "Nintendo continued to sell the revised Wii model and the Wii Mini alongside the Wii U during the Wii U's first release year.",
    "Dorsa has a Bachelor of Music in Composition from California State University, Northridge in 2001, Master of Music in Harpsichord Performance at Cal State Northridge in 2004, and a Doctor of Musical Arts at the University of Michigan, Ann Arbor in 2008."
]

entities_per_sentence = model.predict(sentences)

for entities in entities_per_sentence:
    for entity in entities:
        print(entity["span"], "=>", entity["label"])
    print()
```

--------------------------------

### Evaluate the model on the validation set

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

This code snippet evaluates the trained SpanMarker model on the validation dataset and prints the resulting metrics.

```python
metrics = trainer.evaluate()
metrics
```

--------------------------------

### Initialize SpanMarkerModel

Source: https://tomaarsen.github.io/SpanMarkerNER/_sources/notebooks/getting_started.ipynb.txt

Initializes a SpanMarkerModel from a pre-trained encoder, specifying labels and optional configuration parameters like model_max_length, entity_max_length, and model_card_data for improved model card generation.

```python
from span_marker import SpanMarkerModel, SpanMarkerModelCardData

encoder_id = "bert-base-cased"
model = SpanMarkerModel.from_pretrained(
    # Required arguments
    encoder_id,
    labels=labels,
    # Optional arguments
    model_max_length=256,
    entity_max_length=8,
    # To improve the generated model card
    model_card_data=SpanMarkerModelCardData(
        language=["en"],
        license="cc-by-sa-4.0",
        encoder_id=encoder_id,
        dataset_id=dataset_id,
    )
)
```
Tokens:
Raw