GLiNER

https://urchade.github.io/GLiNER/
Admin
GLiNER is a generalist named entity recognition model using bidirectional transformers that can...
Tokens:428,534
Snippets:2,510
Trust Score:8.3
Update:5 months ago
Show doc for...
Tokens:
Raw
### Start GLiNER Ray Serve Deployment

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/server.html

Initializes and starts the GLiNER Ray Serve deployment. It can optionally run in blocking mode to keep the server alive until interrupted. Requires Ray to be installed and configured.

```python
import ray
from ray import serve as ray_serve

if not ray.is_initialized():
    ray.init(address=config.ray_address, ignore_reinit_error=True)

ray_serve.start(detached=True, http_options={"port": config.http_port})

app = _build_deployment(config)
handle = ray_serve.run(app, name="gliner", route_prefix=config.route_prefix)

logger.info("GLiNER server running at http://localhost:%d%s", config.http_port, config.route_prefix)

if blocking:
    import time
    import signal

    shutdown_event = False

    def handle_signal(_signum, _frame):
        nonlocal shutdown_event
        shutdown_event = True

    signal.signal(signal.SIGINT, handle_signal)
    signal.signal(signal.SIGTERM, handle_signal)

    while not shutdown_event:
        time.sleep(1)

    ray_serve.shutdown()

return handle
```

--------------------------------

### Install GLiNER from Source

Source: https://urchade.github.io/GLiNER/instalation.html

Install the GLiNER package locally after cloning the repository and installing dependencies.

```bash
pip install .
```

--------------------------------

### Install GLiNER

Source: https://urchade.github.io/GLiNER/quickstart.html

Run this command to install the GLiNER framework. Ensure you have pip installed.

```bash
pip install gliner

```

--------------------------------

### Install GLiNER with Serve support

Source: https://urchade.github.io/GLiNER/serving.html

Install GLiNER with the necessary dependencies for serving using uv or pip.

```bash
uv pip install gliner[serve]
```

```bash
pip install gliner ray[serve]
```

--------------------------------

### Install GLiNER from Source

Source: https://urchade.github.io/GLiNER/intro.html

Install GLiNER dependencies and the package itself after cloning the repository.

```bash
cd GLiNER
```

```bash
python -m venv venv
source venv/bin/activate   # On Windows use: venv\Scripts\activate
```

```bash
pip install -r requirements.txt
```

```bash
pip install .
```

--------------------------------

### Install Dependencies from Requirements

Source: https://urchade.github.io/GLiNER/instalation.html

Install all required Python packages listed in the `requirements.txt` file within the activated virtual environment.

```bash
pip install -r requirements.txt
```

--------------------------------

### Start Server

Source: https://urchade.github.io/GLiNER/serving.html

Command to start the GLiNER serving endpoint with a specified model.

```APIDOC
## Start Server

```
python -m gliner.serve --model urchade/gliner_small-v2.1
```
```

--------------------------------

### GLiNER Configuration Initialization

Source: https://urchade.github.io/GLiNER/configs.html

Initialize the GLiNER model with a configuration object. This example shows the basic setup for using the GLiNER library.

```python
from gliner import GLiNERConfig, GLiNER
```

--------------------------------

### Install GLiNER with Training Support

Source: https://urchade.github.io/GLiNER/training.html

Install the GLiNER library with all necessary dependencies for model training. This command ensures you have the required packages to fine-tune models.

```bash
pip install gliner[training]
```

--------------------------------

### Prepare NER Example

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Prepares a single NER example by tokenizing, truncating, and creating span indices and labels. Handles empty token lists and warns if sequences exceed max length.

```python
def prepare_example(self, tokens, ner, classes_to_id):
        """Prepare a single NER example.

        Args:
            tokens: List of tokens in the sentence.
            ner: List of (start, end, label) tuples for entities.
            classes_to_id: Dictionary mapping class names to IDs.

        Returns:
            A dictionary containing the processed example.

        Warnings:
            UserWarning: If sequence length exceeds max_len (gets truncated).
        """
        # Ensure there is always a token list, even if it's empty
        if len(tokens) == 0:
            tokens = ["[PAD]"]

        # Limit the length of tokens based on configuration maximum length
        max_len = self.config.max_len
        if len(tokens) > max_len:
            warnings.warn(f"Sentence of length {len(tokens)} has been truncated to {max_len}", stacklevel=2)
            tokens = tokens[:max_len]

        num_tokens = len(tokens)

        span_idx, span_label = self.prepare_span_idx(ner, classes_to_id, num_tokens)

        example = {
            "tokens": tokens,
            "seq_length": len(tokens),
            "entities": ner,
            "span_idx": span_idx,
            "span_label": span_label,
        }
        return example
```

--------------------------------

### Start GLiNER Ray Serve Deployment

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/server.html

Starts a GLiNER Ray Serve deployment with the specified configuration. This function can optionally block until the server is shut down.

```APIDOC
## serve

### Description
Starts GLiNER Ray Serve deployment.

### Parameters
- **config** (GLiNERServeConfig) - Required - Server configuration.
- **blocking** (bool) - Optional - If True, blocks until the server is shut down.

### Returns
- Ray Serve deployment handle for making predictions.

### Example
```python
from gliner.serve import GLiNERServeConfig, serve

config = GLiNERServeConfig()
serve_handle = serve(config)
```
```

--------------------------------

### Install ONNX Runtime

Source: https://urchade.github.io/GLiNER/convert_to_onnx.html

Install the necessary ONNX Runtime package. Choose the CPU version for general use or the GPU version if you have a compatible NVIDIA GPU.

```bash
pip install onnxruntime  # CPU
# or
pip install onnxruntime-gpu  # GPU

```

--------------------------------

### Install GLiNER via Pip

Source: https://urchade.github.io/GLiNER/instalation.html

Install the base GLiNER library using pip. This is the standard method for most users.

```bash
pip install gliner
```

--------------------------------

### Install GLiNER via Pip

Source: https://urchade.github.io/GLiNER/intro.html

Install the GLiNER Python library using pip. For GPU support, use the `[gpu]` extra.

```bash
!pip install gliner
```

```bash
!pip install gliner[gpu]
```

--------------------------------

### Preprocessing Example for Batch

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Preprocesses individual examples for inclusion in a batch, applying class-to-ID mappings. Handles both list-based and single class mappings.

```python
if isinstance(class_to_ids, list):
            batch = [
                self.preprocess_example(b["tokenized_text"], b[key], class_to_ids[i]) for i, b in enumerate(batch_list)
            ]
        else:
            batch = [self.preprocess_example(b["tokenized_text"], b[key], class_to_ids) for b in batch_list]
```

--------------------------------

### Create Batch Dictionary from Preprocessed Examples

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Creates a batch dictionary from a list of preprocessed span examples, padding sequences and generating masks. Used for model input.

```python
def create_batch_dict(self, batch, class_to_ids, id_to_classes):
        """Create a batch dictionary from preprocessed span examples.

        Args:
            batch: List of preprocessed example dictionaries.
            class_to_ids: List of class-to-ID mappings.
            id_to_classes: List of ID-to-class mappings.

        Returns:
            Dictionary containing:
                - seq_length: Sequence lengths
                - span_idx: Padded span indices
                - tokens: Token strings
                - span_mask: Mask for valid spans
                - span_label: Padded span labels
                - entities: Original NER annotations
                - classes_to_id: Class mappings
                - id_to_classes: Reverse class mappings
        """
        tokens = [el["tokens"] for el in batch]
        entities = [el["entities"] for el in batch]
        span_idx = pad_sequence([b["span_idx"] for b in batch], batch_first=True, padding_value=0)
        span_label = pad_sequence([el["span_label"] for el in batch], batch_first=True, padding_value=-1)
        seq_length = torch.LongTensor([el["seq_length"] for el in batch]).unsqueeze(-1)
        span_mask = span_label != -1

        return {
            "seq_length": seq_length,
            "span_idx": span_idx,
            "tokens": tokens,
            "span_mask": span_mask,
            "span_label": span_label,
            "entities": entities,
            "classes_to_id": class_to_ids,
            "id_to_classes": id_to_classes,
        }
```

--------------------------------

### preprocess_example

Source: https://urchade.github.io/GLiNER/api/gliner.data_processing.processor.html

Preprocesses a single example for token-based prediction.

```APIDOC
## preprocess_example

### Description
Preprocess a single example for token-based prediction.

### Parameters
* **tokens** (list[str]) - List of token strings.
* **ner** (list[tuple]) - List of NER annotations as (start, end, label) tuples.
* **classes_to_id** (dict) - Mapping from class labels to integer IDs.

### Returns
* **tokens** (list[str]) - Token strings
* **seq_length** (int) - Sequence length
* **entities** (list[tuple]) - Original NER annotations
* **span_idx** (Tensor) - Tensor of entity span indices (if represent_spans=True)
* **span_label** (Tensor) - Tensor of entity class IDs (if represent_spans=True)

### Return type
Dictionary containing the preprocessed example.
```

--------------------------------

### Abstract Preprocess Example Method

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Abstract method to preprocess a single example for model input. Subclasses must implement this to define how tokens, NER annotations, and class mappings are converted into model-ready data.

```python
def preprocess_example(
        self, tokens: List[str], ner: List[Tuple[int, int, str]], classes_to_id: Dict[str, int]
    ) -> Dict:
        """Preprocess a single example for model input.

        Args:
            tokens: List of token strings.
            ner: List of NER annotations as (start, end, label) tuples.
            classes_to_id: Mapping from class labels to integer IDs.

        Returns:
            Dictionary containing preprocessed example data.

        Raises:
            NotImplementedError: Must be implemented by subclasses.
        """
        raise NotImplementedError("Subclasses should implement this method")
```

--------------------------------

### Basic GLiNER Training Data Example

Source: https://urchade.github.io/GLiNER/training.html

Provides a concrete example of the basic dataset format for GLiNER training. It shows how to define tokenized text and NER annotations using token-level, inclusive indices.

```python
train_data = [
    {
        "tokenized_text": ["Barack", "Obama", "was", "born", "in", "Hawaii", "."],
        "ner": [
            [0, 1, "person"],      # "Barack Obama" spans tokens 0-1
            [5, 5, "location"]     # "Hawaii" is token 5
        ]
    },
    {
        "tokenized_text": ["Microsoft", "was", "founded", "in", "1975"],
        "ner": [
            [0, 0, "organization"],  # "Microsoft" is token 0
            [4, 4, "date"]           # "1975" is token 4
        ]
    }
]
```

--------------------------------

### Set Up Virtual Environment

Source: https://urchade.github.io/GLiNER/instalation.html

Create and activate a Python virtual environment for installing dependencies. Use `venv\Scripts\activate` on Windows.

```bash
python -m venv venv
source venv/bin/activate   # On Windows use: venv\Scripts\activate
```

--------------------------------

### Get Ground Truth Entities for NER

Source: https://urchade.github.io/GLiNER/_modules/gliner/evaluation/evaluator.html

Converts ground truth entity tuples (start, end, label) into a list format [label, (start, end)] suitable for evaluation.

```python
def get_ground_truth(self, ents):
        """Extract ground truth entities in evaluation format.

        Args:
            ents: List of ground truth entity tuples in format (start, end, label)
                where start and end are word-level indices.

        Returns:
            List of entities in format [[label, (start, end)], ...] suitable
            for evaluation.
        """
        all_ents = []
        for s, e, lab in ents:
            all_ents.append([lab, (s, e)])
        return all_ents
```

--------------------------------

### __init__

Source: https://urchade.github.io/GLiNER/api/gliner.serve.server.html

Builds a configuration and starts the Ray Serve deployment. It can accept a model name/path or a pre-built GLiNERServeConfig.

```APIDOC
## __init__

### Description
Build a config (if not provided) and start the Ray Serve deployment.

### Parameters
* **model** (_str_ | _None_) – Model name or path. Ignored if `config` is provided.
* **config** (_GLiNERServeConfig_ | _None_) – Prebuilt `GLiNERServeConfig`. Mutually exclusive with `model`/`kwargs`.
* **kwargs** – Forwarded to `GLiNERServeConfig` when building one.
```

--------------------------------

### serve

Source: https://urchade.github.io/GLiNER/api/gliner.serve.server.html

Start the GLiNER Ray Serve deployment. This function initializes the server with the provided configuration and returns a handle for making predictions.

```APIDOC
## serve(config, blocking=False)

### Description
Start GLiNER Ray Serve deployment.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **config** (GLiNERServeConfig) - Server configuration.
- **blocking** (bool) - If True, blocks until the server is shut down.

### Request Example
```python
from gliner.serve import GLiNERServeConfig, serve
config = GLiNERServeConfig(model="urchade/gliner_small-v2.1")
handle = serve(config)
# Make predictions
ref = handle.predict.remote("John works at Google", ["person", "org"])
print(ref.result())
```

### Response
#### Success Response (200)
- **deployment_handle**: Ray Serve deployment handle for making predictions.

Return type: Any

#### Response Example
None
```

--------------------------------

### GLiNER Entity Output Format

Source: https://urchade.github.io/GLiNER/architectures.html

Example of the standard GLiNER entity format, including start and end character indices, text, label, and confidence score.

```json
{
    'start': 0,
    'end': 10,
    'text': 'John Smith',
    'label': 'person',
    'score': 0.95
}

```

--------------------------------

### Get Predicted Entities for NER

Source: https://urchade.github.io/GLiNER/_modules/gliner/evaluation/evaluator.html

Converts predicted entity tuples into a list format [label, (start, end)] for evaluation. Handles both object attributes and tuple formats for predictions.

```python
def get_predictions(self, ents):
        """Extract predicted entities in evaluation format.

        Args:
            ents: List of predicted entity tuples in format (start, end, label)
                where start and end are word-level indices.

        Returns:
            List of entities in format [[label, (start, end)], ...] suitable
            for evaluation.
        """
        all_ents = []
        for ent in ents:
            if hasattr(ent, "entity_type"):
                all_ents.append([ent.entity_type, (ent.start, ent.end)])
            else:
                all_ents.append([ent[2], (ent[0], ent[1])])
        return all_ents
```

--------------------------------

### BaseProcessor.preprocess_example

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Abstract method to preprocess a single example for model input. Must be implemented by subclasses.

```APIDOC
## BaseProcessor.preprocess_example

### Description
Preprocess a single example for model input.

### Arguments
- `tokens`: List of token strings.
- `ner`: List of NER annotations as (start, end, label) tuples.
- `classes_to_id`: Mapping from class labels to integer IDs.

### Returns
Dictionary containing preprocessed example data.

### Raises
`NotImplementedError`: Must be implemented by subclasses.
```

--------------------------------

### Install FlashDeBERTa via Pip

Source: https://urchade.github.io/GLiNER/instalation.html

Install or upgrade the `flashdeberta` package using pip. Ensure you have `transformers>=4.51.3` installed.

```bash
pip install flashdeberta -U
```

--------------------------------

### Simple GLiNER Model Training Example

Source: https://urchade.github.io/GLiNER/training.html

Demonstrates a basic workflow for fine-tuning a GLiNER model. Load a pre-trained model, define your training data in the specified format, and initiate the training process. Remember to use separate evaluation data in a real-world scenario.

```python
from gliner import GLiNER

# Load a pretrained model
model = GLiNER.from_pretrained("urchade/gliner_small-v2.1")

# Define your training data
train_data = [
    {
        "tokenized_text": ["Apple", "Inc.", "is", "headquartered", "in", "Cupertino"],
        "ner": [[0, 1, "organization"], [5, 5, "location"]]
    },
    {
        "tokenized_text": ["Steve", "Jobs", "founded", "Apple"],
        "ner": [[0, 1, "person"], [3, 3, "organization"]]
    }
]

# Train the model
trainer = model.train_model(
    train_dataset=train_data,
    eval_dataset=train_data,  # Use separate eval data in practice
    output_dir="./my_model",
    max_steps=1000,
    learning_rate=5e-5,
    per_device_train_batch_size=8,
)

# Save the trained model
trainer.save_model()
```

--------------------------------

### Verify GLiNER Installation

Source: https://urchade.github.io/GLiNER/instalation.html

Import the GLiNER library in a Python script to verify that the installation was successful and print the installed version.

```python
import gliner
print(gliner.__version__)
```

--------------------------------

### Install GLiNER with GPU Support via Pip

Source: https://urchade.github.io/GLiNER/instalation.html

Install GLiNER with GPU support by including the `[gpu]` extra. This also installs the `onnxruntime-gpu` dependency.

```bash
pip install gliner[gpu]
```

--------------------------------

### Start GLiNER Serving

Source: https://urchade.github.io/GLiNER/serving.html

Launch the GLiNER serving endpoint using the command-line interface, specifying the model to serve.

```bash
python -m gliner.serve --model urchade/gliner_small-v2.1
```

--------------------------------

### Start Relation Extraction Server

Source: https://urchade.github.io/GLiNER/serving.html

Command to start a server specifically for GLiNER-RelEx models.

```APIDOC
## Start Relation Extraction Server

### Description
This command starts a GLiNER server configured for relation extraction models. The server automatically detects relation support based on the model configuration.

### Command Example

```bash
python -m gliner.serve \
    --model knowledgator/gliner-relex-large-v1.0 \
    --dtype bfloat16 \
    --max-batch-size 16
```
```

--------------------------------

### Install GLiNER via Conda

Source: https://urchade.github.io/GLiNER/instalation.html

Install GLiNER using the conda package manager from the conda-forge channel.

```bash
conda install -c conda-forge gliner
```

--------------------------------

### Select Entities for an Example

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Selects the appropriate entity types for a given example based on the provided entity specifications. It handles cases where entities are specified per-example, shared across all examples, or when a blank entity token is used for zero-shot scenarios.

```python
def _select_entities(
    self,
    i: int,
    entities: Union[Sequence[Sequence[str]], Dict[int, Sequence[str]], Sequence[str]],
    blank: Optional[str] = None,
) -> List[str]:
    """Select entities for a specific example.

    Args:
        i: Index of the example.
        entities: Entity specifications (see prepare_inputs).
        blank: Optional blank entity token.

    Returns:
        List of entity type strings for this example.
    """
    if blank is not None:
        return [blank]
    if isinstance(entities, dict):
        return list(entities)
    if entities and isinstance(entities[0], (list, tuple, dict)):  # per-item lists
        return list(entities[i])  # type: ignore[index]
    if entities and isinstance(entities[0], str):  # same for all
        return list(entities)  # type: ignore[list-item]
    return []
```

--------------------------------

### Serve GLiNER Deployment

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/server.html

Starts a GLiNER Ray Serve deployment. This function initializes Ray if not already initialized and runs the GLiNER application. It can optionally run in blocking mode to keep the server alive.

```APIDOC
## serve

### Description
Starts a GLiNER Ray Serve deployment.

### Parameters
- **config** (GLiNERServeConfig) - Configuration for the GLiNER server.
- **blocking** (bool, optional) - If True, the server will run in blocking mode, waiting for signals to shut down. Defaults to False.

### Returns
- handle (ray.serve.handle.DeploymentHandle) - The Ray Serve deployment handle.
```

--------------------------------

### Install GLiNER with Tokenizer Dependencies

Source: https://urchade.github.io/GLiNER/intro.html

Install additional dependencies for multi-lingual models or models using the Stanza tokenizer.

```bash
!pip install gliner[tokenizers]
```

```bash
!pip install gliner[stanza]
```

--------------------------------

### __init__

Source: https://urchade.github.io/GLiNER/api/gliner.modeling.base.html

Initialize the token-based uni-encoder model.

```APIDOC
## __init__

### Description
Initialize the token-based uni-encoder model.

### Parameters
* **config** (_Any_) – Model configuration object.
* **from_pretrained** (_bool_) – Whether to load from pretrained weights.
* **cache_dir** (_str_ _|__Path_ _|__None_) – Directory for caching pretrained models.
```

--------------------------------

### Start GLiNER Ray Serve Deployment

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/server.html

Initiates the GLiNER Ray Serve deployment. Optionally blocks until the server is shut down, returning a handle for making predictions.

```python
def serve(
    config: GLiNERServeConfig,
    blocking: bool = False,
) -> Any:
    """Start GLiNER Ray Serve deployment.

    Args:
        config: Server configuration.
        blocking: If True, blocks until the server is shut down.

    Returns:
        Ray Serve deployment handle for making predictions.

    Example:
        >>> from gliner.serve import GLiNERServeConfig, serve

```

--------------------------------

### Navigate to Project Directory

Source: https://urchade.github.io/GLiNER/instalation.html

Change the current directory to the cloned GLiNER repository.

```bash
cd GLiNER
```

--------------------------------

### Populate Relation Matrix for an Example

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Iterates through positive and negative pairs for a single example to populate the adjacency and relation matrices.

```python
for i in range(B):
            N = batch_ents_cpu[i]
            pair_info = all_pairs_info[i]
            adj = torch.zeros(max(N, 1), max(N, 1))

            for pair_idx, (pair, is_positive, relations) in enumerate(pair_info):
                e1, e2 = pair
                adj[e1, e2] = 1.0

                if is_positive:
                    for class_id in relations:
```

--------------------------------

### Install gliner-spacy

Source: https://urchade.github.io/GLiNER/usage.html

Install the gliner-spacy library to integrate GLiNER with spaCy. This is the first step for using GLiNER within a spaCy pipeline.

```bash
pip install gliner-spacy
```

--------------------------------

### create_example

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Preprocesses a single NER example, converting raw text and annotations into a structured dictionary format suitable for batching. It handles tokenization, length truncation, and span index/label preparation.

```APIDOC
## create_example

### Description
Preprocesses a single NER example, converting raw text and annotations into a structured dictionary format suitable for batching. It handles tokenization, length truncation, and span index/label preparation.

### Parameters
- **tokens** (List[str]): A list of token strings representing the input sentence.
- **ner** (List[Tuple[int, int, str]]): A list of tuples, where each tuple represents an entity with (start_index, end_index, label).
- **classes_to_id** (Dict[str, int]): A dictionary mapping entity labels to their corresponding integer IDs.
- **num_tokens** (int): The total number of tokens in the processed sequence.

### Returns
- **example** (Dict[str, Any]): A dictionary containing the processed example with keys:
    - `tokens` (List[str]): The processed list of tokens.
    - `seq_length` (int): The length of the token sequence.
    - `entities` (List[Tuple[int, int, str]]): The original NER annotations.
    - `span_idx` (Tensor): Tensor of entity span indices (if represent_spans=True).
    - `span_label` (Tensor): Tensor of entity class IDs (if represent_spans=True).

### Warnings
- `UserWarning`: If sequence length exceeds `max_len` (gets truncated).
```

--------------------------------

### Initialize GLiNERClient

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/client.html

Instantiate the GLiNERClient to connect to a running Ray Serve deployment. Default values are provided for common configurations.

```python
from gliner.serve import GLiNERClient
client = GLiNERClient()
```

--------------------------------

### Install GLiNER with ONNX Support

Source: https://urchade.github.io/GLiNER/convert_to_onnx.html

Install the GLiNER library with ONNX support using pip. This is a prerequisite for converting models to ONNX format.

```bash
pip install gliner[onnx]
```

--------------------------------

### Processor.process_batch

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Processes a batch of examples for joint entity and relation extraction. It handles data augmentation, dynamic mapping generation, and example preprocessing.

```APIDOC
## Processor.process_batch

### Description
Processes a batch of examples for joint entity and relation extraction. It handles data augmentation, dynamic mapping generation, and example preprocessing.

### Method Signature
```python
def process_batch(self, batch_list, entity_types=None, relation_types=None, ner_negatives=None, rel_negatives=None, class_to_ids=None, id_to_classes=None, rel_class_to_ids=None, rel_id_to_classes=None, key='ner')
```

### Parameters
#### Arguments
- **batch_list** (List[Dict]) - List of dictionaries, where each dictionary represents an example.
- **entity_types** (Optional[List[Union[List[str], str]]]) - Optional list of entity types for mapping.
- **relation_types** (Optional[List[Union[List[str], str]]]) - Optional list of relation types for mapping.
- **ner_negatives** (Optional[List[str]]) - Optional list of negative entity labels.
- **rel_negatives** (Optional[List[str]]) - Optional list of negative relation types.
- **class_to_ids** (Optional[Dict[str, int]]) - Optional mapping from entity class labels to integer IDs.
- **id_to_classes** (Optional[Dict[int, str]]) - Optional mapping from integer IDs to entity class labels.
- **rel_class_to_ids** (Optional[Dict[str, int]]) - Optional mapping from relation class labels to integer IDs.
- **rel_id_to_classes** (Optional[Dict[int, str]]) - Optional mapping from integer IDs to relation class labels.
- **key** (str) - Key for accessing labels in batch (default: 'ner').

### Returns
Dictionary containing collated batch data for joint entity and relation extraction.
```

--------------------------------

### Build Ray Serve Deployment

Source: https://urchade.github.io/GLiNER/_modules/gliner/serve/server.html

Constructs a Ray Serve deployment for the GLiNER model based on the provided configuration. It sets up replicas, resource allocation (GPUs/CPUs), and ongoing request limits.

```python
def _build_deployment(config: GLiNERServeConfig):
    """Build Ray Serve deployment from config."""
    from ray import serve  # noqa: PLC0415

    batch_wait_s = max(config.batch_wait_timeout_ms, 0.0) / 1000.0
    initial_max_batch_size = config.max_batch_size

    @serve.deployment(
        num_replicas=config.num_replicas,
        ray_actor_options={
            "num_gpus": config.num_gpus_per_replica,
            "num_cpus": config.num_cpus_per_replica,
        },
        max_ongoing_requests=config.max_ongoing_requests,
    )
    class GLiNERDeployment:
        def __init__(self, serve_config: GLiNERServeConfig):
            self.server = GLiNERServer(serve_config)
            # Seed Ray's batcher with the pessimistic worst-case size so the
            # first batch is safe. ``_infer_batch`` re-calls ``batch_size_fn``
            # on every dispatch to re-size the batcher based on observed
            # sequence lengths.
            self._infer_batch.set_max_batch_size(self.server.batch_size_fn())
            logger.info(
                "Ray Serve batch size initialized to %d (precompiled: %s)",
                self.server.batch_size_fn(),
                serve_config.precompiled_batch_sizes,
            )

        @serve.batch(
            max_batch_size=initial_max_batch_size,
            batch_wait_timeout_s=batch_wait_s,
        )
        async def _infer_batch(
            self,
            texts: List[str],
            labels_list: List[List[str]],
            relations_list: List[Optional[List[str]]],
            thresholds: List[float],
            relation_thresholds: List[float],
            flat_ner_list: List[bool],
            multi_label_list: List[bool],
        ) -> List[Dict[str, Any]]:
            """Single forward pass over the Ray-accumulated batch.

            Before dispatch, re-sizes Ray's batcher via ``set_max_batch_size``
            using ``batch_size_fn`` on the observed seq length — so the next
            accumulation picks the largest precompiled size that fits.

            Assumes batch requests are homogeneous — labels/thresholds/flags
            are taken from the first request.
            """
            next_max_batch = self.server.batch_size_fn(
                seq_len=self.server.observed_seq_len(
                    texts,
                    labels=labels_list[0] if labels_list else None,
                    relations=relations_list[0] if relations_list else None,
                )
            )
            self._infer_batch.set_max_batch_size(next_max_batch)

            return self.server.predict(
                texts,
                labels_list[0],
                relations=relations_list[0],
                threshold=thresholds[0],
                relation_threshold=relation_thresholds[0],
                flat_ner=flat_ner_list[0],
                multi_label=multi_label_list[0],
            )

        async def predict(
            self,
            text: str,
            labels: List[str],
            relations: Optional[List[str]] = None,
            threshold: Optional[float] = None,
            relation_threshold: Optional[float] = None,
            flat_ner: bool = True,
            multi_label: bool = False,
        ) -> Dict[str, Any]:
            """Single prediction endpoint."""
            if threshold is None:
                threshold = self.server.config.default_threshold
            if relation_threshold is None:
                relation_threshold = self.server.config.default_relation_threshold

            results = await self._infer_batch(
                text,
                labels,
                relations,
                threshold,
                relation_threshold,
                flat_ner,
                multi_label,
            )
            return results

        async def __call__(self, request) -> Dict[str, Any]:
            """Handle HTTP requests."""
            payload = await request.json()
            return await self.predict(
                text=payload["text"],
                labels=payload["labels"],
                relations=payload.get("relations"),
                threshold=payload.get("threshold"),
                relation_threshold=payload.get("relation_threshold"),
                flat_ner=payload.get("flat_ner", True),
                multi_label=payload.get("multi_label", False),
            )

    return GLiNERDeployment.bind(config)
```

--------------------------------

### Docker Deployment

Source: https://urchade.github.io/GLiNER/serving.html

Instructions for building and running the GLiNER server using Docker.

```APIDOC
## Docker
**Build:**
```
docker build -t gliner-serve -f gliner/serve/Dockerfile .

```

**Run:**
```
docker run --gpus all -p 8000:8000 gliner-serve

```

**With custom model:**
```
docker run --gpus all -p 8000:8000 \
  -e GLINER_MODEL=urchade/gliner-medium-v2.1 \
  -e GLINER_ENABLE_FLASHDEBERTA=true \
  gliner-serve

```

**Environment variables:**
Variable | Default | Description  
---|---|---
`GLINER_MODEL` | `urchade/gliner_small-v2.1` | Model name or path  
`GLINER_DEVICE` | `cuda` | Device (cuda/cpu)  
`GLINER_DTYPE` | `bfloat16` | Data type  
`GLINER_MAX_BATCH_SIZE` | `32` | Max batch size  
`GLINER_NUM_REPLICAS` | `1` | Number of replicas  
`GLINER_MEMORY_FRACTION` | `0.8` | GPU memory fraction  
`GLINER_QUANTIZATION` | - | Quantization (`int8` only; use `GLINER_DTYPE` for precision)  
`GLINER_ENABLE_FLASHDEBERTA` | `false` | Enable FlashDeBERTa  
`GLINER_ENABLE_PACKING` | `false` | Enable sequence packing  
`GLINER_DISABLE_COMPILE` | `false` | Disable torch.compile  
`GLINER_ROUTE_PREFIX` | `/gliner` | HTTP route prefix  
```

--------------------------------

### BiEncoder.__init__

Source: https://urchade.github.io/GLiNER/_modules/gliner/modeling/encoder.html

Initializes the BiEncoder, setting up separate encoders for text and labels, and optional projection layers.

```APIDOC
## BiEncoder.__init__

### Description
Initializes the bi-encoder architecture, including separate encoders for text and labels, and optional projection layers.

### Method
`__init__(config: Any, from_pretrained: bool = False, cache_dir: Optional[Union[str, Path]] = None) -> None`

### Parameters
*   `config`: Configuration object containing model hyperparameters, including `labels_encoder` (model name for label encoder) and `hidden_size`.
*   `from_pretrained`: If `True`, loads pretrained weights for both encoders. Defaults to `False`.
*   `cache_dir`: Optional directory for caching downloaded models. Defaults to `None`.
```

--------------------------------

### Install UTCA for Advanced Relation Extraction

Source: https://urchade.github.io/GLiNER/usage.html

Install the UTCA library using pip, which provides advanced capabilities for relation extraction, including integration with GLiNER.

```bash
pip install utca -U
```

--------------------------------

### GLiNER Training Configuration Example

Source: https://urchade.github.io/GLiNER/configs.html

This configuration file outlines parameters for training a GLiNER model, including model architecture, loss function weights, training steps, batch size, learning rates, and data sources. It is suitable for fine-tuning models on datasets with relation annotations.

```yaml
# Model Configuration
model_name: microsoft/deberta-v3-base
relations_layer: biaffine  # Enable relation extraction
triples_layer: distmult
rel_token: "<<REL>>"
embed_rel_token: true
name: "span relex gliner"
max_width: 12
hidden_size: 768
dropout: 0.4
fine_tune: true
span_mode: markerV0

# Loss Configuration
span_loss_coef: 1.0
adjacency_loss_coef: 1.0
relation_loss_coef: 1.0

# Training Parameters
num_steps: 30000
train_batch_size: 6  # Smaller due to relation computation
eval_every: 1000
warmup_ratio: 0.1
scheduler_type: "cosine"

# Loss Configuration
loss_alpha: -1
loss_gamma: 0
label_smoothing: 0
loss_reduction: "sum"

# Learning Rate Configuration
lr_encoder: 1e-5
lr_others: 5e-5
weight_decay_encoder: 0.01
weight_decay_other: 0.01
max_grad_norm: 1.0

# Data Configuration
train_data: "data_with_relations.json"  # Must include relation annotations
prev_path: null
save_total_limit: 3

# Advanced Settings
max_types: 25
max_len: 384
```

--------------------------------

### Create and Run a GLiNER Pipeline

Source: https://urchade.github.io/GLiNER/usage.html

Demonstrates creating a pipeline with entity and relation extraction, then running it on sample text. Configure relation types, filters, and thresholds for specific extraction needs.

```python
from gliner import GLiNER
from gliner.modules import GLiNERPreprocessor, GLiNERRelationExtractionPreprocessor, RenameAttribute

# Assume 'predictor' is a pre-initialized model predictor
# predictor = ... 

pipe = (
    GLiNER(
        predictor=predictor,
        preprocess=GLiNERPreprocessor(threshold=0.7)
    )
    | RenameAttribute("output", "entities")
    | GLiNERRelationExtraction(
        predictor=predictor,
        preprocess=(
            GLiNERPreprocessor(threshold=0.5)
            | GLiNERRelationExtractionPreprocessor()
        )
    )
)

text = """
Microsoft was founded by Bill Gates and Paul Allen on April 4, 1975 to develop 
and sell BASIC interpreters for the Altair 8800. During his career at Microsoft, 
Gates held the positions of chairman, chief executive officer, president and chief 
software architect, while also being the largest individual shareholder until May 2014.
"""

result = pipe.run({
    "text": text,
    "labels": ["organization", "person", "position", "date"],
    "relations": [
        {
            "relation": "founder",
            "pairs_filter": [("organization", "person")],
            "distance_threshold": 100,
        },
        {
            "relation": "inception_date",
            "pairs_filter": [("organization", "date")],
        },
        {
            "relation": "held_position",
            "pairs_filter": [("person", "position")],
        }
    ]
})

for relation in result["output"]:
    source = relation['source']['span']
    target = relation['target']['span']
    rel_type = relation['relation']
    score = relation['score']
    print(f"{source} --[{rel_type}]--> {target} (score: {score:.3f})")

```

```python
# Only extract relations where entities are close together
relations = [
    {
        "relation": "works_for",
        "pairs_filter": [("person", "organization")],
        "distance_threshold": 50,
    }
]

```

```python
# Define complex relation schemas
relations = [
    {
        "relation": "employed_by",
        "pairs_filter": [("person", "organization")],
    },
    {
        "relation": "located_in",
        "pairs_filter": [("organization", "location"), ("person", "location")],
    },
    {
        "relation": "acquired_by",
        "pairs_filter": [("organization", "organization")],
    },
]

```

--------------------------------

### Preprocess Example for Token-Level Prediction

Source: https://urchade.github.io/GLiNER/_modules/gliner/data_processing/processor.html

Preprocesses a single example for token-level encoder-decoder prediction. This method integrates token-level preprocessing with preparations for decoder-based label generation.

```python
    def preprocess_example(self, tokens, ner, classes_to_id):
        """Preprocess a single example for token-level encoder-decoder prediction.

        Uses token-level preprocessing from UniEncoderTokenProcessor while
        preparing for decoder-based label generation.

        Args:
            tokens: List of token strings.

```

--------------------------------

### _calculate_span_score

Source: https://urchade.github.io/GLiNER/_modules/gliner/decoding/decoder.html

Calculates spans and their scores from start, end, and inside predictions. It matches start and end positions of the same class, validates inside scores, and computes final span scores.

```APIDOC
## _calculate_span_score

### Description
Calculate spans and their scores from start/end/inside predictions. Matches start and end positions of the same class, validates inside scores, and computes final span scores.

### Arguments
* `start_idx` (tuple): Tuple of (positions, classes) for start predictions.
* `end_idx` (tuple): Tuple of (positions, classes) for end predictions.
* `scores_inside_i` (torch.Tensor): Inside scores for this sample.
* `start_i` (torch.Tensor): Start scores for this sample.
* `end_i` (torch.Tensor): End scores for this sample.
* `id_to_classes` (Dict[int, str]): Mapping from class IDs to class names.
* `threshold` (float): Confidence threshold.
* `input_spans_i` (Optional[set]): If provided, set of (word_start, word_end) tuples to restrict output to.

### Returns
List[Span]: List of Span objects.
```
Tokens:
Raw