GFM-RAG

https://rmanluo.github.io/gfm-rag/latest/
Admin
GFM-RAG is a graph foundation model-powered RAG pipeline that combines graph neural networks with...
Tokens:255,037
Snippets:1,661
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Example Pre-training Workflow with Wandb

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config?q=

Start pre-training with custom wandb settings including project, group, tags, and name.

```bash
# 1. Start pre-training with custom settings
python -m gfmrag.workflow.kgc_training --config-path config/gfm_rag \
    wandb.project="gfm-rag-experiments" \
    wandb.group="pretraining" \
    wandb.tags=["baseline","hotpot"] \
    wandb.name="pretrain_baseline_v1"
```

--------------------------------

### Install Llama.cpp Backend

Source: https://rmanluo.github.io/gfm-rag/latest/install?q=

Install the llama-cpp-python package to enable Llama.cpp as an LLM backend.

```bash
pip install llama-cpp-python

```

--------------------------------

### Install and Login to Wandb

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config?q=

Install wandb (if not already included) and log in to your wandb account to authenticate.

```bash
# wandb is already included in the dependencies
# Log in to your wandb account
wandb login

```

--------------------------------

### Install Dependencies and Pre-commit Hooks

Source: https://rmanluo.github.io/gfm-rag/latest/DEVELOPING?q=

Installs project dependencies using Poetry and sets up pre-commit hooks for code quality checks. Ensure Poetry is installed before running.

```bash
poetry install
pre-commit install

```

--------------------------------

### Install gfmrag from Source

Source: https://rmanluo.github.io/gfm-rag/latest/install?q=

Clone the repository and install dependencies using Poetry for development or contribution.

```bash
git clone https://github.com/RManLuo/gfm-rag.git
cd gfm-rag
conda create -n gfmrag python=3.12
conda activate gfmrag
conda install cuda-toolkit -c nvidia/label/cuda-12.6.3
poetry install

```

--------------------------------

### Install Ollama Backend

Source: https://rmanluo.github.io/gfm-rag/latest/install?q=

Install the necessary packages for using Ollama as an LLM backend with Langchain.

```bash
pip install langchain-ollama
pip install ollama

```

--------------------------------

### Pre-built Stage1 Layout Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

This layout uses pre-built graph and QA files directly. It is consumed by the framework without rebuilding stage1.

```text
root/
└── data_name/
    └── processed/
        └── stage1/
            ├── nodes.csv
            ├── relations.csv
            ├── edges.csv
            ├── train.json (optional)
            └── test.json (optional)
```

--------------------------------

### Example Fine-tuning Workflow with Wandb

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config?q=

Fine-tune a model using custom wandb settings and load a pre-trained model.

```bash
# 2. Fine-tune using the pre-trained model
python -m gfmrag.workflow.sft_training --config-path config/gfm_rag \
    wandb.project="gfm-rag-experiments" \
    wandb.group="finetuning" \
    wandb.tags=["finetune","hotpot"] \
    wandb.name="finetune_baseline_v1" \
    load_model_from_pretrained="path/to/pretrained/model"
```

--------------------------------

### Start Local vLLM Server

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models?q=

Starts a local vLLM server for embedding generation. It temporarily modifies environment variables to configure the server and restores them afterward.

```python
def _start_vllm_server(self) -> LLM:
        """Start a vLLM server for embedding generation."""

        dist_keys = [
            "RANK",
            "LOCAL_RANK",
            "WORLD_SIZE",
            "LOCAL_WORLD_SIZE",
            "GROUP_RANK",
            "ROLE_RANK",
            "ROLE_NAME",
            "OMP_NUM_THREADS",
            "MASTER_ADDR",
            "MASTER_PORT",
            "TORCHELASTIC_USE_AGENT_STORE",
            "TORCHELASTIC_MAX_RESTARTS",
            "TORCHELASTIC_RUN_ID",
            "TORCH_NCCL_ASYNC_ERROR_HANDLING",
            "TORCHELASTIC_ERROR_FILE",
        ]
        old_env = {}
        for dist_key in dist_keys:
            if dist_key in os.environ:
                old_env[dist_key] = os.environ.pop(dist_key)
        os.environ["CUDA_VISIBLE_DEVICES"] = old_env.get("LOCAL_RANK", "0")
        os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
        text_emb_model = LLM(
            model=self.text_emb_model_name,
            enforce_eager=True,
            task="embed",
            hf_overrides={"is_matryoshka": True},
        )
        # Restore environment variables
        os.environ.pop("CUDA_VISIBLE_DEVICES")
        for dist_key, dist_value in old_env.items():
            if dist_value is not None:
                os.environ[dist_key] = dist_value
        return text_emb_model
```

--------------------------------

### Add EOS Token to Input Examples

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models

Appends the EOS token to each string in a list of input examples. Requires the NVEmbedModel tokenizer and its EOS token to be initialized.

```python
def add_eos(self, input_examples: list[str]) -> list[str]:
        tokenizer = self.text_emb_model.tokenizer
        if tokenizer is None or tokenizer.eos_token is None:
            raise ValueError("NVEmbedModel tokenizer EOS token must be initialized.")
        return [input_example + tokenizer.eos_token for input_example in input_examples]
```

--------------------------------

### Initialize and Use DPRELModel

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/el_model

Instantiate the DPRELModel with a SentenceTransformer model, index entities, and then perform entity linking on mentions. This example demonstrates the core workflow of the model.

```python
model = DPRELModel('sentence-transformers/all-mpnet-base-v2')
model.index(['Paris', 'London', 'Berlin'])
results = model(['paris city'], topk=2)
print(results)
```

--------------------------------

### OpenIE Model Usage Example

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/openie_model?q=

Demonstrates how to use an OpenIE model to extract information from a given text. Instantiate the model and pass the text to its call method.

```python
>>> openie_model = OPENIEModel()
>>> result = openie_model("Emmanuel Macron is the president of France")
>>> print(result)
{'passage': 'Emmanuel Macron is the president of France', 'extracted_entities': ['Emmanuel Macron', 'France'], 'extracted_triples': [['Emmanuel Macron', 'president of', 'France']]}
```

--------------------------------

### Example Workflow: Pre-training and Fine-tuning

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config

This bash script demonstrates a typical workflow for setting up and running experiments with wandb, including pre-training and fine-tuning stages with custom project, group, tags, and names.

```bash
# 1. Start pre-training with custom settings
python -m gfmrag.workflow.kgc_training --config-path config/gfm_rag \
    wandb.project="gfm-rag-experiments" \
    wandb.group="pretraining" \
    wandb.tags=["baseline","hotpot"] \
    wandb.name="pretrain_baseline_v1"

# 2. Fine-tune using the pre-trained model
python -m gfmrag.workflow.sft_training --config-path config/gfm_rag \
    wandb.project="gfm-rag-experiments" \
    wandb.group="finetuning" \
    wandb.tags=["finetune","hotpot"] \
    wandb.name="finetune_baseline_v1" \
    load_model_from_pretrained="path/to/pretrained/model"
```

--------------------------------

### Edge Text Attributes Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Example of text attributes for an edge, including start and end years for a 'lived_in' relation.

```text
start_year: 2009
end_year: 2017
```

--------------------------------

### __init__

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/openie_model?q=

Initializes the LLM-based OpenIE model with specified configurations.

```APIDOC
## __init__

### Description
Initializes the LLM-based OpenIE model with specified configurations.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Method Signature
__init__(self, llm_api='openai', model_name='gpt-4o-mini', max_ner_tokens=1024, max_triples_tokens=4096, n_ctx=None, low_vram=False)

### Parameters
- **llm_api** (Literal['openai', 'nvidia', 'together', 'ollama', 'llama.cpp']) - Optional - The LLM API provider to use. Defaults to "openai".
- **model_name** (str) - Optional - Name of the language model to use. Defaults to "gpt-4o-mini".
- **max_ner_tokens** (int) - Optional - Maximum number of tokens for NER processing. Defaults to 1024.
- **max_triples_tokens** (int) - Optional - Maximum number of tokens for triple extraction. Defaults to 4096.
- **n_ctx** (int | None) - Optional - Context window size (llama.cpp / ollama only).
- **low_vram** (bool) - Optional - Enable low VRAM mode (llama.cpp only). Defaults to False.

### Attributes
- **llm_api** (str) - The selected LLM API provider.
- **model_name** (str) - Name of the language model.
- **max_ner_tokens** (int) - Token limit for NER.
- **max_triples_tokens** (int) - Token limit for triples.
- **client** - Initialized language model client.
```

--------------------------------

### DPRELModel Initialization and Usage

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/el_model

Demonstrates how to initialize the DPRELModel, index entities, and perform entity linking.

```APIDOC
## DPRELModel

### Description
Entity Linking Model based on Dense Passage Retrieval (DPR). This class implements an entity linking model using DPR architecture and SentenceTransformer for encoding entities and computing similarity scores between mentions and candidate entities.

### Parameters
- **model_name** (str) - Required - Name or path of the SentenceTransformer model to use.
- **root** (str) - Optional - Root directory for caching embeddings. Defaults to "tmp".
- **use_cache** (bool) - Optional - Whether to cache and reuse entity embeddings. Defaults to True.
- **normalize** (bool) - Optional - Whether to L2-normalize embeddings. Defaults to True.
- **batch_size** (int) - Optional - Batch size for encoding. Defaults to 32.
- **query_instruct** (str) - Optional - Instruction/prompt prefix for query encoding. Defaults to "".
- **passage_instruct** (str) - Optional - Instruction/prompt prefix for passage encoding. Defaults to "".
- **model_kwargs** (dict) - Optional - Additional kwargs to pass to SentenceTransformer. Defaults to None.

### Methods
- **index(entity_list)**: Indexes a list of entities by computing and caching their embeddings.
- **__call__(ner_entity_list, topk)**: Links named entities to indexed entities and returns top-k matches.

### Example
```python
>>> model = DPRELModel('sentence-transformers/all-mpnet-base-v2')
>>> model.index(['Paris', 'London', 'Berlin'])
>>> results = model(['paris city'], topk=2)
>>> print(results)
{'paris city': [{'entity': 'Paris', 'score': 0.82, 'norm_score': 1.0},
                {'entity': 'London', 'score': 0.35, 'norm_score': 0.43}]}
```
```

--------------------------------

### Relation Text Attributes Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Example of text attributes for a relation, including a description.

```text
name: lived_in
description: A person has a habitual presence in a specific location.
```

--------------------------------

### Entity Node Text Attributes Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Example of text attributes for an entity node.

```text
name: Barack Obama
type: entity
```

--------------------------------

### Initialize Qwen3TextEmbModel

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models?q=

Instantiate the Qwen3TextEmbModel with various configuration options. Specify the model name, normalization, batch size, instruction prompts, truncation dimensions, and vLLM server details.

```python
class Qwen3TextEmbModel(BaseTextEmbModel):
    """A text embedding model class that extends BaseTextEmbModel specifically for Qwen3 embedding models.

    Args:
        text_emb_model_name (str): Name or path of the SentenceTransformer model to use
        normalize (bool, optional): Whether to L2-normalize the embeddings. Defaults to False.
        batch_size (int, optional): Batch size for encoding. Defaults to 32.
        query_instruct (str | None, optional): Instruction/prompt to prepend to queries. Defaults to None.
        passage_instruct (str | None, optional): Instruction/prompt to prepend to passages. Defaults to None.
        truncate_dim (int | None, optional): Dimension to truncate the embeddings to. Defaults to None.
        model_kwargs (dict | None, optional): Additional keyword arguments for the model. Defaults to None.
        tokenizer_kwargs (dict | None, optional): Additional keyword arguments for the tokenizer. Defaults to None.
        api_base (str, optional): Base URL for the vLLM server. If no URL is provided, a local server will be started.
        api_key (str, optional): API key for authentication. Defaults to "EMPTY".
        vllm_timeout (int, optional): Timeout for vLLM requests in seconds. Defaults to 600.

    Attributes:
        client (OpenAI): The OpenAI client for making requests to vLLM server
        async_client (AsyncOpenAI): The async OpenAI client for concurrent requests
        text_emb_model_name (str): Name of the model being used
        normalize (bool): Whether embeddings are L2-normalized
        batch_size (int): Batch size used for encoding
        query_instruct (str | None): Instruction text for queries
        passage_instruct (str | None): Instruction text for passages
        truncate_dim (int | None): Dimension to truncate the embeddings to
        model_kwargs (dict | None): Additional model configuration parameters
        tokenizer_kwargs (dict | None): Additional tokenizer configuration parameters

    Methods:
        encode(text: list[str], is_query: bool = False, show_progress_bar: bool = True) -> torch.Tensor:
            Encodes a list of texts into embeddings.
    """

    def __init__(
        self,
        text_emb_model_name: str,
        normalize: bool = False,
        batch_size: int = 32,
        query_instruct: str | None = None,
        passage_instruct: str | None = None,
        truncate_dim: int | None = None,
        model_kwargs: dict | None = None,
        tokenizer_kwargs: dict | None = None,
        api_base: str | None = None,
        api_key: str = "EMPTY",
        vllm_timeout: int = 600,

```

--------------------------------

### LLMOPENIEModel Initialization and Usage

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/openie_model?q=

Demonstrates how to initialize the LLMOPENIEModel with default or custom parameters and perform OpenIE on a given text.

```APIDOC
## LLMOPENIEModel

### Description
A class for performing Open Information Extraction (OpenIE) using Large Language Models. This class implements OpenIE functionality by performing Named Entity Recognition (NER) and relation extraction using various LLM backends like OpenAI, Together, Ollama, or llama.cpp.

### Parameters
#### Initialization Parameters
- **llm_api** (Literal['openai', 'together', 'ollama', 'llama.cpp']) - Optional - The LLM backend to use. Defaults to "openai".
- **model_name** (str) - Optional - Name of the specific model to use. Defaults to "gpt-4o-mini".
- **max_ner_tokens** (int) - Optional - Maximum number of tokens for NER output. Defaults to 1024.
- **max_triples_tokens** (int) - Optional - Maximum number of tokens for relation triples output. Defaults to 4096.

### Methods
#### `__call__`
- **Description**: Main method to perform complete OpenIE pipeline.
- **Usage**: `openie_model(text)`

### Request Example
```python
openie_model = LLMOPENIEModel()
result = openie_model("Emmanuel Macron is the president of France")
print(result)
```

### Response Example
```json
{
  "passage": "Emmanuel Macron is the president of France",
  "extracted_entities": ["Emmanuel Macron", "France"],
  "extracted_triples": [["Emmanuel Macron", "president of", "France"]]
}
```
```

--------------------------------

### Serve Documentation Locally

Source: https://rmanluo.github.io/gfm-rag/latest/DEVELOPING?q=

Builds and serves the documentation website locally using MkDocs. This command is useful for previewing changes to the documentation.

```bash
mkdocs serve

```

--------------------------------

### Install gfmrag with Pip

Source: https://rmanluo.github.io/gfm-rag/latest/install?q=

A straightforward method to install gfmrag using pip, suitable if you already have a Python environment set up.

```bash
pip install gfmrag

```

--------------------------------

### Install gfmrag with Conda

Source: https://rmanluo.github.io/gfm-rag/latest/install?q=

Use this method to create a new Conda environment and install gfmrag along with CUDA toolkit.

```bash
conda create -n gfmrag python=3.12
conda activate gfmrag
conda install cuda-toolkit -c nvidia/label/cuda-12.6.3
pip install gfmrag

```

--------------------------------

### Initialize QAPromptBuilder

Source: https://rmanluo.github.io/gfm-rag/latest/api/prompt_builder

Instantiate the QAPromptBuilder with a configuration dictionary. The configuration must include system prompt, document prompt, and question prompt templates, and can optionally include few-shot examples.

```python
class QAPromptBuilder:
    """A class for building prompts for question-answering tasks.

    This class constructs formatted prompts for Q&A systems using a configuration-based approach.
    It supports system prompts, document contexts, questions, and optional few-shot examples.

    Args:
        prompt_cfg (DictConfig): Configuration dictionary containing:
            - system_prompt: The system instruction prompt
            - doc_prompt: Template for formatting document context
            - question_prompt: Template for formatting the question
            - examples: List of few-shot examples (optional)

    Methods:
        build_input_prompt(question, retrieved_result, thoughts=None):
            Builds a formatted prompt list for the Q&A system.

            Args:
                question (str): The input question to be answered
                retrieved_result (dict[str, list[dict]]): Retrieved nodes keyed by target type.
                    Each entry contains dicts with keys: id, type, attributes, score.
                thoughts (list, optional): Additional thought process or reasoning steps

            Returns:
                list: A list of dictionaries containing the formatted prompt with roles
                    and content for each component
    """

    def __init__(self, prompt_cfg: DictConfig) -> None:
        self.cfg = prompt_cfg
        self.system_prompt = self.cfg.system_prompt
        self.doc_prompt = self.cfg.doc_prompt
        self.question_prompt = self.cfg.question_prompt
        self.examples = self.cfg.examples
```

--------------------------------

### Document Node Text Attributes Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Example of text attributes for a document node, including title and published year.

```text
name: Obama Biography
type: document
title: The Life of Barack Obama
published_year: 2020
```

--------------------------------

### __init__

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/openie_model

Initializes the LLM-based OpenIE model with specified configurations for LLM API, model name, and token limits.

```APIDOC
## __init__

### Description
Initialize LLM-based OpenIE model.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **llm_api** (Literal['openai', 'nvidia', 'together', 'ollama', 'llama.cpp']) - Optional - The LLM API provider to use. Defaults to "openai".
- **model_name** (str) - Optional - Name of the language model to use. Defaults to "gpt-4o-mini".
- **max_ner_tokens** (int) - Optional - Maximum number of tokens for NER processing. Defaults to 1024.
- **max_triples_tokens** (int) - Optional - Maximum number of tokens for triple extraction. Defaults to 4096.
- **n_ctx** (int | None) - Optional - Context window size (llama.cpp / ollama only)
- **low_vram** (bool) - Optional - Enable low VRAM mode (llama.cpp only). Defaults to False.

### Attributes
- **llm_api** (str) - The selected LLM API provider
- **model_name** (str) - Name of the language model
- **max_ner_tokens** (int) - Token limit for NER
- **max_triples_tokens** (int) - Token limit for triples
- **client** - Initialized language model client

### Request Example
```python
from gfmrag.graph_index_construction.openie_model import LLMOpenIEModel

model = LLMOpenIEModel(llm_api='openai', model_name='gpt-4o-mini')
```

### Response
None
```

--------------------------------

### Raw Documents JSON Example

Source: https://rmanluo.github.io/gfm-rag/latest/getting_started/quick_start

Example of the `documents.json` file, mapping document IDs to their text content. This is required for GFMRetriever to build the graph.

```json
{
  "France": "France is a country in Western Europe. Paris is its capital.",
  "Paris": "Paris is the capital and most populous city of France.",
  "Emmanuel Macron": "Emmanuel Macron has served as president of France since 2017."
}
```

--------------------------------

### Start Local vLLM Server

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models

Starts a local vLLM server for embedding generation. It temporarily modifies environment variables to configure the vLLM worker and then restores them.

```python
def _start_vllm_server(self) -> LLM:
        """Start a vLLM server for embedding generation."""

        dist_keys = [
            "RANK",
            "LOCAL_RANK",
            "WORLD_SIZE",
            "LOCAL_WORLD_SIZE",
            "GROUP_RANK",
            "ROLE_RANK",
            "ROLE_NAME",
            "OMP_NUM_THREADS",
            "MASTER_ADDR",
            "MASTER_PORT",
            "TORCHELASTIC_USE_AGENT_STORE",
            "TORCHELASTIC_MAX_RESTARTS",
            "TORCHELASTIC_RUN_ID",
            "TORCH_NCCL_ASYNC_ERROR_HANDLING",
            "TORCHELASTIC_ERROR_FILE",
        ]
        old_env = {}
        for dist_key in dist_keys:
            if dist_key in os.environ:
                old_env[dist_key] = os.environ.pop(dist_key)
        os.environ["CUDA_VISIBLE_DEVICES"] = old_env.get("LOCAL_RANK", "0")
        os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
        text_emb_model = LLM(
            model=self.text_emb_model_name,
            enforce_eager=True,
            task="embed",
            hf_overrides={"is_matryoshka": True},
        )
        # Restore environment variables
        os.environ.pop("CUDA_VISIBLE_DEVICES")
        for dist_key, dist_value in old_env.items():
            if dist_value is not None:
                os.environ[dist_key] = dist_value
        return text_emb_model

```

--------------------------------

### NVEmbedV2ELModel Initialization and Usage

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/el_model

Demonstrates how to initialize the NVEmbedV2ELModel, index entities, and perform entity linking.

```APIDOC
## NVEmbedV2ELModel

### Description
A DPR-based Entity Linking model specialized for NVEmbed V2 embeddings. This class extends DPRELModel with specific adaptations for handling NVEmbed V2 models, including increased sequence length and right-side padding.

### Attributes
- `model` (model): The underlying model with max_seq_length of 32768 and right-side padding.

### Methods
- `__init__(*args, **kwargs)`: Initializes the DPR Entity Linking model with specific parameters for NVEmbed V2.
- `add_eos(input_examples)`: Adds EOS token to input examples.
- `__call__(ner_entity_list, *args, **kwargs)`: Processes entity list with EOS tokens before linking.

### Example
```python
model = NVEmbedV2ELModel('nvidia/NV-Embed-v2', query_instruct="Instruct: Given a entity, retrieve entities that are semantically equivalent to the given entity\nQuery: ")
model.index(['Paris', 'London', 'Berlin'])
results = model(['paris city'], topk=2)
print(results)
```

### Example Output
```json
{
  "paris city": [
    {
      "entity": "Paris",
      "score": 0.82,
      "norm_score": 1.0
    },
    {
      "entity": "London",
      "score": 0.35,
      "norm_score": 0.43
    }
  ]
}
```
```

--------------------------------

### Optional Test JSON Example

Source: https://rmanluo.github.io/gfm-rag/latest/getting_started/quick_start

Example of the optional `test.json` file, used for plain retrieval, QA, and evaluation. It includes questions, answers, and supporting document IDs.

```json
[
  {
    "id": "toy-1",
    "question": "Who is the president of France?",
    "answer": "Emmanuel Macron",
    "answer_aliases": ["Macron"],
    "supporting_documents": ["France", "Emmanuel Macron"]
  }
]
```

--------------------------------

### Initialize Qwen3TextEmbModel with vLLM

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models?q=

Initializes the model, setting up either a local vLLM server or connecting to an existing API endpoint. Handles environment variables for distributed training and model configuration.

```python
def __init__(self,
            text_emb_model_name: str,
            normalize: bool = False,
            batch_size: int = 32,
            query_instruct: str | None = None,
            passage_instruct: str | None = None,
            truncate_dim: int | None = None,
            model_kwargs: dict | None = None,
            tokenizer_kwargs: dict | None = None,
            api_base: str | None = None,
            api_key: str | None = "EMPTY",
            vllm_timeout: int = 600,
    ) -> None:
        """
        Initialize the BaseTextEmbModel.

        Args:
            text_emb_model_name (str): Name or path of the SentenceTransformer model to use
            normalize (bool, optional): Whether to L2-normalize the embeddings. Defaults to False.
            batch_size (int, optional): Batch size for encoding. Defaults to 32.
            query_instruct (str | None, optional): Instruction/prompt to prepend to queries. Defaults to None.
            passage_instruct (str | None, optional): Instruction/prompt to prepend to passages. Defaults to None.
            truncate_dim (int | None, optional): Dimension to truncate the embeddings to. Defaults to None.
            model_kwargs (dict | None, optional): Additional keyword arguments for the model. Defaults to None.
            tokenizer_kwargs (dict | None, optional): Additional keyword arguments for the tokenizer. Defaults to None.
            api_base (str | None, optional): Base URL for the vLLM server. If no url is provided we would start a local server.
            api_key (str | None, optional): API key for authentication. Defaults to "EMPTY".
            vllm_timeout (int, optional): Timeout for vLLM requests in seconds. Defaults to 600.
        """
        self.text_emb_model_name = text_emb_model_name
        self.normalize = normalize
        self.batch_size = batch_size
        self.query_instruct = query_instruct
        self.passage_instruct = passage_instruct
        self.truncate_dim = truncate_dim
        self.model_kwargs = model_kwargs
        self.tokenizer_kwargs = tokenizer_kwargs
        self.api_base = api_base
        self.api_key = api_key
        self.vllm_timeout = vllm_timeout

        if api_base is None:
            self.text_emb_model = self._start_vllm_server()
        else:
            # Check if API is available
            if not self._is_api_available():
                raise RuntimeError("vLLM API is not available")

            self.client = OpenAI(
                api_key=self.api_key,
                base_url=self.api_base,
            )
```

--------------------------------

### Append EOS Token to Input Examples

Source: https://rmanluo.github.io/gfm-rag/latest/api/graph_construction/el_model

Appends the End of Sequence (EOS) token to each string in a list of input examples. This is a utility function for preparing data before entity linking.

```python
def add_eos(self, input_examples: list[str]) -> list[str]:
    """
    Appends EOS (End of Sequence) token to each input example in the list.

    Args:
        input_examples (list[str]): List of input text strings.

    Returns:
        list[str]: List of input texts with EOS token appended to each example.
    """
    input_examples = [
        input_example + self.model.tokenizer.eos_token
        for input_example in input_examples
    ]
    return input_examples
```

--------------------------------

### KG Constructor Configuration Example

Source: https://rmanluo.github.io/gfm-rag/latest/config/graph_constructor_config?q=

This YAML configuration defines parameters for the KGConstructor class. It specifies models for OpenIE and entity linking, temporary storage locations, processing details, and criteria for adding similarity-based edges.

```yaml
_target_: gfmrag.graph_index_construction.graph_constructors.KGConstructor
open_ie_model: ${openie_model}
el_model: ${el_model}
root: tmp/kg_construction
num_processes: 10
cosine_sim_edges: True
threshold: 0.8
max_sim_neighbors: 100
add_title: True
force: False

```

--------------------------------

### NVEmbedV2.add_eos

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models?q=

Adds an End-Of-Sequence (EOS) token to each input text example.

```APIDOC
## add_eos

### Description
Adds EOS token to each input example.

### Method
`add_eos`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **input_examples** (list[str]) - Required - List of text strings to which EOS tokens will be added.

### Request Example
```python
texts_with_eos = encoder.add_eos(["Hello world", "Another text"])
```

### Response
#### Success Response (200)
- **list[str]** - A list of strings, where each string has the EOS token appended.

#### Response Example
```json
["Hello world<|endoftext|>", "Another text<|endoftext|>"]
```
```

--------------------------------

### Get Processed Directory Path

Source: https://rmanluo.github.io/gfm-rag/latest/api/datasets

Returns the path to the directory where processed data is stored.

```python
@property
    def processed_dir(self) -> str:
        return os.path.join(

```

--------------------------------

### Raw Input Layout Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

This layout includes raw graph source files and optional QA files. The workflow constructs graph and QA files if only raw data is provided.

```text
root/
└── data_name/
    └── raw/
        ├── documents.json
        ├── train.json (optional)
        └── test.json (optional)
```

--------------------------------

### Get Number of Relations

Source: https://rmanluo.github.io/gfm-rag/latest/api/datasets

Returns the number of edge types in the graph, representing the number of relations.

```python
@property
    def num_relations(self) -> int:
        return self.graph.num_edge_types
```

--------------------------------

### Verify Wandb Integration

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config?q=

Check that wandb is installed and accessible by printing its version and API key status.

```python
import wandb

print(f"Wandb version: {wandb.__version__}")
print(f"Wandb is available: {wandb.api.api_key is not None}")
```

--------------------------------

### Configure and Run Supervised Fine-tuning

Source: https://rmanluo.github.io/gfm-rag/latest/experiment/g_reasoner?q=

Set parameters for supervised fine-tuning, including number of GPUs, epochs, batch size, embedding dimensions, and network layers.

```bash
N_GPU=4
N_EPOCH=10
BATCH_SIZE=4
N_DIM=1024
N_LAYERS="[${N_DIM},${N_DIM},${N_DIM},${N_DIM},${N_DIM},${N_DIM}]"
DATA_ROOT="data"
```

--------------------------------

### Run Basic Training with Wandb

Source: https://rmanluo.github.io/gfm-rag/latest/config/wandb_config?q=

Execute pre-training or fine-tuning scripts as usual; wandb integration is enabled by default.

```bash
# Pre-training with wandb logging
python -m gfmrag.workflow.kgc_training --config-path config/gfm_rag

# Fine-tuning with wandb logging
python -m gfmrag.workflow.sft_training --config-path config/gfm_rag

```

--------------------------------

### relations.csv File Format Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Defines relations and their attributes. The 'attributes' field can store descriptions for relations.

```csv
name,attributes
lived_in,"{'description': 'A person has a habitual presence in a specific location.'}"
mentioned_in,"{'description': 'An entity is mentioned in the document}'}
```

--------------------------------

### Initialize Qwen3TextEmbModel with vLLM

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models

Initializes the BaseTextEmbModel, supporting local vLLM server startup or connection to an existing API. Handles model and tokenizer arguments, and optional instruction prefixes for queries and passages.

```python
def __init__(
        self,
        text_emb_model_name: str,
        normalize: bool = False,
        batch_size: int = 32,
        query_instruct: str | None = None,
        passage_instruct: str | None = None,
        truncate_dim: int | None = None,
        model_kwargs: dict | None = None,
        tokenizer_kwargs: dict | None = None,
        api_base: str | None = None,
        api_key: str | None = "EMPTY",
        vllm_timeout: int = 600,
    ) -> None:
        """
        Initialize the BaseTextEmbModel.

        Args:
            text_emb_model_name (str): Name or path of the SentenceTransformer model to use
            normalize (bool, optional): Whether to L2-normalize the embeddings. Defaults to False.
            batch_size (int, optional): Batch size for encoding. Defaults to 32.
            query_instruct (str | None, optional): Instruction/prompt to prepend to queries. Defaults to None.
            passage_instruct (str | None, optional): Instruction/prompt to prepend to passages. Defaults to None.
            truncate_dim (int | None, optional): Dimension to truncate the embeddings to. Defaults to None.
            model_kwargs (dict | None, optional): Additional keyword arguments for the model. Defaults to None.
            tokenizer_kwargs (dict | None, optional): Additional keyword arguments for the tokenizer. Defaults to None.
            api_base (str | None, optional): Base URL for the vLLM server. If no url is provided we would start a local server.
            api_key (str | None, optional): API key for authentication. Defaults to "EMPTY".
            vllm_timeout (int, optional): Timeout for vLLM requests in seconds. Defaults to 600.
        """
        self.text_emb_model_name = text_emb_model_name
        self.normalize = normalize
        self.batch_size = batch_size
        self.query_instruct = query_instruct
        self.passage_instruct = passage_instruct
        self.truncate_dim = truncate_dim
        self.model_kwargs = model_kwargs
        self.tokenizer_kwargs = tokenizer_kwargs
        self.api_base = api_base
        self.api_key = api_key
        self.vllm_timeout = vllm_timeout

        if api_base is None:
            self.text_emb_model = self._start_vllm_server()
        else:
            # Check if API is available
            if not self._is_api_available():
                raise RuntimeError("vLLM API is not available")

            self.client = OpenAI(
                api_key=self.api_key,
                base_url=self.api_base,
            )

```

--------------------------------

### nodes.csv File Format Example

Source: https://rmanluo.github.io/gfm-rag/latest/workflow/data_format

Defines nodes and their attributes. The 'attributes' field is a JSON-formatted string for arbitrary structured attributes.

```csv
name,type,attributes
"Barack Obama","entity","{}"
"White House","entity","{}"
"Obama Biography","document","{'title': 'The Life of Barack Obama', 'published_year': 2020}"
```

--------------------------------

### Initialize Qwen3TextEmbModel

Source: https://rmanluo.github.io/gfm-rag/latest/api/text_emb_models

Instantiate the Qwen3TextEmbModel with various configuration options for text embedding generation using Qwen3 models.

```python
class Qwen3TextEmbModel(BaseTextEmbModel):
    """A text embedding model class that extends BaseTextEmbModel specifically for Qwen3 embedding models.

    Args:
        text_emb_model_name (str): Name or path of the SentenceTransformer model to use
        normalize (bool, optional): Whether to L2-normalize the embeddings. Defaults to False.
        batch_size (int, optional): Batch size for encoding. Defaults to 32.
        query_instruct (str | None, optional): Instruction/prompt to prepend to queries. Defaults to None.
        passage_instruct (str | None, optional): Instruction/prompt to prepend to passages. Defaults to None.
        truncate_dim (int | None, optional): Dimension to truncate the embeddings to. Defaults to None.
        model_kwargs (dict | None, optional): Additional keyword arguments for the model. Defaults to None.
        tokenizer_kwargs (dict | None, optional): Additional keyword arguments for the tokenizer. Defaults to None.
        api_base (str, optional): Base URL for the vLLM server. If no URL is provided, a local server will be started.
        api_key (str, optional): API key for authentication. Defaults to "EMPTY".
        vllm_timeout (int, optional): Timeout for vLLM requests in seconds. Defaults to 600.

    Attributes:
        client (OpenAI): The OpenAI client for making requests to vLLM server
        async_client (AsyncOpenAI): The async OpenAI client for concurrent requests
        text_emb_model_name (str): Name of the model being used
        normalize (bool): Whether embeddings are L2-normalized
        batch_size (int): Batch size used for encoding
        query_instruct (str | None): Instruction text for queries
        passage_instruct (str | None): Instruction text for passages
        truncate_dim (int | None): Dimension to truncate the embeddings to
        model_kwargs (dict | None): Additional model configuration parameters
        tokenizer_kwargs (dict | None): Additional tokenizer configuration parameters

    Methods:
        encode(text: list[str], is_query: bool = False, show_progress_bar: bool = True) -> torch.Tensor:
            Encodes a list of texts into embeddings.
    """

    def __init__(
        self,
        text_emb_model_name: str,
        normalize: bool = False,
        batch_size: int = 32,
        query_instruct: str | None = None,
        passage_instruct: str | None = None,
        truncate_dim: int | None = None,
        model_kwargs: dict | None = None,
        tokenizer_kwargs: dict | None = None,
        api_base: str | None = None,
        api_key: str = "EMPTY",
        vllm_timeout: int = 600,
    ):
        pass
```
Tokens:
Raw