### Install and Run LLM-IE Web Application from Source

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/installation.md

These commands provide an alternative method to install and run the LLM-IE web application by cloning the source repository, installing its Python dependencies, and then executing the main application script. This method requires a local Python environment.

```bash
# Clone source code
git clone https://github.com/daviden1013/llm-ie.git

# Install requirements
pip install -r llm-ie/web_app/requirements.txt

# Run Web App
cd llm-ie/web_app
python run.py
```

--------------------------------

### Defining an Example for Prompt Template

Source: https://github.com/daviden1013/llm-ie/blob/main/demo/prompt_template_writing.ipynb

This snippet defines a string variable `example` containing a structured input-output pair. This example is intended to be inserted into the final prompt template to guide the LLM on the expected extraction format and behavior, demonstrating how to handle diagnosis, date, and status.

```Python
example = """
Input: #### Past Medical History
- Hypertension (diagnosed in 2010)...

Output: [{"Diagnosis": "Hypertension", "Date": "01/01/2010", "Status": "history"}]
"""
```

--------------------------------

### Final Prompt Template with Examples Placeholder

Source: https://github.com/daviden1013/llm-ie/blob/main/demo/prompt_template_writing.ipynb

This multi-line string defines `prompt_template_final`, the complete and refined prompt template. It incorporates all previous elements, including task description, schema, output format, and crucially, a placeholder `{{examples}}` for injecting specific examples to further guide the LLM's extraction process.

```Python
prompt_template_final = """
# Task description
The text below is from a clinical note. Your task is to extract the diagnosis information in a given sentence (provided by user at a time). Note that diagnosis can have associated dates and statuses.

# Schema definition
Your output should contain: 
    Must have, "Diagnosis" which is the name of the diagnosis spelled exactly as in the source document,
    If applicable, "Date" which is the date of the diagnosis following "MM/DD/YYYY" format. If only part of the date is know, default to first day of year/ month.
    If applicable, "Status" which is the status of the diagnosis (e.g. active, resolved),

# Output format definition
Your output should follow JSON format, 
if there are diagnosis mentions:
    [{"entity_text": "<Diagnosis text>", "attr": {"Date": "<date>", "Status": "<status>"}},
    {"entity_text": "<Diagnosis text>", "attr": {"Date": "<date>", "Status": "<status>"}}]
if there is no diagnosis in the given sentence, just output an empty list:
    []

# Examples
{{examples}}

# Additional hints
    1. Your output should be 100% based on the provided content. DO NOT output fake information. 
    2. If there is no specific date or status, just omit the "Date" or "Status" key while still output the Diagnosis.

"""
```

--------------------------------

### Run LLM-IE Web Application with Docker (Default Port)

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/installation.md

These commands pull the latest LLM-IE web application Docker image and run it, mapping the container's port 5000 to the host's port 5000. Access the application via http://localhost:5000 in your browser.

```bash
docker pull daviden1013/llm-ie-web-app:latest
docker run -p 5000:5000 daviden1013/llm-ie-web-app:latest
```

--------------------------------

### Install LLM-IE Python Package via pip

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/installation.md

This command installs the LLM-IE Python package from PyPI. Users must ensure an LLM inference engine is separately installed as this package does not include them, but provides built-in support for several popular engines.

```bash
pip install llm-ie
```

--------------------------------

### Python Code to Access Prompt Writing Guide

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/prompt_templates.md

This Python snippet demonstrates how to programmatically access the prompt writing guide provided by the `DirectFrameExtractor` class. It imports the necessary class and calls the `get_prompt_guide()` static method to retrieve documentation on best practices for prompt template design.

```python
from llm_ie.extractors import DirectFrameExtractor

print(DirectFrameExtractor.get_prompt_guide())
```

--------------------------------

### Run LLM-IE Web Application with Docker (Custom Port)

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/installation.md

This command runs the LLM-IE web application Docker image, mapping the container's port 5000 to a custom host port, such as 8080. This is useful if the default port 5000 is already in use, allowing access via http://localhost:8080.

```bash
docker run -p 8080:5000 daviden1013/llm-ie-web-app:latest
```

--------------------------------

### Install ie-viz Package

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/visualization.md

This command installs the `ie-viz` package, which provides visualization capabilities for `LLMInformationExtractionDocument` objects. It is a prerequisite for using the visualization features.

```cmd
pip install ie-viz
```

--------------------------------

### Install LLM-IE Python Package

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This command installs the LLM-IE library from PyPI. It's important to note that this package does not automatically install LLM inference engines; these must be installed separately according to the project's prerequisites.

```bash
pip install llm-ie
```

--------------------------------

### Example Prompt Template for Binary Relation Extraction

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BinaryRelationExtractor_prompt_guide.txt

This example demonstrates a complete prompt template for a binary relation extraction task. It illustrates how to define the task's purpose, specify the schema for true/false relations, define the required JSON output format, provide guiding hints, and include placeholders for the two entities and the region of interest (ROI) text.

```Prompt Template
    # Task description
    This is a binary relation extraction task. Given a region of interest (ROI) text and two entities from a medical note, indicate the relation existence between the two entities.

    # Schema definition
        True: if there is a relationship between a medication name (one of the entities) and its strength or frequency (the other entity).
        False: Otherwise.

    # Output format definition
    Your output should follow the JSON format:
    {"Relation": "<True or False>"}

    I am only interested in the content between []. Do not explain your answer. 

    # Hints
        1. Your input always contains one medication entity and 1) one strength entity or 2) one frequency entity.
        2. Pay attention to the medication entity and see if the strength or frequency is for it.
        3. If the strength or frequency is for another medication, output False. 
        4. If the strength or frequency is for the same medication but at a different location (span), output False.

    # Entity 1 full information:
    {{frame_1}}

    # Entity 2 full information:
    {{frame_2}}

    # Input placeholders
    ROI Text with the two entities annotated with <entity_1> and <entity_2>:
    "{{roi_text}}"
```

--------------------------------

### Install LLM-IE Web Application using Docker

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/web_application.md

Instructions to pull the LLM-IE web application Docker image from Docker Hub and run it locally, making it accessible on port 5000.

```bash
docker pull daviden1013/llm-ie-web-app:latest
docker run -p 5000:5000 daviden1013/llm-ie-web-app:latest
```

--------------------------------

### Example LLM Prompt Template for Attribute Extraction

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/AttributeExtractor_prompt_guide.txt

This snippet provides a concrete example of a prompt template designed for attribute extraction, demonstrating the application of the previously defined design principles. It includes sections for task description, schema, output format, hints, entity, and context placeholders, guiding the LLM to extract specific attributes in a defined JSON format.

```LLM Prompt
### Task description
This is an attribute extraction task. Given a diagnosis entity and the context, you need to generate attributes for the entity. 

### Schema definition
    "Date" which is the date when the diagnosis was made in MM/DD/YYYY format,
    "Status" which is the current status of the diagnosis (e.g. active, resolved, etc.)

### Output format definition
Your output should follow the JSON format:
{"Date": "<MM/DD/YYYY>", "Status": "<status>"}

I am only interested in the content between []. Do not explain your answer. 

### Hints
- If the date is not complete, use the first available date in the context. For example, if the date is 01/2023, you should return 01/01/2023.
- If the status is not available, you should return "not specified".

### Entity
Information about the entity to extract attributes from:
{{frame}}

### Context
Context for the entity. The <Entity> tags are used to mark the entity in the context.
{{context}}
```

--------------------------------

### Start vLLM OpenAI Compatible Server

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

This command starts the vLLM OpenAI compatible server. It allows specifying the GPUs to use with CUDA_VISIBLE_DEVICES, the number of GPUs for tensor parallelism, and an optional API key. The default port is 8000, which can be changed using --port.

```cmd
CUDA_VISIBLE_DEVICES=<GPU#> vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct --api-key MY_API_KEY --tensor-parallel-size <# of GPUs to use>
```

--------------------------------

### Configure vLLM Inference Engine

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This snippet outlines the steps to use vLLM as an inference engine. It includes starting the vLLM server and then defining the OpenAIInferenceEngine to connect to it, following the OpenAI compatible server specification.

```cmd
vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct
```

```python
from llm_ie.engines import OpenAIInferenceEngine
inference_engine = OpenAIInferenceEngine(base_url="http://localhost:8000/v1",
                                         api_key="EMPTY",
                                         model="meta-llama/Meta-Llama-3.1-8B-Instruct")
```

--------------------------------

### Example Prompt: Extracting Adverse Reactions and Percentages

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt

A complete prompt template example for extracting adverse reactions and their percentages from FDA clinical pharmacology text, demonstrating the application of the prompt design principles for a nested JSON output.

```APIDOC
### Task description
The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.

### Schema definition
Your output should contain:
    "entity_text" which is the name of the clinical trial as spelled in the document,
    If applicable, "Arm" which is the arm within the clinical trial,
    "AdverseReaction" which is the name of the adverse reaction,
    If applicable, "Percentage" which is the occurance of the adverse reaction within the trial and arm,
    "Evidence" which is the EXACT sentence in the text where you found the AdverseReaction from

### Output format definition
Your output should follow JSON format, for example:
[
    {"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}},
    {"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}}
]

### Additional hints
- Your output should be 100% based on the provided content. DO NOT output fake numbers.
- You are expected to generate the output instead of writting code or insturcting other agents to do so.
- If there is no specific arm, just omit the "Arm" key. If the percentage is not reported, just omit the "Percentage" key. The "Evidence" should always be provided.

### Context
The text below is from the Adverse reactions section:
"{{input}}"
```

--------------------------------

### Example: Drug Entity Extraction Output

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt

An example of structured output for drug-related entities, including drug name, strength, frequency, and route, demonstrating a different entity type structure.

```JSON
{"entity_text": "Mesalamine DR", "attr": {"entity_type": "Drug"}},
{"entity_text": "1200 mg", "attr": {"entity_type": "Strength"}},
{"entity_text": "BID", "attr": {"entity_type": "Frequency"}},
{"entity_text": "PO", "attr": {"entity_type": "Route"}}
```

--------------------------------

### Initializing Prompt Editor and Starting Chat

Source: https://github.com/daviden1013/llm-ie/blob/main/demo/prompt_template_writing_via_chat.ipynb

Creates a `PromptEditor` instance, passing the previously defined LLM inference engine and the `SentenceFrameExtractor` as arguments. Subsequently, it initiates an interactive chat session with the prompt editor, allowing users to refine prompt templates.

```python
editor = PromptEditor(llm, SentenceFrameExtractor)
editor.chat()
```

--------------------------------

### Configure OpenAI Inference Engine for Compatible Services

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

Example of setting up OpenAIInferenceEngine for OpenAI-compatible services like OpenRouter, by providing a custom base_url and model name.

```python
from llm_ie.engines import OpenAIInferenceEngine

inference_engine = OpenAIInferenceEngine(base_url="https://openrouter.ai/api/v1", model="meta-llama/llama-4-scout")
```

--------------------------------

### Define and Start LLM Agent for Prompt Engineering

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This Python snippet demonstrates how to set up and initiate an interactive LLM agent for prompt engineering. It defines the inference engine, configures the LLM, and starts a chat session for iterative prompt refinement.

```python
from llm_ie import OpenAIInferenceEngine, BasicLLMConfig, DirectFrameExtractor, PromptEditor, SentenceUnitChunker, SlideWindowContextChunker

# Define a LLM inference engine for the prompt editor
prompt_editor_llm = OpenAIInferenceEngine(base_url="https://openrouter.ai/api/v1", 
                                          model="meta-llama/llama-4-scout", 
                                          api_key=os.getenv("OPENROUTER_API_KEY"),
                                          config=BasicLLMConfig(temperature=0.4, 
                                                                max_new_tokens=4096))
# Define LLM prompt editor
editor = PromptEditor(prompt_editor_llm, DirectFrameExtractor)
# Start chat
editor.chat()
```

--------------------------------

### Example Prompt Template for DirectFrameExtractor

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This is an example of a prompt template drafted by the LLM agent, designed for the DirectFrameExtractor. It outlines the task of extracting diagnoses, dates, and statuses from clinical notes.

```text
### Task description
The paragraph below contains a clinical note with diagnoses listed. Please carefully review it and extract the diagnoses, including the diagnosis date and status.
```

--------------------------------

### Example 2: Simple Event Extraction from Medical Note

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt

Shows a simpler example of 'EVENT' entity extraction from a medical note, illustrating the JSON output for a single event mention.

```Prompt
Input: At that time it appeared well adhered to the underlying skin .
```

```JSON
[
    {"entity_text": "it", "attr": {"entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}}
]
```

--------------------------------

### Configure Azure OpenAI Inference Engine

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This snippet shows how to initialize the AzureOpenAIInferenceEngine. It requires specifying the model and API version, with endpoint and API key setup as per Azure documentation.

```python
from llm_ie.engines import AzureOpenAIInferenceEngine

inference_engine = AzureOpenAIInferenceEngine(model="gpt-4o-mini", 
                                              api_version="<your api version>")
```

--------------------------------

### JSON Example: Event and TIMEX3 Extraction from Medical Note

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicReviewFrameExtractor_prompt_guide.txt

An example demonstrating the extraction of 'EVENT' and 'TIMEX3' entities from a sample medical note, showing how various attributes are populated in the JSON output for each identified entity.

```JSON
[
            {"entity_text": "intravenous fluids", "attr": {"entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}},
            {"entity_text": "decreased", "attr": {"entity_type": "EVENT", "type": "OCCURRENCE", "polarity": "POS", "modality": "FACTUAL"}},
            {"entity_text": "total fluids", "attr": {"entity_type": "EVENT", "type": "TREATMENT", "polarity": "POS", "modality": "FACTUAL"}},
            {"entity_text": "9/7/93 , 1:00 a.m.", "attr": {"entity_type": "TIMEX3", "type": "TIME", "val": "1993-09-07T01:00", "mod": "NA"}},
            {"entity_text": "24 hours", "attr": {"entity_type": "TIMEX3", "type": "DURATION", "val": "PT24H", "mod": "NA"}}
        ]
```

--------------------------------

### Medical Note Entity Recognition Input/Output Examples

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/SentenceReviewFrameExtractor_prompt_guide.txt

Illustrative examples demonstrating the expected input and corresponding JSON output for medical note entity recognition, showcasing how different entity types are extracted and formatted.

```Natural Language
Input: Acetaminophen 650 mg PO BID 5.
```

```JSON
Output: [
                {"entity_text": "Acetaminophen", "attr": {"entity_type": "Drug"}}, 
                {"entity_text": "650 mg", "attr": {"entity_type": "Strength"}}, 
                {"entity_text": "attr": {"PO", "entity_type": "Route"}}, 
                {"entity_text": "BID", "attr": {"entity_type": "Frequency"}}
            ]
```

```Natural Language
Input: Mesalamine DR 1200 mg PO BID 2.
```

```JSON
Output: [
```

--------------------------------

### LLM Prompt Template Design Principles

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt

Defines the six core components for constructing effective prompt templates for Large Language Model-based information extraction, including task description, schema, output format, hints, examples, and context placeholders.

```Text
1. Task Description:  
   Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition). 

2. Schema Definition:  
   List the key concepts that should be extracted, and provide clear definitions for each one. **Must define "entity_text"** which will be used to recognize the entity.

3. Output Format Definition:  
   The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). **Each dictionary must include a key "entity_text" and a key "attr"**. The attributes are placed in the "attr" (e.g., {"entity_text": "<entity_text>", "attr": {"attr1": "<attr1>", "attr2": "<attr2>"}}).

4. Optional: Hints:  
   Provide itemized hints for the information extractors to guide the extraction process. Remind the prompted agent to be truthful. Emphasize that the prompted agent is supposed to perform the task instead of writting code or instruct other agents to do it.

5. Optional: Examples:  
   Include examples in the format:  
    Input: ...  
    Output: ...

6. Context:  
   The template must include a placeholder in the format {{<placeholder_name>}} for the context. The placeholder name can be customized as needed.
```

--------------------------------

### Bash: Example Output of Attribute Extraction Process

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

This example illustrates the detailed console output generated during the attribute extraction process for a single frame. It displays the input frame's metadata, the contextual text used by the LLM, and the final extracted attributes, demonstrating how the hints are applied.

```bash
Frame: id1
{'frame_id': 'id1', 'start': 1026, 'end': 1040, 'entity_text': 'Hyperlipidemia', 'attr': {'Date': '01/01/2015', 'Status': 'not specified'}}

Context:
...tral chest, radiating to the left arm and jaw. He also experienced dyspnea on exertion and occasional palpitations. The patient denied any recent upper respiratory infection, cough, or fever.

#### Past Medical History
- Hypertension (diagnosed in 2010)
- <entity> Hyperlipidemia </entity> (diagnosed in 2015)
- Type 2 Diabetes Mellitus (diagnosed in 2018)

#### Social History
- Former smoker (quit in 2010)
- Occasional alcohol consumption
- Works as an accountant
- Married with two children

#### Family History
- Father: myocardial infarcti...

Extraction:
[{"Date": "01/01/2015", "Status": "not specified"}]
```

--------------------------------

### Installing and Running LLM-IE Web Application with Docker

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

These commands provide instructions for pulling the latest LLM-IE web application Docker image from Docker Hub and running it locally. The application will be accessible via port 5000 on the host machine, enabling no-code access to the LLM-IE features through a web interface.

```bash
docker pull daviden1013/llm-ie-web-app:latest
docker run -p 5000:5000 daviden1013/llm-ie-web-app:latest
```

--------------------------------

### Initialize DirectFrameExtractor with Unit-Context Schema

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

Demonstrates how to initialize the `DirectFrameExtractor` using `SentenceUnitChunker` for unit definition and `SlideWindowContextChunker` for context. This setup helps LLMs process text efficiently by breaking it into manageable units with surrounding context.

```python
from llm_ie import DirectFrameExtractor, SentenceUnitChunker, SlideWindowContextChunker

unit_chunker = SentenceUnitChunker()
context_chunker = SlideWindowContextChunker(window_size=2)
extractor = DirectFrameExtractor(inference_engine=llm, 
                                 unit_chunker=unit_chunker,
                                 context_chunker=context_chunker,
                                 prompt_template=prompt_template)
```

--------------------------------

### Initialize OpenAIInferenceEngine for vLLM

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

This Python snippet demonstrates how to initialize the OpenAIInferenceEngine to connect to a running vLLM server. The base_url should point to the server's address and port, and the model parameter must exactly match the repository name used when starting the vLLM server.

```python
from llm_ie.engines import OpenAIInferenceEngine
inference_engine = OpenAIInferenceEngine(base_url="http://localhost:8000/v1",
                               api_key="MY_API_KEY",
                               model="meta-llama/Meta-Llama-3.1-8B-Instruct")
```

--------------------------------

### Example Prompt: Annotating Multiple Medical Entities

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/ReviewFrameExtractor_prompt_guide.txt

A complete prompt template example for a named entity recognition task in medical notes, demonstrating the annotation of various entity types like Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, and Duration.

```APIDOC
### Task description
This is a named entity recognition task. Given a text chunk from a medical note, annotate the Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, and Duration.

### Schema definition
Your output should contain:
    "entity_text": the exact wording as mentioned in the note.
    "entity_type": type of the entity. It should be one of the "Drug", "Form", "Strength", "Frequency", "Route", "Dosage", "Reason", "ADE", or "Duration".

### Output format definition
Your output should follow JSON format,
if there are one of the entity mentions: Drug, Form, Strength, Frequency, Route, Dosage, Reason, ADE, or Duration:
    [
        {"entity_text": "<Exact entity mentions as in the note>", "attr": {"entity_type": "<entity type as listed above>"}},
        {"entity_text": "<Exact entity mentions as in the note>", "attr": {"entity_type": "<entity type as listed above>"}}
    ]
if there is no entity mentioned in the given note, just output an empty list:
    []

I am only interested in the extracted contents in []. Do NOT explain your answer.

### Examples
Below are some examples:

Input: Acetaminophen 650 mg PO BID 5.
Output: [
                {"entity_text": "Acetaminophen", "attr": {"entity_type": "Drug"}},
                {"entity_text": "650 mg", "attr": {"entity_type": "Strength"}},
                {"entity_text": "PO", "attr": {"entity_type": "Route"}},
                {"entity_text": "BID", "attr": {"entity_type": "Frequency"}}
            ]

Input: Mesalamine DR 1200 mg PO BID 2.
Output: [
```

--------------------------------

### Initialize SentenceFrameExtractor for Sentence-by-Sentence Extraction

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

Shows how to initialize `SentenceFrameExtractor` for sentence-level extraction, allowing configuration of context sentences. Also provides the equivalent detailed setup using `DirectFrameExtractor` with `SentenceUnitChunker` and `SlideWindowContextChunker` for windowed context.

```python
from llm_ie import SentenceFrameExtractor

# slide window of 2 sentences as context
extractor = SentenceFrameExtractor(inference_engine, prompt_template, context_sentences=2)
```

```python
from llm_ie import DirectFrameExtractor, SentenceUnitChunker, SlideWindowContextChunker

unit_chunker = SentenceUnitChunker()
context_chunker = SlideWindowContextChunker(window_size=2)
extractor = DirectFrameExtractor(inference_engine=llm, 
                                 unit_chunker=unit_chunker,
                                 context_chunker=context_chunker,
                                 prompt_template=prompt_template)
```

--------------------------------

### Serve LLMInformationExtractionDocument Visualization

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/visualization.md

This Python snippet demonstrates how to initialize an `LLMInformationExtractionDocument`, add extracted frames and relations, and then start a local Flask server to visualize the document. The visualization is accessible via a web browser on localhost port 5000 by default.

```python
from llm_ie.data_types import LLMInformationExtractionDocument

# Define document
doc = LLMInformationExtractionDocument(doc_id="Medical note",
                                       text=note_text)
# Add extracted frames and relations to document
doc.add_frames(frames)
doc.add_relations(relations)
# Visualize the document
doc.viz_serve()
```

--------------------------------

### Initialize AttributeExtractor and Define Prompt Structure

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

Explains the purpose of `AttributeExtractor` for detailed attribute extraction, especially when many attributes are needed. It shows how to initialize the extractor and provides an example prompt template, highlighting the required `{{frame}}` and `{{context}}` placeholders.

```python
from llm_ie import AttributeExtractor

extractor = AttributeExtractor(
    inference_engine=llm,
    prompt_template=prompt,
)
```

```text
### Task description
This is an attribute extraction task. Given a diagnosis entity and the context, you need to generate attributes for the entity. 

### Schema definition
    "Date" which is the date when the diagnosis was made in MM/DD/YYYY format,
    "Status" which is the current status of the diagnosis (e.g. active, resolved, etc.)

### Output format definition
Your output should follow the JSON format:
{"Date": "<MM/DD/YYYY>", "Status": "<status>"}

I am only interested in the content between []. Do not explain your answer. 
```

--------------------------------

### Example JSON Output for Drug Attributes

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt

Illustrates an initial JSON format for extracting drug-related entities and their attributes from medical text, serving as a reference for similar structured outputs.

```JSON
[
    {"entity_text": "Mesalamine DR", "attr": {"entity_type": "Drug"}},
    {"entity_text": "1200 mg", "attr": {"entity_type": "Strength"}},
    {"entity_text": "BID", "attr": {"entity_type": "Frequency"}},
    {"entity_text": "PO", "attr": {"entity_type": "Route"}}
]
```

--------------------------------

### Initialize ReviewFrameExtractor for Enhanced Extraction

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

Shows how to initialize the `ReviewFrameExtractor`, a subclass of `DirectFrameExtractor`, which adds a review step. This example sets the `review_mode` to 'addition' to boost recall by adding new frames while keeping existing ones. Default `review_prompt` is recommended unless specific needs arise.

```python
from llm_ie import ReviewFrameExtractor, SentenceUnitChunker, SlideWindowContextChunker

unit_chunker = SentenceUnitChunker()
context_chunker = SlideWindowContextChunker(window_size=2)
extractor = ReviewFrameExtractor(inference_engine=llm, 
                                 unit_chunker=unit_chunker,
                                 context_chunker=context_chunker,
                                 prompt_template=prompt_template,
                                 review_mode="addition")
```

--------------------------------

### Test Inference Engine with Ollama

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

This snippet demonstrates how to test an initialized inference engine using the chat() method. It uses OllamaInferenceEngine as an example, sending a simple user message and printing the LLM's response in real-time.

```python
from llm_ie.engines import OllamaInferenceEngine

inference_engine = OllamaInferenceEngine(model_name="llama3.1:8b-instruct-q8_0")
inference_engine.chat(messages=[{"role": "user", "content":"Hi"}], verbose=True)
```

--------------------------------

### Initialize and Extract Frames with SentenceFrameExtractor

Source: https://github.com/daviden1013/llm-ie/blob/main/demo/prompt_template_writing.ipynb

This snippet initializes a SentenceFrameExtractor with an LLM and a prompt template. It then calls the extract_frames method, passing the input text and examples via text_content and specifying document_key for the primary document source. The stream parameter enables streaming results.

```python
extractor = SentenceFrameExtractor(llm, prompt_template_final)

frames = extractor.extract_frames(text_content={"examples":example, "input":note_text},
                                  document_key="input",
                                  stream=True)
```

--------------------------------

### Setting LLM Sampling Parameters in Python

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_config.md

Demonstrates how to configure LLM sampling parameters such as temperature and maximum new tokens by passing a `LLMConfig` instance to the `InferenceEngine` constructor. This example uses `BasicLLMConfig` with `OpenAIInferenceEngine` to control output characteristics.

```python
from llm_ie.engines import OpenAIInferenceEngine, BasicLLMConfig

config = BasicLLMConfig(temperature=0.2, max_new_tokens=4096)
inference_engine = OpenAIInferenceEngine(model="gpt-4o-mini", config=config)
```

--------------------------------

### Inspecting BinaryRelationExtractor Prompt Guide

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

This snippet demonstrates how to retrieve and print the detailed prompt template guidelines for the BinaryRelationExtractor class. It shows the recommended structure and components for designing effective prompts for binary relation extraction tasks.

```python
from llm_ie.extractors import BinaryRelationExtractor

print(BinaryRelationExtractor.get_prompt_guide())
```

--------------------------------

### Initialize and Chat with LLM Prompt Editor Agent

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/quick_start.md

This Python code demonstrates how to initialize an OpenAIInferenceEngine for the prompt editor, specifying the OpenRouter API base URL and a specific LLM model (Llama-4-Scout). It then defines a PromptEditor agent and starts an interactive chat session, allowing users to collaboratively design prompt templates.

```python
from llm_ie import OpenAIInferenceEngine, BasicLLMConfig, DirectFrameExtractor, PromptEditor, SentenceUnitChunker, SlideWindowContextChunker

# Define a LLM inference engine for the prompt editor
prompt_editor_llm = OpenAIInferenceEngine(base_url="https://openrouter.ai/api/v1", 
                                          model="meta-llama/llama-4-scout", 
                                          api_key=os.getenv("OPENROUTER_API_KEY"),
                                          config=BasicLLMConfig(temperature=0.4, 
                                                                max_new_tokens=4096))
# Define LLM prompt editor
editor = PromptEditor(prompt_editor_llm, DirectFrameExtractor)
# Start chat
editor.chat()
```

--------------------------------

### Prompt Template with External Knowledge Injection Example

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/prompt_templates.md

This prompt template illustrates how to inject external 'knowledge' using a placeholder. It defines a task for extracting diagnosis information from a medical note, specifying a schema for diagnosis name, datetime, and status, and requires a JSON output format.

```text

    Below is a medical note. Your task is to extract diagnosis information. 

    # Backgound knowledge
    {{knowledge}}
    Your output should include: 
        "Diagnosis": extract diagnosis names, 
        "Datetime": date/ time of diagnosis, 
        "Status": status of present, history, or family history

    Your output should follow a JSON format:
    [
        {"Diagnosis": <exact words as in the document>, "Datetime": <diagnosis datetime>, "Status": <one of "present", "history">},
        {"Diagnosis": <exact words as in the document>, "Datetime": <diagnosis datetime>, "Status": <one of "present", "history">},
        ...
    ]

    Below is the medical note:
    "{{note}}"
```

--------------------------------

### LLM Prompt Template Design Principles

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicReviewFrameExtractor_prompt_guide.txt

Defines a structured, 6-step approach for designing effective prompt templates for Large Language Models (LLMs) to perform information extraction tasks. It covers defining the task, schema, output format, optional hints, examples, and context placeholders.

```APIDOC
Prompt Template Design:

1. Task Description: Provide a detailed description of the task, including the background and the type of task (e.g., named entity recognition).

2. Schema Definition: List the key concepts that should be extracted, and provide clear definitions for each one. Must define "entity_text" which will be used to recognize the entity.

3. Output Format Definition: The output should be a JSON list, where each element is a dictionary representing a frame (an entity along with its attributes). Each dictionary must include a key "entity_text" and a key "attr". The attributes are placed in the "attr" (e.g., {"entity_text": "<entity_text>", "attr": {"attr1": "<attr1>", "attr2": "<attr2>"}}).

4. Optional: Hints: Provide itemized hints for the information extractors to guide the extraction process. Remind the prompted agent to be truthful. Emphasize that the prompted agent is supposed to perform the task instead of writting code or instruct other agents to do it.

5. Optional: Examples: Include examples in the format: Input: ... Output: ...

6. Context: The template must include a placeholder in the format {{<placeholder_name>}} for the context. The placeholder name can be customized as needed.
```

--------------------------------

### Prompt Example: Extracting Adverse Reactions with Attributes

Source: https://github.com/daviden1013/llm-ie/blob/main/package/llm-ie/src/llm_ie/asset/prompt_guide/BasicFrameExtractor_prompt_guide.txt

A detailed prompt template for extracting adverse reactions and their attributes (clinical trial, arm, percentage, evidence) from FDA clinical pharmacology text, specifying a nested JSON output format.

```Text
### Task description
The paragraph below is from the Food and Drug Administration (FDA) Clinical Pharmacology Section of Labeling for Human Prescription Drug and Biological Products, Adverse reactions section. Please carefully review it and extract the adverse reactions and percentages. Note that each adverse reaction is nested under a clinical trial and potentially an arm. Your output should take that into consideration.

### Schema definition
Your output should contain: 
    "entity_text" which is the name of the clinical trial as spelled in the document, 
    If applicable, "Arm" which is the arm within the clinical trial, 
    "AdverseReaction" which is the name of the adverse reaction,
    If applicable, "Percentage" which is the occurance of the adverse reaction within the trial and arm,
    "Evidence" which is the EXACT sentence in the text where you found the AdverseReaction from

### Output format definition
Your output should follow JSON format, for example:
[{"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}},
{"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}}]

### Additional hints
- Your output should be 100% based on the provided content. DO NOT output fake numbers. 
- You are expected to generate the output instead of writting code or insturcting other agents to do so. 
- If there is no specific arm, just omit the "Arm" key. If the percentage is not reported, just omit the "Percentage" key. The "Evidence" should always be provided.

### Context
Below is full text from the Adverse reactions section:
"{{input}}"
```

```JSON
[
    {"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}},
    {"entity_text": "<Clinical trial name or number>", "attr": {"Arm": "<name of arm>", "AdverseReaction": "<Adverse reaction text>", "Percentage": "<a percent>", "Evidence": "<exact sentence from the text>"}}
]
```

--------------------------------

### Example Output of Extracted Frames

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/quick_start.md

This snippet provides an example of the structured output generated by the `extract_frames` method. Each entry represents a 'frame' with details like `frame_id`, `start` and `end` character positions, the `entity_text` extracted, and a dictionary of `attr` (attributes) associated with the entity.

```json
[
  {"frame_id": "0", "start": 537, "end": 549, "entity_text": "hypertension", "attr": {"Status": ""}},
  {"frame_id": "1", "start": 551, "end": 565, "entity_text": "hyperlipidemia", "attr": {"Status": ""}},
  {"frame_id": "2", "start": 571, "end": 595, "entity_text": "Type 2 diabetes mellitus", "attr": {"Status": ""}},
  {"frame_id": "3", "start": 991, "end": 1003, "entity_text": "Hypertension", "attr": {"Date": "2010", "Status": null}},
  {"frame_id": "4", "start": 1026, "end": 1040, "entity_text": "Hyperlipidemia", "attr": {"Date": "2015", "Status": null}},
  {"frame_id": "5", "start": 1063, "end": 1087, "entity_text": "Type 2 Diabetes Mellitus", "attr": {"Date": "2018", "Status": null}},
  {"frame_id": "6", "start": 1646, "end": 1682, "entity_text": "Jugular venous pressure not elevated", "attr": {}},
  {"frame_id": "7", "start": 1703, "end": 1767, "entity_text": "Clear to auscultation bilaterally, no wheezes, rales, or rhonchi", "attr": {}},
  {"frame_id": "8", "start": 1802, "end": 1823, "entity_text": "no hepatosplenomegaly", "attr": {}},
  {"frame_id": "9", "start": 1926, "end": 1962, "entity_text": "ST-segment depression in leads V4-V6", "attr": {}},
  {"frame_id": "10", "start": 1982, "end": 2004, "entity_text": "Elevated at 0.15 ng/mL", "attr": {"Date": "", "Status": ""}},
  {"frame_id": "11", "start": 2046, "end": 2066, "entity_text": "No acute infiltrates", "attr": {}},
  {"frame_id": "12", "start": 2068, "end": 2093, "entity_text": "normal cardiac silhouette", "attr": {}},
  {"frame_id": "13", "start": 2117, "end": 2150, "entity_text": "Mild left ventricular hypertrophy", "attr": {"Date": "", "Status": ""}},
  {"frame_id": "14", "start": 2321, "end": 2338, "entity_text": "Glucose 180 mg/dL", "attr": {}},
  {"frame_id": "15", "start": 2340, "end": 2350, "entity_text": "HbA1c 7.8%", "attr": {}},
  {"frame_id": "16", "start": 2402, "end": 2431, "entity_text": "acute coronary syndrome (ACS)", "attr": {"Date": null, "Status": null}},
  {"frame_id": "17", "start": 3025, "end": 3033, "entity_text": "Diabetes", "attr": {}},
  {"frame_id": "18", "start": 3925, "end": 3935, "entity_text": "chest pain", "attr": {"Date": "", "Status": ""}}
]
```

--------------------------------

### Initializing and Extracting Relations with BinaryRelationExtractor

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

This snippet demonstrates how to instantiate the BinaryRelationExtractor by providing an inference engine, a prompt template, and an optional pre-processing function. It also shows examples of extracting binary relations from a document, highlighting both concurrent (faster) and streaming (step-by-step) extraction modes.

```python
from llm_ie.extractors import BinaryRelationExtractor

extractor = BinaryRelationExtractor(inference_engine, prompt_template=prompt_template, possible_relation_func=possible_relation_func)
# Extract binary relations with concurrent mode (faster)
relations = extractor.extract_relations(doc, concurrent=True)

# To print out the step-by-step, use the `concurrent=False` and `stream=True` options
relations = extractor.extract_relations(doc, concurrent=False, stream=True)
```

--------------------------------

### BinaryRelationExtractor Prompt Template Design Guidelines

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/extractors.md

This section outlines the essential components and structure for designing effective prompt templates for the BinaryRelationExtractor. It details required elements such as task description, schema definition, output format, and optional hints and examples, along with specific placeholders.

```APIDOC
Prompt Template Design:

1. Task description:
   Provide a detailed description of the task, including the background and the type of task (e.g., binary relation extraction). Mention the region of interest (ROI) text. 
2. Schema definition: 
   List the criterion for relation (True) and for no relation (False).

3. Output format definition:
   The ouptut must be a dictionary with a key "Relation" (i.e., {"Relation": "<True or False>"}).

4. (optional) Hints:
   Provide itemized hints for the information extractors to guide the extraction process.

5. (optional) Examples:
   Include examples in the format:  
    Input: ...  
    Output: ...

6. Entity 1 full information:
   Include a placeholder in the format {{<frame_1>}}

7. Entity 2 full information:
   Include a placeholder in the format {{<frame_2>}}

8. Input placeholders:
   The template must include a placeholder "{{roi_text}}" for the ROI text.


Example:

    # Task description
    This is a binary relation extraction task. Given a region of interest (ROI) text and two entities from a medical note, indicate the relation existence between the two entities.

    # Schema definition
        True: if there is a relationship between a medication name (one of the entities) and its strength or frequency (the other entity).
        False: Otherwise.

    # Output format definition
    Your output should follow the JSON format:
    {"Relation": "<True or False>"}

    I am only interested in the content between []. Do not explain your answer. 

    # Hints
        1. Your input always contains one medication entity and 1) one strength entity or 2) one frequency entity.
        2. Pay attention to the medication entity and see if the strength or frequency is for it.
        3. If the strength or frequency is for another medication, output False. 
        4. If the strength or frequency is for the same medication but at a different location (span), output False.

    # Entity 1 full information:
    {{frame_1}}

    # Entity 2 full information:
    {{frame_2}}

    # Input placeholders
    ROI Text with the two entities annotated with <entity_1> and <entity_2>:
    "{{roi_text}}"
```

--------------------------------

### Configure LiteLLM Inference Engine

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This snippet demonstrates how to initialize the LiteLLMInferenceEngine. It requires specifying the model, base URL, and API key for connecting to a LiteLLM compatible service.

```python
from llm_ie.engines import LiteLLMInferenceEngine

inference_engine = LiteLLMInferenceEngine(model="openai/Llama-3.3-70B-Instruct", base_url="http://localhost:8000/v1", api_key="EMPTY")
```

--------------------------------

### Displaying Extracted Frames in Python

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This snippet shows how to iterate through a list of `frames` and print each frame as a dictionary. The subsequent example output demonstrates the typical structure of an extracted frame, including `frame_id`, `start`, `end`, `entity_text`, and a dictionary of `attr`.

```python
for frame in frames:
    print(frame.to_dict())
```

--------------------------------

### Initialize LiteLLM Inference Engine for Various LLM APIs

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

Demonstrates how to initialize the LiteLLMInferenceEngine for different LLM providers like Huggingface serverless, OpenAI GPT models, local OpenAI-compatible servers, and Ollama. Requires setting environment variables for API keys where applicable.

```python
from llm_ie.engines import LiteLLMInferenceEngine

# Huggingface serverless inferencing
os.environ['HF_TOKEN']
inference_engine = LiteLLMInferenceEngine(model="huggingface/meta-llama/Meta-Llama-3-8B-Instruct")

# OpenAI GPT models
os.environ['OPENAI_API_KEY']
inference_engine = LiteLLMInferenceEngine(model="openai/gpt-4o-mini")

# OpenAI compatible local server
inference_engine = LiteLLMInferenceEngine(model="openai/Llama-3.1-8B-Instruct", base_url="http://localhost:8000/v1", api_key="EMPTY")

# Ollama 
inference_engine = LiteLLMInferenceEngine(model="ollama/llama3.1:8b-instruct-q8_0")
```

--------------------------------

### Configure Llama-cpp-python Inference Engine

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

This snippet shows how to initialize the LlamaCppInferenceEngine. It requires specifying the repository ID and the GGUF filename for the model.

```python
from llm_ie.engines import LlamaCppInferenceEngine

inference_engine = LlamaCppInferenceEngine(repo_id="bullerwins/Meta-Llama-3.1-8B-Instruct-GGUF",
                                           gguf_filename="Meta-Llama-3.1-8B-Instruct-Q8_0.gguf")
```

--------------------------------

### Initialize OpenAI Inference Engine

Source: https://github.com/daviden1013/llm-ie/blob/main/docs/llm_inference_engine.md

Shows how to create an OpenAIInferenceEngine instance using a specified model name. Refer to OpenAI documentation for available models and parameters.

```python
from llm_ie.engines import OpenAIInferenceEngine

inference_engine = OpenAIInferenceEngine(model="gpt-4o-mini")
```

--------------------------------

### Importing LLM-IE Components

Source: https://github.com/daviden1013/llm-ie/blob/main/demo/prompt_template_writing_via_chat.ipynb

Imports necessary classes from the `llm_ie` library, including `OllamaInferenceEngine` for local LLM inference, `SentenceFrameExtractor` for information extraction, and `PromptEditor` for interactive prompt creation.

```python
from llm_ie.engines import OllamaInferenceEngine
from llm_ie.extractors import SentenceFrameExtractor
from llm_ie.prompt_editor import PromptEditor
```

--------------------------------

### Configure OpenAI Inference Engine and Compatible Services

Source: https://github.com/daviden1013/llm-ie/blob/main/README.md

These snippets demonstrate how to initialize the OpenAIInferenceEngine for both direct OpenAI API access and compatible services like OpenRouter. Configurations vary based on whether a custom base URL is needed.

```python
from llm_ie.engines import OpenAIInferenceEngine

inference_engine = OpenAIInferenceEngine(model="gpt-4o-mini")
```

```python
from llm_ie.engines import OpenAIInferenceEngine

inference_engine = OpenAIInferenceEngine(base_url="https://openrouter.ai/api/v1", model="meta-llama/llama-4-scout")
```