### Install iText2KG Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Install the iText2KG library using pip. Ensure you have Python 3.9 or higher installed. ```bash pip install itext2kg ``` -------------------------------- ### OpenAI Chat and Embeddings Models Setup Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Sets up chat and embeddings models using OpenAI. Ensure you have the necessary API key and have followed the LangChain setup tutorials for OpenAI. ```python from langchain_openai import ChatOpenAI, OpenAIEmbeddings openai_api_key = "##" openai_llm_model = llm = ChatOpenAI( api_key = openai_api_key, model="gpt-4o", temperature=0, max_tokens=None, timeout=None, max_retries=2, ) openai_embeddings_model = OpenAIEmbeddings( api_key = openai_api_key , model="text-embedding-3-large", ) ``` -------------------------------- ### Install iText2KG Package Source: https://github.com/auvalab/itext2kg/blob/main/README.md Use this command to install or update the iText2KG package. It ensures you have the latest version for all features. ```bash pip install --update itext2kg ``` -------------------------------- ### Mistral Chat and Embeddings Models Setup Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Sets up chat and embeddings models using Mistral AI. Ensure you have the necessary API key and have followed the LangChain setup tutorials for Mistral. ```python from langchain_mistralai import ChatMistralAI from langchain_mistralai import MistralAIEmbeddings mistral_api_key = "##" mistral_llm_model = ChatMistralAI( api_key = mistral_api_key, model="mistral-large-latest", temperature=0, max_retries=2, ) mistral_embeddings_model = MistralAIEmbeddings( model="mistral-embed", api_key = mistral_api_key ) ``` -------------------------------- ### Custom Pydantic Schema Definition Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Defines custom data models for knowledge graph construction using Pydantic. Includes examples for Author and Article schemas. ```python from typing import List, Optional from pydantic import BaseModel, Field # Define an Author model with name and affiliation fields. class Author(BaseModel): name: str = Field(description="The name of the author") affiliation: str = Field(description="The affiliation of the author") # Define an Article model with various fields describing a scientific article. class Article(BaseModel): title: str = Field(description="The title of the scientific article") authors: List[Author] = Field(description="The list of the article's authors and their affiliation") abstract: str = Field(description="The article's abstract") key_findings: str = Field(description="The key findings of the article") limitation_of_sota: str = Field(description="limitation of the existing work") proposed_solution: str = Field(description="The proposed solution in details") paper_limitations: str = Field(description="The limitations of the proposed solution of the paper") ``` -------------------------------- ### Build Knowledge Graph with iText2KG_Star Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Initializes iText2KG_Star and builds a knowledge graph from text sections. Use this for advanced entity and relationship extraction with configurable thresholds. ```python import asyncio from itext2kg import iText2KG_Star from itext2kg.logging_config import setup_logging, get_logger # Optional: Configure logging setup_logging(level="INFO", log_file="itext2kg.log") logger = get_logger(__name__) async def build_knowledge_graph_star(): # Initialize iText2KG_Star with the llm model and embeddings model itext2kg_star = iText2KG_Star(llm_model=openai_llm_model, embeddings_model=openai_embeddings_model) # Your text sections (can be facts from document distiller or raw text) sections = [ "OpenAI announced ChatGPT agent with new capabilities for autonomous task execution.", "The new model integrates browser tools and terminal access for comprehensive automation.", "ChatGPT agent is rolling out to Pro, Plus, and Team users with enhanced safety measures." ] logger.info("Starting knowledge graph construction with iText2KG_Star...") # Build the knowledge graph - entities are automatically derived from relationships kg = await itext2kg_star.build_graph( sections=sections, ent_threshold=0.8, # Higher threshold for more distinct entities rel_threshold=0.7, # Threshold for relationship merging observation_date="2025-01-15" # Optional: add temporal context ) logger.info(f"Knowledge graph completed! Entities: {len(kg.entities)}, Relationships: {len(kg.relationships)}") return kg # Run the async function kg = asyncio.run(build_knowledge_graph_star()) ``` -------------------------------- ### Build Knowledge Graph with iText2KG Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Initializes the original iText2KG module and builds a knowledge graph from semantic sections. This method can directly use text chunks but may result in noisier graphs. ```python import asyncio from itext2kg import iText2KG from itext2kg.logging_config import setup_logging, get_logger # Optional: Configure logging setup_logging(level="INFO", log_file="itext2kg.log") logger = get_logger(__name__) async def build_knowledge_graph(): # Initialize iText2KG with the llm model and embeddings model. itext2kg = iText2KG(llm_model = openai_llm_model, embeddings_model = openai_embeddings_model) # Format the distilled document into semantic sections. semantic_blocks = [f"{key} - {value}".replace("{", "[").replace("}", "]") for key, value in distilled_doc.items()] logger.info("Starting knowledge graph construction...") # Build the knowledge graph using the semantic sections. # Note: build_graph() is now async and requires await kg = await itext2kg.build_graph(sections=semantic_blocks) logger.info("Knowledge graph construction completed successfully!") return kg # Run the async function kg = asyncio.run(build_knowledge_graph()) ``` -------------------------------- ### Build TKG from Text with OpenAI Models Source: https://github.com/auvalab/itext2kg/blob/main/README.md Demonstrates initializing ATOM with OpenAI's chat and embeddings models and building a knowledge graph from a COVID-19 dataset across different observation times. It also shows how to visualize the resulting graph using Neo4j. ```python import pandas as pd import asyncio import ast # Import LLM and Embeddings models using LangChain wrappers from langchain_openai import ChatOpenAI, OpenAIEmbeddings from itext2kg.atom import Atom from itext2kg import Neo4jStorage # Set up the OpenAI LLM and embeddings models (replace "##" with your API key) openai_api_key = "#" openai_llm_model = ChatOpenAI( api_key=openai_api_key, model="gpt-4.1-2025-04-14", temperature=0, max_tokens=None, timeout=None, max_retries=2, ) openai_embeddings_model = OpenAIEmbeddings( api_key=openai_api_key, model="text-embedding-3-large", ) # Load the 2020-COVID-NYT dataset pickle news_covid = pd.read_pickle("../datasets/atom/nyt_news/2020_nyt_COVID_last_version_ready.pkl") # Define a helper function to convert the dataframe's atomic facts into a dictionary, # where keys are observation dates and values are the combined list of atomic facts for that date. def to_dictionary (df:pd.DataFrame, max_elements: int | None = 20): if isinstance(df['factoids_g_truth'][0], str): df["factoids_g_truth"] = df["factoids_g_truth"].apply(lambda x:ast.literal_eval(x)) grouped_df = df.groupby("date")["factoids_g_truth"].sum().reset_index()[:max_elements] return { str(date): factoids for date, factoids in grouped_df.set_index("date")["factoids_g_truth"].to_dict().items() } # Convert the dataframe into the required dictionary format news_covid_dict = to_dictionary(news_covid) # Initialize the ATOM pipeline with the OpenAI models atom = Atom(llm_model=openai_llm_model, embeddings_model=openai_embeddings_model) # Build the knowledge graph across different observation timestamps kg = await atom.build_graph_from_different_obs_times( atomic_facts_with_obs_timestamps=news_covid_dict, ) # Visualize the resulting knowledge graph using Neo4j URI = "bolt://localhost:7687" USERNAME = "neo4j" PASSWORD = "##" Neo4jStorage(uri=URI, username=USERNAME, password=PASSWORD).visualize_graph(knowledge_graph=kg) ``` -------------------------------- ### iText2KG_Star Initialization Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Initializes the iText2KG_Star for direct relationship extraction from text. This is a recommended, simpler, and more efficient approach. ```python import asyncio from itext2kg import iText2KG_Star from itext2kg.logging_config import setup_logging, get_logger ``` -------------------------------- ### DocumentDistiller Initialization Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Initializes the DocumentDistiller module from the iText2KG package. This module is used for processing raw documents into semantic blocks. ```python import asyncio from itext2kg import DocumentDistiller ``` -------------------------------- ### Dynamic Knowledge Graph Construction from Time-Series Data Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Builds a knowledge graph that evolves over time by processing documents with temporal context. Initializes the graph with the first document and incrementally updates it with subsequent documents. ```python import asyncio from itext2kg import DocumentDistiller, iText2KG_Star from itext2kg.models.schemas import Facts async def build_dynamic_knowledge_graph(): # Initialize components document_distiller = DocumentDistiller(llm_model=openai_llm_model) itext2kg_star = iText2KG_Star(llm_model=openai_llm_model, embeddings_model=openai_embeddings_model) # Sample time-series data (e.g., social media posts, news articles, reports) time_series_data = [ { "observation_date": "2025-01-15", "content": "OpenAI announced ChatGPT agent with autonomous task execution capabilities." }, { "observation_date": "2025-01-16", "content": "ChatGPT agent now integrates browser tools and terminal access for enhanced automation." }, { "observation_date": "2025-01-17", "content": "The new agent is rolling out to Pro, Plus, and Team users with enhanced safety measures." } ] # Extract facts from each time point IE_query = ''' # DIRECTIVES : - Act like an experienced information extractor. - Extract clear, factual statements from the text. ''' # Process first document to initialize the KG facts_0 = await document_distiller.distill( documents=[time_series_data[0]["content"]], IE_query=IE_query, output_data_structure=Facts ) # Build initial knowledge graph kg = await itext2kg_star.build_graph( sections=facts_0.facts, observation_date=time_series_data[0]["observation_date"], ent_threshold=0.8, rel_threshold=0.7 ) # Incrementally update with subsequent documents for i in range(1, len(time_series_data)): print(f"Processing document {i+1} from {time_series_data[i]['observation_date']}") # Extract facts from current document facts = await document_distiller.distill( documents=[time_series_data[i]["content"]], IE_query=IE_query, output_data_structure=Facts ) # Update the knowledge graph incrementally kg = await itext2kg_star.build_graph( sections=facts.facts, observation_date=time_series_data[i]["observation_date"], existing_knowledge_graph=kg.model_copy(), # Pass existing KG for incremental updates ent_threshold=0.8, rel_threshold=0.7 ) print(f"Dynamic KG completed! Entities: {len(kg.entities)}, Relationships: {len(kg.relationships)}") # Each relationship now contains observation_dates showing when it was first observed for rel in kg.relationships: if rel.properties.observation_dates: print(f"Relationship '{rel.name}' first observed: {rel.properties.observation_dates[0]}") return kg # Run the dynamic KG construction dynamic_kg = asyncio.run(build_dynamic_knowledge_graph()) ``` -------------------------------- ### Facts-Based Knowledge Graph Construction Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Extracts structured facts from documents using the DocumentDistiller and a specific query, then aggregates these facts. Requires an LLM model. ```python import asyncio from itext2kg import DocumentDistiller from itext2kg.models.schemas import Facts async def extract_facts(): # Initialize the DocumentDistiller document_distiller = DocumentDistiller(llm_model=openai_llm_model) # Your documents documents = ["OpenAI announced ChatGPT agent with new capabilities...", "The new model can perform complex tasks autonomously..."] # Extract facts from each document IE_query = ''' # DIRECTIVES : - Act like an experienced information extractor. - Extract clear, factual statements from the text. ''' facts_list = await asyncio.gather(*[ document_distiller.distill( documents=[doc], IE_query=IE_query, output_data_structure=Facts ) for doc in documents ]) return facts_list # Run the async function facts = asyncio.run(extract_facts()) ``` -------------------------------- ### Integrate and Visualize Knowledge Graph with Neo4j Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Integrates extracted entities and relationships into a Neo4j graph database and provides a visualization. Use this to explore and analyze structured data using Neo4j's capabilities. ```python from itext2kg.graph_integration import Neo4jStorage URI = "bolt://localhost:7687" USERNAME = "neo4j" PASSWORD = "###" # Note: Graph visualization remains synchronous graph_integrator = Neo4jStorage(uri=URI, username=USERNAME, password=PASSWORD) graph_integrator.visualize_graph(knowledge_graph=kg) ``` -------------------------------- ### Document Distillation with Custom Schema Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md Distills documents using a custom Article schema and an information extraction query. Requires an initialized LLM model and a list of documents. ```python from itext2kg.models.schemas import Article async def main(): # Initialize the DocumentDistiller with llm model. document_distiller = DocumentDistiller(llm_model = openai_llm_model) # List of documents to be distilled. documents = ["doc1", "doc2", "doc3"] # Information extraction query. IE_query = ''' # DIRECTIVES : - Act like an experienced information extractor. - You have a chunk of a scientific paper. - If you do not find the right information, keep its place empty. ''' # Distill the documents using the defined query and output data structure. # Note: distill() is now async and requires await distilled_doc = await document_distiller.distill(documents=documents, IE_query=IE_query, output_data_structure=Article) return distilled_doc # Run the async function distilled_doc = asyncio.run(main()) ``` -------------------------------- ### ATOM Project Citation Source: https://github.com/auvalab/itext2kg/blob/main/README.md BibTeX entry for citing the ATOM paper. Use this in your LaTeX documents to reference the work. ```bibtex @article{lairgi2024atom, title={ATOM: AdapTive and OptiMized dynamic temporal knowledge graph construction using LLMs}, author={Lairgi, Yassir and Moncla, Ludovic and Benabdeslem, Khalid and Cazabet, R{\'e}my and Cl{\'e}au, Pierre}, journal={arXiv preprint arXiv:2510.22590}, year={2025}, url={https://arxiv.org/abs/2510.22590}, eprint={2510.22590}, archivePrefix={arXiv}, primaryClass={cs.AI} } ``` -------------------------------- ### iText2KG Citation Source: https://github.com/auvalab/itext2kg/blob/main/README_itext2kg.md BibTeX entry for citing the iText2KG paper. Use this when referencing the project in academic publications. ```bibtex @article{lairgi2024itext2kg, title={iText2KG: Incremental Knowledge Graphs Construction Using Large Language Models}, author={Lairgi, Yassir and Moncla, Ludovic and Cazabet, R{\'e}my and Benabdeslem, Khalid and Cl{\'e}au, Pierre}, journal={arXiv preprint arXiv:2409.03284}, year={2024}, note={Accepted at The International Web Information Systems Engineering conference (WISE) 2024}, url={https://arxiv.org/abs/2409.03284}, eprint={2409.03284}, archivePrefix={arXiv}, primaryClass={cs.AI} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.