### Pip Installation with Mirror Source Source: https://github.com/zjunlp/oneke/blob/main/README.md Demonstrates how to install Python packages using pip with a specified mirror source to circumvent network issues, particularly useful for environments with restricted network access or slow default repositories. ```Shell pip install -i [mirror-source] ... ``` -------------------------------- ### Execute OneKE Web UI Source: https://github.com/zjunlp/oneke/blob/main/README.md This command starts the OneKE web user interface. Ensure `gradio 4.44.0` is installed before execution. The UI will be accessible via a web browser. ```bash python src/webui.py ``` -------------------------------- ### Navigate to OneKE Project Directory Source: https://github.com/zjunlp/oneke/blob/main/README.md Changes the current working directory to the cloned OneKE project directory. All subsequent commands for setup should be executed from this location. ```bash cd OneKE ``` -------------------------------- ### Install Required Python Packages Source: https://github.com/zjunlp/oneke/blob/main/README.md Installs all necessary Python packages listed in the 'requirements.txt' file. If network issues occur, consider configuring a domestic mirror for pip to ensure successful installation. ```bash pip install -r requirements.txt # If you encounter network issues, consider setting up a domestic mirror for pip. ``` -------------------------------- ### Start VLLM Service for Local Model Deployment Source: https://github.com/zjunlp/oneke/blob/main/README.md Shell command to start the VLLM service for a locally deployed model, referencing a YAML configuration file. This service must be running before executing extraction tasks that utilize VLLM. ```Shell config_file=your_yaml_file_path # REMEMBER to set vllm_serve to TRUE! python src/models/vllm_serve.py --config $config_file # deploy local model via vllm, executed in the OneKE directory ``` -------------------------------- ### OneKE Web UI Initial Page Buttons Source: https://github.com/zjunlp/oneke/blob/main/README.md Describes the main function buttons available on the OneKE web UI's initial page for quick start, task submission, and clearing. ```APIDOC Buttons: 🎲 Quick Start with an Example 🎲: Quickly get a simple example to try OneKE. Submit: After configuring your customized tasks, click this button to run. Clear: When a task is completed, click this button to restore. ``` -------------------------------- ### Configure Local LLM Deployment with Transformer in YAML Source: https://github.com/zjunlp/oneke/blob/main/README.md Example YAML configuration for deploying a local LLM (e.g., LLaMA) using the Transformer library, specifying the model category and path, and explicitly disabling VLLM. ```YAML model: category: LLaMA # model category, chosen from LLaMA, Qwen, ChatGLM, MiniCPM, OneKE. model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct # model name to download from huggingface or use the local model path. vllm_serve: false # whether to use the vllm. Default set to false. ``` -------------------------------- ### Activate OneKE Conda Virtual Environment Source: https://github.com/zjunlp/oneke/blob/main/README.md Activates the 'oneke' Conda virtual environment, making its Python interpreter and installed packages available for use. This step is crucial before installing dependencies or running the project. ```bash conda activate oneke ``` -------------------------------- ### OneKE YAML Configuration for Web News Extraction Source: https://github.com/zjunlp/oneke/blob/main/README.md Example YAML configuration file for performing web news knowledge extraction. It defines model parameters (category, name, API key, base URL) and extraction settings (task type, instruction, input file path, output schema, mode, update case, show trajectory). ```yaml # model configuration model: category: DeepSeek # model category, chosen from ChatGPT, DeepSeek, LLaMA, Qwen, ChatGLM, MiniCPM, OneKE. model_name_or_path: deepseek-chat # model name, chosen from deepseek-chat and deepseek-reasoner. Choose deepseek-chat to use DeepSeek-V3 or choose deepseek-reasoner to use DeepSeek-R1. api_key: your_api_key # your API key for the model with API service. No need for open-source models. base_url: https://api.deepseek.com # base URL for the API service. No need for open-source models. # extraction configuration extraction: task: Base # task type, chosen from Base, NER, RE, EE. instruction: Extract key information from the given text. # description for the task. No need for NER, RE, EE task. use_file: true # whether to use a file for the input text. Default set to false. file_path: ./data/input_files/Tulsi_Gabbard_News.html # path to the input file. No need if use_file is set to false. output_schema: NewsReport # output schema for the extraction task. Selected the from schema repository. mode: customized # extraction mode, chosen from quick, detailed, customized. Default set to quick. See src/config.yaml for more details. update_case: false # whether to update the case repository. Default set to false. show_trajectory: false # whether to display the extracted intermediate steps ``` -------------------------------- ### Configure DeepSeek API Service Model in YAML Source: https://github.com/zjunlp/oneke/blob/main/README.md Example YAML configuration for using DeepSeek's API service, specifying model category, name, API key, and base URL for information extraction tasks. ```YAML model: category: DeepSeek # model category, chosen from ChatGPT and DeepSeek model_name_or_path: deepseek-chat # model name, chosen from deepseek-chat and deepseek-reasoner. Choose deepseek-chat to use DeepSeek-V3 or choose deepseek-reasoner to use DeepSeek-R1. api_key: your_api_key # your API key for the model with API service. base_url: https://api.deepseek.com # base URL for the API service. No need for open-source models. ``` -------------------------------- ### Configure Local LLM Deployment with VLLM in YAML Source: https://github.com/zjunlp/oneke/blob/main/README.md Example YAML configuration for deploying a local LLM (e.g., DeepSeek-R1 series) using VLLM, specifying the model category and path, and enabling VLLM. Note that DeepSeek-R1 models only support VLLM deployment. ```YAML model: category: DeepSeek # model category model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct # model name to download from huggingface or use the local model path. vllm_serve: true # whether to use the vllm. Default set to false. ``` -------------------------------- ### Access OneKE Web UI in Browser Source: https://github.com/zjunlp/oneke/blob/main/README.md After starting the web UI, open this URL in your browser to access the OneKE web interface. The default port for Gradio is 7860. ```text http://127.0.0.1:7860 ``` -------------------------------- ### Configure Knowledge Graph Construction in YAML Source: https://github.com/zjunlp/oneke/blob/main/README.md Example YAML configuration for integrating triple extraction results into a Knowledge Graph. This snippet shows how to define database connection parameters for Neo4j, including URL, username, and password, within the `construct` field of the configuration file. This field is optional and should be removed if KG construction is not intended. ```YAML construct: # (Optional) If you want to construct a Knowledge Graph, you need to set the construct field, or you must delete this field. database: Neo4j # your database type. url: neo4j://localhost:7687 # your database URL,Neo4j's default port is 7687. username: your_username # your database username. password: "your_password" # your database password. ``` -------------------------------- ### Create Conda Virtual Environment for OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md Creates a new Conda virtual environment named 'oneke' with Python 3.9. This environment isolates project dependencies and prevents conflicts with other Python installations. ```bash conda create -n oneke python=3.9 ``` -------------------------------- ### Event Extraction Type and Argument Constraints Source: https://github.com/zjunlp/oneke/blob/main/README.md This table defines the structured constraints for various event types, listing their associated arguments. These definitions guide the Event Extraction model in identifying and categorizing extracted information, ensuring consistency with predefined schemas. ```APIDOC Event Type: phishing Event Arguments: damage amount, attack pattern, tool, victim, place, attacker, purpose, trusted entity, time Event Type: data breach Event Arguments: damage amount, attack pattern, number of data, number of victim, tool, compromised data, victim, place, attacker, purpose, time Event Type: ransom Event Arguments: damage amount, attack pattern, payment method, tool, victim, place, attacker, price, time Event Type: discover vulnerability Event Arguments: vulnerable system, vulnerability, vulnerable system owner, vulnerable system version, supported platform, common vulnerabilities and exposures, capabilities, time, discoverer Event Type: patch vulnerability Event Arguments: vulnerable system, vulnerability, issues addressed, vulnerable system version, releaser, supported platform, common vulnerabilities and exposures, patch number, time, patch ``` -------------------------------- ### Clone OneKE Repository with Git Source: https://github.com/zjunlp/oneke/blob/main/README.md Clones the OneKE project repository from GitHub to your local machine. This is the first step to set up the project. ```bash git clone https://github.com/zjunlp/OneKE.git ``` -------------------------------- ### Clone OneKE Repository Source: https://github.com/zjunlp/oneke/blob/main/README.md Clones the OneKE project repository from GitHub to your local machine. This is the initial step to set up the development environment. ```bash git clone https://github.com/zjunlp/OneKE.git ``` -------------------------------- ### Deploy Local Models with vLLM and Run OneKE Extraction Source: https://github.com/zjunlp/oneke/blob/main/README.md Deploys local models using vLLM for serving, then initiates the OneKE knowledge extraction process. Requires vllm_serve to be set to TRUE in the configuration file. ```bash config_file=your_yaml_file_path # REMEMBER to set vllm_serve to TRUE! python src/models/vllm_serve.py --config $config_file # deploy local model via vllm, executed in the OneKE directory python src/run.py --config $config_file # start extraction, executed in the OneKE directory ``` -------------------------------- ### OneKE Web UI Instruction and Truth Setting Source: https://github.com/zjunlp/oneke/blob/main/README.md Details how to set extraction instructions and provide ground truth for updating the case repository in the OneKE web UI. ```APIDOC Instruction and Truth Fields: πŸ•ΉοΈ Instruction: Enter the desired extraction instruction, for example: "Please help me extract all the person names". πŸ’° Update Case: Check this box if you want to update the Case Repository, then you need to provide your truth. πŸͺ™ Truth: You can enter the truth you want LLMs to know, for example: "{\"relation_list\": [{\"head\": \"Guinea\", \"tail\": \"Conakry\", \"relation\": \"country capital\"}]}". ``` -------------------------------- ### Execute Triple Extraction Script Source: https://github.com/zjunlp/oneke/blob/main/README.md Command to run the triple extraction process using a specified configuration file. This script initiates the extraction based on the settings defined in `Triple2KG.yaml`. ```Bash config_file=./examples/config/Triple2KG.yaml python src/run.py --config $config_file ``` -------------------------------- ### Launch OneKE Docker Container with Local Models Source: https://github.com/zjunlp/oneke/blob/main/README.md Launches a Docker container, similar to the basic run, but additionally maps a local model path to a container path, allowing the use of locally deployed models within the container. ```bash docker run --gpus all \ -v ./OneKE:/app/OneKE \ -v your_local_model_path:/app/model/your_model_name \ -it oneke:v4 /bin/bash ``` -------------------------------- ### Run Web News Extraction Task Source: https://github.com/zjunlp/oneke/blob/main/README.md This snippet shows how to execute the customized Web News Extraction task using a specified configuration file. It involves setting the `config_file` variable and running the `src/run.py` script with that configuration. ```bash config_file=./examples/config/NewsExtraction.yaml python src/run.py --config $config_file ``` -------------------------------- ### Launch OneKE Docker Container Source: https://github.com/zjunlp/oneke/blob/main/README.md Launches a Docker container from the OneKE v4 image, mapping the local OneKE directory to /app/OneKE inside the container and enabling GPU support. The container enters an interactive bash session. ```bash docker run --gpus all \ -v ./OneKE:/app/OneKE \ -it oneke:v4 /bin/bash ``` -------------------------------- ### OneKE Web UI Text Input Options Source: https://github.com/zjunlp/oneke/blob/main/README.md Describes how to provide input text to the OneKE web UI, either by direct entry or file upload. ```APIDOC Text Input Options: πŸ“‚ Use File: Check to enable file upload. πŸ“– Upload a File: Drop or click to upload files (e.g., .pdf, .html) after 'Use File' is checked. πŸ“– Text: Enter text directly in any language. ``` -------------------------------- ### Run OneKE Knowledge Extraction via CLI Source: https://github.com/zjunlp/oneke/blob/main/README.md Executes the OneKE knowledge extraction process using a specified YAML configuration file. This command should be run from the OneKE directory. ```bash config_file=your_yaml_file_path # configuration file path, use the container path if inside a container python src/run.py --config $config_file # start extraction, executed in the OneKE directory ``` -------------------------------- ### OneKE Web UI Model Configuration Source: https://github.com/zjunlp/oneke/blob/main/README.md Details the fields for configuring the extraction model within the OneKE web UI, including model name, API key, and base URL. ```APIDOC Model Configuration Fields: πŸͺ„ Enter your Model: Input model name (e.g., gpt-4o-mini, o3-mini, deepseek-chat) or local model path. πŸ”‘ Enter your API-Key: Provide your model's API key (not needed for local models). πŸ”— Enter your Base-URL: Specify a custom Base-URL (leave empty for default). ``` -------------------------------- ### Run Named Entity Recognition (NER) Task with OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md This shell script executes the Named Entity Recognition (NER) task using the OneKE framework. It sets the path to the configuration file (`./examples/config/NER.yaml`), which defines the model and extraction settings, and then runs the main Python script (`src/run.py`) with this configuration. ```bash config_file=./examples/config/NER.yaml python src/run.py --config $config_file ``` -------------------------------- ### OneKE Web UI Task Configuration Source: https://github.com/zjunlp/oneke/blob/main/README.md Explains the options for configuring extraction tasks in the OneKE web UI, covering task type and extraction mode. ```APIDOC Task Configuration Fields: 🎯 Select your Task: Choose between Traditional IE (NER, RE, EE, Triple) and Open Domain IE (Web News, Book Knowledge, custom). 🧭 Select your Mode: Select extraction method (predefined agent combinations or custom). Recommended modes: 'direct mode' for longer texts, 'standard mode' for shorter high-accuracy tasks. 'Not Required' for no agent. ``` -------------------------------- ### Configure YAML for File Content Input Source: https://github.com/zjunlp/oneke/blob/main/README.md This YAML configuration shows how to set up the extraction task to read content from a specified file. The `use_file` flag is set to `true`, and the `file_path` field points to the input file. ```yaml use_file: true file_path: ./data/input_files/Tulsi_Gabbard_News.html ``` -------------------------------- ### Pull OneKE Docker Image Source: https://github.com/zjunlp/oneke/blob/main/README.md Pulls the OneKE Docker image (version v4) from the zjunlp mirror repository. This image provides a pre-configured environment for OneKE. ```bash docker pull zjunlp/oneke:v4 ``` -------------------------------- ### Access Neo4j Browser Interface Source: https://github.com/zjunlp/oneke/blob/main/README.md The default web interface URL for accessing the Neo4j graph database through a browser for visualization and interaction. ```Plain Text http://localhost:7474/browser ``` -------------------------------- ### Configure YAML for Plain Text Input Source: https://github.com/zjunlp/oneke/blob/main/README.md This YAML configuration demonstrates how to set up the extraction task to process plain text directly. The `use_file` flag is set to `false`, and the text to be extracted is provided in the `text` field. ```yaml use_file: false text: Finally , every other year , ELRA organizes a major conference LREC , the International Language Resources and Evaluation Conference . ``` -------------------------------- ### Execute OneKE Event Extraction Script Source: https://github.com/zjunlp/oneke/blob/main/README.md This bash script runs the Event Extraction task using the OneKE framework. It takes a configuration file as input, which specifies the model and extraction settings, and initiates the extraction process. ```bash config_file=./examples/config/EE.yaml python src/run.py --config $config_file ``` -------------------------------- ### BibTeX Citation for OneKE Project Source: https://github.com/zjunlp/oneke/blob/main/README.md Provides the BibTeX entry for citing the OneKE project, a Dockerized Schema-Guided LLM Agent-based Knowledge Extraction System, in academic work. This citation should be used when referencing the project in publications. ```bibtex @inproceedings{luo2025oneke, title={OneKE: A Dockerized Schema-Guided LLM Agent-based Knowledge Extraction System}, author={Luo, Yujie and Ru, Xiangyuan and Liu, Kangwei and Yuan, Lin and Sun, Mengshu and Zhang, Ningyu and Liang, Lei and Zhang, Zhiqiang and Zhou, Jun and Wei, Lanning and others}, booktitle={Companion Proceedings of the ACM on Web Conference 2025}, pages={2871--2874}, year={2025} } ``` -------------------------------- ### OneKE Extraction Method Support API Source: https://github.com/zjunlp/oneke/blob/main/README.md This section outlines the available extraction methods categorized by agents (Schema Agent, Extraction Agent, Reflection Agent) within the OneKE system. It describes the purpose of each method, allowing users to combine them for information extraction tasks. ```APIDOC Extraction Method Support: Schema Agent: Default Schema: Use the default JSON output format. Predefined Schema: Utilize the predefined output schema retrieved from the knowledge base. Self Schema Deduction: Generate the output schema by inferring from the task description and the source text. Extraction Agent: Direct IE: Directly extract information from the given text based on the task description. Case Retrieval: Retrieve similar good cases from the knowledge base to aid in the extraction. Reflection Agent: No Reflection: Directly return the extraction results. Case Reflection: Use the self-consistency approach, and if inconsistencies appear, reflect on the original answer by retrieving similar bad cases from the knowledge base. ``` -------------------------------- ### Set Custom Extraction Mode in OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md This YAML snippet shows how to set the `mode` to `customized` in an external configuration file like `examples/customized.yaml`. This activates the custom extraction methods defined in `src/config.yaml`, enabling the customized extraction experience. ```YAML # examples/customized.yaml mode: customized ``` -------------------------------- ### Run Relation Extraction Task with OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md This shell script executes the relation extraction task using the specified configuration file. It uses `python src/run.py` with the `--config` argument pointing to the `RE.yaml` file. ```bash config_file=./examples/config/RE.yaml python src/run.py --config $config_file ``` -------------------------------- ### OneKE Web UI Result Display Source: https://github.com/zjunlp/oneke/blob/main/README.md Explains how results are displayed in the OneKE web UI after task submission, including generated schema, final answer, and error messages. ```APIDOC Result Display Sections: πŸ€” Generated Schema: This is the extraction schema automatically generated by OneKE for your task, presented in a structured Python class format. πŸ˜‰ Final Answer: This is the final extraction result, presented in structured JSON format. πŸ˜΅β€πŸ’« Error capture: This section displays all error messages caught during the process, such as network issues, etc. ``` -------------------------------- ### Configure Custom Extraction Methods in OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md This YAML snippet demonstrates how to configure custom extraction methods within the `src/config.yaml` file. It sets specific functions for `schema_agent`, `extraction_agent`, and `reflection_agent` under the `customized` section, allowing users to define their preferred information extraction pipeline. ```YAML # src/config.yaml customized: schema_agent: get_deduced_schema extraction_agent: extract_information_direct reflection_agent: reflect_with_case ``` -------------------------------- ### Configure Case Repository Update with Truth Data in OneKE Source: https://github.com/zjunlp/oneke/blob/main/README.md This YAML snippet demonstrates how to configure the automatic update of the Case Repository in OneKE. By setting `update_case` to `true` and providing `truth` data (e.g., a relation list), the system compares extraction results with the truth, generates analysis, and stores the case, aiding in self-consistency and reflection. ```YAML # examples/config/RE.yaml truth: {"relation_list": [{"head": "Guinea", "tail": "Conakry", "relation": "country capital"}]} # Truth data for the relation update_case: true ``` -------------------------------- ### Define Custom Pydantic Schema in OneKE Schema Repository Source: https://github.com/zjunlp/oneke/blob/main/README.md This Python code snippet illustrates how to add a new Pydantic schema, `ChemicalSubstance` and `ChemicalList`, to the `src/modules/knowledge_base/schema_repository.py` file. This allows the OneKE system to use a predefined output schema for information extraction, ensuring strict adherence to the defined data structure. ```Python # src/modules/knowledge_base/schema_repository.py class ChemicalSubstance(BaseModel): name: str = Field(description="Name of the chemical substance") formula: str = Field(description="Molecular formula") appearance: str = Field(description="Physical appearance") uses: List[str] = Field(description="Primary uses") hazards: str = Field(description="Hazard classification") class ChemicalList(BaseModel): chemicals: List[ChemicalSubstance] = Field(description="List of chemicals") ``` -------------------------------- ### Define Triple Extraction Constraints in Python Source: https://github.com/zjunlp/oneke/blob/main/README.md Demonstrates how to define constraints for triple extraction, specifying entity types, relation types, or a combination of subject, relation, and object entity types. The `constraint` parameter can be a single list for entity types, a nested list for entity and relation types, or a triple-nested list for subject, relation, and object entity types. ```Python ["Person", "Place", "Event", "property"] ``` ```Python [["Person", "Place", "Event", "property"], ["Interpersonal", "Located", "Ownership", "Action"]] ``` ```Python [["Person"], ["Interpersonal", "Ownership"], ["Person", "property"]] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.