### Start SpeechT5 Service (Gaudi2 HPU)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/tts/src/README.md

Installs necessary packages, starts the SpeechT5 service on a Gaudi2 HPU, and tests it. Requires `optimum[habana]` to be installed.

```bash
pip install optimum[habana]

cd comps/third_parties/speecht5/src
nohup python speecht5_server.py --device=hpu &
curl http://localhost:7055/v1/tts -XPOST -d '{"text": "Who are you?"}' -H 'Content-Type: application/json'
```

--------------------------------

### Start Guardrails Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/guardrails/README.md

Execute the Python script to start the Guardrails microservice. Ensure all requirements are installed.

```bash
python opea_guardrails_microservice.py
```

--------------------------------

### Setup Environment Variables for Microservice

Source: https://github.com/opea-project/genaicomps/blob/main/comps/feedback_management/deployment/docker_compose/README.md

Configure necessary environment variables for proxy settings and MongoDB connection details before starting the microservice.

```bash
export http_proxy=${your_http_proxy}
export https_proxy=${your_http_proxy}
export OPEA_STORE_NAME="mongodb"
export MONGO_HOST=${MONGO_HOST}
export MONGO_HOST=27017
export DB_NAME=${DB_NAME}
export COLLECTION_NAME=${COLLECTION_NAME}
```

--------------------------------

### Start Language Detection Microservice

Source: https://github.com/opea-project/genaicomps/blob/main/comps/language_detection/deployment/docker_compose/README.md

Execute this command to start the Language Detection microservice. This script should be run after all dependencies are installed.

```bash
python opea_language_detection_microservice.py
```

--------------------------------

### Install XTune on Native

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/README.md

Installs XTune and its dependencies on a native system. Ensure git is installed and follow the dependency installation guide for Arc 770 drivers.

```bash
conda create -n xtune python=3.10 -y
conda activate xtune
apt install -y rsync
# open webui as default
bash prepare_xtune.sh
# this way it will not open webui
# bash prepare_xtune.sh false
```

--------------------------------

### Install OPEA Components Library

Source: https://context7.com/opea-project/genaicomps/llms.txt

Install the OPEA components library using pip or build from source by cloning the repository.

```bash
# Install from PyPI
pip install opea-comps

# Or build from source
git clone https://github.com/opea-project/GenAIComps
cd GenAIComps
pip install -e .
```

--------------------------------

### Start Toxicity Detection Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/toxicity_detection/README.md

Execute the main Python script to start the toxicity detection microservice. Ensure all Python dependencies are installed first.

```bash
python toxicity_detection.py
```

--------------------------------

### Start Bias Detection Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/bias_detection/README.md

Execute the Python script to start the bias detection microservice. Ensure Python is installed and requirements are met.

```bash
python bias_detection.py
```

--------------------------------

### Run Dataprep Microservice with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_neo4j_llamaindex.md

Starts the Dataprep microservice along with Neo4j, TGI, and TEI services using Docker Compose. This setup is necessary for LLM-based graph extraction and Neo4j integration.

```bash
cd comps/dataprep/deployment/docker_compose
docker compose -f compose.yaml up -d

```

--------------------------------

### Run MCP Client Example

Source: https://github.com/opea-project/genaicomps/blob/main/comps/cores/mcp/README.md

Example demonstrating connection to both SSE and Stdio MCP servers and execution of tools. This script requires asyncio and the Opea MCP components.

```python
import asyncio
from comps.cores.mcp.config import OpeaMCPConfig, OpeaMCPSSEServerConfig, OpeaMCPStdioServerConfig
from comps.cores.mcp.manager import OpeaMCPToolsManager


async def main():
    config = OpeaMCPConfig(
        sse_servers=[
            OpeaMCPSSEServerConfig(url="http://localhost:8931/sse"),
        ],
        stdio_servers=[
            OpeaMCPStdioServerConfig(name="mcp-simple-tool", command="uv", args=["run", "mcp-simple-tool"]),
        ],
    )

    async with await OpeaMCPToolsManager.create(config) as manager:
        # Execute tools exposed by the servers
        result = await manager.execute_tool("browser_snapshot", {})
        print(result)

        result = await manager.execute_tool("fetch", {"url": "https://opea.dev/"})
        print(result)


# Run the async function
asyncio.run(main())
```

--------------------------------

### Start Text-to-SQL Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2query/src/README_sql.md

Executes the main Python script to start the Text-to-SQL microservice. Ensure all environment variables and dependencies are set up.

```bash
python3 opea_text2query_microservice.py
```

--------------------------------

### Start ArangoDB Server with Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_arangodb.md

Launches an ArangoDB instance locally using Docker. Ensure Docker is installed before running.

```bash
docker run -d -p 8529:8529 -e ARANGO_ROOT_PASSWORD=test arangodb/arangodb:latest
```

--------------------------------

### Start GPT-SoVITS in Air-Gapped Environment

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/gpt-sovits/src/README.md

Starts the GPT-SoVITS service in an air-gapped environment. Pre-download models and set llm_download to False. Mounts local pretrained models.

```bash
export llm_download="False"
git clone https://huggingface.co/lj1995/GPT-SoVITS pretrained_models
docker run -itd -p 9880:9880 -e llm_download=$llm_download -v ./pretrained_models/:/home/user/GPT-SoVITS/GPT_SoVITS/pretrained_models/ opea/gpt-sovits:latest
```

--------------------------------

### Start Services with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/rerankings/src/README_videoqna.md

Starts the reranking-videoqna service using Docker Compose in detached mode. Ensure you are in the correct directory.

```bash
docker compose up reranking-videoqna -d
```

--------------------------------

### Install Python Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/pii_detection/README.md

Navigate to the PII detection microservice directory and install the required Python packages.

```bash
cd $OPEA_GENAICOMPS_ROOT/comps/guardrails/src/pii_detection
pip install -r requirements.txt
```

--------------------------------

### Start General ASR Microservice

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_paraformer.md

Starts the general ASR microservice. Navigate to the project root and execute the microservice script, followed by a check script.

```bash
cd ../../..
python opea_asr_microservice.py
python check_asr_server.py
```

--------------------------------

### Install Python Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/toxicity_detection/README.md

Navigate to the toxicity detection microservice directory and install the necessary Python packages using the requirements.txt file.

```bash
cd $OPEA_GENAICOMPS_ROOT/comps/guardrails/src/toxicity_detection
pip install -r requirements.txt
```

--------------------------------

### Install Optimum for Habana

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_whisper.md

Installs the Optimum library with Habana support for running on Gaudi2 HPU.

```bash
pip install optimum[habana]
```

--------------------------------

### Start PII Detection Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/pii_detection/README.md

Execute the Python script to start the PII detection microservice locally.

```bash
python opea_pii_detection_microservice.py
```

--------------------------------

### Start Text-to-SQL Service on Xeon CPU

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2sql/src/README.md

Navigate to the Docker Compose directory and start the text2sql service in detached mode.

```bash
cd comps/text2sql/deployment/docker_compose
docker compose -f compose.yaml up text2sql -d
```

--------------------------------

### Configure and Start Web Retriever Service

Source: https://github.com/opea-project/genaicomps/blob/main/comps/web_retrievers/src/README.md

Sets environment variables for the TEI endpoint and search engine API keys, then starts the web retriever microservice using Docker.

```bash
# set TEI endpoint
export TEI_EMBEDDING_ENDPOINT="http://${your_ip}:6060"

# set search engine env variables
export GOOGLE_API_KEY=xxx
export GOOGLE_CSE_ID=xxx
```

```bash
docker run -d --name="web-retriever-server" -p 7077:7077 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy -e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT -e GOOGLE_API_KEY=$GOOGLE_API_KEY -e GOOGLE_CSE_ID=$GOOGLE_CSE_ID opea/web-retriever:latest
```

--------------------------------

### Start GPT-SoVITS Service

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/gpt-sovits/src/README.md

Starts the GPT-SoVITS microservice using Docker. The llm_download environment variable controls model downloading. Port 9880 is exposed.

```bash
export llm_download=True
docker run -itd -p 9880:9880 -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e llm_download=$llm_download opea/gpt-sovits:latest
```

--------------------------------

### Install Intel oneAPI Base Toolkit

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/adaclip_readme.md

Downloads and silently installs the Intel oneAPI Base Toolkit. This is a prerequisite for using Intel GPUs with PyTorch.

```sh
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/dfc4a434-838c-4450-a6fe-2fa903b75aa7/intel-oneapi-base-toolkit-2025.0.1.46_offline.sh
sudo sh ./intel-oneapi-base-toolkit-2025.0.1.46_offline.sh -a --silent --cli --eula accept
```

--------------------------------

### Dataprep Microservice Setup and Run

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_pgvector.md

Instructions for setting up environment variables, building the Docker image, and running the Dataprep Microservice using Docker or Docker Compose.

```APIDOC
## Environment Variables Setup

### Description
Set the following environment variables before building or running the Docker image.

### Environment Variables
- **PG_CONNECTION_STRING** (string) - Required - PostgreSQL connection string for PGVector.
- **INDEX_NAME** (string) - Required - Name of the index to be used in PGVector.
- **TEI_EMBEDDING_ENDPOINT** (string) - Required - Endpoint for the Text Embedding Inference (TEI) service.
- **HF_TOKEN** (string) - Required - Hugging Face API token for accessing models.

### Example
```bash
export PG_CONNECTION_STRING=postgresql+psycopg2://testuser:testpwd@${your_ip}:5432/vectordb
export INDEX_NAME=${your_index_name}
export TEI_EMBEDDING_ENDPOINT=${your_tei_embedding_endpoint}
export HF_TOKEN=${your_hf_api_token}
```

## Build Docker Image

### Description
Build the Docker image for the Dataprep Microservice.

### Command
```bash
cd GenAIComps
docker build -t opea/dataprep:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/src/Dockerfile .
```

## Run Docker Container

### Option A: Run with CLI

#### Description
Run the Dataprep Microservice using a Docker command.

#### Command
```bash
docker run  --name="dataprep-pgvector" -p 6007:6007 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e PG_CONNECTION_STRING=$PG_CONNECTION_STRING  -e INDEX_NAME=$INDEX_NAME -e EMBED_MODEL=${EMBED_MODEL} -e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT -e HF_TOKEN=$HF_TOKEN -e DATAPREP_COMPONENT_NAME="OPEA_DATAPREP_PGVECTOR" opea/dataprep:latest
```

### Option B: Run with Docker Compose

#### Description
Run the Dataprep Microservice using Docker Compose.

#### Command
```bash
cd comps/dataprep/deployment/docker_compose
docker compose -f compose_pgvector.yaml up -d
```
```

--------------------------------

### Start Retriever Service with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/retrievers/src/README_opensearch.md

Starts the retriever microservice using Python. Ensure the TEI_EMBEDDING_ENDPOINT and RETRIEVER_COMPONENT_NAME environment variables are set correctly.

```bash
export TEI_EMBEDDING_ENDPOINT="http://${your_ip}:6060"
export RETRIEVER_COMPONENT_NAME="OPEA_RETRIEVER_OPENSEARCH"
export HF_TOKEN=${your_hf_token}
python opea_retrievers_microservice.py
```

--------------------------------

### Start LVM Service with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/predictionguard/src/README.md

Starts the LVM microservice using a Python script. This is the primary method for running the service locally.

```bash
python lvm.py
```

--------------------------------

### Start Finetuning Service

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/README.md

Launches the fine-tuning microservice. Ensure the HF_TOKEN environment variable is set. Optionally, set FINETUNING_COMPONENT_NAME to specify the component.

```bash
export HF_TOKEN=${your_huggingface_token}
# export FINETUNING_COMPONENT_NAME="which component you want to run"
# export FINETUNING_COMPONENT_NAME="OPEA_FINETUNING" or export FINETUNING_COMPONENT_NAME="XTUNE_FINETUNING"
python opea_finetuning_microservice.py
```

--------------------------------

### Start Qdrant Server with Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_qdrant.md

Run the Qdrant vector database server using Docker. Mount a local volume for persistent storage.

```bash
docker run -p 6333:6333 -p 6334:6334 -v ./qdrant_storage:/qdrant/storage:z qdrant/qdrant
```

--------------------------------

### Dataprep Get API Response Example

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_elasticsearch.md

This is an example JSON response from the `/v1/dataprep/get` API, showing the structure of uploaded file information.

```json
[
  {
    "name": "uploaded_file_1.txt",
    "id": "uploaded_file_1.txt",
    "type": "File",
    "parent": ""
  },
  {
    "name": "uploaded_file_2.txt",
    "id": "uploaded_file_2.txt",
    "type": "File",
    "parent": ""
  }
]
```

--------------------------------

### Start Microservice with Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/clip/src/README.md

Instructions for building the Docker image and running the microservice using Docker Compose.

```APIDOC
## Build Docker Image

### Build Langchain Docker

```bash
cd GenAIComps/
docker build -t opea/embedding-multimodal-clip:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/third_parties/clip/src/Dockerfile .
```

## Run Docker with Docker Compose

```bash
cd comps/third_parties/clip/deployment/docker_compose/
docker compose -f compose_intel_cpu.yaml up -d
```
```

--------------------------------

### Start Services with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/embeddings/src/README_ovms.md

Navigates to the Docker Compose directory and starts the OVMS embedding server and the embedding microservice in detached mode.

```bash
cd comps/embeddings/deployment/docker_compose/ 
docker compose up ovms-embedding-server -d
```

--------------------------------

### Start Ray Head Node

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/README.md

Starts a Ray head node locally. This is the first step in setting up a Ray cluster for distributed computing.

```bash
ray start --head
```

--------------------------------

### Source Environment Setup

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2kg/src/README.md

Source this script to set up environment variables for your device and user. If skipped, variables must be exported individually.

```bash
source comps/text2kg/src/environment_setup.sh
```

--------------------------------

### Start LLaVA Service (Gaudi2 HPU)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/llava/src/README.md

Starts the LLaVA service in the background on a Gaudi2 HPU. This command assumes optimum[habana] is installed.

```bash
# Start LLaVA service
nohup python llava_server.py &
# Test
python check_llava_server.py
```

--------------------------------

### Start Microservice with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/chathistory/deployment/docker_compose/README.md

Starts the Chat History microservice and its dependencies using Docker Compose. This command assumes you are in the deployment directory and have a compose.yaml file configured.

```bash
docker compose -f ../deployment/docker_compose/compose.yaml up -d chathistory-mongo
```

--------------------------------

### Example MCP Client Usage in Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/prompt_registry/deployment/docker_compose/README.md

Demonstrates how to use the Prompt Registry microservice as an MCP tool for AI agents, including creating and retrieving prompts.

```python
from mcp.client.session import ClientSession
from mcp.client.sse import sse_client


async def use_prompt_registry():
    async with sse_client("http://localhost:6018/sse") as streams:
        async with ClientSession(*streams) as session:
            # Initialize connection
            await session.initialize()

            # Create a prompt
            result = await session.call_tool(
                "create_prompt",
                {"request": {"prompt_text": "Explain quantum computing in simple terms", "user": "ai_agent_001"}},
            )

            # Retrieve prompts
            prompts = await session.call_tool("get_prompt", {"request": {"user": "ai_agent_001"}})

```

--------------------------------

### Install PyTorch with IPEX on Intel GPU

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/adaclip_readme.md

Installs PyTorch, torchvision, torchaudio, and Intel Extension for PyTorch (IPEX) for XPU devices. This setup is for Intel Arc GPUs.

```sh
conda create -n adaclip_py310 python=3.10 -y
conda activate adaclip_py310
python -m pip install torch==2.5.1+cxx11.abi torchvision==0.20.1+cxx11.abi torchaudio==2.5.1+cxx11.abi intel-extension-for-pytorch==2.5.10+xpu oneccl_bind_pt==2.5.0+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
pip install -r requirements.txt
```

--------------------------------

### Launch Stdio MCP Server

Source: https://github.com/opea-project/genaicomps/blob/main/comps/cores/mcp/README.md

Launch a simple Stdio MCP server by cloning the Python SDK repository and running the example server. Ensure you are in the correct directory.

```bash
git clone https://github.com/modelcontextprotocol/python-sdk.git
cd python-sdk/examples/servers/simple-tool/mcp_simple_tool
uv run mcp-simple-tool
```

--------------------------------

### Dataprep Get API Response Structure

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_mariadb.md

Example JSON response structure from the `/v1/dataprep/get` API, showing uploaded files and their metadata.

```json
[
  {
    "name": "uploaded_file_1.txt",
    "id": "uploaded_file_1.txt",
    "type": "File",
    "parent": ""
  },
  {
    "name": "uploaded_file_2.txt",
    "id": "uploaded_file_2.txt",
    "type": "File",
    "parent": ""
  }
]

```

--------------------------------

### Start Microservice with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/prompt_registry/deployment/docker_compose/README.md

Deploys the Prompt Registry microservice and its MongoDB dependency using Docker Compose.

```bash
docker compose -f ../deployment/docker_compose/compose.yaml up -d promptregistry-mongo
```

--------------------------------

### Consume OPEA LLM Service (Non-Streaming)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/llms/src/text-generation/README_ovms.md

Sends a POST request to the `/v1/chat/completions` endpoint of the OPEA LLM microservice to get a text generation response. This example does not use streaming.

```bash
curl http://localhost:9000/v1/chat/completions\
  -X POST \
  -d '{"messages":"What is Deep Learning?"}' \
  -H 'Content-Type: application/json'
```

--------------------------------

### Start Text-to-Image Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2image/src/README.md

Run the text-to-image microservice using a Python script. Ensure the MODEL and HF_TOKEN environment variables are set.

```bash
python opea_text2image_microservice.py --bf16 --model_name_or_path $MODEL --token $HF_TOKEN
```

--------------------------------

### Start Prompt Template Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/prompt_template/src/README.md

Launches the Prompt Template microservice using a Python script. This is an alternative to using Docker.

```bash
python opea_prompt_template_microservice.py
```

--------------------------------

### Consume LVM Service API with cURL

Source: https://github.com/opea-project/genaicomps/blob/main/comps/lvms/src/README_vllm_ipex.md

Send a POST request to the LVM service endpoint using cURL to get a chat completion. This example includes an image URL and a text prompt for multimodal understanding.

```bash
curl http://localhost:41091/v1/chat/completions -XPOST -H "Content-Type: application/json" -d '{
    "model": "Qwen/Qwen2.5-VL-7B-Instruct",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Describe the image."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg"
            }
          }
        ]
      }
    ],
    "max_tokens": 512
  }'
```

--------------------------------

### Install Intel GPU Driver

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/install_dependency.md

Installs the Intel graphics driver by adding the Intel GPU repository, updating package lists, and installing the i915 DKMS module. A reboot is required after installation.

```bash
# Install the Intel graphics GPG public key
wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
    sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg

# Configure the repositories.intel.com package repository
echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy unified" | \
    sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list

# Update the package repository metadata
apt update

# Install i915
apt install intel-i915-dkms
reboot
```

--------------------------------

### Setup Environment Variables

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2query/src/README_cypher.md

Set up necessary environment variables for the Text-to-Cypher microservice, including Neo4j credentials and service ports. Ensure HF_TOKEN is also exported.

```bash
ip_address=$(hostname -I | awk '{print $1}')
export HF_TOKEN=${HF_TOKEN}
export NEO4J_USER=neo4j
export NEO4J_PASSWORD=neo4jtest
export NEO4J_URL="bolt://${ip_address}:7687"
export TEXT2CYPHER_PORT=11801
export TEXT2QUERY_COMPONENT_NAME="OPEA_TEXT2QUERY_CYPHER"
```

--------------------------------

### Install PyTorch on NVIDIA

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/adaclip_readme.md

Installs PyTorch, torchvision, and torchaudio for CUDA 11.7 within a conda environment. Ensure you have the correct CUDA toolkit installed.

```sh
conda activate adaclip_py310_nv
conda create -n adaclip_py310_nv python=3.10 -y
conda install -y pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 cudatoolkit=11.7 -c pytorch -c conda-forge
pip install -r requirements.txt
```

--------------------------------

### Start Microservice with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/rerankings/src/README_videoqna.md

Deploys the Videoqna Reranking Service and the Reranking Microservice using Docker Compose.

```APIDOC
## Start Microservice with Docker Compose

### Description
Deploys both the Videoqna Reranking Service and the Reranking Microservice using Docker Compose.

### Steps

1. Set environment variables:
   ```bash
    export TEI_RERANKING_PORT=12006
    export RERANK_PORT=8000
    export host_ip=$(hostname -I | awk '{print $1}')
    export TEI_RERANKING_ENDPOINT="http://${host_ip}:${TEI_RERANKING_PORT}"
    export TAG=comps
   ```

2. Navigate to the Docker Compose directory:
   ```bash
   cd comps/rerankings/deployment/docker_compose
   ```

3. Start the services:
   ```bash
    docker compose up reranking-videoqna -d
   ```
```

--------------------------------

### Start Whisper Service on CPU

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_whisper.md

Starts the Whisper ASR service in the background on a CPU. It's recommended to check if port 7066 is free before starting.

```bash
cd comps/third_parties/whisper/src
nohup python whisper_server.py --device=cpu &
python check_whisper_server.py
```

--------------------------------

### Start FunASR Paraformer Service (CPU)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_paraformer.md

Starts the FunASR Paraformer microservice on CPU. Navigate to the FunASR source directory and run the server. Ensure port 7066 is available.

```bash
cd comps/third_parties/funasr/src
nohup python funasr_server.py --device=cpu &
python check_funasr_server.py
```

--------------------------------

### Install Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/opensearch/src/README.md

Installs Docker Compose on Linux systems. Ensure Docker is installed first. Replace `$(uname -s)` and `$(uname -m)` with your system's OS and architecture if needed.

```bash
sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
```

```bash
sudo chmod +x /usr/local/bin/docker-compose
```

--------------------------------

### Install gpt-sovits Helm Chart

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/gpt-sovits/deployment/kubernetes/README.md

Use this command to install the gpt-sovits Helm chart on a Kubernetes cluster. Ensure you have Helm installed and your Hugging Face token is set as an environment variable. This command deploys on Xeon.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install gpt-sovits oci://ghcr.io/opea-project/charts/gpt-sovits  --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Setup Environment Variables

Source: https://github.com/opea-project/genaicomps/blob/main/comps/prompt_registry/deployment/docker_compose/README_redis.md

Environment variables required for setting up the Prompt Registry microservice.

```APIDOC
## Setup Environment Variables

This section outlines the environment variables needed to configure the Prompt Registry microservice.

### Environment Variables

- **http_proxy** (string) - Your HTTP proxy server address.
- **https_proxy** (string) - Your HTTPS proxy server address.
- **OPEA_STORE_NAME** (string) - The name of the store to use (e.g., "redis"). Defaults to "redis".
- **REDIS_URL** (string) - The connection URL for the Redis instance. Defaults to "redis://localhost:6379".
- **INDEX_NAME** (string) - The name of the Redis index. Defaults to "opea:index".
- **DOC_PREFIX** (string) - The prefix for document keys in Redis. Defaults to "doc:".
- **AUTO_CREATE_INDEX** (boolean) - Whether to automatically create the index. Defaults to "true".
- **ENABLE_MCP** (boolean) - Set to `true` to enable MCP support. Defaults to `false`.
```

--------------------------------

### Install Dataprep Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/README.md

Installs necessary packages for the Dataprep Microservice. Ensure you have apt-get available.

```bash
apt-get update
apt-get install libreoffice
```

--------------------------------

### Run Fine-tuning Microservice

Source: https://context7.com/opea-project/genaicomps/llms.txt

Launches the fine-tuning microservice. Requires the HF_TOKEN environment variable to be set for accessing Hugging Face models.

```bash
export HF_TOKEN=${your_huggingface_token}
docker run -d --name="finetuning-server" \
  -p 8015:8015 \
  -e HF_TOKEN=$HF_TOKEN \
  opea/finetuning:latest
```

--------------------------------

### Start ASR Microservice

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_whisper.md

Starts the main ASR microservice. This should be run while the Whisper service is active.

```bash
cd ../..
python opea_asr_microservice.py
python check_asr_server.py
```

--------------------------------

### Launch Services with Docker Compose (CPU)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2kg/src/README.md

Start the text2kg service and its dependencies using docker-compose for CPU execution. Logs are redirected to a file.

```bash
docker compose -f compose.yaml -f custom-override.yml up ${service_name}  -d > ${LOG_PATH}/start_services_with_compose.log
```

--------------------------------

### Install TTS Microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/tts/deployment/kubernetes/README.md

Use this command to install the TTS microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. This command assumes you have a 'cpu-values.yaml' file for configuration.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install tts oci://ghcr.io/opea-project/charts/tts --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Launch vLLM Service with OpenVINO (Intel CPU)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/third_parties/vllm/README.md

Start the vLLM model server using the OpenVINO backend for Intel CPUs. This command assumes the Docker image has been built.

```bash
bash launch_vllm_service_openvino.sh
```

--------------------------------

### Install GenAIComps from Pypi

Source: https://github.com/opea-project/genaicomps/blob/main/README.md

Use this command to install the GenAIComps package directly from the Python Package Index.

```bash
pip install opea-comps
```

--------------------------------

### Start Optuna Dashboard for Visualization

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/Qwen-VL_README.md

Launch the Optuna dashboard to visualize hyperparameter optimization results. Ensure port 8084 is open and accessible.

```bash
sudo ufw allow 8084
optuna-dashboard --host 0.0.0.0 --port 8084 sqlite:///./saves/optuna_tuning/optuna_study.db
```

--------------------------------

### Install web-retriever microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/web_retrievers/deployment/kubernetes/README.md

Use this command to install the web-retriever microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. The '-f cpu-values.yaml' flag specifies configuration values.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install web-retriever oci://ghcr.io/opea-project/charts/web-retriever --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Start Retriever Service with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/retrievers/src/README_milvus.md

Launches the Retriever Microservice using Python. Ensure environment variables are set and the embedding endpoint is correctly configured.

```bash
export TEI_EMBEDDING_ENDPOINT="http://${your_ip}:${your_embedding_port}"
export RETRIEVER_COMPONENT_NAME="OPEA_RETRIEVER_MILVUS"
python opea_retrievers_microservice.py
```

--------------------------------

### Start Neo4J Server with Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/dataprep/src/README_neo4j_langchain.md

Launches a Neo4J instance locally using Docker, mapping ports and mounting volumes for data and plugins. It also configures authentication and enables the APOC plugin.

```bash
docker run \
    -p 7474:7474 -p 7687:7687 \
    -v $PWD/data:/data -v $PWD/plugins:/plugins \
    --name neo4j-apoc \
    -d \
    -e NEO4J_AUTH=neo4j/password \
    -e NEO4J_PLUGINS=\["apoc"\]  \
    neo4j:latest
```

--------------------------------

### Install chathistory microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/chathistory/deployment/kubernetes/README.md

Use this command to install the chathistory microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. The '-f cpu-values.yaml' flag specifies configuration values.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install chathistory-usvc oci://ghcr.io/opea-project/charts/chathistory-usvc --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Install LVM Microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/lvms/deployment/kubernetes/README.md

Use this command to install the LVM microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. The '-f cpu-values.yaml' flag specifies custom configuration values.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install lvm oci://ghcr.io/opea-project/charts/lvm-uservice --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Install Embedding Microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/embeddings/deployment/kubernetes/README.md

Use this command to install the embedding microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. This command also uses a 'cpu-values.yaml' file for specific configurations.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install embedding-usvc oci://ghcr.io/opea-project/charts/embedding-usvc --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Start Text-to-SQL Service with Docker Compose (Gaudi2)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2query/src/README_sql.md

Starts the Text-to-SQL microservice using Docker Compose, optimized for Gaudi2 HPUs. Assumes the compose.yaml file is in the correct directory.

```bash
cd comps/text2sql/deployment/docker_compose
docker compose -f compose.yaml up text2query-sql-gaudi -d
```

--------------------------------

### Setup Environment Variables

Source: https://github.com/opea-project/genaicomps/blob/main/comps/agent/src/README.md

Sets up essential environment variables for the agent microservice deployment, including IP address, model name, Hugging Face token, and local model volume.

```bash
export ip_address=$(hostname -I | awk '{print $1}')
export model="meta-llama/Meta-Llama-3.1-70B-Instruct"
export HF_TOKEN=${HF_TOKEN}
export vllm_volume=${YOUR_LOCAL_DIR_FOR_MODELS}
```

--------------------------------

### Install Guardrails Microservice Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/guardrails/README.md

Install the necessary Python packages for the Guardrails microservice. Ensure OPEA_GENAICOMPS_ROOT is set.

```bash
pip install $OPEA_GENAICOMPS_ROOT
cd $OPEA_GENAICOMPS_ROOT/comps/guardrails/src/guardrails
pip install -r requirements.txt
```

--------------------------------

### Configure and Start vLLM-IPEX Microservice with Docker Compose

Source: https://github.com/opea-project/genaicomps/blob/main/comps/llms/src/text-generation/README_vllm_ipex.md

Set environment variables for registry, tag, group IDs, cache, model, quantization, port, and device selection. Then, navigate to the Docker Compose directory and start the text generation service.

```bash
# Use image: intel/llm-scaler-vllm:0.10.0-b4
export REGISTRY=intel
export TAG=0.10.0-b4

export VIDEO_GROUP_ID=$(getent group video | awk -F: '{printf "%s\n", $3}')
export RENDER_GROUP_ID=$(getent group render | awk -F: '{printf "%s\n", $3}')

HF_HOME=${HF_HOME:=~/.cache/huggingface}
export HF_HOME

export MAX_MODEL_LEN=20000
export LLM_MODEL_ID=Qwen/Qwen3-8B-AWQ
export LOAD_QUANTIZATION=awq
export VLLM_PORT=41090

# Single-Arc GPU, select GPU index as needed
export ONEAPI_DEVICE_SELECTOR="level_zero:0"
export TENSOR_PARALLEL_SIZE=1
# Multi-Arc GPU, select GPU indices as needed
# export ONEAPI_DEVICE_SELECTOR="level_zero:0;level_zero:1"
# export TENSOR_PARALLEL_SIZE=2
```

```bash
cd comps/llms/deployment/docker_compose/ 
docker compose -f compose_text-generation.yaml up textgen-vllm-ipex-service -d
```

--------------------------------

### Setup Environment Variables for Microservice

Source: https://github.com/opea-project/genaicomps/blob/main/comps/llms/src/text-generation/README.md

Set up essential environment variables required to run the OpeaTextGen microservice. These include ports, host IP, HuggingFace token, and the LLM model to be used.

```bash
export LLM_ENDPOINT_PORT=8008
export TEXTGEN_PORT=9000
export host_ip=${host_ip}
export HF_TOKEN=${HF_TOKEN}
export LLM_ENDPOINT="http://${host_ip}:${LLM_ENDPOINT_PORT}"
export LLM_MODEL_ID="Intel/neural-chat-7b-v3-3"
```

--------------------------------

### EmbedDoc Example

Source: https://context7.com/opea-project/genaicomps/llms.txt

Example of creating an EmbedDoc, which includes text, an embedding vector, search type, and retrieval parameters.

```python
# Example: Creating an EmbedDoc
embed_doc = EmbedDoc(
    text="What is machine learning?",
    embedding=[0.1, 0.2, 0.3, ...],  # 768-dim vector
    search_type="similarity",
    k=4,
    score_threshold=0.2
)
```

--------------------------------

### Pretraining Dataset Format Example

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/README.md

Example of the JSON format for a pretraining dataset. Each entry should contain a 'text' field.

```json
{"text": "A girl with a blue tank top sitting watching three dogs."}
{"text": "A boy with a blue tank top sitting watching three dogs."}
```

--------------------------------

### Install LLM Microservice using Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/llms/deployment/kubernetes/README.md

Use this command to install the LLM microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. This command also references a 'cpu-values.yaml' file for specific configurations.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install llm oci://ghcr.io/opea-project/charts/llm-uservice --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Run FAQGen Microservice with Docker CLI

Source: https://github.com/opea-project/genaicomps/blob/main/comps/llms/src/faq-generation/README.md

Start the FAQGen microservice using the Docker CLI after ensuring the backend LLM service is running. Choose the appropriate FAQGen_COMPONENT_NAME based on your backend.

```bash
export FAQGen_COMPONENT_NAME="OpeaFaqGenTgi" # or "OpeaFaqGenvLLM"
docker run -d \
    --name="llm-faqgen-server" \
    -p 9000:9000 \
    --ipc=host \
    -e http_proxy=$http_proxy \
    -e https_proxy=$https_proxy \
    -e LLM_MODEL_ID=$LLM_MODEL_ID \
    -e LLM_ENDPOINT=$LLM_ENDPOINT \
    -e HF_TOKEN=$HF_TOKEN \
    -e FAQGen_COMPONENT_NAME=$FAQGen_COMPONENT_NAME \
    opea/llm-faqgen:latest
```

--------------------------------

### Start Text-to-SQL Service with Docker Compose (Xeon)

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2query/src/README_sql.md

Starts the Text-to-SQL microservice using Docker Compose, specifically for Xeon CPUs. Assumes the compose.yaml file is in the correct directory.

```bash
cd comps/text2query/deployment/docker_compose
docker compose -f compose.yaml up text2query-sql -d
```

--------------------------------

### Install ASR Microservice with Helm

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/deployment/kubernetes/README.md

Use this command to install the ASR microservice on your Kubernetes cluster. Ensure you have Helm installed and replace 'insert-your-huggingface-token-here' with your actual Hugging Face token. The '-f cpu-values.yaml' flag specifies configuration values for CPU deployment.

```bash
export HFTOKEN="insert-your-huggingface-token-here"
helm install asr oci://ghcr.io/opea-project/charts/asr --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml
```

--------------------------------

### Run Train-Only Mode for Qwen2.5-VL with Overrides

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/Qwen-VL_README.md

Execute the finetuning process for Qwen2.5-VL, overriding the video FPS for evaluation to manage resource usage.

```bash
python optuna_tuning.py \
--config_file ./qwen_vl_configs/config_finetune_only.json \
--override \
  base.model_name=Qwen2.5-VL-7B-Instruct \
  evaluation.video_fps=0.05 \

```

--------------------------------

### Install Python Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_whisper.md

Installs necessary Python packages from a requirements file. Ensure you are in the project's root directory.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install Python Requirements

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_paraformer.md

Installs the required Python packages for the project using pip. Ensure you are in the correct directory before running.

```bash
pip install -r requirements-cpu.txt
```

--------------------------------

### Start Text-to-Image Service on Xeon using Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2image/src/README.md

Run the text-to-image service container on Xeon hardware. This command maps the necessary ports and sets environment variables including proxy, Hugging Face token, and the selected model.

```bash
docker run --ipc=host -p 9379:9379 -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e HF_TOKEN=$HF_TOKEN -e MODEL=$MODEL opea/text2image:latest
```

--------------------------------

### Start Text-to-SQL Service on Gaudi2 HPU

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2sql/src/README.md

Navigate to the Docker Compose directory and start the text2sql-gaudi service in detached mode.

```bash
cd comps/text2sql/deployment/docker_compose
docker compose -f compose.yaml up text2sql-gaudi -d
```

--------------------------------

### Navigate to Finetuning Directory

Source: https://github.com/opea-project/genaicomps/blob/main/comps/finetuning/src/integrations/xtune/doc/adaclip_readme.md

Change the current directory to the AdaCLIP finetuning scripts.

```shell
cd src/llamafactory/adaclip_finetune
```

--------------------------------

### Start Text-to-Cypher Microservice with Python

Source: https://github.com/opea-project/genaicomps/blob/main/comps/text2cypher/src/README.md

Run the Text-to-Cypher microservice using its Python script. This is an alternative to using Docker.

```bash
python3 opea_text2cypher_microservice.py
```

--------------------------------

### Start Prediction Guard Microservice with Docker

Source: https://github.com/opea-project/genaicomps/blob/main/comps/guardrails/src/prompt_injection/README.md

Starts the prediction guard microservice in detached mode. Requires PREDICTIONGUARD_API_KEY to be set.

```bash
docker run -d --name="guardrails-injection-predictionguard" -p 9085:9085 -e PREDICTIONGUARD_API_KEY=$PREDICTIONGUARD_API_KEY opea/guardrails-injection-predictionguard:latest
```

--------------------------------

### Stop and Remove Manually Started Containers

Source: https://github.com/opea-project/genaicomps/blob/main/comps/asr/src/README_paraformer.md

Stop and remove the FunASR Paraformer and ASR service containers that were started manually.

```bash
docker stop funasr-paraformer-service asr-funasr-paraformer-service
docker rm funasr-paraformer-service asr-funasr-paraformer-service
```