### Install Backend Dependencies

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Navigate to the backend directory and install Python dependencies. Remember to copy and populate the .env file with your API keys.

```bash
cd backend
pip install -r requirements.txt
cp .env.example .env   # Add your API keys
uvicorn main:app --reload
```

--------------------------------

### Install Frontend Dependencies

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Navigate to the frontend directory and install Node.js dependencies to run the Next.js application.

```bash
cd frontend
npm install
npm run dev
```

--------------------------------

### Docker Compose Commands

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Bash commands for building, starting, and tearing down Docker services defined in docker-compose.yml. Use `docker-compose up --build` to start with a fresh build, `docker-compose up -d` to run in the background, and `docker-compose down -v` to remove services and volumes.

```bash
# Build and start both services
docker-compose up --build

# Run in the background
docker-compose up -d

# Tear down and remove volumes
docker-compose down -v
```

--------------------------------

### Trigger Connector Sync

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Starts an asynchronous sync job that pulls documents from the external store into the local RAG index.

```bash
curl -X POST http://localhost:8000/api/connectors/connector-uuid/sync
# {"status": "sync_started"}
```

--------------------------------

### Get current settings

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Retrieves the current configuration settings, including the active LLM provider, model names, and all RAG pipeline parameters.

```APIDOC
## GET /api/settings

### Description
Returns the active LLM provider, model names, and all RAG pipeline parameters.

### Method
GET

### Endpoint
/api/settings

### Response Example
```json
{
  "llm_provider": "groq",
  "openai_model": "gpt-4o-mini",
  "watsonx_model": "ibm/granite-13b-chat-v2",
  "chunk_size": 1000,
  "chunk_overlap": 200,
  "retrieval_top_k": 10,
  "rerank_top_k": 8,
  "bm25_weight": 0.4,
  "vector_weight": 0.6,
  "use_hybrid_search": false,
  "use_multi_query": false,
  "use_hyde": false,
  "use_reranking": true
}
```
```

--------------------------------

### Get current config

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Retrieves the current configuration settings for the RAG system, including LLM provider and RAG parameters.

```APIDOC
## GET /api/settings

### Description
Get current config

### Method
GET

### Endpoint
/api/settings

### Response
#### Success Response (200)
- **settings** (object) - The current configuration settings.
```

--------------------------------

### Get Current Settings

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Retrieves the current configuration of the RAG system, including the active LLM provider, model names, and all RAG pipeline parameters.

```bash
curl http://localhost:8000/api/settings

# Response
{
  "llm_provider": "groq",
  "openai_model": "gpt-4o-mini",
  "watsonx_model": "ibm/granite-13b-chat-v2",
  "chunk_size": 1000,
  "chunk_overlap": 200,
  "retrieval_top_k": 10,
  "rerank_top_k": 8,
  "bm25_weight": 0.4,
  "vector_weight": 0.6,
  "use_hybrid_search": false,
  "use_multi_query": false,
  "use_hyde": false,
  "use_reranking": true
}
```

--------------------------------

### Frontend API Client Usage

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Examples of how to use the TypeScript helper functions to interact with the RAG Forge API for uploading documents, checking status, and sending messages.

```APIDOC
## Frontend API Client (`frontend/lib/api.ts`)

TypeScript helper functions that wrap every REST endpoint. The base URL is read from `NEXT_PUBLIC_API_URL` (defaults to `http://localhost:8000`).

### Upload files
```typescript
const [file] = (document.getElementById("input") as HTMLInputElement).files!;
const [doc] = await uploadDocuments([file]);
console.log("Pending doc:", doc.id, doc.status);
```

### Poll document status
```typescript
let status = await getDocumentStatus(doc.id);
while (status.status !== "completed" && status.status !== "failed") {
  await new Promise((r) => setTimeout(r, 1000));
  status = await getDocumentStatus(doc.id);
}
```

### Send a message and get a response
```typescript
const conv = await createConversation("Analysis session");
const resp = await sendMessage("Summarise the uploaded document", conv.id);
console.log(resp.message);
resp.sources.forEach((s) => console.log(s.doc_name, s.relevance_score));
```

### Update settings
```typescript
await updateSettings({
  llm_provider: "gemini",
  google_api_key: "AIza...",
  use_reranking: true,
  chunk_size: 800,
});
```
```

--------------------------------

### GET /api/documents

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Returns all indexed documents ordered by creation date (newest first).

```APIDOC
## GET /api/documents

### Description
Returns all indexed documents ordered by creation date (newest first).

### Method
`GET`

### Endpoint
`/api/documents`

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the document.
- **filename** (string) - The name of the document file.
- **file_type** (string) - The file extension (e.g., '.pdf').
- **file_size** (integer) - The size of the file in bytes.
- **chunk_count** (integer) - The number of chunks the document was split into.
- **status** (string) - The processing status of the document (e.g., 'completed').
- **source_type** (string) - The type of the source (e.g., 'file').
- **progress** (integer) - The completion percentage of the document processing.
- **created_at** (string) - The timestamp when the document was created.

#### Response Example
```json
[
  {
    "id": "doc-uuid-1234",
    "filename": "report.pdf",
    "file_type": ".pdf",
    "file_size": 204800,
    "chunk_count": 47,
    "status": "completed",
    "source_type": "file",
    "progress": 100,
    "created_at": "2024-11-01T10:00:00"
  }
]
```
```

--------------------------------

### Get conversation with messages

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Retrieves a specific conversation along with its associated messages.

```APIDOC
## GET /api/conversations/{id}

### Description
Get conversation with messages

### Method
GET

### Endpoint
/api/conversations/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the conversation to retrieve.
```

--------------------------------

### Get Hybrid Retriever (BM25 + Vector)

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Construct an `EnsembleRetriever` using `get_hybrid_retriever()` that merges BM25 keyword matching with ChromaDB vector similarity. Adjust `settings.bm25_weight` and `settings.vector_weight` to control the balance. `settings.retrieval_top_k` determines the number of documents to retrieve.

```python
from rag.retrieval import get_hybrid_retriever
from config import settings

settings.bm25_weight = 0.4
settings.vector_weight = 0.6
settings.retrieval_top_k = 8

retriever = get_hybrid_retriever()
docs = retriever.invoke("authentication token expiry policy")
for doc in docs:
    print(f"  [{doc.metadata.get('source_file')}] {doc.page_content[:100]}")
```

--------------------------------

### Get Conversation with Messages

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Retrieves a specific conversation along with its entire message history. Source citations for assistant messages are included.

```bash
curl http://localhost:8000/api/conversations/conv-uuid-5678
```

--------------------------------

### GET /api/conversations/{conv_id}

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Returns a conversation along with its full message history, including the source citations attached to each assistant message.

```APIDOC
## GET /api/conversations/{conv_id}

### Description
Returns a conversation along with its full message history, including the source citations attached to each assistant message.

### Method
`GET`

### Endpoint
`/api/conversations/{conv_id}`

### Parameters
#### Path Parameters
- **conv_id** (string) - Required - The unique identifier of the conversation.

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the conversation.
- **title** (string) - The title of the conversation.
- **messages** (array) - An array of message objects within the conversation.
  - **role** (string) - The role of the message sender (`user` or `assistant`).
  - **content** (string) - The content of the message.
  - **citations** (array) - An array of source citations for assistant messages.
    - **id** (string) - Identifier for the source document.
    - **name** (string) - Name of the source document.
    - **page** (integer) - Page number within the source document.
- **created_at** (string) - The timestamp when the conversation was created.
- **updated_at** (string) - The timestamp when the conversation was last updated.

### Request Example
```bash
curl http://localhost:8000/api/conversations/conv-uuid-5678
```
```

--------------------------------

### GET /api/documents/{doc_id}/status

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Returns the current processing status and progress percentage. Use this as a polling fallback when WebSocket connection is not available.

```APIDOC
## GET /api/documents/{doc_id}/status

### Description
Returns the current processing status and progress percentage. Use this as a polling fallback when WebSocket connection is not available.

### Method
`GET`

### Endpoint
`/api/documents/{doc_id}/status`

### Parameters
#### Path Parameters
- **doc_id** (string) - Required - The unique identifier of the document.

### Response
#### Success Response (200)
- **status** (string) - The current processing status (`processing`, `completed`, `failed`).
- **progress** (integer) - The completion percentage (0-100).
- **error_message** (string | null) - An error message if the status is `failed`, otherwise null.

#### Response Example (While processing)
```json
{"status": "processing", "progress": 70, "error_message": null}
```

#### Response Example (After completion)
```json
{"status": "completed", "progress": 100, "error_message": null}
```

#### Response Example (On failure)
```json
{"status": "failed", "progress": 0, "error_message": "Unsupported file type: .xyz"}
```
```

--------------------------------

### Backend Project Structure

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Overview of the backend directory structure, highlighting key components like FastAPI app, configuration, database, LLM providers, RAG engine, and API routers.

```tree
backend/
├── main.py                  # FastAPI app + WebSocket endpoint
├── config.py                # Pydantic settings
├── database.py              # SQLite (conversations, messages, documents)
├── providers/               # LLM abstraction (OpenAI, Groq, Gemini, WatsonX)
├── rag/                     # RAG engine, chunking, retrieval, prompts
├── ingestion/               # Multi-format document loader + processor
├── vectorstore/             # ChromaDB operations
├── routers/                 # API routes (chat, documents, conversations, settings)
└── models/                  # Pydantic schemas
```

--------------------------------

### Read and Mutate Application Settings

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Demonstrates reading configuration values from the `settings` object and modifying them at runtime. This is useful for dynamic configuration changes.

```python
from config import settings

# Read
print(settings.llm_provider)       # "groq"
print(settings.chunk_size)         # 1000

# Mutate at runtime (used by PUT /api/settings)
settings.llm_provider = "openai"
settings.openai_api_key = "sk-..."
settings.use_hybrid_search = True
```

--------------------------------

### GET /api/conversations

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Returns all conversations ordered by last-updated time (most recent first).

```APIDOC
## GET /api/conversations

### Description
Returns all conversations ordered by last-updated time (most recent first).

### Method
`GET`

### Endpoint
`/api/conversations`

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the conversation.
- **title** (string) - The title of the conversation.
- **created_at** (string) - The timestamp when the conversation was created.
- **updated_at** (string) - The timestamp when the conversation was last updated.

#### Response Example
```json
[
  {
    "id": "conv-uuid-5678",
    "title": "What are the key findings in...",
    "created_at": "2024-11-01T10:10:00",
    "updated_at": "2024-11-01T10:15:00"
  }
]
```
```

--------------------------------

### Configure Application Settings with `Settings`

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Manages application configuration using Pydantic `BaseSettings`. Settings can be defined in a `.env` file or environment variables and mutated at runtime.

```bash
# backend/.env
LLM_PROVIDER=groq
GROQ_API_KEY=gsk_...

OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini

GOOGLE_API_KEY=AIza...
GEMINI_MODEL=gemini-2.0-flash

WATSONX_API_KEY=...
WATSONX_PROJECT_ID=...
WATSONX_URL=https://us-south.ml.cloud.ibm.com
WATSONX_MODEL=ibm/granite-13b-chat-v2

CHROMA_PERSIST_DIR=./chromadb
SQLITE_DB_PATH=./data/ragforge.db

CHUNK_SIZE=1000
CHUNK_OVERLAP=200
RETRIEVAL_TOP_K=10
RERANK_TOP_K=8
BM25_WEIGHT=0.4
VECTOR_WEIGHT=0.6

USE_HYBRID_SEARCH=false
USE_MULTI_QUERY=false
USE_HYDE=false
USE_RERANKING=true
```

--------------------------------

### Docker Deployment

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Instructions for deploying RAG Forge using Docker Compose, including building and running the services.

```APIDOC
## Docker deployment

Run the complete stack (backend on port 8000, frontend on port 3000) with a single command.

```yaml
# docker-compose.yml (excerpt)
services:
  backend:
    build:
      context: .
      dockerfile: Dockerfile.backend
    ports:
      - "8000:8000"
    env_file: ./backend/.env
    volumes:
      - ./backend/data:/app/data
      - ./backend/chromadb:/app/chromadb

  frontend:
    build:
      context: .
      dockerfile: Dockerfile.frontend
    ports:
      - "3000:3000"
    environment:
      - NEXT_PUBLIC_API_URL=http://backend:8000
      - NEXT_PUBLIC_WS_URL=ws://backend:8000
    depends_on:
      - backend
```

```bash
# Build and start both services
docker-compose up --build

# Run in the background
docker-compose up -d

# Tear down and remove volumes
docker-compose down -v
```
```

--------------------------------

### Update Settings at Runtime

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Allows patching of RAG system settings. Changes are applied immediately without requiring a server restart. Provide API keys only when changing providers.

```bash
# Switch to OpenAI with a new key
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{
    "llm_provider": "openai",
    "openai_api_key": "sk-...",
    "openai_model": "gpt-4o"
  }'
```

```bash
# Enable hybrid search + reranking with custom chunk size
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{
    "use_hybrid_search": true,
    "use_reranking": true,
    "chunk_size": 800,
    "chunk_overlap": 150,
    "retrieval_top_k": 12,
    "rerank_top_k": 6,
    "bm25_weight": 0.3,
    "vector_weight": 0.7
  }'
```

```bash
# Enable HyDE (Hypothetical Document Embeddings) retrieval
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{"use_hyde": true, "use_hybrid_search": false}'
```

--------------------------------

### Test Connector Connection

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Verifies connectivity to a registered external vector store and updates its status to 'connected' on success.

```bash
curl -X POST http://localhost:8000/api/connectors/connector-uuid/test
# Success:  {"ok": true,  "message": "Connected successfully"}
# Failure:  {"ok": false, "message": "Connection refused"}
```

--------------------------------

### Tech Stack Overview

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Summary of technologies used across the frontend, backend, LLM providers, vector store, database, and deployment.

```markdown
| Layer | Technologies |
|-------|-------------|
| Frontend | Next.js 14, React 18, TypeScript, Tailwind CSS |
| Backend | FastAPI, Python 3.11, LangChain, Pydantic |
| LLM Providers | Groq, OpenAI, Google Gemini, IBM WatsonX |
| Vector Store | ChromaDB with HuggingFace embeddings |
| Database | SQLite (aiosqlite) |
| Deployment | Docker, docker-compose |
```

--------------------------------

### Frontend Project Structure

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Overview of the frontend directory structure, detailing the Next.js App Router, components, API client, and TypeScript interfaces.

```tree
frontend/
├── app/                     # Next.js 14 App Router
├── components/              # Chat, Sidebar, Documents, Settings, UI
├── lib/                     # API client, WebSocket hook
└── types/                   # TypeScript interfaces
```

--------------------------------

### Load Files in Multiple Formats with `load_file()`

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Ingests content from local files into LangChain `Document` objects. The format is inferred from the file extension. Raises `ValueError` for unsupported types.

```python
from ingestion.loader import load_file

# Load a PDF
pdf_docs = load_file("/tmp/annual_report.pdf")
print(f"Loaded {len(pdf_docs)} pages from PDF")

# Load a CSV
csv_docs = load_file("/tmp/sales_data.csv")
print(f"Loaded {len(csv_docs)} rows from CSV")

# Load a JSON file
json_docs = load_file("/tmp/config.json")
for doc in json_docs:
    print(doc.metadata["source_file"], "—", doc.page_content[:80])

# Unsupported format raises ValueError
try:
    load_file("/tmp/archive.zip")
except ValueError as e:
    print(e)  # Unsupported file type: .zip
```

--------------------------------

### Run with Docker Compose

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Use Docker Compose to build and run the entire RAG Forge application, including the backend, frontend, and database.

```bash
docker-compose up
```

--------------------------------

### Send message, get RAG response

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Sends a user message to the RAG engine and returns a response. This is the primary endpoint for interacting with the RAG system.

```APIDOC
## POST /api/chat

### Description
Send message, get RAG response

### Method
POST

### Endpoint
/api/chat

### Request Body
- **message** (string) - Required - The user's message to the RAG system.

### Response
#### Success Response (200)
- **response** (string) - The RAG-generated response to the user's message.
- **sources** (array) - List of sources used to generate the response.
```

--------------------------------

### Update settings at runtime

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Allows patching of settings fields at runtime. Changes are applied immediately without requiring a server restart. API keys should only be provided when changing LLM providers.

```APIDOC
## PUT /api/settings

### Description
Patches any combination of settings fields. Changes take effect immediately on the next request — no server restart required. Provide API keys only when changing providers.

### Method
PUT

### Endpoint
/api/settings

### Request Body
- **llm_provider** (string) - Optional - The LLM provider to use (e.g., "openai", "groq").
- **openai_api_key** (string) - Optional - Your OpenAI API key.
- **openai_model** (string) - Optional - The OpenAI model to use.
- **watsonx_model** (string) - Optional - The WatsonX model to use.
- **chunk_size** (integer) - Optional - The size of document chunks.
- **chunk_overlap** (integer) - Optional - The overlap between document chunks.
- **retrieval_top_k** (integer) - Optional - The number of top results to retrieve.
- **rerank_top_k** (integer) - Optional - The number of top results to rerank.
- **bm25_weight** (number) - Optional - Weight for BM25 retrieval.
- **vector_weight** (number) - Optional - Weight for vector search.
- **use_hybrid_search** (boolean) - Optional - Whether to use hybrid search.
- **use_multi_query** (boolean) - Optional - Whether to use multi-query generation.
- **use_hyde** (boolean) - Optional - Whether to use Hypothetical Document Embeddings.
- **use_reranking** (boolean) - Optional - Whether to use reranking.

### Request Example
```bash
# Switch to OpenAI with a new key
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{
    "llm_provider": "openai",
    "openai_api_key": "sk-...",
    "openai_model": "gpt-4o"
  }'

# Enable hybrid search + reranking with custom chunk size
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{
    "use_hybrid_search": true,
    "use_reranking": true,
    "chunk_size": 800,
    "chunk_overlap": 150,
    "retrieval_top_k": 12,
    "rerank_top_k": 6,
    "bm25_weight": 0.3,
    "vector_weight": 0.7
  }'

# Enable HyDE (Hypothetical Document Embeddings) retrieval
curl -X PUT http://localhost:8000/api/settings \
  -H "Content-Type: application/json" \
  -d '{"use_hyde": true, "use_hybrid_search": false}'
```
```

--------------------------------

### RAGEngine.query()

Source: https://context7.com/abinovarghese/ragforge/llms.txt

The core Python method for performing programmatic RAG queries. It can be used directly within scripts or background jobs for single or multi-turn conversations.

```APIDOC
## RAGEngine.query()

### Description
The core Python class that powers `/api/chat`. Can be used directly in scripts or background jobs.

### Method Signature
`async def query(question: str, conversation_id: Optional[str] = None) -> Tuple[str, List[Source]]`

### Parameters
- **question** (string) - Required - The user's query or prompt.
- **conversation_id** (string) - Optional - The ID of an existing conversation to maintain context for multi-turn interactions.

### Returns
- **answer** (string) - The generated answer to the query.
- **sources** (List[Source]) - A list of sources used to generate the answer, each containing `doc_name`, `page`, `chunk_text`, and `relevance_score`.

### Request Example
```python
import asyncio
from rag.engine import rag_engine

async def main():
    # Single-turn question (no conversation context)
    answer, sources = await rag_engine.query(
        "What is the project's primary architecture?"
    )
    print(answer)
    for src in sources:
        print(f"  [{src.doc_name} p.{src.page}] score={src.relevance_score:.4f}")
        print(f"  {src.chunk_text[:120]}...")

    # Multi-turn: pass conversation_id to include chat history
    answer2, sources2 = await rag_engine.query(
        "Can you give a concrete example?",
        conversation_id="conv-uuid-5678"
    )
    print(answer2)

asyncio.run(main())
```
```

--------------------------------

### Create External Vector Store Connector

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Registers a connection to an external vector store (Chroma remote, Pinecone, or Weaviate) for syncing additional document collections.

```bash
# Register a remote Chroma instance
curl -X POST http://localhost:8000/api/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Chroma",
    "type": "chroma_remote",
    "config": {
      "host": "chroma.internal",
      "port": "8001",
      "collection": "company_docs"
    }
  }'
```

```bash
# Register a Pinecone index
curl -X POST http://localhost:8000/api/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Pinecone KB",
    "type": "pinecone",
    "config": {
      "api_key": "pcsk_...",
      "index_name": "knowledge-base"
    }
  }'
```

--------------------------------

### Programmatic RAG Query with RAGEngine

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Utilizes the RAGEngine.query() method for programmatic RAG queries. Supports single-turn questions and multi-turn conversations by passing a conversation_id.

```python
import asyncio
from rag.engine import rag_engine

async def main():
    # Single-turn question (no conversation context)
    answer, sources = await rag_engine.query(
        "What is the project's primary architecture?"
    )
    print(answer)
    for src in sources:
        print(f"  [{src.doc_name} p.{src.page}] score={src.relevance_score:.4f}")
        print(f"  {src.chunk_text[:120]}...")

    # Multi-turn: pass conversation_id to include chat history
    answer2, sources2 = await rag_engine.query(
        "Can you give a concrete example?",
        conversation_id="conv-uuid-5678"
    )
    print(answer2)

asyncio.run(main())
```

--------------------------------

### Docker Compose Configuration

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Excerpt from docker-compose.yml defining backend and frontend services, including build contexts, ports, environment variables, and dependencies. Ensure NEXT_PUBLIC_API_URL and NEXT_PUBLIC_WS_URL are set for the frontend.

```yaml
# docker-compose.yml (excerpt)
services:
  backend:
    build:
      context: .
      dockerfile: Dockerfile.backend
    ports:
      - "8000:8000"
    env_file: ./backend/.env
    volumes:
      - ./backend/data:/app/data
      - ./backend/chromadb:/app/chromadb

  frontend:
    build:
      context: .
      dockerfile: Dockerfile.frontend
    ports:
      - "3000:3000"
    environment:
      - NEXT_PUBLIC_API_URL=http://backend:8000
      - NEXT_PUBLIC_WS_URL=ws://backend:8000
    depends_on:
      - backend
```

--------------------------------

### Upload Documents for Ingestion

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Upload one or more files using `multipart/form-data`. Files are processed asynchronously; track progress via the ingestion WebSocket. The response includes document details and initial status.

```bash
# Upload a single PDF
curl -X POST http://localhost:8000/api/documents/upload \
  -F "files=@/path/to/report.pdf"

# Upload multiple files at once
curl -X POST http://localhost:8000/api/documents/upload \
  -F "files=@report.pdf" \
  -F "files=@data.xlsx" \
  -F "files=@notes.md"

# Response (array of DocumentOut objects, one per file)
[
  {
    "id": "doc-uuid-1234",
    "filename": "report.pdf",
    "file_type": ".pdf",
    "file_size": 204800,
    "chunk_count": 0,
    "status": "pending",
    "source_type": "file",
    "progress": 0,
    "created_at": "2024-11-01T10:00:00"
  }
]
```

--------------------------------

### `RAGEngine.stream_query()`

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Provides an asynchronous generator interface for streaming RAG query results, yielding tokens and sources. This is suitable for real-time applications like WebSocket endpoints.

```APIDOC
## `RAGEngine.stream_query()` — Programmatic streaming RAG query

Async generator version of `query()`. Yields `token`, `sources`, and (implicitly) terminates after sources. Used by the WebSocket endpoint.

```python
import asyncio
from rag.engine import rag_engine

async def main():
    full_text = ""
    async for event in rag_engine.stream_query(
        "Explain the chunking strategy used",
        conversation_id="conv-uuid-5678"
    ):
        if event["type"] == "token":
            full_text += event["content"]
            print(event["content"], end="", flush=True)
        elif event["type"] == "sources":
            print("\n\n--- Sources ---")
            for src in event["sources"]:
                print(f"  {src['doc_name']}: {src['relevance_score']}")
    print("\n\nFull response length:", len(full_text))

asyncio.run(main())
```
```

--------------------------------

### Stream Query Results with RAGEngine

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Use `stream_query` for an async generator version of `query()`. It yields tokens and sources, suitable for WebSocket endpoints. Ensure `rag.engine` is imported.

```python
import asyncio
from rag.engine import rag_engine

async def main():
    full_text = ""
    async for event in rag_engine.stream_query(
        "Explain the chunking strategy used",
        conversation_id="conv-uuid-5678"
    ):
        if event["type"] == "token":
            full_text += event["content"]
            print(event["content"], end="", flush=True)
        elif event["type"] == "sources":
            print("\n\n--- Sources ---")
            for src in event["sources"]:
                print(f"  {src['doc_name']}: {src['relevance_score']}")
    print("\n\nFull response length:", len(full_text))

asyncio.run(main())
```

--------------------------------

### Ingest Document from URL

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Use this endpoint to fetch a webpage, extract its text, and index it. Set `deep_crawl` to `true` for recursive crawling of a site, with an optional `max_depth` parameter.

```bash
# Ingest a single page
curl -X POST http://localhost:8000/api/documents/url \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/docs/overview", "deep_crawl": false}'
```

```bash
# Deep-crawl an entire documentation site (max_depth=2)
curl -X POST http://localhost:8000/api/documents/url \
  -H "Content-Type: application/json" \
  -d '{"url": "https://docs.example.com/", "deep_crawl": true}'
```

```json
{
  "id": "url-doc-uuid",
  "filename": "https://example.com/docs/overview",
  "file_type": "url",
  "status": "pending",
  "source_type": "url",
  "source_url": "https://example.com/docs/overview",
  "progress": 0,
  "created_at": "2024-11-01T10:05:00"
}
```

--------------------------------

### Create external vector store connector

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Registers a connection to an external vector store, such as Chroma remote, Pinecone, or Weaviate, for syncing additional document collections.

```APIDOC
## POST /api/connectors

### Description
Registers a connection to an external vector store (Chroma remote, Pinecone, or Weaviate) for syncing additional document collections.

### Method
POST

### Endpoint
/api/connectors

### Parameters
#### Request Body
- **name** (string) - Required - A user-friendly name for the connector.
- **type** (string) - Required - The type of the vector store (e.g., "chroma_remote", "pinecone", "weaviate").
- **config** (object) - Required - Configuration details specific to the vector store type.
  - **host** (string) - Required for "chroma_remote" - The hostname of the Chroma instance.
  - **port** (string) - Required for "chroma_remote" - The port of the Chroma instance.
  - **collection** (string) - Required for "chroma_remote" - The name of the Chroma collection.
  - **api_key** (string) - Required for "pinecone" - Your Pinecone API key.
  - **index_name** (string) - Required for "pinecone" - The name of the Pinecone index.

### Request Example
```bash
# Register a remote Chroma instance
curl -X POST http://localhost:8000/api/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Chroma",
    "type": "chroma_remote",
    "config": {
      "host": "chroma.internal",
      "port": "8001",
      "collection": "company_docs"
    }
  }'

# Register a Pinecone index
curl -X POST http://localhost:8000/api/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Pinecone KB",
    "type": "pinecone",
    "config": {
      "api_key": "pcsk_...",
      "index_name": "knowledge-base"
    }
  }'
```
```

--------------------------------

### Frontend API Client Usage (TypeScript)

Source: https://context7.com/abinovarghese/ragforge/llms.txt

TypeScript helper functions for interacting with REST endpoints. The base URL is read from NEXT_PUBLIC_API_URL. Demonstrates uploading documents, polling for status, sending messages, and updating settings.

```typescript
import {
  sendMessage,
  uploadDocuments,
  ingestURL,
  listDocuments,
  deleteDocument,
  getDocumentStatus,
  listConversations,
  createConversation,
  getConversation,
  deleteConversation,
  getSettings,
  updateSettings,
  listConnectors,
  createConnector,
  testConnector,
  syncConnector,
  deleteConnector,
} from "@/lib/api";

// Upload files
const [file] = (document.getElementById("input") as HTMLInputElement).files!;
const [doc] = await uploadDocuments([file]);
console.log("Pending doc:", doc.id, doc.status);

// Poll until done
let status = await getDocumentStatus(doc.id);
while (status.status !== "completed" && status.status !== "failed") {
  await new Promise((r) => setTimeout(r, 1000));
  status = await getDocumentStatus(doc.id);
}

// Ask a question (blocking)
const conv = await createConversation("Analysis session");
const resp = await sendMessage("Summarise the uploaded document", conv.id);
console.log(resp.message);
resp.sources.forEach((s) => console.log(s.doc_name, s.relevance_score));

// Update settings to switch providers
await updateSettings({
  llm_provider: "gemini",
  google_api_key: "AIza...",
  use_reranking: true,
  chunk_size: 800,
});
```

--------------------------------

### Upload Documents

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Accepts one or more files via `multipart/form-data`. Each file is saved, chunked, embedded, and indexed asynchronously. Returns immediately with `status: "pending"` documents; track progress via the ingestion WebSocket.

```APIDOC
## POST /api/documents/upload — Upload documents

### Description
Accepts one or more files via `multipart/form-data`. Each file is saved, chunked, embedded, and indexed asynchronously. Returns immediately with `status: "pending"` documents; track progress via the ingestion WebSocket.

### Method
POST

### Endpoint
/api/documents/upload

#### Parameters
#### Form Data
- **files** (file) - Required - One or more files to upload.

### Request Example
```bash
# Upload a single PDF
curl -X POST http://localhost:8000/api/documents/upload \
  -F "files=@/path/to/report.pdf"

# Upload multiple files at once
curl -X POST http://localhost:8000/api/documents/upload \
  -F "files=@report.pdf" \
  -F "files=@data.xlsx" \
  -F "files=@notes.md"
```

### Response
#### Success Response (200)
- Returns an array of DocumentOut objects, one per file.
  - **id** (string) - Unique identifier for the document.
  - **filename** (string) - The original name of the file.
  - **file_type** (string) - The extension of the file (e.g., ".pdf").
  - **file_size** (integer) - The size of the file in bytes.
  - **chunk_count** (integer) - The number of chunks the document has been split into.
  - **status** (string) - The current status of the document processing (e.g., "pending").
  - **source_type** (string) - The type of source, e.g., "file".
  - **progress** (integer) - The progress of the ingestion process (0-100).
  - **created_at** (string) - The timestamp when the document was created.

### Response Example
```json
[
  {
    "id": "doc-uuid-1234",
    "filename": "report.pdf",
    "file_type": ".pdf",
    "file_size": 204800,
    "chunk_count": 0,
    "status": "pending",
    "source_type": "file",
    "progress": 0,
    "created_at": "2024-11-01T10:00:00"
  }
]
```
```

--------------------------------

### Ingest from URL

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Ingests content from a given URL and adds it to the RAG system.

```APIDOC
## POST /api/documents/url

### Description
Ingest from URL

### Method
POST

### Endpoint
/api/documents/url

### Request Body
- **url** (string) - Required - The URL to ingest content from.
```

--------------------------------

### List All Indexed Documents

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Retrieves a list of all documents currently indexed in the system, ordered by their creation date with the newest first. The response includes metadata for each document.

```bash
curl http://localhost:8000/api/documents
```

```json
[
  {
    "id": "doc-uuid-1234",
    "filename": "report.pdf",
    "file_type": ".pdf",
    "file_size": 204800,
    "chunk_count": 47,
    "status": "completed",
    "source_type": "file",
    "progress": 100,
    "created_at": "2024-11-01T10:00:00"
  }
]
```

--------------------------------

### Test connector connection

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Verifies the connectivity to a registered external vector store. Updates the connector's status to 'connected' upon successful verification.

```APIDOC
## POST /api/connectors/{id}/test

### Description
Verifies connectivity to a registered external vector store and updates its status to `"connected"` on success.

### Method
POST

### Endpoint
/api/connectors/{id}/test

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the connector to test.

### Request Example
```bash
curl -X POST http://localhost:8000/api/connectors/connector-uuid/test
```

### Response Example
```json
# Success:  {"ok": true,  "message": "Connected successfully"}
# Failure:  {"ok": false, "message": "Connection refused"}
```
```

--------------------------------

### Create a New Conversation

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Creates a new, empty conversation. An optional custom title can be provided in the request body.

```bash
curl -X POST http://localhost:8000/api/conversations \
  -H "Content-Type: application/json" \
  -d '{"title": "Q4 Report Analysis"}'
```

```json
# Response
{
  "id": "conv-uuid-new",
  "title": "Q4 Report Analysis",
  "created_at": "2024-11-01T10:20:00",
  "updated_at": "2024-11-01T10:20:00"
}
```

--------------------------------

### Create new conversation

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Creates a new, empty conversation.

```APIDOC
## POST /api/conversations

### Description
Create new conversation

### Method
POST

### Endpoint
/api/conversations

### Response
#### Success Response (200)
- **id** (string) - The ID of the newly created conversation.
```

--------------------------------

### API Endpoints for Chat and Document Management

Source: https://github.com/abinovarghese/ragforge/blob/main/README.md

Reference for backend API endpoints, including methods for sending messages, managing documents, handling conversations, and updating settings.

```markdown
| Method | Endpoint | Purpose |
|--------|----------|---------|
| POST | `/api/chat` | Send message, get RAG response |
| WS | `/ws/chat/{id}` | Streaming chat via WebSocket |
| POST | `/api/documents/upload` | Upload document(s) |
| GET | `/api/documents` | List all documents |
| DELETE | `/api/documents/{id}` | Delete document + vectors |
| POST | `/api/documents/url` | Ingest from URL |
| GET | `/api/conversations` | List conversations |
| POST | `/api/conversations` | Create new conversation |
| GET | `/api/conversations/{id}` | Get conversation with messages |
| DELETE | `/api/conversations/{id}` | Delete conversation |
| GET | `/api/settings` | Get current config |
| PUT | `/api/settings` | Update provider/RAG settings |
```

--------------------------------

### POST /api/documents/url

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Fetches a webpage (or recursively crawls a site when `deep_crawl` is `true`), extracts text, and indexes it into the vector store.

```APIDOC
## POST /api/documents/url

### Description
Fetches a webpage (or recursively crawls a site when `deep_crawl` is `true`), extracts text, and indexes it into the vector store.

### Method
`POST`

### Endpoint
`/api/documents/url`

### Parameters
#### Request Body
- **url** (string) - Required - The URL of the webpage or site to ingest.
- **deep_crawl** (boolean) - Optional - If true, recursively crawls the site. Defaults to false.

### Request Example
```bash
# Ingest a single page
curl -X POST http://localhost:8000/api/documents/url \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/docs/overview", "deep_crawl": false}'

# Deep-crawl an entire documentation site (max_depth=2)
curl -X POST http://localhost:8000/api/documents/url \
  -H "Content-Type: application/json" \
  -d '{"url": "https://docs.example.com/", "deep_crawl": true}'
```

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the ingested document.
- **filename** (string) - The source identifier (e.g., URL).
- **file_type** (string) - The type of the source (e.g., 'url').
- **status** (string) - The initial status of the ingestion job (e.g., 'pending').
- **source_type** (string) - The type of the source (e.g., 'url').
- **source_url** (string) - The original URL provided for ingestion.
- **progress** (integer) - The initial progress percentage (usually 0).
- **created_at** (string) - The timestamp when the ingestion job was created.

#### Response Example
```json
{
  "id": "url-doc-uuid",
  "filename": "https://example.com/docs/overview",
  "file_type": "url",
  "status": "pending",
  "source_type": "url",
  "source_url": "https://example.com/docs/overview",
  "progress": 0,
  "created_at": "2024-11-01T10:05:00"
}
```
```

--------------------------------

### WebSocket /ws/ingest/{doc_id}

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Streams real-time progress events for a document being processed. Connect immediately after uploading to show a live progress bar. The connection closes automatically when processing finishes.

```APIDOC
## WebSocket /ws/ingest/{doc_id}

### Description
Streams real-time progress events for a document being processed. Connect immediately after uploading to show a live progress bar. The connection closes automatically when processing finishes.

### Connection URL
`ws://<host>:<port>/ws/ingest/{doc_id}`

### Events
- **stage**: The current processing stage (`loading` | `chunking` | `embedding` | `indexing` | `complete` | `error`)
- **progress**: The completion percentage (0-100)
- **detail**: A human-readable string describing the current status
- **error**: An error message (only present when `stage` is `error`)

### Example Usage (Browser)
```javascript
const ws = new WebSocket("ws://localhost:8000/ws/ingest/doc-uuid-1234");

ws.onmessage = ({ data }) => {
  const event = JSON.parse(data);
  console.log(`[${event.stage}] ${event.progress}% — ${event.detail}`);
};
```
```

--------------------------------

### `multi_query_retrieve()`

Source: https://context7.com/abinovarghese/ragforge/llms.txt

Enhances retrieval recall by generating multiple rephrased versions of the user's query using an LLM. Each variant is used to query the hybrid retriever, and the results are merged and deduplicated.

```APIDOC
## `multi_query_retrieve()` — Multi-query retrieval

Generates three LLM-rephrased variants of the user query, runs each through the hybrid retriever, and returns a deduplicated merged result set. Improves recall for ambiguous questions.

```python
from rag.retrieval import multi_query_retrieve
from providers.factory import get_llm

llm = get_llm()
docs = multi_query_retrieve("how do I reset my password?", llm)
print(f"Retrieved {len(docs)} unique chunks across query variants")
for doc in docs[:3]:
    print(f"  {doc.page_content[:120]}")
```
```