### DeepContext Complete Workflow Example (Python)

Source: https://context7.com/pranavms13/deepcontext/llms.txt

An end-to-end example demonstrating the full DeepContext workflow: initializing components like ContentFetcher, VectorStore, Chunkers, and LibraryStore. It also includes optional setup for authentication credentials.

```python
from deepcontext import ContentFetcher, VectorStore
from deepcontext.chunker import ChunkConfig, MarkdownCodeChunker
from deepcontext.store import LibraryStore, derive_library_id, library_id_to_collection_name
import os

# Optional: Set up authentication
os.environ["GITHUB_TOKEN"] = "ghp_your_token_here"
os.environ["CONFLUENCE_EMAIL"] = "you@company.com"
os.environ["CONFLUENCE_TOKEN"] = "your_token"

# Initialize components
config = ChunkConfig(max_chunk_size=1500, min_chunk_size=150)
chunker = MarkdownCodeChunker(config)
library_store = LibraryStore(host="localhost", port=6333)
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Installs the necessary project dependencies using the 'uv sync' command. This ensures all required libraries are available for running DeepContext.

```bash
uv sync
```

--------------------------------

### Start Qdrant with Docker or Podman

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Instructions to start the Qdrant vector database using Docker Compose or Podman Compose. Qdrant is a dependency for DeepContext's vector search functionality.

```bash
# Using Docker
docker compose up -d

# Or using Podman
podman compose up -d
```

--------------------------------

### Deepcontext Serve CLI

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Start the HTTP API server (FastAPI + Uvicorn) with the MCP HTTP server mounted at `/mcp`. Options include host, port, and reload.

```APIDOC
## `deepcontext serve`

### Description
Start the HTTP API server (FastAPI + Uvicorn) with the MCP HTTP server mounted at `/mcp`.

### Method
```bash
`uv run deepcontext serve --host 0.0.0.0 --port 8000`
```

### Options
- `--host` (string): Host to bind to (default: `0.0.0.0`).
- `--port` (integer): Port to bind to (default: `8000`).
- `--reload`: Enable auto-reload for development.
```

--------------------------------

### HTTP API Server

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Instructions for starting the FastAPI-based HTTP service, which includes the background worker and MCP server. Access API documentation at `/docs`.

```APIDOC
## HTTP API Server

Start the FastAPI-based HTTP service with MCP server.

```bash
# Start the API server (includes background worker and MCP server)
uv run deepcontext serve --host 0.0.0.0 --port 8000

# With auto-reload for development
uv run deepcontext serve --reload

# Access API docs at http://localhost:8000/docs
# MCP server available at http://localhost:8000/mcp
```
```

--------------------------------

### Start DeepContext HTTP API Server CLI

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Launches the DeepContext HTTP API server using FastAPI and Uvicorn. The MCP HTTP server is mounted at the `/mcp` endpoint. Supports configuring host, port, and auto-reload for development.

```bash
uv run deepcontext serve --host 0.0.0.0 --port 8000
uv run deepcontext serve --reload
```

--------------------------------

### Start DeepContext HTTP API Server

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Commands to start the FastAPI-based HTTP API server for DeepContext. This server integrates the background worker and MCP server, and can be run with or without auto-reload for development.

```bash
# Start the API server (includes background worker and MCP server)
uv run deepcontext serve --host 0.0.0.0 --port 8000

```

```bash
# With auto-reload for development
uv run deepcontext serve --reload

```

--------------------------------

### Python: Fetch Confluence Spaces

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Provides Python code examples for fetching all pages from a Confluence space using the ContentFetcher. It includes setting up authentication via environment variables and retrieving page metadata like ID, version, and space name.

```python
from deepcontext import ContentFetcher
import os

# Set up authentication
os.environ["CONFLUENCE_EMAIL"] = "you@company.com"
os.environ["CONFLUENCE_TOKEN"] = "your_api_token"

with ContentFetcher() as fetcher:
    # Fetch all pages from a space
    docs = fetcher.fetch_confluence_space(
        base_url="https://company.atlassian.net/wiki",
        space_key="ENGINEERING",
        limit=50
    )

    print(f"Fetched {len(docs)} pages from Confluence")

    for doc in docs:
        metadata = doc.metadata
        print(f"- {doc.title}")
        print(f"  Page ID: {metadata['confluence_page_id']}")
        print(f"  Version: {metadata['version']}")
        print(f"  Space: {metadata['space_name']}")

```

--------------------------------

### Example Search Result Output

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Illustrates the format of search results returned by DeepContext, including the source file, semantic score, and an extracted code block with its language.

```text
Search Results for: how to redirect in middleware

────────────────────────────────────────────────────────────

### Example
Source: /path/to/middleware.md
Score: 0.6693

Example typescript code demonstrating implementation.

┌──────────────────────────────────────────────────────────┐
│ import { NextResponse } from 'next/server'               │
│ import type { NextRequest } from 'next/server'           │
│                                                          │
│ export function middleware(request: NextRequest) {       │
│   return NextResponse.redirect(new URL('/home', ...))
│ }
│                                                          │
│ export const config = {                                  │
│   matcher: '/about/:path*',                              │
│ }
└──────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────
```

--------------------------------

### Use DeepContext MCP Server with Anthropic AI (Python)

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Illustrates how to integrate DeepContext as an MCP tool for AI assistants, specifically using the Anthropic client. It shows examples for resolving library IDs and searching documentation within a library.

```python
# The MCP server is available in two modes:

# 1. HTTP transport (streamable-http) - mounted at /mcp when running the API server
# Start the server:
# uv run deepcontext serve --host 0.0.0.0 --port 8000

# Configure your MCP client to use:
# http://localhost:8000/mcp

# 2. Stdio transport - for local command-line MCP clients
# Run the stdio server:
# uv run deepcontext-mcp

# The MCP server provides two main tools:

# - resolve-library-id: Find library IDs by name
#   Example: Search for "next.js" returns /vercel/next.js

# - get-library-docs: Search documentation within a library
#   Example: Query "middleware" in /vercel/next.js returns relevant chunks

# Example MCP tool usage (client-side):
import anthropic

client = anthropic.Anthropic()

# Use the resolve-library-id tool
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    tools=[
        {
            "name": "resolve-library-id",
            "description": "Resolve a library name to its ID",
            "input_schema": {
                "type": "object",
                "properties": {
                    "libraryName": {"type": "string"}
                }
            }
        }
    ],
    messages=[{"role": "user", "content": "Find the library ID for Next.js"}]
)

# Use the get-library-docs tool
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=4096,
    tools=[
        {
            "name": "get-library-docs",
            "description": "Search library documentation",
            "input_schema": {
                "type": "object",
                "properties": {
                    "libraryId": {"type": "string"},
                    "query": {"type": "string"},
                    "limit": {"type": "integer", "default": 5}
                }
            }
        }
    ],
    messages=[{"role": "user", "content": "How do I use middleware in Next.js?"}]
)
```

--------------------------------

### Run HTTP API + MCP Server

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Starts DeepContext as a long-lived HTTP service, providing a REST API for ingestion, search, stats, and library metadata, as well as an MCP server for streamable-http MCP clients. Uses FastAPI and Uvicorn.

```bash
# Start the API + MCP service (FastAPI + Uvicorn)
uv run deepcontext serve --host 0.0.0.0 --port 8000

# Open interactive docs
#   http://localhost:8000/docs

# For stdio-based MCP clients
uv run deepcontext-mcp
```

--------------------------------

### Filter VectorStore Searches and Manage Collections

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This example illustrates advanced features of VectorStore, including filtering search results by source type (e.g., GitHub) and programming language (e.g., TypeScript). It also covers collection management operations such as retrieving statistics, deleting chunks by source or document ID, and clearing the entire collection. These operations help in refining search relevance and maintaining the integrity of the vector database.

```python
from deepcontext import VectorStore
from deepcontext.models import SourceType

store = VectorStore(collection_name="docs")

# Search with filters
results = store.search(
    query="async functions",
    limit=10,
    source_type=SourceType.GITHUB,
    language="typescript"
)

print(f"Found {len(results)} TypeScript examples from GitHub")

# Get collection statistics
stats = store.get_stats()
print(f"Collection: {stats['collection_name']}")
print(f"Total points: {stats['points_count']}")
print(f"Indexed vectors: {stats['indexed_vectors_count']}")
print(f"Status: {stats['status']}")

# Delete chunks by source
deleted = store.delete_by_source("https://example.com/old-docs")
print(f"Deleted {deleted} chunks from old source")

# Delete by document ID
deleted = store.delete_by_document_id("abc123def456")
print(f"Deleted {deleted} chunks from document")

# Clear entire collection
store.clear()
print("Collection cleared and recreated")

store.close()
```

--------------------------------

### Python: Fetch GitHub Repositories

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Illustrates how to fetch all documentation files from a GitHub repository using the ContentFetcher. It covers fetching the entire repository, specifying paths and branches, filtering by file extensions, and using a GitHub token for increased rate limits.

```python
from deepcontext import ContentFetcher

with ContentFetcher() as fetcher:
    # Fetch entire repository (default: .md and .mdx files)
    docs = fetcher.fetch_github_repo("vercel/next.js")
    print(f"Fetched {len(docs)} documents")

    # Fetch specific path and branch
    docs = fetcher.fetch_github_repo(
        repo="facebook/react",
        branch="main",
        path="docs",
        extensions=[ ".md", ".mdx", ".txt" ]
    )

    for doc in docs[:5]:
        print(f"- {doc.title} ({doc.metadata['path']})")

# With GitHub token for higher rate limits
import os
os.environ["GITHUB_TOKEN"] = "ghp_your_token_here"

with ContentFetcher() as fetcher:
    docs = fetcher.fetch_github_repo("microsoft/typescript", path="docs")
    print(f"Successfully fetched {len(docs)} files")

```

--------------------------------

### Initialize and Index Chunks into Vector Store (Python)

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This code sets up a VectorStore client, likely connecting to a Qdrant instance. It determines a library ID and collection name, then indexes the previously generated chunks. Finally, it updates metadata for the library, including document and chunk counts.

```python
from deepcontext.vector_store import VectorStore
from deepcontext.indexing import derive_library_id, library_id_to_collection_name
from deepcontext.library_store import library_store

# Determine library ID and collection for indexing
library_id = derive_library_id("vercel/next.js", "repo")
collection_name = library_id_to_collection_name(library_id)

# Initialize vector store with per-library collection
store = VectorStore(
    host="localhost",
    port=6333,
    collection_name=collection_name
)

# Index all chunks
# Assuming 'all_chunks' is populated from the previous step
indexed = store.index_chunks(all_chunks, batch_size=100)
print(f"Indexed {indexed} chunks into collection: {collection_name}")

# Update library metadata
library_store.update_library_metadata(
    library_id=library_id,
    documents_count=len(github_docs + website_docs + confluence_docs), # Assuming these are defined from previous step
    chunks_count=indexed
)
```

--------------------------------

### Get Job Details

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Retrieves detailed information for a specific job, including its status, progress, and any associated error messages.

```APIDOC
## GET /job/{job_id}

### Description
Shows detailed information for a specific job, including its status, counts, timestamps, and any error messages.

### Method
GET

### Endpoint
`/job/<job_id>`

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The unique identifier for the job.

### Request Example
```bash
# Get details for a job with ID '123e4567-e89b-12d3-a456-426614174000'
uv run deepcontext job 123e4567-e89b-12d3-a456-426614174000
```

### Response
(Details on the structure of the job details response would typically be here, but are not provided in the source text.)
```

--------------------------------

### Python: Fetch Entire Websites

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Demonstrates using ContentFetcher to crawl entire websites, either by using sitemap.xml or by following links. It shows options for setting maximum pages, filtering URLs with regex patterns, and disabling sitemap usage in favor of direct crawling with a progress callback.

```python
from deepcontext import ContentFetcher

with ContentFetcher() as fetcher:
    # Use sitemap automatically
    docs = fetcher.fetch_website(
        base_url="https://ui.shadcn.com/docs",
        max_pages=100
    )
    print(f"Fetched {len(docs)} pages from sitemap")

    # With URL pattern filter (regex)
    docs = fetcher.fetch_website(
        base_url="https://nextjs.org/docs",
        max_pages=50,
        url_pattern=r"/docs/app/",
        use_sitemap=True
    )

    # Force crawling instead of sitemap with progress callback
    def progress(url, current, total):
        print(f"[{current}/{total}] Fetching {url[:60]}...")

    docs = fetcher.fetch_website(
        base_url="https://example.com/docs",
        max_pages=30,
        use_sitemap=False,
        progress_callback=progress
    )

    for doc in docs:
        print(f"- {doc.title} ({len(doc.content)} chars)")

```

--------------------------------

### Run DeepContext Background Worker CLI

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Starts a background worker that processes ingestion jobs from a queue. Supports options to process a single job once or configure polling intervals. This worker complements the HTTP API.

```bash
deepcontext worker
deepcontext worker --once
deepcontext worker --poll-interval 5.0
```

--------------------------------

### Index and Search Chunks with VectorStore in Qdrant

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This snippet shows how to initialize a VectorStore connected to Qdrant, fetch a document, chunk it using MarkdownCodeChunker, and then index these chunks. It demonstrates performing a semantic search with a query and displaying the results, including score, title, description, source, and code blocks. The VectorStore is configured with connection details, collection name, and an embedding model.

```python
from deepcontext import ContentFetcher, VectorStore
from deepcontext.chunker import MarkdownCodeChunker
from deepcontext.models import SourceType

# Initialize vector store
store = VectorStore(
    host="localhost",
    port=6333,
    collection_name="my_docs",
    embedding_model="BAAI/bge-small-en-v1.5"
)

# Fetch, chunk, and index content
with ContentFetcher() as fetcher:
    doc = fetcher.fetch("https://nextjs.org/docs/app/building-your-application/routing/middleware")

chunker = MarkdownCodeChunker()
chunks = chunker.chunk_document(doc)

# Index chunks
indexed = store.index_chunks(chunks)
print(f"Indexed {indexed} chunks")

# Search for similar content
results = store.search(
    query="how to redirect in middleware",
    limit=5
)

for result in results:
    print(f"\nScore: {result.score:.4f}")
    print(f"Title: {result.chunk.title}")
    print(f"Description: {result.chunk.description}")
    print(f"Source: {result.chunk.source}")

    # Display code blocks
    for code in result.chunk.code_blocks:
        print(f"\nCode ({result.chunk.language}):")
        print(code[:200])

store.close()
```

--------------------------------

### Ingest Content via CLI

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Demonstrates various ways to ingest content into DeepContext using the command-line interface. Supports local files, GitHub URLs, webpages, Confluence pages and spaces, and entire websites.

```bash
# From a local markdown file
uv run deepcontext ingest ./docs/README.md

# From a GitHub URL
uv run deepcontext ingest https://github.com/vercel/next.js/blob/canary/contributing/core/developing.md

# From a webpage
uv run deepcontext ingest https://nextjs.org/docs/app/building-your-application/routing/middleware

# From a Confluence page (requires auth)
export CONFLUENCE_EMAIL=you@company.com
export CONFLUENCE_TOKEN=your_api_token
uv run deepcontext ingest https://company.atlassian.net/wiki/spaces/DOCS/pages/123456/Page-Title

# From an entire Confluence space
uv run deepcontext ingest-confluence https://company.atlassian.net/wiki DOCS --limit 50

# From an entire website (uses sitemap or crawls links)
uv run deepcontext ingest-website https://ui.shadcn.com/docs --max-pages 50
```

--------------------------------

### ContentFetcher - Fetch GitHub Repositories

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Fetch all documentation files from a GitHub repository using `fetch_github_repo`. Supports specifying branches, paths, file extensions, and authentication tokens for increased rate limits.

```APIDOC
## Python API - ContentFetcher - Fetch GitHub Repositories

Fetch all documentation files from a GitHub repository using gitingest.

```python
from deepcontext import ContentFetcher

with ContentFetcher() as fetcher:
    # Fetch entire repository (default: .md and .mdx files)
    docs = fetcher.fetch_github_repo("vercel/next.js")
    print(f"Fetched {len(docs)} documents")

    # Fetch specific path and branch
    docs = fetcher.fetch_github_repo(
        repo="facebook/react",
        branch="main",
        path="docs",
        extensions=[".md", ".mdx", ".txt"]
    )

    for doc in docs[:5]:
        print(f"- {doc.title} ({doc.metadata['path']})")

# With GitHub token for higher rate limits
import os
os.environ["GITHUB_TOKEN"] = "ghp_your_token_here"

with ContentFetcher() as fetcher:
    docs = fetcher.fetch_github_repo("microsoft/typescript", path="docs")
    print(f"Successfully fetched {len(docs)} files")
```
```

--------------------------------

### Queue Website for Ingestion

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Queues a website for background ingestion. Options mirror `ingest-website`.

```APIDOC
## POST /queue-website

### Description
Queues a website for background ingestion. This is the queued version of `deepcontext ingest-website`.

### Method
POST

### Endpoint
`/queue-website <url>`

### Parameters
#### Path Parameters
- **url** (string) - Required - The base URL of the website to ingest.

#### Query Parameters
- **max-pages** (integer) - Optional - Maximum number of pages to fetch. Defaults to 100.
- **pattern** (string) - Optional - A URL pattern (regex) to filter pages.
- **no-sitemap** (boolean) - Optional - If set, prevents the command from trying to use `sitemap.xml` and forces link crawling.
- **collection** (string) - Optional - Qdrant collection name. Defaults to "deepcontext_chunks".
- **host** (string) - Optional - Qdrant host. Defaults to "localhost".
- **port** (integer) - Optional - Qdrant port. Defaults to 6333.

### Request Example
```bash
uv run deepcontext queue-website https://example.com/docs
```

### Response
Prints a **job ID** that can be used with `deepcontext jobs` and `deepcontext job`.
(Details on success/error responses would typically be here, but are not provided in the source text.)
```

--------------------------------

### Ingest Data from GitHub, Websites, and Confluence (Python)

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This snippet demonstrates how to use the ContentFetcher to ingest documentation from a GitHub repository, a website, and a Confluence space. It then chunks the retrieved documents using a chunker and stores them in a list. Error handling is included for the chunking process.

```python
from deepcontext.retrieval import ContentFetcher
from deepcontext.indexing import chunker

# Initialize ContentFetcher
with ContentFetcher() as fetcher:
    all_chunks = []

    # Fetch GitHub repository
    github_docs = fetcher.fetch_github_repo(
        repo="vercel/next.js",
        path="docs",
        extensions=[ ".md", ".mdx" ]
    )
    print(f"Fetched {len(github_docs)} files from GitHub")

    # Fetch website
    website_docs = fetcher.fetch_website(
        base_url="https://ui.shadcn.com/docs",
        max_pages=50
    )
    print(f"Fetched {len(website_docs)} pages from website")

    # Fetch Confluence space
    confluence_docs = fetcher.fetch_confluence_space(
        base_url="https://company.atlassian.net/wiki",
        space_key="TEAM",
        limit=25
    )
    print(f"Fetched {len(confluence_docs)} pages from Confluence")

    # Chunk all documents
    for doc in github_docs + website_docs + confluence_docs:
        try:
            chunks = chunker.chunk_document(doc)
            all_chunks.extend(chunks)
        except Exception as e:
            print(f"Error chunking {doc.title}: {e}")

    print(f"\nCreated {len(all_chunks)} total chunks")
```

--------------------------------

### Python: Fetch Content from Multiple Sources

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Demonstrates using the ContentFetcher class in Python to retrieve content from various sources like local files, GitHub, webpages, and Confluence. It shows basic usage with context managers, custom timeouts, and authentication for Confluence.

```python
from deepcontext import ContentFetcher

# Basic usage with context manager
with ContentFetcher() as fetcher:
    # Local markdown file
    doc = fetcher.fetch("./docs/README.md")
    print(f"Title: {doc.title}")
    print(f"Content length: {len(doc.content)}")

    # GitHub file
    doc = fetcher.fetch("https://github.com/vercel/next.js/blob/canary/docs/api-reference/components/image.md")
    print(f"Source: {doc.source_type}")

    # Webpage
    doc = fetcher.fetch("https://nextjs.org/docs/app/building-your-application/routing")
    print(f"Metadata: {doc.metadata}")

# With custom timeout
fetcher = ContentFetcher(timeout=60.0)
doc = fetcher.fetch("https://example.com/slow-page")
fetcher.close()

# Confluence with authentication
import os
os.environ["CONFLUENCE_EMAIL"] = "you@company.com"
os.environ["CONFLUENCE_TOKEN"] = "your_token"

with ContentFetcher() as fetcher:
    doc = fetcher.fetch("https://company.atlassian.net/wiki/spaces/DOCS/pages/123456")
    print(f"Confluence page: {doc.title}")
    print(f"Version: {doc.metadata.get('version')}")

```

--------------------------------

### Ingest GitHub Repository with DeepContext

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Ingests documentation from a specified GitHub repository. Supports branch, path, and file extension filtering. Requires GITHUB_TOKEN for enhanced rate limits. Outputs to a Qdrant collection.

```bash
# Create token at https://github.com/settings/tokens with 'public_repo' scope
export GITHUB_TOKEN=ghp_your_token_here
uv run deepcontext ingest-repo vercel/next.js --path docs
```

--------------------------------

### Python API - Content Fetching and Chunking

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Demonstrates fetching content from various sources (webpages, GitHub, Confluence) and chunking it using configurable parameters.

```APIDOC
## Python API

### Fetching Content
```python
from deepcontext import ContentFetcher, VectorStore
from deepcontext.chunker import ChunkConfig, DocumentChunker

# Fetch content
with ContentFetcher() as fetcher:
    doc = fetcher.fetch("https://github.com/vercel/next.js/blob/canary/docs/...")

# Chunk with code awareness
config = ChunkConfig(
    max_chunk_size=2000,  # Max characters per chunk
    min_chunk_size=200,   # Min characters for a chunk
    overlap_size=100,     # Overlap between chunks
)
chunker = DocumentChunker(config)
chunks = chunker.chunk_document(doc)

# Store and search
store = VectorStore()
store.index_chunks(chunks)

results = store.search("middleware authentication")
for result in results:
    print(result.chunk.to_display_format())
    print(f"Score: {result.score}")

store.close()
```

### Fetching Multiple Documents
```python
from deepcontext import ContentFetcher

with ContentFetcher() as fetcher:
    # Fetch entire GitHub repo
    docs = fetcher.fetch_github_repo("vercel/next.js", path="docs")
    
    # Fetch Confluence space
    docs = fetcher.fetch_confluence_space(
        base_url="https://company.atlassian.net/wiki",
        space_key="DOCS",
        limit=50,
    )
    
    # Fetch entire website
    docs = fetcher.fetch_website(
        base_url="https://ui.shadcn.com/docs",
        max_pages=100,
        url_pattern=r"/docs/",  # Optional regex filter
    )
```
```

--------------------------------

### DeepContext Ingest CLI Options

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Details the options available for the `deepcontext ingest` command, allowing customization of chunk size, similarity threshold, code-aware processing, and Qdrant connection parameters.

```bash
### `deepcontext ingest <source>`

Ingest content from any supported source.

**SOURCE** can be:
- A local file path (e.g., `./docs/README.md`)
- A GitHub URL (e.g., `https://github.com/vercel/next.js/blob/canary/docs/`)
- A Confluence page (e.g., `https://company.atlassian.net/wiki/spaces/DOCS/pages/123456`)
- A webpage URL (e.g., `https://example.com/docs`)

Options:
- `--chunk-size`: Maximum chunk size in tokens (default: 1024)
- `--threshold`: Similarity threshold for semantic chunking (default: 0.7)
- `--code-aware/--no-code-aware`: Use code-aware chunking (default: enabled)
- `--collection`: Qdrant collection name (default: deepcontext_chunks)
- `--host`: Qdrant host (default: localhost)
- `--port`: Qdrant port (default: 6333)
```

--------------------------------

### ContentFetcher - Fetch Entire Websites

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Crawl websites using `fetch_website`, supporting sitemap.xml or link following. Allows filtering by URL pattern and provides a progress callback for monitoring.

```APIDOC
## Python API - ContentFetcher - Fetch Entire Websites

Crawl websites using sitemap.xml or link following with optional filtering.

```python
from deepcontext import ContentFetcher

with ContentFetcher() as fetcher:
    # Use sitemap automatically
    docs = fetcher.fetch_website(
        base_url="https://ui.shadcn.com/docs",
        max_pages=100
    )
    print(f"Fetched {len(docs)} pages from sitemap")

    # With URL pattern filter (regex)
    docs = fetcher.fetch_website(
        base_url="https://nextjs.org/docs",
        max_pages=50,
        url_pattern=r"/docs/app/",
        use_sitemap=True
    )

    # Force crawling instead of sitemap with progress callback
    def progress(url, current, total):
        print(f"[{current}/{total}] Fetching {url[:60]}...")

    docs = fetcher.fetch_website(
        base_url="https://example.com/docs",
        max_pages=30,
        use_sitemap=False,
        progress_callback=progress
    )

    for doc in docs:
        print(f"- {doc.title} ({len(doc.content)} chars)")
```
```

--------------------------------

### Manage Library Metadata with LibraryStore

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This snippet demonstrates the usage of LibraryStore for managing library metadata and per-library collections. It shows how to derive a library ID from a source (like a GitHub repository), convert this ID to a collection name, list available libraries, and retrieve details for a specific library. This is useful for organizing and querying data related to different software libraries or projects.

```python
from deepcontext.store import LibraryStore, derive_library_id, library_id_to_collection_name

# Initialize library store
library_store = LibraryStore(
    host="localhost",
    port=6333
)

# Derive library ID from source
library_id = derive_library_id("vercel/next.js", "repo")
print(f"Library ID: {library_id}")  # Output: /vercel/next.js

# Convert library ID to collection name
collection_name = library_id_to_collection_name(library_id)
print(f"Collection: {collection_name}")  # Output: lib_vercel_next_js

# List all libraries
libraries = library_store.list_libraries(limit=50)
for lib in libraries:
    print(f"- {lib.library_id} ({lib.name})")
    print(f"  Collection: {lib.collection_name}")
    print(f"  Source: {lib.source}")
    print(f"  Chunks: {lib.chunks_count}")

# Get specific library
library = library_store.get_library("/vercel/next.js")
if library:
    print(f"Found library: {library.name}")
    print(f"Documents: {library.documents_count}")
    print(f"Chunks: {library.chunks_count}")
```

--------------------------------

### Manage DeepContext Libraries (Python)

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Demonstrates basic library management operations using the DeepContext Python client, including updating metadata, deleting libraries, and closing the connection. Assumes a running LibraryStore instance.

```python
library_store.update_library_metadata(
    library_id="/vercel/next.js",
    documents_count=150,
    chunks_count=450
)

library_store.delete_library("/vercel/next.js")

library_store.close()
```

--------------------------------

### Queue GitHub Repository for Ingestion

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Queues a GitHub repository for background ingestion. Options mirror `ingest-repo`.

```APIDOC
## POST /queue-repo

### Description
Queues a GitHub repository for background ingestion. This is the queued version of `deepcontext ingest-repo`.

### Method
POST

### Endpoint
`/queue-repo <owner/repo>`

### Parameters
#### Path Parameters
- **owner/repo** (string) - Required - The owner and name of the repository (e.g., "vercel/next.js").

#### Query Parameters
- **branch** (string) - Optional - The branch to fetch from. Defaults to auto-detection.
- **path** (string) - Optional - The path within the repo to start from.
- **extensions** (string) - Optional - File extensions to fetch. Defaults to ".md,.mdx".
- **collection** (string) - Optional - Qdrant collection name. Defaults to "deepcontext_chunks".
- **host** (string) - Optional - Qdrant host. Defaults to "localhost".
- **port** (integer) - Optional - Qdrant port. Defaults to 6333.

### Request Example
```bash
uv run deepcontext queue-repo vercel/next.js --path docs
```

### Response
Prints a **job ID** that can be used with `deepcontext jobs` and `deepcontext job`.
(Details on success/error responses would typically be here, but are not provided in the source text.)
```

--------------------------------

### Ingest Entire Website with DeepContext CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Ingest content from websites using `deepcontext ingest-website`. It can automatically detect and use `sitemap.xml` or perform link crawling. Options include filtering by URL patterns, setting a maximum number of pages, and forcing link crawling.

```bash
# Automatically find and use sitemap.xml
uv run deepcontext ingest-website https://ui.shadcn.com/docs

# Filter by URL pattern (regex)
uv run deepcontext ingest-website https://nextjs.org/docs --pattern "/docs/app/" --max-pages 100

# Force link crawling instead of sitemap
uv run deepcontext ingest-website https://example.com/docs --no-sitemap --max-pages 50
```

--------------------------------

### ContentFetcher - Fetch Confluence Spaces

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Fetch all pages from a Confluence space with pagination using `fetch_confluence_space`. Requires Confluence email and API token for authentication.

```APIDOC
## Python API - ContentFetcher - Fetch Confluence Spaces

Fetch all pages from a Confluence space with pagination.

```python
from deepcontext import ContentFetcher
import os

# Set up authentication
os.environ["CONFLUENCE_EMAIL"] = "you@company.com"
os.environ["CONFLUENCE_TOKEN"] = "your_api_token"

with ContentFetcher() as fetcher:
    # Fetch all pages from a space
    docs = fetcher.fetch_confluence_space(
        base_url="https://company.atlassian.net/wiki",
        space_key="ENGINEERING",
        limit=50
    )

    print(f"Fetched {len(docs)} pages from Confluence")

    for doc in docs:
        metadata = doc.metadata
        print(f"- {doc.title}")
        print(f"  Page ID: {metadata['confluence_page_id']}")
        print(f"  Version: {metadata['version']}")
        print(f"  Space: {metadata['space_name']}")
```
```

--------------------------------

### Ingest GitHub Repository with DeepContext CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Ingest all documentation files from a GitHub repository using `deepcontext ingest-repo`. Supports filtering by path, specifying branches, custom file extensions, and utilizing GitHub tokens for increased rate limits. Defaults to processing .md and .mdx files.

```bash
# Ingest entire repository (default: .md and .mdx files)
uv run deepcontext ingest-repo vercel/next.js

# Ingest specific path within repository
uv run deepcontext ingest-repo vercel/next.js --path docs --branch canary

# Custom file extensions
uv run deepcontext ingest-repo facebook/react --extensions .md,.mdx,.txt

# With GitHub token for higher rate limits (5,000/hour vs 60/hour)
export GITHUB_TOKEN=ghp_your_token_here
uv run deepcontext ingest-repo vercel/next.js --path docs
```

--------------------------------

### Manage Vector Store with DeepContext CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Manage the Qdrant vector store using `deepcontext stats` to view collection statistics and `deepcontext clear` to remove all data. Statistics can be viewed for specific collections and custom host/port configurations.

```bash
# Show collection statistics
uv run deepcontext stats

# Clear all data (requires confirmation)
uv run deepcontext clear

# Check stats for custom collection
uv run deepcontext stats --collection custom_docs --host localhost --port 6333
```

--------------------------------

### ContentFetcher - Fetch from Multiple Sources

Source: https://context7.com/pranavms13/deepcontext/llms.txt

The `ContentFetcher` class allows fetching content from various sources including local files, GitHub repositories, webpages, and Confluence spaces. It supports basic usage with a context manager and custom timeouts.

```APIDOC
## Python API - ContentFetcher - Fetch from Multiple Sources

Fetch content from local files, GitHub, webpages, and Confluence.

```python
from deepcontext import ContentFetcher

# Basic usage with context manager
with ContentFetcher() as fetcher:
    # Local markdown file
    doc = fetcher.fetch("./docs/README.md")
    print(f"Title: {doc.title}")
    print(f"Content length: {len(doc.content)}")

    # GitHub file
    doc = fetcher.fetch("https://github.com/vercel/next.js/blob/canary/docs/api-reference/components/image.md")
    print(f"Source: {doc.source_type}")

    # Webpage
    doc = fetcher.fetch("https://nextjs.org/docs/app/building-your-application/routing")
    print(f"Metadata: {doc.metadata}")

# With custom timeout
fetcher = ContentFetcher(timeout=60.0)
doc = fetcher.fetch("https://example.com/slow-page")
fetcher.close()

# Confluence with authentication
import os
os.environ["CONFLUENCE_EMAIL"] = "you@company.com"
os.environ["CONFLUENCE_TOKEN"] = "your_token"

with ContentFetcher() as fetcher:
    doc = fetcher.fetch("https://company.atlassian.net/wiki/spaces/DOCS/pages/123456")
    print(f"Confluence page: {doc.title}")
    print(f"Version: {doc.metadata.get('version')}")
```
```

--------------------------------

### Ingest Confluence Space with DeepContext CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Ingest all pages from a Confluence space using `deepcontext ingest-confluence`. Requires setting `CONFLUENCE_EMAIL` and `CONFLUENCE_TOKEN` environment variables for authentication. Supports custom page limits and specifying a custom Qdrant collection.

```bash
# Set up authentication
export CONFLUENCE_EMAIL=you@company.com
export CONFLUENCE_TOKEN=your_api_token

# Ingest entire space (default limit: 100 pages)
uv run deepcontext ingest-confluence https://company.atlassian.net/wiki ENGINEERING

# Custom page limit
uv run deepcontext ingest-confluence https://company.atlassian.net/wiki DOCS --limit 50

# Store in custom collection
uv run deepcontext ingest-confluence https://company.atlassian.net/wiki TEAM --collection team_docs
```

--------------------------------

### DeepContext CLI Commands

Source: https://context7.com/pranavms13/deepcontext/llms.txt

This section outlines the various commands available for interacting with DeepContext via the command line interface for managing jobs and workers.

```APIDOC
## DeepContext CLI Commands

### List all jobs
```bash
uv run deepcontext jobs
```

### View specific job details
```bash
uv run deepcontext job job_abc123
```

### Run the background worker (processes queued jobs)
```bash
uv run deepcontext worker
```

### Run worker once and exit
```bash
uv run deepcontext worker --once
```

### Cancel a job
```bash
uv run deepcontext cancel-job job_abc123
```

### Clear completed and failed jobs
```bash
uv run deepcontext clear-jobs
```
```

--------------------------------

### Ingest Single Source with DeepContext CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Ingest content from local markdown files, GitHub URLs, webpages, or Confluence pages using the `deepcontext ingest` command. Custom chunking parameters like size, threshold, and code-aware processing can be specified. Authentication is required for Confluence.

```bash
# Local markdown file
uv run deepcontext ingest ./docs/README.md

# GitHub file
uv run deepcontext ingest https://github.com/vercel/next.js/blob/canary/contributing/core/developing.md

# Webpage
uv run deepcontext ingest https://nextjs.org/docs/app/building-your-application/routing/middleware

# Confluence page (requires CONFLUENCE_EMAIL and CONFLUENCE_TOKEN env vars)
export CONFLUENCE_EMAIL=you@company.com
export CONFLUENCE_TOKEN=your_api_token
uv run deepcontext ingest https://company.atlassian.net/wiki/spaces/DOCS/pages/123456/Page-Title

# With custom chunking parameters
uv run deepcontext ingest ./docs/README.md --chunk-size 2000 --threshold 0.8 --no-code-aware
```

--------------------------------

### Manage Vector Store with DeepContext

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Provides commands to view statistics of the vector store or clear all data from it. Both operations can target a specific Qdrant collection.

```bash
# Show statistics
uv run deepcontext stats

# Clear all data (requires confirmation)
uv run deepcontext clear
```

--------------------------------

### Ingest Confluence Space with DeepContext

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Ingests all pages from a Confluence space. Requires CONFLUENCE_EMAIL and CONFLUENCE_TOKEN environment variables for authentication. Supports limiting the number of pages fetched and specifying the output Qdrant collection.

```bash
export CONFLUENCE_EMAIL=you@company.com
export CONFLUENCE_TOKEN=your_api_token
uv run deepcontext ingest-confluence https://company.atlassian.net/wiki ENGINEERING
```

--------------------------------

### Manage DeepContext Jobs

Source: https://github.com/pranavms13/deepcontext/blob/main/README.md

Provides commands to list all queued jobs with optional status and limit filters, or to view detailed information about a specific job using its ID.

```bash
# List all pending jobs
uv run deepcontext jobs --status pending

# Show details for a specific job
uv run deepcontext job abcdef12-3456-7890-abcd-ef1234567890
```

--------------------------------

### HTTP API - Programmatic Access

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Use the HTTP API to queue jobs, search programmatically, and manage libraries.

```APIDOC
## POST /ingest/repo

### Description
Queue a repository for ingestion.

### Method
POST

### Endpoint
/ingest/repo

### Parameters
#### Query Parameters
None

#### Request Body
- **repo** (string) - Required - The repository to ingest (e.g., "vercel/next.js").
- **path** (string) - Optional - The specific path within the repository to ingest (e.g., "docs").
- **extensions** (string) - Optional - Comma-separated list of file extensions to include (e.g., ".md,.mdx").

### Request Example
```json
{
    "repo": "vercel/next.js",
    "path": "docs",
    "extensions": ".md,.mdx"
}
```

### Response
#### Success Response (200)
- **id** (string) - The ID of the queued job.

#### Response Example
```json
{
    "id": "job_id_123"
}
```

## GET /jobs/{job_id}

### Description
Check the status of a queued job.

### Method
GET

### Endpoint
/jobs/{job_id}

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The ID of the job to check.

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
(No request body)
```

### Response
#### Success Response (200)
- **status** (string) - The current status of the job (e.g., "completed", "pending", "failed").

#### Response Example
```json
{
    "status": "completed"
}
```

## POST /search

### Description
Search for documentation within a specified library.

### Method
POST

### Endpoint
/search

### Parameters
#### Query Parameters
None

#### Request Body
- **query** (string) - Required - The search query.
- **library_id** (string) - Required - The ID of the library to search within.
- **limit** (integer) - Optional - The maximum number of results to return (defaults to 5).

### Request Example
```json
{
    "query": "how to use middleware",
    "library_id": "/vercel/next.js",
    "limit": 5
}
```

### Response
#### Success Response (200)
- **total** (integer) - The total number of results found.
- **results** (array) - A list of search results.
  - **title** (string) - The title of the result.
  - **score** (float) - The relevance score of the result.

#### Response Example
```json
{
    "total": 3,
    "results": [
        {
            "title": "Using Middleware in Next.js",
            "score": 0.95
        },
        {
            "title": "Middleware Explained",
            "score": 0.88
        }
    ]
}
```

## GET /libraries

### Description
List all available libraries.

### Method
GET

### Endpoint
/libraries

### Parameters
None

### Request Example
```json
(No request body)
```

### Response
#### Success Response (200)
- An array of library objects, where each object contains library details.

#### Response Example
```json
[
    {
        "name": "Next.js",
        "id": "/vercel/next.js",
        "chunks_count": 450
    }
]
```

## GET /libraries/{library_id}

### Description
Retrieve details for a specific library.

### Method
GET

### Endpoint
/libraries/{library_id}

### Parameters
#### Path Parameters
- **library_id** (string) - Required - The ID of the library to retrieve.

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
(No request body)
```

### Response
#### Success Response (200)
- **name** (string) - The name of the library.
- **id** (string) - The ID of the library.
- **chunks_count** (integer) - The number of chunks in the library.

#### Response Example
```json
{
    "name": "Next.js",
    "id": "/vercel/next.js",
    "chunks_count": 450
}
```

## DELETE /libraries/{library_id}

### Description
Delete a library and its associated collection.

### Method
DELETE

### Endpoint
/libraries/{library_id}

### Parameters
#### Path Parameters
- **library_id** (string) - Required - The ID of the library to delete.

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
(No request body)
```

### Response
#### Success Response (200)
- **status** (string) - Indicates the result of the deletion operation (e.g., "deleted").

#### Response Example
```json
{
    "status": "deleted"
}
```
```

--------------------------------

### Manage DeepContext Jobs using CLI

Source: https://context7.com/pranavms13/deepcontext/llms.txt

Command-line interface commands to manage jobs and workers within DeepContext. This includes listing, viewing, running, and canceling jobs, as well as clearing completed or failed tasks.

```bash
uv run deepcontext jobs

```

```bash
uv run deepcontext job job_abc123

```

```bash
uv run deepcontext worker

```

```bash
uv run deepcontext worker --once

```

```bash
uv run deepcontext cancel-job job_abc123

```

```bash
uv run deepcontext clear-jobs

```