raghilda

https://posit-dev.github.io/raghilda/
Admin
raghilda is a Python package for implementing Retrieval-Augmented Generation (RAG) workflows,...
Tokens:67,990
Snippets:662
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install raghilda from GitHub

Source: https://posit-dev.github.io/raghilda

Alternative installation command to install directly from the GitHub repository.

```bash
pip install git+https://github.com/posit-dev/raghilda.git
```

--------------------------------

### OpenAIStore usage example

Source: https://posit-dev.github.io/raghilda/reference/store.OpenAIStore.md

Complete example showing how to create or connect to an OpenAIStore, insert a MarkdownDocument, and retrieve similar chunks. Demonstrates the main workflow with the raghilda API.

```python
from raghilda.store import OpenAIStore

# Create a new store
store = OpenAIStore.create(name="my-store")

# Or connect to an existing store
store = OpenAIStore.connect(store_id="vs_abc123")

# Insert documents
from raghilda.document import MarkdownDocument
doc = MarkdownDocument(content="# Hello\nWorld", origin="example.md")
store.upsert(doc)

# Retrieve similar chunks
chunks = store.retrieve("greeting", top_k=5)
```

--------------------------------

### Setup development environment

Source: https://posit-dev.github.io/raghilda/contributing.md

Commands to synchronize the project dependencies and activate the virtual environment.

```bash
uv sync --all-groups --extra test
source .venv/bin/activate
```

--------------------------------

### WebCrawler example: crawl with scope and cache

Source: https://posit-dev.github.io/raghilda/reference/crawl.WebCrawler.md

Demonstrates creating a `WebCrawler` with disk caching and a `CrawlScope` to crawl the Quarto guide, yielding `MarkdownDocument` objects. Requires `from datetime import timedelta` and `from raghilda.crawl import CrawlScope, WebCrawler`.

```python
from datetime import timedelta

from raghilda.crawl import CrawlScope, WebCrawler

crawler = WebCrawler(cache_dir=True, cache_stale_after=timedelta(days=1))
scope = CrawlScope(
    roots=["https://quarto.org/docs/guide/"],
    depth=2,
    include_patterns=[r"^https://quarto\.org/docs/guide/"],
    include_types=["html"],
)

for document in crawler.markdown_documents(scope):
    print(document.origin)
```

--------------------------------

### Install skill with npx

Source: https://posit-dev.github.io/raghilda/skills.html

Use this command to install the skill for any compatible AI agent.

```bash
npx skills add https://posit-dev.github.io/raghilda/
```

--------------------------------

### Implement a RAG pipeline with raghilda

Source: https://posit-dev.github.io/raghilda

Example demonstrating the full RAG workflow: creating a store, indexing documentation pages, and performing a retrieval query.

```python
from raghilda.store import DuckDBStore
from raghilda.embedding import EmbeddingOpenAI
from raghilda.scrape import find_links
from raghilda.read import read_as_markdown
from raghilda.chunker import MarkdownChunker

# Create a store with embeddings
store = DuckDBStore.create(
    location="chatlas.db",
    embed=EmbeddingOpenAI(),
)

# Find and index pages from the chatlas documentation
links = find_links("https://posit-dev.github.io/chatlas/")
chunker = MarkdownChunker()

# Read, chunk, and store each page
for link in links:
    document = read_as_markdown(link)
    chunked_document = chunker.chunk(document)
    store.upsert(chunked_document)

# Build indexes before retrieval
store.build_index()

# Retrieve relevant chunks
chunks = store.retrieve("How do I stream a response?", top_k=5)
for chunk in chunks:
    print(chunk.text)
```

--------------------------------

### DuckDBStore full example: create, upsert, index, retrieve

Source: https://posit-dev.github.io/raghilda/reference/store.DuckDBStore.md

Complete example demonstrating how to create a DuckDBStore with an OpenAI embedding provider, insert a chunked Markdown document, build indexes, and retrieve similar chunks. Requires the raghilda package and an OpenAI API key.

```python
from raghilda.store import DuckDBStore
from raghilda.embedding import EmbeddingOpenAI

# Create a new store with embeddings
store = DuckDBStore.create(
    location="my_store.db",
    embed=EmbeddingOpenAI(),
)

# Insert a chunked document
from raghilda.document import MarkdownDocument
from raghilda.chunker import MarkdownChunker

doc = MarkdownDocument(
    origin="https://example.com/doc1.md",
    content="# Example

This is a sample document.",
)
store.upsert(MarkdownChunker().chunk(doc))

# Build indexes before retrieval
store.build_index()

# Retrieve similar chunks
chunks = store.retrieve("How do I use this?", top_k=5)
```

--------------------------------

### Install raghilda with sentence-transformers

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingSentenceTransformers.md

Install raghilda with sentence-transformers support. Required before using the provider.

```bash
pip install raghilda[sentence-transformers]
```

--------------------------------

### Create, upsert, and retrieve with ChromaDBStore

Source: https://posit-dev.github.io/raghilda/reference/store.ChromaDBStore.md

Complete example of creating a ChromaDB store, upserting a markdown document, and retrieving the top 3 chunks for a query.

```python
from raghilda.store import ChromaDBStore

store = ChromaDBStore.create(location="raghilda_chroma", name="docs")

store.upsert(markdown_doc)
chunks = store.retrieve("hello world", top_k=3)
```

--------------------------------

### Install skill for Codex or OpenCode

Source: https://posit-dev.github.io/raghilda/skills.html

Instruct the agent to fetch the skill file directly.

```text
Tell the agent:  
Fetch the skill file at https://posit-dev.github.io/raghilda/skill.md and follow the instructions.
```

--------------------------------

### Install raghilda via pip

Source: https://posit-dev.github.io/raghilda

Standard installation command for the raghilda package.

```bash
pip install raghilda
```

--------------------------------

### Install ChromaDB dependency

Source: https://posit-dev.github.io/raghilda/user-guide/chromadb-store.md

Install the standalone ChromaDB package.

```bash
pip install chromadb
```

--------------------------------

### Crawl a website with WebCrawler

Source: https://posit-dev.github.io/raghilda/user-guide/crawling-and-ingestion.md

This example demonstrates configuring a WebCrawler with caching and concurrency, defining a CrawlScope for a website, and ingesting the resulting documents into a DuckDBStore.

```python
from datetime import timedelta

from raghilda.chunker import MarkdownChunker
from raghilda.crawl import CrawlScope, WebCrawler
from raghilda.embedding import EmbeddingOpenAI
from raghilda.store import DuckDBStore

store = DuckDBStore.create(
    location="docs.db",
    embed=EmbeddingOpenAI(),
    name="docs",
    overwrite=True,
)

crawler = WebCrawler(
    cache_dir=True,
    cache_stale_after=timedelta(days=1),
    max_workers=4,
)
scope = CrawlScope(
    roots=["https://quarto.org/docs/guide/"],
    depth=2,
    include_patterns=["https://quarto.org/docs/guide/**"],
    exclude_patterns=["**/reference/**"],
    include_types=["html"],
)

chunker = MarkdownChunker(chunk_size=1600, target_overlap=0.5)

summary = store.ingest(
    crawler.markdown_documents(scope),
    prepare=chunker.chunk,
    max_workers=4,
)
store.build_index()

print(summary)
```

--------------------------------

### Install raghilda with ChromaDB support

Source: https://posit-dev.github.io/raghilda/user-guide/chromadb-store.md

Install the raghilda package with the optional ChromaDB dependency included.

```bash
pip install "raghilda[chromadb]"
```

--------------------------------

### Start console mode with chat.console()

Source: https://posit-dev.github.io/raghilda/user-guide/chatlas-integration.md

Opens an interactive REPL where you can ask questions and see tool calls in real time. Type exit or press Ctrl+C to quit.

```python
chat.console()
```

--------------------------------

### Example: Generate embeddings with EmbeddingOpenAI

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingOpenAI.md

Imports EmbeddingOpenAI, creates a provider with the default model, and embeds a list of texts. Prints the number of embeddings, the dimension of the first embedding, and the first ten values of the embedding vector.

```python
from raghilda.embedding import EmbeddingOpenAI

provider = EmbeddingOpenAI(model="text-embedding-3-small")
embeddings = provider.embed(["hello world", "testing embeddings"])
print(len(embeddings))
print(len(embeddings[0]))  # Dimension of the embedding
print(embeddings[0][:10])  # The embedding vector
```

--------------------------------

### Embedding examples with EmbeddingCohere and EmbedInputType

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingCohere.md

Creates a provider with the default model, embeds documents using input_type=EmbedInputType.DOCUMENT, and embeds a query using input_type=EmbedInputType.QUERY. The query embedding should be used for search while the document embedding is for indexing.

```python
from raghilda.embedding import EmbeddingCohere, EmbedInputType

provider = EmbeddingCohere(model="embed-english-v3.0")

# Embed documents for indexing
doc_embeddings = provider.embed(
    ["Hello world", "Testing embeddings"],
    input_type=EmbedInputType.DOCUMENT
)

# Embed a query for search
query_embedding = provider.embed(
    ["How do I test embeddings?"],
    input_type=EmbedInputType.QUERY
)
```

--------------------------------

### Configure In-Memory vs Persistent Storage

Source: https://posit-dev.github.io/raghilda/user-guide/chromadb-store.md

Demonstrates how to initialize a store for in-memory usage versus persistent disk storage.

```python
# In-memory store (data lost when process ends)
store = ChromaDBStore.create(location=":memory:", embed=EmbeddingOpenAI())

# Persistent store (data saved to disk)
store = ChromaDBStore.create(location="./my_store", embed=EmbeddingOpenAI())
```

--------------------------------

### create()

Source: https://posit-dev.github.io/raghilda/reference/store.ChromaDBStore.md

Create a new ChromaDB store. Returns a newly created store instance.

```APIDOC
## create()

### Description
Create a new ChromaDB store.

### Method
create

### Endpoint
create(location=None, *, overwrite=False, name=None, title=None, embed=None, collection_metadata=None, attributes=None, client=None)

### Parameters
#### Parameters
- **location** (str | Path | None) - Optional - Path where ChromaDB will persist its data. Use ":memory:" or None for an in-memory store.
- **overwrite** (bool) - Optional - Whether to overwrite an existing collection with the same name.
- **name** (str | None) - Optional - Collection name for the store.
- **title** (str | None) - Optional - Human-readable title for the store.
- **embed** (EmbeddingProvider | chromadb.api.types.EmbeddingFunction[chromadb.api.types.Documents] | None) - Optional - Optional embedding function. Can be either a raghilda EmbeddingProvider (e.g., EmbeddingOpenAI, EmbeddingCohere) or a ChromaDB embedding function. Raghilda providers are adapted internally to Chroma-compatible embedding functions. If None, Chroma's default embedding function is used.
- **collection_metadata** (dict[str, Any] | None) - Optional - Additional metadata to attach to the Chroma collection.
- **attributes** (AttributesSchemaSpec | None) - Optional - Optional schema for user-defined attribute columns. Attribute names use identifier-style syntax. Chroma also provides built-in filterable columns: `chunk_id`, `start_index`, `end_index`, `char_count`, `context`, and `origin`.
- **client** (Any) - Optional - Optional pre-configured Chroma client (e.g., HttpClient).

### Returns
- **ChromaDBStore** - A newly created store instance.

```

--------------------------------

### Custom embedding provider example

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingProvider.md

Example of subclassing EmbeddingProvider and registering it with @register_embedding_provider. Implements embed, get_config, and from_config methods.

```python
from raghilda.embedding import EmbeddingProvider, register_embedding_provider

@register_embedding_provider("MyCustomEmbedding")
class MyCustomEmbedding(EmbeddingProvider):
    def __init__(self, model: str = "default", api_key: str | None = None):
        self.model = model
        self.api_key = api_key
        # Initialize your embedding client here

    def embed(self, x, input_type=None):
        # Return list of embedding vectors
        ...

    def get_config(self):
        # Return config dict (exclude sensitive values like api_key)
        return {"type": "MyCustomEmbedding", "model": self.model}

    @classmethod
    def from_config(cls, config):
        return cls(model=config.get("model", "default"))
```

--------------------------------

### Initialize PostgreSQLStore

Source: https://posit-dev.github.io/raghilda/reference/store.PostgreSQLStore.md

Basic instantiation of the PostgreSQLStore class.

```python
store.PostgreSQLStore(
    con,
    metadata,
    schema,
)
```

--------------------------------

### Example: find_links() usage

Source: https://posit-dev.github.io/raghilda/reference/scrape.find_links.md

Example usage of find_links() to discover pages under a documentation section, one level deep, with children_only=True. Prints the number of found pages.

```python
from raghilda.scrape import find_links

# Discover pages under a documentation section, one level deep
links = find_links(
    "https://quarto.org/docs/guide/",
    depth=1,
    children_only=True,
)
print(f"Found {len(links)} pages")
```

--------------------------------

### create()

Source: https://posit-dev.github.io/raghilda/reference/store.BaseStore.md

Creates a new store and returns a newly created store instance.

```APIDOC
## create()

### Description
Create a new store.

### Method
create

### Signature
create(*args, **kwargs)

### Returns
- **BaseStore** - A newly created store instance.
```

--------------------------------

### Importing the built-in EmbeddingSentenceTransformers provider

Source: https://posit-dev.github.io/raghilda/user-guide/custom-embedding-providers.md

Shows how to import the built-in EmbeddingSentenceTransformers provider. This is the recommended approach for real use, requiring the extra installation `pip install "raghilda[sentence-transformers]"`.

```python
from raghilda.embedding import EmbeddingSentenceTransformers
```

--------------------------------

### connect()

Source: https://posit-dev.github.io/raghilda/reference/store.BaseStore.md

Connects to an existing store and returns a connected store instance.

```APIDOC
## connect()

### Description
Connect to an existing store.

### Method
connect

### Signature
connect(*args, **kwargs)

### Returns
- **BaseStore** - A connected store instance.
```

--------------------------------

### Initialize document.Document

Source: https://posit-dev.github.io/raghilda/reference/document.Document.md

Creates a new Document instance with content, an optional origin, and optional attributes.

```python
document.Document(
    content,
    origin=None,
    attributes=None,
)
```

--------------------------------

### Usage: instantiate types.ChunkLike()

Source: https://posit-dev.github.io/raghilda/reference/types.ChunkLike.md

Shows the usage pattern for the ChunkLike structural type in a code example. This is a type alias for any object exposing the required chunk fields, and raghilda normalizes it with `Chunk.from_any()`. The snippet simply calls `types.ChunkLike()` as an example.

```python
types.ChunkLike()
```

--------------------------------

### connect()

Source: https://posit-dev.github.io/raghilda/reference/store.ChromaDBStore.md

Connect to an existing ChromaDB store. Returns a connected store instance.

```APIDOC
## connect()

### Description
Connect to an existing ChromaDB store.

### Method
connect

### Endpoint
connect(name, location=None, *, client=None)

### Parameters
#### Parameters
- **name** (str) - Required - Collection name for the store.
- **location** (str | Path | None) - Optional - Path where ChromaDB persists its data. Use ":memory:" or None for an in-memory store.
- **client** (Any) - Optional - Optional pre-configured Chroma client (e.g., HttpClient).

### Returns
- **ChromaDBStore** - A connected store instance.

```

--------------------------------

### scrape.find_links()

Source: https://posit-dev.github.io/raghilda/reference/scrape.find_links.md

Documentation for the `scrape.find_links()` function, including its parameters, return value, and usage example.

```APIDOC
## `scrape.find_links()`

### Description
Discover links by crawling one or more starting pages. It follows links up to a specified depth and returns a deduplicated list of absolute URLs. It reads both HTML pages (following `<a href>` links) and XML sitemaps (collecting their `<loc>` entries).

### Function Signature
```python
scrape.find_links(
    x,
    depth=0,
    children_only=False,
    progress=True,
    *,
    url_filter=None,
    validate=False,
    **request_kwargs
)
```

### Parameters
- **x** (`str | Path | Sequence[str | Path]`) - Required - Starting URL(s) or path(s): a single string or `Path`, or a sequence of them.
- **depth** (`int`, default `0`) - Optional - Maximum traversal depth from each starting page. `0` inspects the starting pages only, `1` also follows their direct links, and so on.
- **children_only** (`bool`, default `False`) - Optional - When `True`, only links at or below the starting URL are kept and followed (for example links under `https://site/docs/` when you start there). This is the easiest way to stay within one section of a site.
- **progress** (`bool`, default `True`) - Optional - Whether to display a `tqdm` progress bar while crawling.
- **url_filter** (`Callable[[set[str]], list[str]] | None`, default `None`) - Optional - Optional callback that receives the set of links found on a page and returns the (possibly smaller) list of URLs to keep and follow. Use it to apply custom include/exclude rules.
- **validate** (`bool`, default `False`) - Optional - When `True`, check that each URL is reachable (with an HTTP `HEAD` request) before including it in the results.
- **request_kwargs** (`Any`, default `{}`) - Optional - Extra keyword arguments forwarded to `requests.Session.get` (and to `head` during validation) when fetching pages.

### Returns
- **list[str]** - A deduplicated list of absolute URLs discovered during the crawl. The crawl completes before this list is returned.

### Example
```python
from raghilda.scrape import find_links

# Discover pages under a documentation section, one level deep
links = find_links(
    "https://quarto.org/docs/guide/",
    depth=1,
    children_only=True,
)
print(f"Found {len(links)} pages")
```
```

--------------------------------

### from_config() method usage

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingProvider.md

Shows the signature of the from_config method.

```python
from_config(config)
```

--------------------------------

### Instantiate ChunkerLike type

Source: https://posit-dev.github.io/raghilda/reference/types.ChunkerLike.md

Usage example for the ChunkerLike type. This is a structural type, so any object with the required methods can be used where ChunkerLike is expected.

```python
types.ChunkerLike()
```

--------------------------------

### Create ChromaDBStore with raghilda EmbeddingOpenAI

Source: https://posit-dev.github.io/raghilda/user-guide/chromadb-embedding-functions.md

Initializes a ChromaDBStore using the raghilda EmbeddingOpenAI provider.

```python
from raghilda.embedding import EmbeddingOpenAI
from raghilda.store import ChromaDBStore

# Create a store with a raghilda embedding provider
store = ChromaDBStore.create(
    location="my_store",
    name="documents",
    embed=EmbeddingOpenAI(model="text-embedding-3-small"),
)
```

--------------------------------

### Create a new store

Source: https://posit-dev.github.io/raghilda/reference/store.BaseStore.md

Creates a new store. Returns a newly created BaseStore instance.

```python
create(*args, **kwargs)
```

--------------------------------

### Instantiate BaseCrawler

Source: https://posit-dev.github.io/raghilda/reference/crawl.BaseCrawler.md

Creates an instance of the abstract base class. This is a placeholder usage example; actual crawlers are subclasses like DirectoryCrawler, WebCrawler, or CloudflareCrawler.

```python
crawl.BaseCrawler()
```

--------------------------------

### Execute complete RAG workflow

Source: https://posit-dev.github.io/raghilda/user-guide/getting-started.md

A full script demonstrating store creation, document scraping, chunking, indexing, and retrieval.

```python
from raghilda.store import DuckDBStore
from raghilda.embedding import EmbeddingOpenAI
from raghilda.scrape import find_links

# 1. Create store
store = DuckDBStore.create(
    location="quarto_docs.db",
    embed=EmbeddingOpenAI(),
    name="quarto",
    title="Quarto Documentation",
    overwrite=True,
)

# 2. Find documents
links = find_links(
    "https://quarto.org/docs/guide/",
    depth=1,
    children_only=True,
)

# 3. Prepare and insert documents
from raghilda.read import read_as_markdown
from raghilda.chunker import MarkdownChunker

chunker = MarkdownChunker()
for link in links:
    store.upsert(chunker.chunk(read_as_markdown(link)))

# 4. Build indexes
store.build_index()

# 5. Retrieve
chunks = store.retrieve("How do I create a presentation?", top_k=3)
for chunk in chunks:
    print(f"\n## {chunk.context}")
    print(chunk.text[:300])
```

--------------------------------

### Ingest documents with store.ingest()

Source: https://posit-dev.github.io/raghilda/user-guide/crawling-and-ingestion.md

This example demonstrates how to use store.ingest() to process markdown documents, specifying a chunking function and worker count, and then printing the resulting summary.

```python
summary = store.ingest(
    crawler.markdown_documents(scope, cache_force_refresh=True),
    prepare=chunker.chunk,
    max_workers=4,
)

print(f"Inserted: {summary.inserted}")
print(f"Replaced: {summary.replaced}")
print(f"Skipped: {summary.skipped}")
```

--------------------------------

### Create a persistent ChromaDBStore

Source: https://posit-dev.github.io/raghilda/user-guide/chromadb-store.md

Initialize a new persistent ChromaDB store using the ChromaDBStore.create method.

```python
from raghilda.store import ChromaDBStore
from raghilda.embedding import EmbeddingOpenAI

# Create a persistent store
store = ChromaDBStore.create(
    location="my_vector_store",
    name="documents",
    embed=EmbeddingOpenAI(),
)
```

--------------------------------

### Convert an object into a Chunk with to_chunk()

Source: https://posit-dev.github.io/raghilda/reference/types.IntoChunk.md

Usage example for the to_chunk() method. Implement this method to return a raghilda Chunk, allowing raghilda to accept your type directly.

```python
to_chunk()
```

--------------------------------

### connect() method signature

Source: https://posit-dev.github.io/raghilda/reference/store.ChromaDBStore.md

Signature for the connect() method. Connects to an existing ChromaDB store. Use ":memory:" or None for an in-memory store.

```python
connect(name, location=None, *, client=None)
```

--------------------------------

### Full RAG chat session implementation

Source: https://posit-dev.github.io/raghilda/user-guide/chatlas-integration.md

A complete script that builds a DuckDB store from a documentation site and starts an interactive chat session using ChatOpenAI.

```python
from pathlib import Path

from chatlas import ChatOpenAI

from raghilda.chunker import MarkdownChunker
from raghilda.crawl import CrawlScope, WebCrawler
from raghilda.embedding import EmbeddingOpenAI
from raghilda.store import DuckDBStore

DB_PATH = Path("chatlas_docs.db")


def build_store() -> DuckDBStore:
    store = DuckDBStore.create(
        location=str(DB_PATH),
        embed=EmbeddingOpenAI(),
        name="chatlas_docs",
        title="Chatlas Documentation",
        overwrite=True,
    )
    crawler = WebCrawler(cache_dir=True, max_workers=4)
    scope = CrawlScope(
        roots=["https://posit-dev.github.io/chatlas/"],
        depth=1,
        include_types=["html"],
    )
    chunker = MarkdownChunker()
    summary = store.ingest(
        crawler.markdown_documents(scope),
        prepare=chunker.chunk,
        max_workers=4,
    )
    store.build_index()
    print(f"Indexed {summary.inserted} documents")
    return store


def get_store() -> DuckDBStore:
    if DB_PATH.exists():
        return DuckDBStore.connect(str(DB_PATH), read_only=True)
    return build_store()


def main():
    import json

    store = get_store()

    def search_chatlas_docs(query: str, num_results: int = 5) -> str:
        """
        Search the chatlas documentation.

        Use this tool when the user asks about chatlas features,
        API usage, model providers, tool calling, or streaming.

        Parameters
        ----------
        query
            A description of what to look for.
        num_results
            Number of passages to return (default of 5).
        """
        chunks = store.retrieve(query, top_k=num_results, deoverlap=True)
        return json.dumps(
            [{"text": chunk.text, "context": chunk.context} for chunk in chunks]
        )

    chat = ChatOpenAI(
        model="gpt-5.5",
        system_prompt=(
            "You answer questions about the chatlas Python library. "
            "Always use the search tool before answering."
        ),
    )
    chat.register_tool(search_chatlas_docs)
    chat.console()


if __name__ == "__main__":
    main()
```

--------------------------------

### ChromaDBStore constructor usage

Source: https://posit-dev.github.io/raghilda/reference/store.ChromaDBStore.md

Shows the constructor signature for ChromaDBStore. The client, collection, and metadata arguments are placeholders; use the create() or connect() methods instead.

```python
store.ChromaDBStore(
    client,
    collection,
    metadata,
)
```

--------------------------------

### Construct an IntoChunk with types.IntoChunk()

Source: https://posit-dev.github.io/raghilda/reference/types.IntoChunk.md

Usage example for the IntoChunk structural type. Objects implementing to_chunk() can be accepted by raghilda directly, and Chunk.from_any() calls to_chunk() when given an IntoChunk.

```python
types.IntoChunk()
```

--------------------------------

### Crawl a website with find_links

Source: https://posit-dev.github.io/raghilda/user-guide/getting-started.md

Uses the find_links function to discover pages from a starting URL. The depth parameter limits the crawl depth, and children_only restricts results to sub-paths.

```python
from raghilda.scrape import find_links

links = find_links(
    "https://quarto.org/docs/guide/",
    depth=1,  # follow links 1 level deep from the starting page
    children_only=True,
)
print(f"Found {len(links)} pages")
```

```text
Found 2 pages
```

--------------------------------

### Build and refresh a documentation store

Source: https://posit-dev.github.io/raghilda/user-guide/cloudflare-crawler.md

A complete script demonstrating how to build a DuckDB store from a JavaScript-rendered site and refresh it using caching to minimize redundant API calls.

```python
import os
from datetime import timedelta
from pathlib import Path

from raghilda.chunker import MarkdownChunker
from raghilda.crawl import CloudflareCrawler, CrawlScope
from raghilda.embedding import EmbeddingOpenAI
from raghilda.store import DuckDBStore

DB_PATH = Path("spa_docs.db")


def build_store() -> DuckDBStore:
    store = DuckDBStore.create(
        location=str(DB_PATH),
        embed=EmbeddingOpenAI(),
        name="spa_docs",
        title="SPA Documentation",
        overwrite=True,
    )

    crawler = CloudflareCrawler(
        account_id=os.environ["CLOUDFLARE_ACCOUNT_ID"],
        api_token=os.environ["CLOUDFLARE_API_TOKEN"],
        cache_dir=True,
        cache_stale_after=timedelta(days=1),
        render=True,
        source="all",
        max_workers=4,
    )

    scope = CrawlScope(
        roots=["https://my-spa-docs.example.com/"],
        depth=2,
        include_patterns=["https://my-spa-docs.example.com/**"],
        exclude_patterns=["https://my-spa-docs.example.com/internal/**"],
        limit=1000,
    )

    chunker = MarkdownChunker(chunk_size=1600, target_overlap=0.5)

    summary = store.ingest(
        crawler.markdown_documents(scope),
        prepare=chunker.chunk,
        max_workers=4,
    )
    
    store.build_index()
    
    print(f"Inserted: {summary.inserted}")
    print(f"Replaced: {summary.replaced}")
    print(f"Skipped: {summary.skipped}")
    
    return store


def refresh_store() -> None:
    store = DuckDBStore.connect(str(DB_PATH))
    
    crawler = CloudflareCrawler(
        account_id=os.environ["CLOUDFLARE_ACCOUNT_ID"],
        api_token=os.environ["CLOUDFLARE_API_TOKEN"],
        cache_dir=True,
        cache_stale_after=timedelta(days=1),
        render=True,
        max_workers=4,
    )
    
    scope = CrawlScope(
        roots=["https://my-spa-docs.example.com/"],
        depth=2,
        include_patterns=["https://my-spa-docs.example.com/**"],
        limit=1000,
    )
    
    chunker = MarkdownChunker(chunk_size=1600, target_overlap=0.5)
    
    summary = store.ingest(
        crawler.markdown_documents(scope),
        prepare=chunker.chunk,
        max_workers=4,
    )
    
    print(f"Refresh complete: {summary.inserted} new, {summary.replaced} updated, {summary.skipped} unchanged")


if __name__ == "__main__":
    if DB_PATH.exists():
        refresh_store()
    else:
        build_store()
```

--------------------------------

### create()

Source: https://posit-dev.github.io/raghilda/reference/store.DuckDBStore.md

Creates a new DuckDB store. Returns a newly created store instance.

```APIDOC
## create()

### Description
Create a new DuckDB store.

### Usage
```python
create(
    location, embed, overwrite=False, name=None, title=None, attributes=None
)
```

### Parameters
- **location** (str | Path) - Required - Path where the DuckDB database file will be created.
- **embed** (EmbeddingProvider | None) - Required - Embedding provider for generating vector embeddings. If None, only full-text search will be available.
- **overwrite** (bool) - Optional - Whether to overwrite an existing database at the location. Defaults to False.
- **name** (str | None) - Optional - Internal name for the store.
- **title** (str | None) - Optional - Human-readable title for the store.
- **attributes** (AttributesSchemaSpec | None) - Optional - Optional schema for user-defined attribute columns stored per chunk. Example: `{"tenant": str, "priority": int}`. Attribute names use identifier-style syntax. The following names are reserved and cannot be used as attributes: `chunk_id`, `context`, `embedding`, `origin`, `text`, `start_index`, `end_index`, `char_count`, `metric_name`, and `metric_value`.

### Returns
- **DuckDBStore** - A newly created store instance.
```

--------------------------------

### Retrieve raw page data with fetch_raw()

Source: https://posit-dev.github.io/raghilda/user-guide/cloudflare-crawler.md

Use fetch_raw() to get the FetchedSource object for a specific URL, providing access to metadata like HTTP status codes and content paths.

```python
source = crawler.fetch_raw("https://example.com/docs/getting-started")
print(f"Status: {source.status_code}")
print(f"Fetched at: {source.fetched_at}")
print(f"Body at: {source.body_path}")
```

--------------------------------

### Configure incremental updates with modified_since

Source: https://posit-dev.github.io/raghilda/user-guide/cloudflare-crawler.md

Use the modified_since parameter to restrict crawls to pages modified after a specific Unix timestamp. This example fetches pages modified within the last 7 days.

```python
import time

# Only include pages modified in the last 7 days
one_week_ago = int(time.time()) - (7 * 24 * 60 * 60)

crawler = CloudflareCrawler(
    account_id=os.environ["CLOUDFLARE_ACCOUNT_ID"],
    api_token=os.environ["CLOUDFLARE_API_TOKEN"],
    cache_dir=True,
    modified_since=one_week_ago,
)
```

--------------------------------

### from_config()

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingProvider.md

Creates a provider instance from a configuration dict. The config should come from get_config() and contain all parameters needed to recreate the provider.

```APIDOC
## from_config(config)

### Description
Create a provider instance from a configuration dict.

### Method
from_config

### Parameters
- **config** (dict[str, Any]) - Required - Configuration dict from get_config().

### Returns
- **EmbeddingProvider** - A new instance of the provider.
```

--------------------------------

### Configure ChatGoogle for prototyping

Source: https://posit-dev.github.io/raghilda/user-guide/chatlas-integration.md

Initializes a ChatGoogle instance with the specified model and registers a search tool.

```python
from chatlas import ChatGoogle

chat = ChatGoogle(model="gemini-3.5-flash")
chat.register_tool(search_docs)
```

--------------------------------

### create()

Source: https://posit-dev.github.io/raghilda/reference/store.OpenAIStore.md

Create a new OpenAI vector store. This method returns a newly created store instance.

```APIDOC
## create()

### Description
Create a new OpenAI vector store.

### Method
create

### Endpoint
create(base_url="https://api.openai.com/v1", api_key=None, *, attributes=None, metadata=None, **kwargs)

### Parameters
- **base_url** (str) - Optional - Base URL for the OpenAI API. Default: "https://api.openai.com/v1".
- **api_key** (str | None) - Optional - OpenAI API key. If None, uses the OPENAI_API_KEY environment variable.
- **attributes** (AttributesSchemaSpec | None) - Optional - Optional schema for user-defined attribute columns. Attribute names use identifier-style syntax. OpenAIStore filters only support declared attributes.
- **metadata** (Mapping[str, str] | None) - Optional - Additional metadata to attach to the OpenAI vector store resource.
- **kwargs** - Optional - Additional arguments passed to the vector store creation (e.g., name, expires_after).

### Returns
- **OpenAIStore** - A newly created store instance.
```

--------------------------------

### Create and access a Metric instance

Source: https://posit-dev.github.io/raghilda/reference/chunk.Metric.md

Example of importing Metric from raghilda.chunk, creating a metric with name 'similarity' and value 0.95, then printing its name and value. The output shows 'similarity: 0.95' after an ONNX runtime warning.

```python
from raghilda.chunk import Metric

similarity = Metric(name="similarity", value=0.95)
print(f"{similarity.name}: {similarity.value}")
```

--------------------------------

### get_config()

Source: https://posit-dev.github.io/raghilda/reference/embedding.EmbeddingProvider.md

Gets the configuration dict for this provider. The config should contain all parameters needed to recreate the provider, except for sensitive values like API keys. It must include a 'type' key with the registered name of the provider.

```APIDOC
## get_config()

### Description
Get the configuration dict for this provider.

### Method
get_config

### Returns
- **dict** - Configuration dict that can be passed to from_config().
```

--------------------------------

### Define document sources from local files

Source: https://posit-dev.github.io/raghilda/user-guide/getting-started.md

Demonstrates how to define a list of local files or use glob patterns with pathlib to identify documents.

```python
# Local files
links = ["docs/guide.md", "docs/reference.md", "docs/tutorial.md"]

# Or use glob patterns with pathlib
from pathlib import Path
links = list(Path("docs").glob("**/*.md"))
```

--------------------------------

### Custom local embedding provider with sentence-transformers

Source: https://posit-dev.github.io/raghilda/user-guide/custom-embedding-providers.md

Illustrative example of a custom embedding provider that wraps a local sentence-transformers model. It handles both document and query input types, validates input, and provides configuration methods. Useful as a template for wrapping unsupported local models.

```python
from typing import Any, Sequence
from sentence_transformers import SentenceTransformer
from raghilda.embedding import (
    EmbeddingProvider,
    EmbedInputType,
    register_embedding_provider,
)

@register_embedding_provider("EmbeddingLocal")
class EmbeddingLocal(EmbeddingProvider):
    """Embedding provider using local sentence-transformers models."""

    def __init__(self, model: str = "all-MiniLM-L6-v2"):
        self.model_name = model
        self._model = SentenceTransformer(model)

    def embed(
        self,
        x: Sequence[str],
        input_type: EmbedInputType = EmbedInputType.DOCUMENT,
    ) -> Sequence[Sequence[float]]:
        if isinstance(x, str):
            raise TypeError("Input must be a sequence of strings")

        if len(x) == 0:
            return []

        # sentence-transformers doesn't distinguish query vs document
        embeddings = self._model.encode(list(x))
        return [emb.tolist() for emb in embeddings]

    def get_config(self) -> dict[str, Any]:
        return {
            "type": "EmbeddingLocal",
            "model": self.model_name,
        }

    @classmethod
    def from_config(cls, config: dict[str, Any]) -> "EmbeddingLocal":
        return cls(model=config.get("model", "all-MiniLM-L6-v2"))
```
Tokens:
Raw