### Connection setup example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/neo4j.mdx

Example of setting up a ConnectionFactory for Neo4j with a ContextKey.

```python
from collections.abc import AsyncIterator
from cocoindex.connectors import neo4j
import cocoindex as coco

KG_DB: coco.ContextKey[neo4j.ConnectionFactory] = coco.ContextKey("kg_db")

@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    builder.provide(
        KG_DB,
        neo4j.ConnectionFactory(
            uri="bolt://localhost:7687",
            auth=("neo4j", "cocoindex"),
            database="neo4j",
        ),
    )
    yield
```

--------------------------------

### Initialize a new CocoIndex project

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

Creates a new project directory with essential files and installs it locally.

```bash
cocoindex init my-project
cd my-project
```

--------------------------------

### Multitenancy example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/neo4j.mdx

Example demonstrating multitenancy setup with separate ContextKeys and ConnectionFactories for different Neo4j databases.

```python
KG_DB: coco.ContextKey[neo4j.ConnectionFactory] = coco.ContextKey("kg_db")
APIS_DB: coco.ContextKey[neo4j.ConnectionFactory] = coco.ContextKey("apis_db")

@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    uri = "bolt://localhost:7687"
    auth = ("neo4j", "cocoindex")
    builder.provide(KG_DB, neo4j.ConnectionFactory(uri=uri, auth=auth, database="kg"))
    builder.provide(APIS_DB, neo4j.ConnectionFactory(uri=uri, auth=auth, database="apis"))
    yield
```

--------------------------------

### Setup Test Environment

Source: https://github.com/cocoindex-io/cocoindex/blob/main/python/tests/cli/README.md

Navigate to the CLI test directory and clean up any existing test artifacts before starting manual tests. This ensures a clean slate for testing.

```bash
cd python/tests/cli

# Clean up all test artifacts before starting
rm -rf cocoindex*.db db1 db2 db_alpha out_*

```

--------------------------------

### Setup Local Development Environment with Prek Hooks

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/contributing/setup_dev_environment.mdx

Installs and enables prek hooks for automatic checks before each commit. Optionally, install all extra dependencies.

```sh
uv run prek install
```

```sh
uv sync --all-extras
```

--------------------------------

### Install Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/amazon_s3_embedding/README.md

Installs the project dependencies, including the editable install.

```sh
pip install -e .
```

--------------------------------

### Running Your Pipeline Commands

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

Common commands for installing dependencies, updating pipelines, running in live mode, showing component paths, and resetting the project.

```bash
pip install -e .
cocoindex update main.py
cocoindex update main.py -L
cocoindex show main.py
cocoindex drop main.py -f
```

--------------------------------

### Build Command for the Rust Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/rust/multi-codebase-summarization/README.md

Builds the Rust project in release mode.

```sh
cd examples/rust/multi-codebase-summarization
cargo build --release
```

--------------------------------

### Start Local Postgres Instance

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/amazon_s3_embedding/README.md

Starts a local PostgreSQL instance with the pgvector extension using Docker Compose.

```sh
docker compose -f ../../dev/postgres.yaml up -d
```

--------------------------------

### Live bucket watching example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/oci_object_storage.mdx

Example setup for live incremental bucket watching using OCI Streaming and Kafka connector.

```python
from confluent_kafka.aio import AIOConsumer
from cocoindex.connectors import kafka, oci_object_storage

consumer = AIOConsumer({
    "bootstrap.servers": "<oci-streaming-bootstrap-host>:9092",
    "group.id": "my-group",
    "enable.auto.commit": "false",
})
topic_stream = kafka.topic_as_stream(consumer, ["object-storage-events"])

walker = oci_object_storage.list_objects(
    client, "my-namespace", "my-bucket",
    path_matcher=PatternFilePathMatcher(included_patterns=["**/*.md"]),
    live_stream=topic_stream.payloads(),
)
await coco.mount_each(process_file, walker.items(), target)
```

--------------------------------

### Multitenancy setup

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/falkordb.mdx

Example demonstrating how to set up multiple FalkorDB connections for different graphs on the same Redis instance.

```python
KG_DB: coco.ContextKey[falkordb.ConnectionFactory] = coco.ContextKey("kg_db")
APIS_DB: coco.ContextKey[falkordb.ConnectionFactory] = coco.ContextKey("apis_db")

@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    uri = "falkor://localhost:6379"
    builder.provide(KG_DB, falkordb.ConnectionFactory(uri=uri, graph="knowledge_graph"))
    builder.provide(APIS_DB, falkordb.ConnectionFactory(uri=uri, graph="apis_graph"))
    yield
```

--------------------------------

### Run the transformation

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/rust/files-transform/README.md

Runs the file transformation example against sample data.

```sh
cargo run -- ../../files_transform/data ./output_html
```

--------------------------------

### Install Rust

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/contributing/setup_dev_environment.mdx

Installs Rust using the official script. Ensure Rust is up-to-date by running `rustup update` if already installed.

```sh
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

```sh
rustup update
```

--------------------------------

### Run Examples with Local CocoIndex

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/contributing/setup_dev_environment.mdx

Navigates to an example directory and runs it using the local, editable version of CocoIndex. The `coco-dev-run` function ensures the example uses your local build.

```sh
# Navigate to an example directory
cd examples/text_embedding

# Run with your local cocoindex changes
coco-dev-run cocoindex update main
```

--------------------------------

### Start frontend

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/image_search/README.md

Starts the React frontend development server.

```sh
cd frontend
npm install
npm run dev
```

--------------------------------

### Usage Command to Summarize Python Examples

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/rust/multi-codebase-summarization/README.md

Runs the compiled Rust program to summarize Python examples in the repository, outputting results to the 'output' directory.

```sh
cargo run -- ../.. ./output
```

--------------------------------

### Example client creation

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/qdrant.mdx

Example of creating a Qdrant client instance.

```python
client = qdrant.create_client("http://localhost:6333")
```

--------------------------------

### Install CocoIndex and dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/example-posts/multi-codebase-summarization.md

Command to install CocoIndex and necessary Python libraries.

```bash
pip install 'cocoindex>=1.0.0' instructor litellm pydantic
```

--------------------------------

### Set Environment Variables

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/audio_to_text/README.md

Sets the necessary environment variables for LiteLLM and PostgreSQL connection.

```shell
export POSTGRES_URL="postgres://cocoindex:cocoindex@localhost/cocoindex"
export OPENAI_API_KEY="..."
```

--------------------------------

### SQLite connect examples

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/sqlite.mdx

Examples demonstrating how to use the sqlite.connect() function with different options.

```python
managed_conn = sqlite.connect("mydb.sqlite")  # Auto-loads sqlite-vec if available
# Or for in-memory:
managed_conn = sqlite.connect(":memory:")
# Or explicitly require vector support:
managed_conn = sqlite.connect("mydb.sqlite", load_vec=True)
# Or disable auto-loading:
managed_conn = sqlite.connect("mydb.sqlite", load_vec=False)
```

--------------------------------

### Set Up Environment Variables

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/patient_intake_extraction_baml/README.md

Create a .env file to store your API keys. Replace 'your_api_key_here' with your actual Gemini API key.

```sh
echo "GEMINI_API_KEY=your_api_key_here" > .env
```

--------------------------------

### Environment Variables Configuration

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

Example `.env` file content for configuring CocoIndex, PostgreSQL, Qdrant, and API keys.

```dotenv
# CocoIndex internal database (required)
COCOINDEX_DB=./cocoindex.db

# PostgreSQL (if using)
POSTGRES_URL=postgres://user:pass@localhost/db

# Qdrant (if using)
QDRANT_URL=http://localhost:6333

# API keys (if using LLM extraction)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
```

--------------------------------

### Tiny Scenario Example Directory Structure

Source: https://github.com/cocoindex-io/cocoindex/blob/main/benchmarks/file_summarization/README.md

Example directory structure for a tiny scenario with 3 projects.

```tree
dataset/
├─ project_000/
│  ├─ src/module_00.rs        (4 sections)
│  ├─ src/module_01.rs        (4 sections)
│  ├─ python/worker_00.py     (4 sections)
│  ├─ docs/guide_00.md        (3 sections)
│  └─ Cargo_00.toml           (2 sections)
├─ project_001/ ...
└─ project_002/ ...

output/
├─ projects/project_000.json   ← one JSON per project
├─ projects/project_001.json
├─ projects/project_002.json
└─ manifest.json               ← index of everything
```

--------------------------------

### Target Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/localfs.mdx

An example demonstrating how to set up an output directory target and process files, writing transformed content to the target.

```python
import pathlib
import cocoindex as coco
from cocoindex.connectors import localfs
from cocoindex.resources.file import FileLike, PatternFilePathMatcher

SOURCE_DIR = coco.ContextKey[pathlib.Path]("source_dir")
OUTPUT_DIR = coco.ContextKey[pathlib.Path]("output_dir")

@coco.fn
async def app_main() -> None:
    # Declare output directory target using context key
    target = await localfs.mount_dir_target(OUTPUT_DIR)

    # Process files and write outputs
    await coco.mount_each(process_file, localfs.walk_dir(SOURCE_DIR, recursive=True).items(), target)

@coco.fn(memo=True)
async def process_file(file: FileLike, target: localfs.DirTarget) -> None:
    # Transform the file
    content = (await file.read_text()).upper()

    # Write to output with same relative path
    target.declare_file(
        filename=file.file_path.path,
        content=content,
        create_parent_dirs=True,
    )
```

--------------------------------

### Running the Benchmarks

Source: https://github.com/cocoindex-io/cocoindex/blob/main/benchmarks/file_summarization/README.md

Command-line examples for running the file summarization benchmarks with different configurations for scenarios, profiles, scales, and output formats.

```sh
# fastest smoke check — tiny dataset, one scenario
./run.sh --scenario codebase --profile mixed --scale tiny

# medium is where the real numbers start
./run.sh --scenario all --profile mixed --scale medium --trials 3

# full matrix (all scenarios × all profiles)
./run.sh --scenario all --profile all --scale large

# the 10k+ file matrix behind BENCHMARK_REPORT.md
./run.sh --scenario all --profile all --scale xlarge

# JSON output for piping into something else
./run.sh --scenario all --profile all --scale tiny --format json
```

--------------------------------

### Chunk Splitting Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/common_resources/data_types.mdx

Example demonstrating how to split text into chunks using RecursiveSplitter and print their start positions and text.

```python
from cocoindex.ops.text import RecursiveSplitter

splitter = RecursiveSplitter()
chunks = splitter.split(text, chunk_size=2000, chunk_overlap=500, language="markdown")

for chunk in chunks:
    print(f"[{chunk.start.line}:{chunk.start.column}] {chunk.text[:50]}...")

```

--------------------------------

### Command Line Usage

Source: https://github.com/cocoindex-io/cocoindex/blob/main/dev/agent-skills/upgrade-examples/SKILL.md

The command to initiate the upgrade process, specifying the target version.

```bash
/upgrade-examples <version>
```

--------------------------------

### Run Qdrant

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/image_search/README.md

Starts a Qdrant instance.

```sh
docker run -d -p 6334:6334 -p 6333:6333 qdrant/qdrant
```

--------------------------------

### Relation Table Example (Knowledge Graph)

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/falkordb.mdx

An example illustrating the setup and usage of relation tables for knowledge graphs, including defining entities and relationships.

```python
@dataclass
class Entity:
    value: str


@dataclass
class RelationshipRow:
    id: str
    predicate: str


@coco.fn
async def kg_app_main() -> None:
    documents = await falkordb.mount_table_target(
        KG_DB, "Document",
        await falkordb.TableSchema.from_class(Document, primary_key="filename"),
        primary_key="filename",
    )
    entities = await falkordb.mount_table_target(
        KG_DB, "Entity",
        await falkordb.TableSchema.from_class(Entity, primary_key="value"),
        primary_key="value",
    )
    relationships = await falkordb.mount_relation_target(
        KG_DB, "RELATIONSHIP",
        entities, entities,
        await falkordb.TableSchema.from_class(RelationshipRow, primary_key="id"),
        primary_key="id",
    )

    # populate ...
    documents.declare_record(row=Document(filename="overview.md", title="Overview", summary="..."))
    entities.declare_record(row=Entity(value="CocoIndex"))
    entities.declare_record(row=Entity(value="FalkorDB"))
    relationships.declare_relation(
        from_id="CocoIndex",
        to_id="FalkorDB",
        record=RelationshipRow(id="rel-1", predicate="writes_to"),
    )


kg_app = coco.App(coco.AppConfig(name="kg_app"), kg_app_main)
```

--------------------------------

### Live File Watching Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/localfs.mdx

Example demonstrating how to use `live=True` with `items()` and `mount_each()` for automatic incremental file watching.

```python
files = localfs.walk_dir(
    sourcedir,
    recursive=True,
    path_matcher=PatternFilePathMatcher(included_patterns=["**/*.md"]),
    live=True,
)
await coco.mount_each(process_file, files.items(), target)
```

--------------------------------

### Prepare Source Data

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/postgres_source/README.md

Creates the source table `source_products` with sample data in PostgreSQL.

```sh
psql "postgres://cocoindex:cocoindex@localhost/cocoindex" -f ./prepare_source_data.sql
```

--------------------------------

### Connection setup with ConnectionFactory

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/falkordb.mdx

Example of setting up a ConnectionFactory for FalkorDB within a lifespan context.

```python
from collections.abc import AsyncIterator
from cocoindex.connectors import falkordb
import cocoindex as coco

KG_DB: coco.ContextKey[falkordb.ConnectionFactory] = coco.ContextKey("kg_db")

@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    builder.provide(
        KG_DB,
        falkordb.ConnectionFactory(
            uri="falkor://localhost:6379",
            graph="knowledge_graph",
        ),
    )
    yield
```

--------------------------------

### Kafka Configuration Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/kafka_to_lancedb/README.md

Configure Kafka connection details, topic, and group ID for the pipeline.

```env
KAFKA_BOOTSTRAP_SERVERS=localhost:9092
KAFKA_TOPIC=cocoindex-example-csv-rows
KAFKA_GROUP_ID=kafka-to-lancedb
LANCEDB_URI=./lancedb_data
```

--------------------------------

### Full example of mounting a table target with data and vector index

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/postgres.mdx

A comprehensive example demonstrating the setup of a Postgres table, including defining the schema from a class, providing data, and declaring a vector index.

```python
import asyncpg
import cocoindex as coco
from cocoindex.connectors import postgres

DATABASE_URL = "postgresql://localhost/mydb"

PG_DB = coco.ContextKey[asyncpg.Pool]("main_db")

@dataclass
class OutputProduct:
    category: str
    name: str
    description: str
    embedding: Annotated[NDArray, embedder]

@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    async with await asyncpg.create_pool(DATABASE_URL) as pool:
        builder.provide(PG_DB, pool)
        yield

@coco.fn
async def app_main() -> None:
    # Declare table target state
    table = await postgres.mount_table_target(
        PG_DB,
        "products",
        await postgres.TableSchema.from_class(
            OutputProduct,
            primary_key=["category", "name"],
        ),
    )

    # Declare rows
    for product in products:
        table.declare_row(row=product)

    # Declare a vector index on the embedding column
    table.declare_vector_index(
        column="embedding",
        metric="cosine",
        method="hnsw",
    )
```

--------------------------------

### Set up Qdrant with Docker

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_database.md

Use this Docker Compose configuration to set up a Qdrant instance. It exposes the necessary ports for communication and uses a volume for persistent storage.

```bash
cat > docker-compose.yml <<EOF
version: '3.8'

services:
  qdrant:
    image: qdrant/qdrant:latest
    container_name: cocoindex-qdrant
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - qdrant_data:/qdrant/storage

volumes:
  qdrant_data:
EOF

docker-compose up -d
```

--------------------------------

### Sync tests environment setup

Source: https://github.com/cocoindex-io/cocoindex/blob/main/AGENTS.md

Example of setting up a CocoIndex test environment for synchronous tests by calling `common.create_test_env` at the module level.

```python
coco_env = common.create_test_env(__file__)
```

--------------------------------

### Qdrant Connection Setup

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/connectors.md

Sets up a Qdrant client for use within the CocoIndex environment. Requires the Qdrant client to be running locally.

```python
from cocoindex.connectors import qdrant

from qdrant_client import QdrantClient
QDRANT_DB = coco.ContextKey[QdrantClient]("qdrant_db")

@coco.lifespan
def coco_lifespan(builder: coco.EnvironmentBuilder) -> Iterator[None]:
    client = qdrant.create_client("http://localhost:6333")
    builder.provide(QDRANT_DB, client)
    yield
```

--------------------------------

### Kafka Integration Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex and confluent-kafka for Kafka integration.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "confluent-kafka",
]
```

--------------------------------

### Create projects directory structure

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/example-posts/multi-codebase-summarization.md

Example directory structure for input Python projects.

```bash
mkdir projects

```

```bash
projects/
├── my_project_1/
│   ├── main.py
│   └── utils.py
├── my_project_2/
│   └── app.py
└── ...

```

--------------------------------

### Qdrant Integration Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex and qdrant-client for Qdrant integration.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "qdrant-client",
]
```

--------------------------------

### File Watching with localfs and LiveMapView

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/programming_guide/live_mode.mdx

This example shows how to use the `localfs` connector with `live=True` to watch for file system changes and process markdown files.

```python
@coco.fn
async def app_main(sourcedir: pathlib.Path, outdir: pathlib.Path) -> None:
    files = localfs.walk_dir(
        sourcedir, recursive=True,
        path_matcher=PatternFilePathMatcher(included_patterns=["**/*.md"]),
        live=True,  # items() returns a LiveMapView
    )
    await coco.mount_each(process_file, files.items(), outdir)

app = coco.App(coco.AppConfig(name="FilesTransform"), app_main, sourcedir=..., outdir=...)
app.update_blocking(live=True)
```

--------------------------------

### LanceDB Integration Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex and lancedb for LanceDB integration.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "lancedb",
]
```

--------------------------------

### Set up environment variables

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/text_embedding_turbopuffer/README.md

Copy the example environment file and fill in your Turbopuffer API key.

```shell
cp .env.example .env
# then edit .env and set TURBOPUFFER_API_KEY=tpuf_...
```

--------------------------------

### SQLite Integration Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex and sqlite-vec for SQLite integration.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "sqlite-vec",
]
```

--------------------------------

### PostgreSQL Integration Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex and asyncpg for PostgreSQL integration.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "asyncpg",
]
```

--------------------------------

### Set up SurrealDB using Docker

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_database.md

Run a SurrealDB instance using the official Docker image. This command starts the database with root user and password credentials.

```bash
docker run --rm -p 8000:8000 surrealdb/surrealdb:latest \
  start --user root --pass root
```

--------------------------------

### Vector Embedding Pipeline Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex, sentence-transformers, and asyncpg.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "sentence-transformers",
    "asyncpg",
]
```

--------------------------------

### Complete pipeline example using context

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/programming_guide/context.mdx

A full pipeline demonstrating context key definition, value provision in lifespan, and context retrieval in processing functions.

```python
from __future__ import annotations

import pathlib
from dataclasses import dataclass
from typing import AsyncIterator, Annotated

import asyncpg
from numpy.typing import NDArray

import cocoindex as coco
from cocoindex.connectors import localfs, postgres
from cocoindex.ops.text import RecursiveSplitter
from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
from cocoindex.resources.chunk import Chunk
from cocoindex.resources.file import FileLike, PatternFilePathMatcher
from cocoindex.resources.id import IdGenerator

DATABASE_URL = "postgres://cocoindex:cocoindex@localhost/cocoindex"
EMBED_MODEL = "sentence-transformers/all-MiniLM-L6-v2"

# 1. Define context keys at module level
PG_DB = coco.ContextKey[asyncpg.Pool]("text_embedding_db")
EMBEDDER = coco.ContextKey[SentenceTransformerEmbedder]("embedder", detect_change=True)

_splitter = RecursiveSplitter()


# 2. Provide values in the lifespan
@coco.lifespan
async def coco_lifespan(builder: coco.EnvironmentBuilder) -> AsyncIterator[None]:
    async with await asyncpg.create_pool(DATABASE_URL) as pool:
        builder.provide(PG_DB, pool)
        builder.provide(EMBEDDER, SentenceTransformerEmbedder(EMBED_MODEL))
        yield


# 3. Use EMBEDDER in type annotations (for vector column schema)
@dataclass
class DocEmbedding:
    id: int
    filename: str
    text: str
    embedding: Annotated[NDArray, EMBEDDER]  # dimension resolved from context


# 4. Retrieve values in processing functions
@coco.fn
async def process_chunk(
    chunk: Chunk,
    filename: pathlib.PurePath,
    id_gen: IdGenerator,
    table: postgres.TableTarget[DocEmbedding],
) -> None:
    table.declare_row(
        row=DocEmbedding(
            id=await id_gen.next_id(chunk.text),
            filename=str(filename),
            text=chunk.text,
            embedding=await coco.use_context(EMBEDDER).embed(chunk.text),
        ),
    )


@coco.fn(memo=True)
async def process_file(
    file: FileLike,
    table: postgres.TableTarget[DocEmbedding],
) -> None:
    text = await file.read_text()
    chunks = _splitter.split(text, chunk_size=2000, chunk_overlap=500, language="markdown")
    id_gen = IdGenerator()
    await coco.map(process_chunk, chunks, file.file_path.path, id_gen, table)


# 5. PG_DB used directly by the connector (resolved internally)
@coco.fn
async def app_main(sourcedir: pathlib.Path) -> None:
    table = await postgres.mount_table_target(
        PG_DB,
        table_name="doc_embeddings",
        table_schema=await postgres.TableSchema.from_class(
            DocEmbedding, primary_key=["id"],
        ),
    )
    files = localfs.walk_dir(
        sourcedir,
        recursive=True,
        path_matcher=PatternFilePathMatcher(included_patterns=["**/*.md"]),
    )
    await coco.mount_each(process_file, files.items(), table)


app = coco.App(
    coco.AppConfig(name="TextEmbedding"),
    app_main,
    sourcedir=pathlib.Path("./markdown_files"),
)
```

--------------------------------

### CocoIndex lifespan setup with SQLite connection

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/sqlite.mdx

Example of setting up a SQLite connection within the CocoIndex lifespan and providing it via a ContextKey.

```python
import cocoindex as coco

SQLITE_DB = coco.ContextKey[sqlite.ManagedConnection]("main_db")

@coco.lifespan
def coco_lifespan(builder: coco.EnvironmentBuilder) -> Iterator[None]:
    with sqlite.managed_connection("mydb.sqlite") as conn:
        builder.provide(SQLITE_DB, conn)
        yield
```

--------------------------------

### Timing Table Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/benchmarks/file_summarization/README.md

Example output of the timing table generated by `./run.sh --format table`, showing performance metrics for different phases and profiles.

```bash
./run.sh --format table

Scenario: codebase | Profile: cpu
phase    rust_ms  py_ms  ratio  rust_miss  py_miss  rust_out  py_out
cold       193.0 10173.7  52.71        ...
warm       129.1   798.4   6.18        ...
edit       142.7   837.1   5.87        ...
shape      165.6   871.6   5.26        ...
```

--------------------------------

### Manual Settings in Lifespan

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

Python code snippet demonstrating how to manually set database path within the lifespan context.

```python
@coco.lifespan
def coco_lifespan(builder: coco.EnvironmentBuilder) -> Iterator[None]:
    builder.settings.db_path = pathlib.Path("./custom.db")
    yield
```

--------------------------------

### main.py

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/getting_started/quickstart.mdx

This function defines the main app logic for walking through source directories and mounting processing components.

```python
@coco.fn
async def app_main(sourcedir: pathlib.Path, outdir: pathlib.Path) -> None:
    files = localfs.walk_dir(
        sourcedir,
        recursive=True,
        path_matcher=PatternFilePathMatcher(included_patterns=["**/*.pdf"]),
    )
    await coco.mount_each(process_file, files.items(), outdir)
```

--------------------------------

### Type Definitions for Provider

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/advanced_topics/custom_target_connector.mdx

Example of starting type definitions for a custom target provider, including NamedTuple, Collection, and dataclass imports.

```python
from typing import NamedTuple, Collection
from dataclasses import dataclass
import cocoindex as coco

# Key: StableKey is used to identify target states — it's a union type:
```

--------------------------------

### Local File System Walk Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/localfs.mdx

An example demonstrating how to walk through directories recursively and process each markdown file found.

```python
import pathlib
import cocoindex as coco
from cocoindex.connectors import localfs
from cocoindex.resources.file import FileLike, PatternFilePathMatcher

SOURCE_DIR = coco.ContextKey[pathlib.Path]("source_dir")

@coco.fn
async def app_main() -> None:
    matcher = PatternFilePathMatcher(included_patterns=["**/*.md"])

    async for file in localfs.walk_dir(SOURCE_DIR, recursive=True, path_matcher=matcher):
        await coco.mount(
            coco.component_subpath("file", str(file.file_path.path)),
            process_file,
            file,
        )

@coco.fn(memo=True)
async def process_file(file: FileLike) -> None:
    text = await file.read_text()
    # ... process the file content ...
```

--------------------------------

### Environment Variables Setup

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/meeting_notes_graph_falkordb/README.md

Example environment variables to set for the application.

```sh
export OPENAI_API_KEY=sk-...
export GOOGLE_SERVICE_ACCOUNT_CREDENTIAL=/absolute/path/to/service_account.json
export GOOGLE_DRIVE_ROOT_FOLDER_IDS=folderId1,folderId2
export FALKORDB_URI=falkor://localhost:6379
export FALKORDB_GRAPH=meeting_notes
export LLM_MODEL=openai/gpt-5.4
export RESOLUTION_LLM_MODEL=openai/gpt-5-mini   # used for entity resolution
```

--------------------------------

### Keyed Iteration with items()

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/oci_object_storage.mdx

Example of using the `items()` method on the `OCIWalker` to get key-value pairs of file paths and file objects.

```python
async for key, file in oci_object_storage.list_objects(client, "ns", "my-bucket").items():
    content = await file.read()
```

--------------------------------

### Environment Variables for LLM API Key and Configuration

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/rust/multi-codebase-summarization/README.md

Sets the OpenAI API key and optional overrides for the LLM model and base URL.

```sh
export LLM_API_KEY="sk-..."
# Optional overrides:
export LLM_MODEL="gpt-4o-mini"         # default
export LLM_BASE_URL="https://api.openai.com/v1"  # default
```

--------------------------------

### Generate Stable IDs for Items

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/common_resources/id_generation.mdx

Use `generate_id` to get the same integer ID for the same item key across different runs. IDs start from 1; 0 is reserved.

```python
from cocoindex.resources.id import generate_id, generate_uuid

async def process_item(item: Item) -> Row:
    # Same item.key always gets the same ID
    item_id = await generate_id(item.key)
    return Row(id=item_id, data=item.data)
```

--------------------------------

### Commit and push changes

Source: https://github.com/cocoindex-io/cocoindex/blob/main/dev/agent-skills/upgrade-examples/SKILL.md

Commands to stage, commit, and push the changes to the Git repository.

```bash
git add examples/*/pyproject.toml
git commit -m "chore: upgrade examples deps to cocoindex-VERSION"
git push -u origin ex-dep-VERSION
```

--------------------------------

### LLM-Based Extraction Dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_project.md

TOML configuration for project dependencies including cocoindex, litellm, instructor, pydantic, and asyncpg for LLM-based extraction.

```toml
[project]
dependencies = [
    "cocoindex>=1.0.0",
    "litellm",
    "instructor",
    "pydantic>=2.0",
    "asyncpg",
]
```

--------------------------------

### CLI Subcommand: init

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/cli.mdx

Initialize a new CocoIndex project.

```APIDOC
## `init`

Initialize a new CocoIndex project.

Creates a new project directory with starter files: 1. main.py (Main
application file) 2. pyproject.toml (Project metadata and dependencies) 3.
README.md (Quick start guide)

`PROJECT_NAME`: Name of the project (defaults to current directory name if
not specified).

### Usage:

```bash
cocoindex init [OPTIONS] [PROJECT_NAME]
```

### Options:

| Option | Description |
|--------|-------------|
| `--dir DIRECTORY` | Directory to create the project in. |
| `--help` | Show this message and exit. |
```

--------------------------------

### Async tests environment setup with async resources

Source: https://github.com/cocoindex-io/cocoindex/blob/main/AGENTS.md

Example of setting up a CocoIndex test environment for asynchronous tests with async resources, using an async fixture to bind the Environment to the test's event loop and provide async resources like a database pool.

```python
import pytest
import pytest_asyncio
import asyncpg
from typing import Any

# Assuming common and DB_KEY are defined elsewhere
# from cocoindex.testing import common
# DB_KEY = "db_pool"

@pytest_asyncio.fixture
async def pg_env(request: pytest.FixtureRequest) -> Any:
    dsn = "your_database_dsn"
    pool = await asyncpg.create_pool(dsn)
    coco_env = common.create_test_env(__file__, suffix=request.node.name)
    coco_env.context_provider.provide(DB_KEY, pool)
    yield pool, coco_env
    await pool.close()
```

--------------------------------

### Install CocoIndex using uv

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/getting_started/installation.mdx

Install CocoIndex using the uv package installer.

```sh
uv add cocoindex
```

--------------------------------

### CLI Subcommand: ls

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/cli.mdx

List all apps.

```APIDOC
## `ls`

List all apps.

If `APP_TARGET` (`path/to/app.py` or `module`) is provided, lists apps
defined in that module and their persisted status, grouped by environment.

If `APP_TARGET` is omitted and `--db` is provided, lists all apps from the
specified database.

### Usage:

```bash
cocoindex ls [OPTIONS] [APP_TARGET]
```

### Options:

| Option | Description |
|--------|-------------|
| `--db TEXT` | Path to database to list apps from (only used when APP_TARGET is not specified). |
| `--help` | Show this message and exit. |
```

--------------------------------

### Environment Variables Setup

Source: https://github.com/cocoindex-io/cocoindex/blob/main/examples/meeting_notes_graph_neo4j/README.md

Example environment variables to configure the CocoIndex flow, including API keys, Google Drive credentials, Neo4j connection details, and LLM models. It also shows how to load these variables from a .env file.

```sh
export OPENAI_API_KEY=sk-...
export GOOGLE_SERVICE_ACCOUNT_CREDENTIAL=/absolute/path/to/service_account.json
export GOOGLE_DRIVE_ROOT_FOLDER_IDS=folderId1,folderId2
export NEO4J_URI=bolt://localhost:7687
export NEO4J_USER=neo4j
export NEO4J_PASSWORD=cocoindex
export NEO4J_DATABASE=neo4j
export LLM_MODEL=openai/gpt-5.4
export RESOLUTION_LLM_MODEL=openai/gpt-5-mini   # used for entity resolution
```

```sh
set -a && source .env && set +a
```

--------------------------------

### Declare File Example

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/localfs.mdx

Example showing how to declare a single file target for writing content to the file system.

```python
OUTPUT_DIR = coco.ContextKey[pathlib.Path]("output_dir")

@coco.fn
def app_main() -> None:
    coco.mount(
        localfs.declare_file,
        localfs.FilePath("readme.txt", base_dir=OUTPUT_DIR),
        content="Hello, world!",
        create_parent_dirs=True,
    )
```

--------------------------------

### Install CocoIndex using Poetry

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/getting_started/installation.mdx

Install CocoIndex using the Poetry package manager.

```sh
poetry add cocoindex
```

--------------------------------

### Register Resources with Environment Builder

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/api_reference.md

Use builder.provide() to register resources in the context, typically done during the application's lifespan setup. builder.provide_with() and provide_async_with() are used for context managers.

```python
builder.provide(PG_DB, pool)
builder.provide_with(KEY, context_manager)         # Sync CM
await builder.provide_async_with(KEY, async_cm)    # Async CM
```

--------------------------------

### Install Postgres connector dependencies

Source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/src/content/docs/connectors/postgres.mdx

This connector requires additional dependencies. Install with: `pip install cocoindex[postgres]`

```bash
pip install cocoindex[postgres]
```

--------------------------------

### Connect to Qdrant Cloud

Source: https://github.com/cocoindex-io/cocoindex/blob/main/skills/cocoindex/references/setup_database.md

Create a Qdrant client instance for connecting to Qdrant Cloud. Provide your cluster URL and API key. gRPC is preferred for performance.

```python
client = qdrant.create_client(
    url="https://your-cluster.qdrant.io",
    api_key="your-api-key",
    prefer_grpc=True,
)
```