### KVault Quick Start Example
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/kvault.md
Demonstrates basic KVault initialization, storing different data types (dictionary, NumPy array, raw bytes), and retrieving them.
```python
from kohakuvault import KVault
import numpy as np
kv = KVault("app.db") # auto-pack is on by default
kv["config"] = {"timeout": 30, "retries": 3} # auto MessagePack
kv["embedding"] = np.random.randn(768).astype(np.float32)
kv["image.jpg"] = open("image.jpg", "rb").read() # raw bytes stay raw
assert kv["config"]["timeout"] == 30
assert kv["embedding"].shape == (768,)
```
--------------------------------
### KohakuVault Development Workflow Setup
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/README.md
Sets up the development environment by installing KohakuVault in editable mode with development dependencies. This command is typically run once.
```bash
pip install -e .[dev]
maturin develop
```
--------------------------------
### Create and Use Single-Column TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Illustrates the default single-column TextVault setup and performing an exact match lookup.
```python
tv = TextVault("data.db") # defaults to columns=["content"]
tv.insert("Hello world", {"meta": "data"})
value = tv["Hello world"] # exact match lookup
```
--------------------------------
### Install KohakuVault for Development
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/README.md
Installs KohakuVault in editable mode with development dependencies, including tools for building PyO3 extensions. Run 'maturin develop' after changing Rust code.
```bash
pip install -e .[dev]
# everytime you change the rust code, you should run this command
maturin develop
```
--------------------------------
### Install KohakuVault Package
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/README.md
Installs the KohakuVault library using pip. This is the standard command for adding the package to your Python environment.
```bash
pip install kohakuvault
```
--------------------------------
### TextVault Auto-Packing Examples
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Demonstrates storing various Python objects as values in TextVault, leveraging MessagePack and DataPacker.
```python
# Store any Python object as values
tv.insert("text", {"dict": "value"}) # MessagePack
tv.insert("text", [1, 2, 3]) # MessagePack
tv.insert("text", np.array([1, 2, 3])) # DataPacker vec:*
tv.insert("text", 42) # DataPacker i64
tv.insert("text", b"raw bytes") # Raw (no header)
```
--------------------------------
### KohakuVault Core Functionality Example
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/README.md
Demonstrates the primary usage of KVault for storing raw bytes and auto-packed metadata, TextVault for full-text search, ColumnVault for incremental columnar logging, and VectorKVault for k-NN search within the same SQLite database.
```python
from kohakuvault import KVault, TextVault, ColumnVault, VectorKVault, CSBTree
import numpy as np
DB = "test.db"
# Store media + metadata in one file
kv = KVault(DB, table="media")
kv["video:42"] = b"abcdefg" # raw bytes stay raw
kv["video:42:meta"] = {"fps": 60, "tags": ["tutorial", "gpu"]} # auto MessagePack
# Full-text search with BM25 ranking (great for RAG)
tv = TextVault(DB, table="docs")
tv.insert("Machine learning is a subset of AI", {"category": "tech"})
for doc_id, score, meta in tv.search("machine learning", k=5):
print(f"Doc {doc_id}: score={score:.4f}")
# Incremental columnar logging (no file rewrites)
cols = ColumnVault(kv) # or ColumnVault(DB)
ids = cols.ensure("ids", "i64")
latency = cols.ensure("latency_ms", "f64")
embeddings = cols.ensure("embeddings", "vec:f32:384")
with cols.cache(cap_bytes=8 << 20):
ids.append(len(ids))
latency.append(12.3)
embeddings.append(np.random.randn(384).astype(np.float32))
# Ordered metadata via embedded CSBTree
index = CSBTree()
index.insert(latency[-1], ids[-1]) # log-latency -> run-id
# Vector similarity search (sqlite-vec inside the same DB)
search = VectorKVault(DB, table="runs", dimensions=384, metric="cosine")
for idx, emb in enumerate(embeddings[:10]):
search.insert(emb, str(idx).encode())
for rank, (rid, distance, run_id) in enumerate(search.search(embeddings[-1], k=3), 1):
print(rank, distance, run_id.decode())
```
--------------------------------
### Initialize and Use VectorKVault for Similarity Search
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/vectors.md
Set up `VectorKVault` for k-NN search with specified dimensions, metric, and vector type. This example demonstrates inserting a vector, performing a search, and retrieving results.
```python
from kohakuvault import VectorKVault
import numpy as np
vkv = VectorKVault("search.db", table="search", dimensions=384, metric="cosine", vector_type="f32")
vector = np.random.randn(384).astype(np.float32)
doc_id = vkv.insert(vector, b"document payload")
results = vkv.search(vector, k=10)
for row_id, distance, payload in results:
print(row_id, distance, payload[:32])
closest = vkv.get(vector)
vector2, payload2 = vkv.get_by_id(doc_id)
vkv.update(doc_id, vector=new_vector)
vkv.delete(doc_id)
```
--------------------------------
### TextVault Safe Query Escaping Examples
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Demonstrates how TextVault automatically escapes special characters in queries for safe literal matching.
```python
tv.search("What is this?") # Works with ?
tv.search("C++ programming") # Works with +
tv.search("test@email.com") # Works with @
tv.search('He said "hello"') # Works with quotes
```
--------------------------------
### Get Document by Exact Key (Single-Column)
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Retrieves a document from a single-column TextVault using its exact text content as the key.
```python
value = tv.get("exact document text")
# or dict-style:
value = tv["exact document text"]
```
--------------------------------
### TextVault Utility Methods
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Provides utility methods for TextVault, including getting the document count, checking existence, retrieving information, listing keys, and clearing all documents.
```python
len(tv) # Document count
tv.count() # Same as len()
tv.count_matches("query") # Count matching documents
tv.exists(doc_id) # Check if ID exists
doc_id in tv # Same as exists()
tv.info() # {"table": "...", "columns": [...], "count": N}
tv.columns # ["content"] or ["title", "body", "tags"]
tv.keys(limit=100, offset=0) # Paginated list of rowids
tv.clear() # Delete all documents
```
--------------------------------
### Create Image Vector Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Example of creating a column to store image data represented as unsigned 8-bit integer vectors with a 28x28 shape.
```python
images = vault.create_column("mnist", "vec:u8:28:28")
```
--------------------------------
### Get Document by ID from TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Retrieves a document from TextVault using its unique ID, returning either text or a dictionary of columns.
```python
text, value = tv.get_by_id(doc_id)
# Single column: text is string
# Multi-column: text is dict {"title": "...", "body": "...", "tags": "..."}
```
--------------------------------
### Create Arbitrary-Shape Vector Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/vectors.md
Use `vec:f32` for vectors where dimensions can vary. This example shows appending vectors with different shapes to a generic column.
```python
generic = cv.create_column("generic", "vec:f32")
generic.append(np.random.randn(100).astype(np.float32))
generic.append(np.random.randn(10, 20).astype(np.float32))
```
--------------------------------
### Create and Extend Fixed-Shape Vector Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/vectors.md
Use `vec:f32:768` for storing vectors of a consistent shape, such as embeddings. This example demonstrates creating a column and extending it with random float32 vectors.
```python
from kohakuvault import ColumnVault
import numpy as np
cv = ColumnVault("vectors.db")
embeddings = cv.create_column("bert", "vec:f32:768")
embeddings.extend(np.random.randn(10_000, 768).astype(np.float32))
```
--------------------------------
### Create and Use Columns
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Demonstrates creating a new column, extending it with data, and reusing an existing column.
```python
from kohakuvault import ColumnVault
import numpy as np
vault = ColumnVault("analytics.db")
counts = vault.create_column("counts", "i64")
counts.extend(range(10))
# Reuse existing column or create it lazily
metrics = vault.ensure("metrics", "msgpack")
```
--------------------------------
### Create and Search Multi-Column TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Shows how to set up a multi-column TextVault for structured documents and search specific columns.
```python
tv = TextVault("data.db", columns=["title", "body", "tags"])
tv.insert(
{"title": "Introduction to ML", "body": "Machine learning is...", "tags": "ml ai tutorial"},
{"author": "John", "date": "2024-01-01"}
)
# Search specific columns
results = tv.search("introduction", column="title")
results = tv.search("tutorial", column="tags")
```
--------------------------------
### Create and Search Single-Column TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Demonstrates creating a single-column TextVault, inserting a document, and performing a BM25 ranked search.
```python
from kohakuvault import TextVault
# Create a single-column text vault
tv = TextVault("documents.db")
# Insert documents with any Python value
doc_id = tv.insert("Machine learning is a subset of AI", {"category": "tech", "importance": 5})
# Search with BM25 ranking
results = tv.search("machine learning", k=10)
for doc_id, score, value in results:
print(f"ID {doc_id}: score={score:.4f}, value={value}")
```
--------------------------------
### Instantiate DataPacker with Fixed and Variable Size Types
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/datapacker.md
Demonstrates creating DataPacker instances for fixed-size and variable-size data types. Fixed-size types have a defined element size, while variable-size types do not.
```python
from kohakuvault import DataPacker
fixed = DataPacker("str:32:utf8")
assert fixed.elem_size == 32 and not fixed.is_varsize
var = DataPacker("vec:f32")
assert var.elem_size == 0 and var.is_varsize
```
--------------------------------
### KohakuVault Formatting, Linting, and Testing
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/README.md
Runs code formatting with 'black', Rust code formatting and linting with 'cargo fmt' and 'cargo clippy', builds the PyO3 extension in release mode, and executes Python tests with 'pytest'.
```bash
black .
cargo fmt
cargo clippy
maturin develop --release
pytest
```
--------------------------------
### Pack and Unpack Data with DataPacker
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/datapacker.md
Shows how to pack a dictionary into a byte blob and then unpack it using a DataPacker instance. Offsets can be used to read multiple values from a single blob.
```python
packer = DataPacker("msgpack")
blob = packer.pack({"user": 1, "tags": ["vip", "beta"]})
restored = packer.unpack(blob, offset=0)
```
--------------------------------
### ColumnVault and Column Caching
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Demonstrates using the cache context manager for both the entire vault and individual columns. Useful for optimizing ingest operations by controlling cache size and flush thresholds.
```python
with vault.cache(cap_bytes=32 << 20, flush_threshold=8 << 20):
counts.extend(range(1_000_000))
col = vault["counts"]
with col.cache(cap_bytes=4 << 20):
col.append(123)
```
--------------------------------
### KVault Streaming API for Files
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/kvault.md
Demonstrates uploading large files using `put_file` and downloading them using `get_to_file` without loading the entire content into memory. This is useful for handling large blobs efficiently.
```python
# Upload large blobs without loading them fully in memory
with open("movie.mp4", "rb") as reader:
kv.put_file("movie:2024", reader, chunk_size=4 << 20)
with open("copy.mp4", "wb") as writer:
kv.get_to_file("movie:2024", writer, chunk_size=4 << 20)
```
--------------------------------
### Create and Extend Vector Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Shows how to create a column for fixed-shape float vectors and extend it with random data.
```python
embeddings = vault.create_column("embeddings", "vec:f32:768")
embeddings.extend(np.random.randn(10_000, 768).astype(np.float32))
```
--------------------------------
### RAG Pipeline with Hybrid Search
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Sets up TextVault and VectorKVault for a RAG pipeline, indexing documents and their embeddings, and implementing a hybrid search function combining BM25 and vector similarity.
```python
from kohakuvault import TextVault, VectorKVault
import numpy as np
# Documents with text search
docs = TextVault("rag.db", table="documents")
# Embeddings with vector search
vecs = VectorKVault("rag.db", table="embeddings", dimensions=384, metric="cosine")
# Index documents
for idx, doc in enumerate(documents):
# Store full document with metadata
doc_id = docs.insert(doc["text"], {"source": doc["source"], "date": doc["date"]})
# Store embedding linked to doc_id
embedding = model.encode(doc["text"])
vecs.insert(embedding.astype(np.float32), str(doc_id).encode())
# Hybrid search: combine BM25 + vector similarity
def hybrid_search(query: str, k: int = 10, alpha: float = 0.5):
# BM25 search
bm25_results = {doc_id: score for doc_id, score, _ in docs.search(query, k=k*2)}
# Vector search
query_vec = model.encode(query).astype(np.float32)
vec_results = {int(doc_id): 1/(1+dist) for _, dist, doc_id in vecs.search(query_vec, k=k*2)}
# Combine scores
all_ids = set(bm25_results) | set(vec_results)
combined = []
for doc_id in all_ids:
bm25_score = bm25_results.get(doc_id, 0)
vec_score = vec_results.get(doc_id, 0)
combined.append((doc_id, alpha * bm25_score + (1-alpha) * vec_score))
return sorted(combined, key=lambda x: x[1], reverse=True)[:k]
```
--------------------------------
### KVault Optimization with PRAGMA and VACUUM
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
Performs database optimization by running SQLite's PRAGMA optimize and VACUUM commands. This helps in compacting the database file and improving query performance.
```python
KVault.optimize()
```
--------------------------------
### KVault Auto-Pack Configuration
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/kvault.md
Shows how to enable or disable the auto-pack feature, which automatically serializes Python objects into compact binary formats. Use `use_pickle=True` to allow Pickle serialization for unsupported types.
```python
kv.enable_auto_pack(use_pickle=True) # default
kv.disable_auto_pack() # revert to bytes-only
kv.auto_pack_enabled() # -> bool
```
--------------------------------
### Extend and Modify Fixed-Size Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Shows how to extend a fixed-size column with initial values and then modify a slice of it.
```python
scores = vault.create_column("scores", "f64")
scores.extend([95.5, 87.3, 99.1])
scores[1:3] = [90.0, 91.5]
```
--------------------------------
### Batch Pack and Unpack Integer and Vector Data
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/datapacker.md
Illustrates using the `pack_many` and `unpack_many` methods for efficient bulk processing of data. This is suitable for large lists of fixed-size types or vectors.
```python
from kohakuvault import DataPacker
import numpy as np
pack_i64 = DataPacker("i64")
values = list(range(1_000))
buffer = pack_i64.pack_many(values)
assert pack_i64.unpack_many(buffer, count=len(values)) == values
pack_vec = DataPacker("vec:f32:768")
vectors = [np.random.randn(768).astype(np.float32) for _ in range(1_000)]
buffer = pack_vec.pack_many(vectors)
restored = pack_vec.unpack_many(buffer, count=len(vectors))
```
--------------------------------
### Create Variable-Size Column with JSON Schema Validation
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Demonstrates creating a DataPacker with a JSON schema for validating structured data before appending.
```python
from kohakuvault import DataPacker
schema = {"type": "object", "required": ["id", "name"]}
packer = DataPacker.with_json_schema(schema)
```
--------------------------------
### Append and Extend Variable-Size Column
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/columnvault.md
Illustrates appending single structured items and extending with multiple items to a variable-size column.
```python
events = vault.ensure("events", "msgpack")
events.append({"type": "login", "user": 42})
events.extend({"type": "purchase", "amount": a} for a in amounts)
```
--------------------------------
### KVault Caching and Concurrency Control
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/kvault.md
Illustrates using KVault's write-back cache for buffering writes and managing concurrency. The `cache` context manager allows specifying capacity and flush thresholds, while `lock_cache` provides temporary blocking of background flushes.
```python
with kv.cache(cap_bytes=64 << 20, flush_threshold=8 << 20) as buffered:
for i in range(200_000):
buffered[f"log:{i}"] = b"..."
kv.enable_cache(flush_interval=5.0) # background daemon flush
with kv.lock_cache(): # block daemon flush temporarily
mutate_many_keys()
kv.flush_cache()
```
--------------------------------
### DataPacker with JSON Schema Validation
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/datapacker.md
Demonstrates creating a DataPacker instance that validates data against a provided JSON schema. Invalid data will raise a ValueError during packing.
```python
schema = {
"type": "object",
"properties": {
"id": {"type": "integer"},
"name": {"type": "string"},
},
"required": ["id", "name"],
}
packer = DataPacker.with_json_schema(schema)
packer.pack({"id": 1, "name": "Rin"})
packer.pack({"id": "oops"}) # raises ValueError
```
--------------------------------
### Insert Document into TextVault (Multi-Column)
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Inserts a structured document with separate searchable fields and metadata into a multi-column TextVault.
```python
# Multi-column
doc_id = tv.insert(
{"title": "Hello", "body": "World", "tags": "greeting"},
{"created": "2024-01-01"}
)
```
--------------------------------
### Manage Auto-Pack Status
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Check, disable, and re-enable TextVault's auto-pack feature. Auto-packing is enabled by default and can be configured to use pickle for custom objects.
```python
# Check status
tv.auto_pack_enabled() # True by default
# Disable for bytes-only mode
tv.disable_auto_pack()
# Re-enable
tv.enable_auto_pack(use_pickle=True) # use_pickle allows custom objects
```
--------------------------------
### Insert Document into TextVault (Single-Column)
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Inserts a document with associated metadata into a single-column TextVault.
```python
# Single column
doc_id = tv.insert("document text", {"key": "value"})
```
--------------------------------
### KVault and ColumnVault WAL Checkpointing
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
Initiates a Write-Ahead Logging (WAL) checkpoint operation for KVault or ColumnVault instances. This operation is crucial for maintaining database integrity and performance by merging WAL data into the main database file.
```python
KVault.checkpoint()
ColumnVault.checkpoint()
```
--------------------------------
### Storing Documents, Embeddings, and Titles
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/vectors.md
This snippet initializes KVault, ColumnVault, and creates columns for embeddings and titles. It then iterates through a dataset, storing document text, titles, and embeddings. Finally, it sets up a VectorKVault for similarity search and inserts the embeddings.
```python
from kohakuvault import KVault, ColumnVault, VectorKVault
import numpy as np
db = "semantic.db"
kv = KVault(db, table="documents")
cv = ColumnVault(kv)
emb = cv.create_column("embeddings", "vec:f32:384")
titles = cv.create_column("titles", "str:utf8")
for idx, (title, text, embedding) in enumerate(dataset):
kv[f"doc:{idx}"] = text.encode()
titles.append(title)
emb.append(embedding.astype(np.float32))
search = VectorKVault(db, table="search_index", dimensions=384, metric="cosine")
for idx, embedding in enumerate(emb[-len(dataset):]):
search.insert(embedding, str(idx).encode())
query = model.encode("machine learning").astype(np.float32)
for rank, (row_id, dist, doc_idx_bytes) in enumerate(search.search(query, k=5), 1):
doc_idx = int(doc_idx_bytes)
print(rank, dist, titles[doc_idx], kv[f"doc:{doc_idx}"][:80])
```
--------------------------------
### Enabling Raw Byte Headers
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
Controls whether raw byte data stored in KVault should be prefixed with headers. Headers are rarely needed and can be disabled to save space.
```python
KVault.enable_headers()
```
--------------------------------
### Hybrid Layout: Metadata + Blobs
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/structured_columns.md
Store binary payloads in KVault and structured metadata in a MessagePack column. This separates large blobs from filterable metadata, improving performance.
```python
from kohakuvault import KVault, ColumnVault
kv = KVault("media.db")
cols = ColumnVault(kv)
meta = cols.ensure("media_meta", "msgpack")
for asset in assets:
kv[f"blob:{asset['id']}"] = asset["bytes"]
meta.append({"id": asset["id"], "size": len(asset["bytes"]), "mime": asset["mime"]})
```
--------------------------------
### TextVault Search with Snippets
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Fetches search results with highlighted snippets showing context around matches.
```python
results = tv.search_with_snippets(
"machine learning",
k=10,
snippet_tokens=15, # words around match
highlight_start="", # custom highlight markers
highlight_end=""
)
for doc_id, score, snippet, value in results:
print(f"Snippet: {snippet}") # "...is a machine learning algorithm that..."
```
--------------------------------
### TextVault Search with Text Content
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Retrieves the indexed text content alongside search results.
```python
results = tv.search_with_text("machine learning", k=5)
for doc_id, score, text, value in results:
print(f"Text: {text}")
print(f"Value: {value}")
```
--------------------------------
### Hybrid Layout: Secondary Indexes with CSBTree
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/structured_columns.md
Build in-memory indexes using CSBTree for efficient lookups on column data. This pattern keeps the canonical record in a column while providing fast access via an index.
```python
from kohakuvault import ColumnVault, CSBTree
cols = ColumnVault("events.db")
events = cols.ensure("events", "msgpack")
index = CSBTree()
row_id = len(events)
events.append({"user": 42, "type": "login"})
index.insert(42, row_id)
for _, rid in index.range(42, 42):
handle(events[rid])
```
--------------------------------
### Batch Operations with DataPacker for Vectors
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/vectors.md
Utilize `DataPacker` for efficient preprocessing and marshaling of vectors. This snippet shows packing and unpacking a large batch of fixed-shape vectors.
```python
from kohakuvault import DataPacker
packer = DataPacker("vec:f32:768")
vectors = [np.random.randn(768).astype(np.float32) for _ in range(10_000)]
buffer = packer.pack_many(vectors)
unpacked = packer.unpack_many(buffer, count=len(vectors))
```
--------------------------------
### TextVault Raw FTS5 Syntax Search
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Shows how to disable default escaping to use raw FTS5 query operators for advanced searching.
```python
tv.search("hello AND world", escape=False) # Both terms required
tv.search("python OR java", escape=False) # Either term
tv.search("hello NOT goodbye", escape=False) # Exclusion
tv.search("mach*", escape=False) # Prefix matching
tv.search("NEAR(hello world, 5)", escape=False) # Proximity search
```
--------------------------------
### Basic TextVault Search
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Performs a basic BM25 ranked search and prints the document ID and score.
```python
results = tv.search("hello world", k=10)
for doc_id, score, value in results:
print(f"Doc {doc_id}: {score:.4f}")
```
--------------------------------
### Auto-Pack Header Format
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
The 10-byte header used to prefix non-raw auto-packed values in the KVault table. It includes version, encoding, and flags for safe mixing of raw and packed data.
```rust
|0x89 0x4B|version|encoding|flags|reserved(3)|0x56 0x4B|
```
--------------------------------
### Update Document in TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Updates the text content, metadata value, or both for an existing document in TextVault using its ID.
```python
# Update text only
tv.update(doc_id, texts="new document text")
# Update value only
tv.update(doc_id, value={"new": "metadata"})
# Update both
tv.update(doc_id, texts="new text", value={"new": "value"})
```
--------------------------------
### Enforce JSON Schema with DataPacker
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/structured_columns.md
Use DataPacker with a JSON schema to validate MessagePack payloads before they are written to SQLite. Failures raise ValueError, allowing for pre-write validation in Python.
```python
from kohakuvault import ColumnVault, DataPacker
schema = {
"type": "object",
"properties": {
"id": {"type": "integer"},
"name": {"type": "string"},
"tags": {"type": "array", "items": {"type": "string"}},
},
"required": ["id", "name"],
}
packer = DataPacker.with_json_schema(schema)
vault = ColumnVault("users.db")
users = vault.ensure("users", "msgpack")
users.append({"id": 1, "name": "Rin"})
```
--------------------------------
### TextVault Schema Definition
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Defines the SQL schema created by TextVault, consisting of an FTS5 virtual table for indexed text and a separate values table for storing blob values.
```sql
-- What TextVault creates:
CREATE VIRTUAL TABLE text_vault USING fts5(content, value_ref UNINDEXED);
CREATE TABLE text_vault_values (id INTEGER PRIMARY KEY, value BLOB);
```
--------------------------------
### Retrieve Document by ID
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Retrieves a document by its ID and prints the type of the decoded value. The value can be a dictionary, list, numpy array, or other types.
```python
text, value = tv.get_by_id(doc_id)
print(type(value)) # , , , etc.
```
--------------------------------
### Flushing Cached Writes to SQLite
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
Forces any buffered writes held in memory caches for vaults or columns to be immediately written to the underlying SQLite database. This ensures data durability.
```python
flush_cache()
```
--------------------------------
### Disabling Raw Byte Headers
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/architecture.md
Disables the automatic addition of headers to raw byte data stored in KVault. This is the default behavior and is useful when minimizing storage footprint.
```python
KVault.disable_headers()
```
--------------------------------
### Delete Document from TextVault
Source: https://github.com/kohakublueleaf/kohakuvault/blob/main/docs/textvault.md
Removes a document from TextVault using its ID, or by exact key for single-column exact match lookups.
```python
tv.delete(doc_id)
# or for single-column with exact match:
del tv["exact document text"]
```
=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.