TopK

https://docs.topk.io/
Admin
TopK is a search engine for the AI era that offers vector search, keyword search, metadata...
Tokens:28,396
Snippets:128
Trust Score:7.8
Update:1 year ago
Show doc for...
Tokens:
Raw
### Initialize TopK Client

Source: https://docs.topk.io/quickstart

Initializes a TopK Client instance with the provided API key and region. This is the first step to interact with TopK services.

```Python
from topk_sdk import Client

client = Client(api_key="YOUR_TOPK_API_KEY", region="aws-us-east-1-elastica")
```

--------------------------------

### Create a Collection with Semantic Index

Source: https://docs.topk.io/quickstart

Creates a new collection named 'books' with a schema that includes a 'title' field indexed for semantic search. This enables both semantic and keyword search capabilities on the 'title' field.

```Python
from topk_sdk.schema import text, semantic_index

client.collections().create(
    "books",
    schema={
        "title": text().required().index(semantic_index()),
    },
)
```

--------------------------------

### Perform Semantic Search and Rerank

Source: https://docs.topk.io/quickstart

Executes a semantic search query on the 'books' collection to find documents semantically similar to 'classic American novel'. Results are scored, sorted by similarity, limited to the top 10, and optionally reranked.

```Python
from topk_sdk.query import select, fn, field

results = client.collection("books").query(
    select(
        "title",
        # fn.semantic_similarity() scores documents by how semantically similar their titles are to "classic American novel"
        title_similarity=fn.semantic_similarity("title", "classic American novel"),
    )
    # Sort results by `title_similarity` (computed in the `select()` function above), and limit to top 10 results
    .topk(field("title_similarity"), 10)
    # Rerank results using the built-in reranking model
    .rerank()
)
```

--------------------------------

### Upsert Documents into Collection

Source: https://docs.topk.io/quickstart

Adds or updates documents in the 'books' collection. Each document must include a unique '_id' field and can contain other fields like 'title'.

```Python
client.collection("books").upsert(
    [
        {"_id": "gatsby", "title": "The Great Gatsby"},
        {"_id": "1984", "title": "1984"},
        {"_id": "catcher", "title": "The Catcher in the Rye"},
    ],
)
```

--------------------------------

### Install TopK SDK

Source: https://docs.topk.io/installation

Installs the TopK SDK using pip. This is the initial step to begin using the SDK in a Python environment.

```bash
pip install topk-sdk
```

--------------------------------

### Install TopK SDK

Source: https://docs.topk.io/documents/helper-functions

Instructions for installing the TopK Python or Javascript SDK to interact with the search API.

```bash
pip install topk-sdk
# or
npm install topk-sdk
```

--------------------------------

### Delete a Collection

Source: https://docs.topk.io/quickstart

Deletes the specified collection, removing all its data and schema. This action is irreversible.

```Python
client.collections().delete("books")
```

--------------------------------

### List Collections in a Project (Javascript)

Source: https://docs.topk.io/collections/list

Demonstrates how to list all collections within a project using the TopK.io client. This function returns a list of Collection objects.

```Javascript
const collections = client.collections().list();
```

--------------------------------

### Example Document Response Structure

Source: https://docs.topk.io/documents/get

Illustrates the structure of the response when retrieving documents by their IDs. Each document is returned as an object with its associated data, keyed by its ID. Documents not found are omitted.

```JSON
{
    "lotr": {
		"_id": "lotr",
        "title": "The Lord of the Rings",
        "author": "J.R.R. Tolkien",
        "published_year": 1954
    },
    "moby": {
		"_id": "moby",
        "title": "Moby Dick",
        "author": "Herman Melville",
        "published_year": 1851
    }
}
```

--------------------------------

### List Collections in a Project (Python)

Source: https://docs.topk.io/collections/list

Demonstrates how to list all collections within a project using the TopK.io client. This function returns a list of Collection objects.

```Python
collections = client.collections().list()
```

--------------------------------

### Create Collection with Keyword Index

Source: https://docs.topk.io/concepts/keyword-search

Defines a collection named 'books' with 'title' and 'description' fields, both indexed for keyword search using TopK's SDK. This setup is necessary before performing keyword searches.

```Python
from topk_sdk.schema import text, keyword_index

client.collections().create(
    "books",
    schema={
        "title": text().index(keyword_index()),
        "description": text().index(keyword_index()),
    },
)
```

```Javascript
client.collections().create(
    "books",
    schema={
        "title": text().index(keyword_index()),
        "description": text().index(keyword_index()),
    },
)
```

--------------------------------

### Starts With Operator (starts_with)

Source: https://docs.topk.io/documents/query

The 'starts_with' operator matches documents where a string field begins with a specified prefix. Useful for scoping queries, e.g., by tenant ID.

```Python
.filter(
    field("_id").starts_with("tenant_123/")
)

```

```Javascript
.filter(
    field("_id").starts_with("tenant_123/")
)

```

--------------------------------

### Implement Complex Search with Filtering and Reranking (Python)

Source: https://docs.topk.io/concepts/true-hybrid-search

This Python example shows a complex search query on a 'books' collection. It combines keyword search (BM25), semantic similarity on the 'title' field, metadata filtering by 'published_year', and a custom weighted top-k selection followed by reranking.

```Python
from topk_sdk.query import select, field, fn, match

docs = client.collection("books").query(
    select(
        "title",
        # Score documents using BM25 algorithm
        text_score=fn.bm25_score(),
        # Compute semantic similarity between the provided query and the `title` field.
        title_similarity=fn.semantic_similarity("title", "catcher"),
    )
    # Filter documents that contain the `great` keyword
    .filter(match("great"))
    # Filtering by metadata
    .filter(field("published_year") > 1980)
    # Return top 10 documents with the highest combined score
    .topk(field("text_score") * 0.2 + field("title_similarity") * 0.8, 10)
    # Rerank the documents
    .rerank()
)
```

--------------------------------

### Get Collection by Name (Javascript)

Source: https://docs.topk.io/collections/get

Retrieves a specific collection from a project by its name using the TopK.io client. The `get()` function takes the collection's name as an argument and returns a Collection object.

```Javascript
collection = client.collections().get("books")
```

--------------------------------

### Get Collection Details (Javascript)

Source: https://docs.topk.io/collections/list

Iterates through a list of Collection objects and prints their details, including name, organization ID, project ID, region, and schema.

```Javascript
collections.forEach(collection => {
    console.log(`Collection name: ${collection.name}`);
    console.log(`Organization ID: ${collection.org_id}`);
    console.log(`Project ID: ${collection.project_id}`);
    console.log(`Region: ${collection.region}`);
    console.log(`Schema: ${JSON.stringify(collection.schema)}`);
});

```

--------------------------------

### Advanced Field Selection with Expressions

Source: https://docs.topk.io/documents/query

Demonstrates selecting fields using TopK's expression language, allowing chaining of logical and mathematical operations. Examples include multiplying a field, checking age conditions, and range comparisons for publication years.

```Python
from topk_sdk.query import select, field

select(
  weight_in_grams=field("weight").mul(1000),
  is_adult=field("age").gt(18),
  published_in_nineteenth_century=field("published_year") >= 1800 & field("published_year") < 1900,
)

```

--------------------------------

### Get Collection by Name (Python)

Source: https://docs.topk.io/collections/get

Retrieves a specific collection from a project by its name using the TopK.io client. The `get()` function takes the collection's name as an argument and returns a Collection object.

```Python
collection = client.collections().get("books")
```

--------------------------------

### Get Collection Details (Python)

Source: https://docs.topk.io/collections/list

Iterates through a list of Collection objects and prints their details, including name, organization ID, project ID, region, and schema.

```Python
for collection in collections:
    print(f"Collection name: {collection.name}")
    print(f"Organization ID: {collection.org_id}")
    print(f"Project ID: {collection.project_id}")
    print(f"Region: {collection.region}")
    print(f"Schema: {collection.schema}")
```

--------------------------------

### Insert Document with All Supported Types in TopK

Source: https://docs.topk.io/documents/upsert

Provides an example of inserting a single document into the 'books' collection, showcasing the usage of all supported data types, including various vector formats and lists, with their corresponding TopK data constructors.

```Python
from topk_sdk.data import f32_vector, u8_vector, binary_vector, f32_sparse_vector, u8_sparse_vector, bytes

client.collection("books").upsert([
  {
    "_id": "1",
    "title": "The Great Gatsby",
    "published_year": 1925,
    "price": 10.99,
    "is_published": true,
    "f32_embedding": f32_vector([0.12, 0.67, 0.82, 0.53]),
    "u8_embedding": u8_vector([0, 1, 2, 3]),
    "binary_embedding": binary_vector([0, 1, 1, 0]),
    "f32_sparse_embedding": f32_sparse_vector({0: 0.12, 6: 0.67, 17: 0.82, 97: 0.53}),
    "u8_sparse_embedding": u8_sparse_vector({0: 12, 6: 67, 17: 82, 97: 53}),
    "bytes": bytes([0, 1, 1, 0]),
    "tags": ["dream", "illusion", "desire"],
  }
])
```

```Javascript
client.collection("books").upsert([
  {
    "_id": "1",
    "title": "The Great Gatsby",
    "published_year": 1925,
    "price": 10.99,
    "is_published": true,
    "f32_embedding": f32_vector([0.12, 0.67, 0.82, 0.53]),
    "u8_embedding": u8_vector([0, 1, 2, 3]),
    "binary_embedding": binary_vector([0, 1, 1, 0]),
    "f32_sparse_embedding": f32_sparse_vector({0: 0.12, 6: 0.67, 17: 0.82, 97: 0.53}),
    "u8_sparse_embedding": u8_sparse_vector({0: 12, 6: 67, 17: 82, 97: 53}),
    "bytes": bytes([0, 1, 1, 0]),
    "tags": ["dream", "illusion", "desire"],
  }
])
```

--------------------------------

### Upsert Document and Query with LSN (Python)

Source: https://docs.topk.io/documents/query

Demonstrates how to upsert a document and then query it using the obtained LSN to ensure strong consistency. This involves calling the upsert method to get the LSN and then passing it to the query method.

```Python
# Upsert a document and get the LSN
lsn = client.collection("books").upsert([
    {"_id": "1984", "title": "1984", "author": "George Orwell", "year": 1949}
])

# Query with LSN to ensure consistency
results = client.collection("books").query(
    select("title", "author", "year")
    .filter(field("author") == "George Orwell")
    .topk(field("year"), 10),
    lsn=lsn
)

```

--------------------------------

### Define Unsigned 8-bit Vector Field

Source: https://docs.topk.io/collections/create

Provides an example of defining an unsigned 8-bit vector field with a specified dimension using `u8_vector()`.

```Python
from topk_sdk.schema import u8_vector

"title_embedding": u8_vector(dimension=1536)
```

--------------------------------

### Define Collection with Keyword and Semantic Indexes (Python)

Source: https://docs.topk.io/concepts/true-hybrid-search

This snippet demonstrates how to create a collection named 'articles' with a 'title' field indexed for keyword retrieval and a 'content' field indexed for semantic search using TopK's SDK.

```Python
from topk_sdk.schema import text, keyword_index, semantic_index

client.collections().create(
    "articles",
    schema={
        "title": text().required().index(keyword_index()),  # Keyword-based retrieval
        "content": text().index(semantic_index()),  # Semantic search
    },
)
```

--------------------------------

### Applying reranking to improve search results

Source: https://docs.topk.io/concepts/reranking

Shows how to integrate the rerank() function into a search query to boost the relevance of results. This example reranks documents based on the 'content' field using the query 'how to reset a router'.

```Python
client.collection(name).query(
    select(
        "title",
        "content",
        title_score=fn.bm25_score(),
    )
    .filter(match("how to reset a router", field="title"))
    .topk(field("title_score"), 10)
    .rerank(
        fields=["content"],
        query="how to reset a router"
    )
)
```

--------------------------------

### Define Boolean Field

Source: https://docs.topk.io/collections/create

Provides an example of defining a boolean field in a TopK collection schema using the `bool()` function.

```Python
from topk_sdk.schema import bool

"is_published": bool()
```

--------------------------------

### Access Collection Properties (Javascript)

Source: https://docs.topk.io/collections/get

Prints various properties of a retrieved Collection object, including its name, organization ID, project ID, region, and schema. This demonstrates how to access the details of a collection after fetching it.

```Javascript
console.log(`Collection name: ${collection.name}`);
console.log(`Organization ID: ${collection.org_id}`);
console.log(`Project ID: ${collection.project_id}`);
console.log(`Region: ${collection.region}`);
console.log(`Schema: ${collection.schema}`);
```

--------------------------------

### Access Collection Properties (Python)

Source: https://docs.topk.io/collections/get

Prints various properties of a retrieved Collection object, including its name, organization ID, project ID, region, and schema. This demonstrates how to access the details of a collection after fetching it.

```Python
print(f"Collection name: {collection.name}")
print(f"Organization ID: {collection.org_id}")
print(f"Project ID: {collection.project_id}")
print(f"Region: {collection.region}")
print(f"Schema: {collection.schema}")
```

--------------------------------

### Combine Semantic and Keyword Search

Source: https://docs.topk.io/concepts/semantic-search

This example shows how to combine semantic search with keyword search in TopK. It uses `semantic_similarity()` for contextual meaning and `bm25_score()` for keyword relevance, with a custom scoring function for weighted results.

```Python
from topk_sdk.query import select, field, fn, match

docs = client.collection("books").query(
    select(
        "title",
        title_similarity=fn.semantic_similarity("title", "catcher"),
        text_score=fn.bm25_score(),  # Keyword-based relevance
    )
    .filter(match("classic"))  # Ensure the book contains the keyword "classic" in any of the text-indexed fields
    .topk(field("title_similarity") * 0.7 + field("text_score") * 0.3, 10) # Add 70% weight to semantic similarity and 30% weight to keyword relevance
    .rerank()
)

```

--------------------------------

### Get Top K Results with topk Function

Source: https://docs.topk.io/documents/query

The `topk()` function returns the top `k` results based on a logical expression and sort order. It requires `k` (number of results) and `asc` (boolean for ascending order) as parameters.

```Python
.topk(field("title_similarity"), 10, asc=False)
```

```Javascript
.topk(field("title_similarity"), 10, asc=False)
```

--------------------------------

### Get Minimum Value with min Operator

Source: https://docs.topk.io/documents/query

The `min` operator returns the smaller of two values, used for clamping or setting upper bounds. It works with scalar values, fields, or expressions and uses lexicographic order for strings. Requires importing `min`.

```Python
from topk_sdk.query import min

# Clamp BM25 scores to a maximum of 2.0
select(
    clamped_score=min(field("bm25_score"), 2.0)
)

# Take the lower of critic score vs user rating
select(
    conservative_score=min(field("critic_score"), field("user_rating"))
)
```

--------------------------------

### Initialize TopK Client

Source: https://docs.topk.io/installation

Initializes the TopK client with the provided API key and region. The client is essential for interacting with TopK services.

```python
from topk_sdk import Client

client = Client(api_key="YOUR_TOPK_API_KEY", region="aws-us-east-1-elastica")
```

--------------------------------

### Create TopK Collection with Schema

Source: https://docs.topk.io/collections/create

Demonstrates how to create a TopK collection named 'books' with a defined schema including text, vector, and integer fields, along with indexing configurations.

```Python
from topk_sdk.schema import int, text, f32_vector, vector_index, keyword_index

client.collections().create(
    "books",
    schema={
        "title": text().required().index(keyword_index()),
        "title_embedding": f32_vector(dimension=1536)
            .required()
            .index(vector_index(metric="euclidean")),
        "published_year": int().required(),
    },
)
```

--------------------------------

### Get Documents by IDs

Source: https://docs.topk.io/documents/get

Retrieves multiple documents from a collection by providing a list of their IDs to the `get()` function. The function returns a key-value mapping where keys are document IDs and values are the corresponding document objects.

```Python
docs = ctx.client.collection("books").get(["lotr", "moby"])
```

```Javascript
const docs = ctx.client.collection("books").get(["lotr", "moby"])
```

--------------------------------

### Create TopK Collection with All Supported Types

Source: https://docs.topk.io/documents/upsert

Shows how to create a new collection named 'books' with a schema that includes all supported TopK data types, such as text, integers, floats, booleans, various vector types, bytes, and lists. It also demonstrates setting up indexes for these fields.

```Python
from topk_sdk.schema import (
    int, text, float, bool, f32_vector, u8_vector, binary_vector, f32_sparse_vector, u8_sparse_vector, bytes, list
)

client.collections().create(
    "books",
    schema={
        "title": text(),
        "published_year": int(),
        "price": float(),
        "is_published": bool(),
        "f32_embedding": f32_vector(dimension=1536).index(vector_index(metric="cosine")),
        "u8_embedding": u8_vector(dimension=1536).index(vector_index(metric="euclidean")),
        "binary_embedding": binary_vector(dimension=1536).index(vector_index(metric="hamming")),
        "f32_sparse_embedding": f32_sparse_vector().index(vector_index(metric="dot_product")),
        "u8_sparse_embedding": u8_sparse_vector().index(vector_index(metric="dot_product")),
        "bytes": bytes(),
        "tags": list(value_type="text").index(keyword_index()),
    },
)
```

```Javascript
client.collections().create(
    "books",
    schema={
        "title": text(),
        "published_year": int(),
        "price": float(),
        "is_published": bool(),
        "f32_embedding": f32_vector(dimension=1536).index(vector_index(metric="cosine")),
        "u8_embedding": u8_vector(dimension=1536).index(vector_index(metric="euclidean")),
        "binary_embedding": binary_vector(dimension=1536).index(vector_index(metric="hamming")),
        "f32_sparse_embedding": f32_sparse_vector().index(vector_index(metric="dot_product")),
        "u8_sparse_embedding": u8_sparse_vector().index(vector_index(metric="dot_product")),
        "bytes": bytes(),
        "tags": list(value_type="text").index(keyword_index()),
    },
)
```

--------------------------------

### Configure Vite for SSR with topk-js

Source: https://docs.topk.io/installation

Configures Vite to treat 'topk-js' as an external dependency for server-side rendering (SSR) environments like Remix, Nuxt, SvelteKit, and Astro. This prevents build-time issues.

```typescript
import { defineConfig } from 'vite';

export default defineConfig({
  // ...
  ssr: {
    external: ["topk-js"],
  },
  optimizeDeps: {
    exclude: ["topk-js"],
  },
});
```

--------------------------------

### Query Documents by Tenant ID Prefix (Python)

Source: https://docs.topk.io/concepts/multi-tenancy

Shows how to query documents within a specific tenant's scope using the startsWith() filter on the _id field. This allows retrieval of all documents belonging to a particular tenant.

```Python
client.collection("books").query(
    select(
        "title",
    )
    .filter(field("_id").startsWith("tenant-1234"))
)
```

--------------------------------

### Configure Next.js for SSR with topk-js

Source: https://docs.topk.io/installation

Configures Next.js to mark 'topk-js' as an external package for server-side rendering (SSR), particularly for projects using the App Router and Server Components. This ensures compatibility and avoids bundling issues.

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  /* config options here */
  serverExternalPackages: ["topk-js"],
};

export default nextConfig;
```

--------------------------------

### Initialize Select Stage in Python

Source: https://docs.topk.io/documents/query

Initializes the select stage of a TopK query in Python, allowing selection of static or computed fields. It uses the `select()` and `field()` functions from the `topk_sdk.query` module.

```Python
from topk_sdk.query import select, field

client.collection("books").query(
  select(
    "published_year", # elect the static fields directly
    title=field("title"),
  )
  ...
)
```

--------------------------------

### Rerank Query Results

Source: https://docs.topk.io/documents/query

The `rerank()` function is used to rerank the results of a query. Further details can be found in the reranking guide.

```Python
.rerank()
```

```Javascript
.rerank()
```

--------------------------------

### Filter Documents by Metadata

Source: https://docs.topk.io/documents/query

Filters documents based on metadata fields, such as 'published_year'. This example shows how to select documents published after a specific year using a comparison operator.

```Python
.filter(
    field("published_year") > 1980
)

```

--------------------------------

### Using the rerank() function in TopK

Source: https://docs.topk.io/concepts/reranking

Demonstrates how to call the rerank() function with various parameters to improve search result relevance. It shows both a basic call and a more detailed call specifying model, query, fields, and topk_multiple.

```Python
.rerank()

# or

.rerank(
    model="cohere/rerank-v3.5",
    query="catcher",
    fields=["title", "description"],
    topk_multiple=2
)
```

```Javascript
.rerank()

# or

.rerank(
    model="cohere/rerank-v3.5",
    query="catcher",
    fields=["title", "description"],
    topk_multiple=2
)
```

--------------------------------

### Get Absolute Value with abs Operator

Source: https://docs.topk.io/documents/query

The `abs` operator returns the absolute value of a number, useful for calculating distances or differences. It requires importing `abs` from `topk_sdk.query`.

```Python
from topk_sdk.query import abs

# Find books published closest to 1990
select(
    delta=abs(field("published_year").sub(1990))
)
```

--------------------------------

### Select Fields and Create Computed Fields in Python

Source: https://docs.topk.io/documents/query

Demonstrates selecting fields, renaming them, and creating computed fields using arithmetic operations within the `select()` function in Python. This allows for flexible data extraction and transformation.

```Python
from topk_sdk.query import select, field

docs = client.collection("books").query(
    select(
        "title", # the actual "title" field from the document
        year=field("published_year"), # renamed field
        year_plus_ten=field("published_year") + 10, # computed field
    )
)
```

--------------------------------

### TopK Balanced Consistency Query (Python/Javascript)

Source: https://docs.topk.io/concepts/consistency

Perform a query using the default Balanced Consistency mode in TopK. This mode offers a balance between data freshness and performance, with a small delay for recent writes.

```Python
client.collection("my_collection").query(
    query,
    # no need to specify consistency mode
)
```

```Javascript
client.collection("my_collection").query(
    query,
    // no need to specify consistency mode
)
```

--------------------------------

### Query with Custom Scoring Function (Python)

Source: https://docs.topk.io/concepts/true-hybrid-search

This Python snippet shows how to query documents by combining semantic similarity with a precomputed importance score. The topk function applies weights to these scores to rank the results.

```Python
from topk_sdk.query import select, field, fn

docs = client.collection("documents").query(
    select(
        content_score=fn.semantic_similarity("content", "machine learning applications"),
    )
    .topk(0.8 * field("importance") + 0.2 * field("content_score"), 10)
)

```

--------------------------------

### TopK Indexed Consistency Query (Python/Javascript)

Source: https://docs.topk.io/concepts/consistency

Execute a query with Indexed Consistency in TopK. This mode prioritizes query performance by only considering fully compacted files, potentially skipping the latest writes.

```Python
client.collection("my_collection").query(
    ..., # query
    consistency="indexed",
)
```

```Javascript
client.collection("my_collection").query(
    ..., // query
    consistency="indexed",
)
```

--------------------------------

### Pass i64_list in Python

Source: https://docs.topk.io/documents/data-constructors

Use the `i64_list()` helper function to pass a list of 64-bit signed integers.

```Python
from topk_sdk.data import i64_list

i64_list([0, 1, 2, 3])
```

--------------------------------

### TopK Strong Consistency Query (Python/Javascript)

Source: https://docs.topk.io/concepts/consistency

Make a query with Strong Consistency in TopK. This mode guarantees that all queries reflect the latest writes, ensuring maximum data freshness at the cost of higher latency and expense.

```Python
client.collection("my_collection").query(
    ..., # query
    consistency="strong",
)
```

```Javascript
client.collection("my_collection").query(
    ..., // query
    consistency="strong",
)
```

--------------------------------

### Creating a collection and upserting documents

Source: https://docs.topk.io/concepts/reranking

Provides Python code to create a new collection named 'documents' with 'title' and 'content' fields, both indexed for semantic search. It then demonstrates how to upsert sample documents into this collection.

```Python
client.collections().create(
    "documents",
    schema={
        "title": text().index(semantic_index()).required(),
        "content": text().index(semantic_index()).required(),
    }
)

client.collection(name).upsert([
    {
        "_id": "1",
        "title": "How to reset router settings to fix intermittent connectivity",
        "content": "To fix intermittent connectivity issues related to your device, you need to unplug your modem or router for 10 seconds, then plug them back in. This will cause a reset of the network hardware and often resolves temporary disruptions.",
    },
    {
        "_id": "2",
        "title": "How to reset your device",
        "content": "Factory reset procedures vary by device. Press and hold the reset button for 10 seconds. This clears all settings, returning the router to default configuration.",
    }
])
```

--------------------------------

### Pass i32_list in Python

Source: https://docs.topk.io/documents/data-constructors

Use the `i32_list()` helper function to pass a list of 32-bit signed integers.

```Python
from topk_sdk.data import i32_list

i32_list([0, 1, 2, 3])
```

--------------------------------

### Define Collection with Custom Scoring Fields (Python)

Source: https://docs.topk.io/concepts/true-hybrid-search

This Python snippet illustrates how to create a collection named 'documents' that includes a 'content' field for semantic search and an 'importance' field of type float, intended for use in custom scoring functions.

```Python
from topk_sdk.schema import text, float, semantic_index

client.collections().create(
    "documents",
    schema={
        "content": text().index(semantic_index()),  # Semantic search
        "importance": float().required(),  # Precomputed importance score
    },
)
```

--------------------------------

### Combine Sparse Vector Search with Metadata Filtering (Python)

Source: https://docs.topk.io/concepts/sparse-vector-search

This example demonstrates how to combine sparse vector search with metadata filtering in TopK. It adds a `filter` stage to the query to narrow down results based on a specific metadata field, in this case, `published_year`.

```Python
from topk_sdk.query import select, field, fn
from topk_sdk.data import f32_sparse_vector

docs = client.collection("books").query(
    select(
        "title",
        title_score=fn.vector_distance(
          "title_embedding",
          f32_sparse_vector({0: 0.12, 6: 0.67, ...}),
        )
        published_year=field("published_year"),
    )
    .filter(field("published_year") > 2000)
    .topk(field("title_score"), 10)
)
```

--------------------------------

### Create Keyword Index

Source: https://docs.topk.io/collections/create

The keyword_index() function creates a keyword index on a text field, allowing for keyword searches. This is typically used in conjunction with the text() data constructor.

```Python
from topk_sdk.schema import text, keyword_index

"title": text().index(keyword_index())
```

--------------------------------

### Store Documents with Tenant ID Prefix (Python)

Source: https://docs.topk.io/concepts/multi-tenancy

Demonstrates how to store documents in a TopK collection by prepending the tenant ID to the document's _id. This method is used to isolate data for specific tenants.

```Python
client.collection("books").upsert([
    {"_id": "tenant-1234_1", "title": "The Great Gatsby", "author": "F. Scott Fitzgerald"},
    {"_id": "tenant-1234_2", "title": "To Kill a Mockingbird", "author": "Harper Lee"},
])
```

--------------------------------

### Perform Hybrid Search with Keyword and Semantic Scoring (Python)

Source: https://docs.topk.io/concepts/true-hybrid-search

This Python code performs a hybrid search on the 'articles' collection. It combines semantic similarity from the 'content' field with BM25 keyword scores, applying a filter for specific keywords and a weighted top-k selection.

```Python
from topk_sdk.query import select, field, fn, match

docs = client.collection("articles").query(
    select(
        "title",
        content_similarity=fn.semantic_similarity("content", "climate change policies"),
        text_score=fn.bm25_score(),
    )
    .filter(match("carbon") | match("renewable energy"))  # Ensure keyword relevance
    .topk(field("content_similarity") * 0.6 + field("text_score") * 0.4, 10)
)
```

--------------------------------

### Get Maximum Value with max Operator

Source: https://docs.topk.io/documents/query

The `max` operator returns the larger of two values, used for clamping or setting lower bounds. It supports scalar values, fields, or expressions and uses lexicographic order for strings. Requires importing `max`.

```Python
from topk_sdk.query import max

# Ensure minimum relevance score of 1.5
select(
    boosted_score=max(field("relevance_score"), 1.5)
)

# Take the higher of critic score vs user rating
select(
    best_score=max(field("critic_score"), field("user_rating"))
)
```

--------------------------------

### Evaluate all expressions in an array with 'all'

Source: https://docs.topk.io/documents/query

The 'all()' helper evaluates to true if each expression in the array is true, equivalent to a logical AND across all expressions.

```Python
from topk_sdk.query import all, field

.filter(
    all([
        field("published_year") >= 1900,
        field("published_year") <= 2000,
        field("title").is_not_null()
    ])
)

# This is equivalent to:
# field("published_year") >= 1900 & field("published_year") <= 2000 & field("title").is_not_null()
```

```Javascript
// Javascript equivalent would follow a similar pattern using the SDK's methods
```

--------------------------------

### Pass f64_list in Python

Source: https://docs.topk.io/documents/data-constructors

Use the `f64_list()` helper function to pass a list of 64-bit floating-point numbers.

```Python
from topk_sdk.data import f64_list

f64_list([0.12, 0.67, 0.82, 0.53])
```

--------------------------------

### Conditional selection with 'choose' operator

Source: https://docs.topk.io/documents/query

The 'choose' operator evaluates a condition and returns the first argument if the condition is true, otherwise it returns the second argument.

```Python
select(
  summary=(field("book_type") == "fiction").choose(
      field("plot_summary"),
      field("technical_summary")
  )
)
```

```Javascript
// Javascript equivalent would follow a similar pattern using the SDK's methods
```

--------------------------------

### Match Multiple Keywords with 'all' Parameter

Source: https://docs.topk.io/documents/query

Configures the 'match()' function to require all specified terms to be present in the search. Uses a delimiter (non-alphanumeric character) to separate terms and the 'all=True' parameter to enforce an AND condition.

```Python
from topk_sdk.query import match

.filter(match("catcher|rye", field="title", all=True))

```

--------------------------------

### Pass u32_list in Python

Source: https://docs.topk.io/documents/data-constructors

Use the `u32_list()` helper function to pass a list of 32-bit unsigned integers.

```Python
from topk_sdk.data import u32_list

u32_list([0, 1, 2, 3])
```

--------------------------------

### Perform Keyword Search with BM25

Source: https://docs.topk.io/concepts/keyword-search

Executes a keyword search on the 'books' collection, filtering for documents matching 'great' in the 'title' or 'novel' in any keyword-indexed field. It selects the title, description, and BM25 text score, returning the top 10 results.

```Python
from topk_sdk.query import select, fn, match, field

docs = client.collection("books").query(
    select(
        "title",
        "description",
        # Score documents using BM25 algorithm
        text_score=fn.bm25_score(),
    )
    # Filter documents that have the `great` keyword in the `title` field
    # or the `novel` in any of the text-indexed fields.
    .filter(
        match("great", field="title") | match("novel")
    )
    # Return top 10 documents with the highest text score
    .topk(field("text_score"), 10)
)
```

```Javascript
client.collection("books").query(
    select(
        "title",
        "description",
        # Score documents using BM25 algorithm
        text_score=fn.bm25_score(),
    )
    # Filter documents that have the `great` keyword in the `title` field
    # or the `novel` in any of the text-indexed fields.
    .filter(
        match("great", field="title") | match("novel")
    )
    # Return top 10 documents with the highest text score
    .topk(field("text_score"), 10)
)
```

--------------------------------

### Define Binary Vector Field

Source: https://docs.topk.io/collections/create

Shows how to define a binary vector field, specifying the dimension in bytes, using `binary_vector()`.

```Python
from topk_sdk.schema import binary_vector

"title_embedding": binary_vector(dimension=128)
```

--------------------------------

### Perform Semantic Search Query

Source: https://docs.topk.io/concepts/semantic-search

This snippet illustrates how to perform a semantic search query in TopK. It uses the `semantic_similarity()` function to find documents semantically similar to a given query and ranks them using `.topk()` and `.rerank()`.

```Python
from topk_sdk.query import select, field, fn

docs = client.collection("books").query(
    select(
        "title",
        title_similarity=fn.semantic_similarity("title", "classic American novel"),
    )
    .topk(field("title_similarity"), 10)
    .rerank()
)

```

--------------------------------

### Performing lexical search with BM25 scoring

Source: https://docs.topk.io/concepts/reranking

Illustrates how to perform a lexical search using the BM25 scoring method on the 'title' field. The query searches for 'how to reset a router' and retrieves the top 10 results based on the calculated title score.

```Python
client.collection(name).query(
    select(
        "title",
        "content",
        title_score=fn.bm25_score()
    )
    .filter(match("how to reset a router", field="title"))
    .topk(field("title_score"), 10)
)
```

--------------------------------

### Combine Keyword Search and Metadata Filtering

Source: https://docs.topk.io/documents/query

Demonstrates combining keyword searches with metadata filtering in a single query. Multiple filter stages can be chained to apply complex criteria, such as matching a keyword and filtering by publication year ranges or specific years.

```Python
.filter(
    match("catcher")
)
.filter(
    field("published_year") == 1997 | (field("published_year") >= 1920 & field("published_year") <= 1980)
)

```

--------------------------------

### Pass bytes in Python

Source: https://docs.topk.io/documents/data-constructors

Use the `bytes()` helper function to pass a byte object. This function takes a list of byte values.

```Python
from topk_sdk.data import bytes

bytes([0, 1, 1, 0])
```

--------------------------------

### Keyword Search with Match Function

Source: https://docs.topk.io/documents/query

Performs keyword searches using the 'match()' function. This function can search for single terms, multiple terms (with delimiters), on specific fields, and with custom weights.

```Python
from topk_sdk.query import match

.filter(
    match("catcher")
)

```

--------------------------------

### Calculate Exponential Function with exp Operator

Source: https://docs.topk.io/documents/query

The `exp` operator calculates e raised to the power of the input (e^x), useful for exponential scaling and boosting. It's applied using the `.exp()` method.

```Python
# Apply exponential boosting to BM25 scores
select(
    boosted_score=(field("bm25_score") * 1.5).exp()
)
```

```Javascript
# Apply exponential boosting to BM25 scores
select(
    boosted_score=(field("bm25_score") * 1.5).exp()
)
```
Tokens:
Raw