Tantivy Py

https://tantivy-py.readthedocs.io/
Admin
tantivy-py is a Python wrapper and binding for the Tantivy full-text search engine library,...
Tokens:33,612
Snippets:257
Trust Score:9.6
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Setup: Create test index and add documents

Source: https://tantivy-py.readthedocs.io/en/latest/reference.html

Creates an in-memory index with an integer field 'doc_id' and text fields 'title' and 'body', adds two documents, commits, and reloads. Required before running the query examples.

```python
import os
from tantivy import SchemaBuilder, Index, Document
schema = (
    SchemaBuilder()
        .add_integer_field("doc_id", indexed=True, stored=True)
        .add_text_field("title", stored=True)
        .add_text_field("body")
        .build()
)
index = Index(schema=schema, path=None)
writer = index.writer(heap_size=15_000_000, num_threads=1)
doc = Document()
doc.add_integer("doc_id", 1)
doc.add_text("title", "The Old Man and the Sea")
doc.add_text(
    "body",
    (
        "He was an old man who fished alone in a skiff in"
        "the Gulf Stream and he had gone eighty-four days "
        "now without taking a fish."
    ),
)
writer.add_document(doc)

doc = Document()
doc.add_integer("doc_id", 2)
doc.add_text("title", "The Old Man and the Sea II")
doc.add_text("body", "He was an old man who sailed alone.")

writer.add_document(doc)
writer.commit()
index.reload()

```

--------------------------------

### Install dev dependencies for documentation

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Install all optional dev dependencies, including MkDocs, into your virtual environment.

```bash
pip install .[dev]
```

--------------------------------

### Install tantivy-py with pip

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Install tantivy-py from PyPI using pip. If no binary wheel is available, the bindings will be built from source, requiring Rust to be installed.

```bash
pip install tantivy
```

--------------------------------

### Install Rust toolchain with rustup

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Install the Rust toolchain via rustup, required for building tantivy-py from source.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

--------------------------------

### Set up development environment with nox

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Install nox and build the bindings in a virtual environment.

```bash
python3 -m pip install nox
nox
```

--------------------------------

### Example JSON output from explain()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Example output from the explain() method's to_json() call. Shows the final score value, a description, and details of each contributing term with its weight and boost.

```json
{
  "value": 2.5,
  "description": "sum of:",
  "details": [
    {
      "value": 2.0,
      "description": "weight(title:fish) with boost 2.0"
    },
    {
      "value": 0.5,
      "description": "weight(body:days)"
    }
  ]
}

```

--------------------------------

### Build a Tantivy schema with SchemaBuilder

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

These three code blocks are displayed sequentially in the documentation as a single example of building a Tantivy schema. The first block creates a SchemaBuilder, the second adds two text fields (title and body), and the third builds the schema. The example is part of the SchemaBuilder class documentation.

```python
>>> builder = tantivy.SchemaBuilder()
```

```python
>>> title = builder.add_text_field("title", stored=True)
>>> body = builder.add_text_field("body")
```

```python
>>> schema = builder.build()
```

--------------------------------

### Index and search documents with tantivy

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Shows the full workflow of building a schema, creating an index, adding documents, committing, and searching. The example uses a text field with stored=True and a non-stored body field, then parses a JSON document and asserts that the search for "sea whale" returns exactly one result.

```python
>>> import json
>>> import tantivy
>>> builder = tantivy.SchemaBuilder()
>>> title = builder.add_text_field("title", stored=True)
>>> body = builder.add_text_field("body")
>>> schema = builder.build()
>>> index = tantivy.Index(schema)
>>> doc = tantivy.Document()
>>> doc.add_text(title, "The Old Man and the Sea")
>>> doc.add_text(body, ("He was an old man who fished alone in a "
                        "skiff in the Gulf Stream and he had gone "
                        "eighty-four days now without taking a fish."))
>>> writer.add_document(doc)
>>> doc = schema.parse_document(json.dumps({
       "title": ["Frankenstein", "The Modern Prometheus"],
       "body": ("You will rejoice to hear that no disaster has "
                "accompanied the commencement of an enterprise which "
                "you have regarded with such evil forebodings.  "
                "I arrived here yesterday, and my first task is to "
                "assure my dear sister of my welfare and increasing "
                "confidence in the success of my undertaking.")
}))
>>> writer.add_document(doc)
>>> writer.commit()
>>> reader = index.reader()
>>> searcher = reader.searcher()
>>> query = index.parse_query("sea whale", [title, body])
>>> result = searcher.search(query, 10)
>>> assert len(result) == 1
```

--------------------------------

### Doctest example for Tantivy Document

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Example of a doctest in a Rust docstring for the Tantivy Document struct. The doctest creates a document, adds text fields, and prints the representation, which is compared to the expected output.

```rust
/// Tantivy's Document is the object that can be indexed and then searched for.
///
/// Documents are fundamentally a collection of unordered tuples
/// (field_name, value). In this list, one field may appear more than once.
///
/// Example:
///     >>> doc = tantivy.Document()
///     >>> doc.add_text("title", "The Old Man and the Sea")
///     >>> doc.add_text("body", ("He was an old man who fished alone in a "
///     ...                       "skiff in the Gulf Stream and he had gone "
///     ...                       "eighty-four days now without taking a fish."))
///     >>> doc
///     Document(body=[He was an ],title=[The Old Ma])
///
#[pyclass(module = "tantivy")]
#[derive(Clone, Default, PartialEq)]
pub(crate) struct Document {
    pub(crate) field_values: BTreeMap<String, Vec<Value>>,
}
```

--------------------------------

### Build custom grouping with separate queries

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Shows how to achieve custom grouping by building groups as separate queries first and then combining them. This example creates (man OR sea) and then ANDs it with old.

```python
# old AND (man OR sea)
man_or_sea = query_man.or_should_match(query_sea)
combined = query_old.and_must_match(man_or_sea)
assert len(searcher.search(combined, 3).hits) > 0
```

--------------------------------

### Build a TextAnalyzer with TextAnalyzerBuilder

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Builds a TextAnalyzer by chaining filters onto a TextAnalyzerBuilder initialized with a simple tokenizer. The example applies lowercase and ngram filters before calling build().

```python
my_analyzer: TextAnalyzer = (
    TextAnalyzerBuilder(Tokenizer.simple())
    .filter(Filter.lowercase())
    .filter(Filter.ngram())
    .build()
)
```

--------------------------------

### Get highlighted ranges with highlighted()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Use the highlighted() method to get ranges of matched terms. The offsets are relative to the fragment, not the original document, so use the fragment for slicing. The example asserts the first highlight corresponds to 'days'.

```python
highlights = snippet.highlighted()
first_highlight = highlights[0]
assert first_highlight.start == 93
assert first_highlight.end == 97
assert fragment[first_highlight.start:first_highlight.end] == "days"

```

--------------------------------

### get_all

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Gets all values associated with the given field.

```APIDOC
## get_all(field_name: str) -> list[typing.Any]

### Description
Gets all values associated with the given field.

### Method
get_all

### Parameters
- **field_name** (str) - Required - The field for which we would like to get the values.

### Returns
- **list[Any]** - A list of values. The type of the value depends on the field.
```

--------------------------------

### Add a document using writer as context manager

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Alternative to the previous example using the writer as a context manager. Both commit() and wait_merging_threads() are called when the with-block is exited.

```python
with index.writer() as writer:
    writer.add_document(tantivy.Document(
        doc_id=1,
        title=["The Old Man and the Sea"],
        body=["""He was an old man who fished alone in a skiff in the Gulf Stream and he had gone eighty-four days now without taking a fish."""],
))

```

--------------------------------

### get_first

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Gets the first value associated with the given field.

```APIDOC
## get_first(field_name: str) -> Any | None

### Description
Gets the first value associated with the given field.

### Method
get_first

### Parameters
- **field_name** (str) - Required - The field for which we would like to get the value.

### Returns
- **Any | None** - The value if one is found, otherwise None. The type of the value depends on the field.
```

--------------------------------

### Combine queries with and_must_match, and_must_not_match, or_should_match

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Shows the three boolean helper methods: and_must_match (AND), and_must_not_match (AND NOT), and or_should_match (OR). Each returns a new Query, leaving the originals untouched. The example uses term queries on a "title" field and asserts search results.

```python
from tantivy import Query

query_old = Query.term_query(schema, "title", "old")
query_man = Query.term_query(schema, "title", "man")
query_whale = Query.term_query(schema, "title", "whale")

# Match documents whose title contains both "old" AND "man"
combined = query_old.and_must_match(query_man)
(score, doc_address) = searcher.search(combined, 3).hits[0]
assert searcher.doc(doc_address)["title"] == ["The Old Man and the Sea"]

# Match documents whose title contains "old" but NOT "whale"
combined = query_old.and_must_not_match(query_whale)
assert len(searcher.search(combined, 3).hits) > 0

# Match documents whose title contains "old" OR "whale"
combined = query_old.or_should_match(query_whale)
assert len(searcher.search(combined, 3).hits) > 0
```

--------------------------------

### Generate HTML snippet with to_html()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Generate a marked-up HTML snippet with to_html(). The matched terms are wrapped in <b> tags. The example asserts the exact HTML output.

```python
html_snippet = snippet.to_html()
assert html_snippet == (
    "He was an old man who fished alone in a skiff in the "
    "Gulf Stream and he had gone eighty-four <b>days</b> now "
    "without taking a <b>fish</b>"
)

```

--------------------------------

### Explain query scoring with explain() and to_json()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Use the explain() method on a query to get a JSON representation of the scoring details for a matched document. The to_json() method returns a pretty-printed JSON string showing the final score, breakdown, matched clauses, and term contributions.

```python
# Let's search with the complex_query built earlier and get the top result
result = searcher.search(complex_query, 10)
if result.hits:
    score, doc_address = result.hits[0]

    # Get an explanation for why this document matched
    explanation = complex_query.explain(searcher, doc_address)

    # The explanation provides a JSON representation of the scoring details
    explanation_json = explanation.to_json()
    print(explanation_json)

```

--------------------------------

### Set up development environment with Makefile

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Build the bindings using the provided Makefile.

```bash
make
```

--------------------------------

### Run tests with make test

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Run the test suite using the Makefile.

```bash
make test
```

--------------------------------

### Apply multiple queries with argument unpacking

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Demonstrates passing a list of queries to a boolean helper using argument unpacking (*queries). The comment notes it is equivalent to query_old.and_must_match(query_man, query_sea).

```python
query_sea = Query.term_query(schema, "title", "sea")

# Equivalent to query_old.and_must_match(query_man, query_sea)
queries = [query_man, query_sea]
combined = query_old.and_must_match(*queries)
assert len(searcher.search(combined, 3).hits) > 0
```

--------------------------------

### Reload index and get searcher

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Reloads the index to ensure it points to the last commit and obtains a searcher for querying.

```python
# Reload the index to ensure it points to the last commit.
index.reload()
searcher = index.searcher()

```

--------------------------------

### Create persistent index on disk with tantivy-py

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Shows how to create a persistent index by passing the path parameter to tantivy.Index. The snippet uses a temporary directory and creates the index directory before instantiating the index.

```python
tmpdir = tempfile.TemporaryDirectory()
index_path = pathlib.Path(tmpdir.name) / "index"
index_path.mkdir()
persistent_index = tantivy.Index(schema, path=str(index_path))

```

--------------------------------

### Get snippet fragment with fragment()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Retrieve the fragment of the document text from the snippet. The fragment is typically a window around the matched terms.

```python
fragment = snippet.fragment()

```

--------------------------------

### Build Document with constructor list values

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Shows building a Document by passing field values as lists directly to the constructor. This is a simpler alternative to using add_text.

```python
>>> doc = tantivy.Document(title=["The Old Man and the Sea"], body=["..."])
```

--------------------------------

### Declare schema and create in-memory index with tantivy-py

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Shows how to declare a schema with text and integer fields, build it, and create an in-memory index using tantivy.Index(schema). The snippet is the foundation for further index operations.

```python
import tempfile
import pathlib
import tantivy

# Declaring our schema.
schema_builder = tantivy.SchemaBuilder()
schema_builder.add_text_field("title", stored=True)
schema_builder.add_text_field("body", stored=True)
schema_builder.add_integer_field("doc_id",stored=True)
schema = schema_builder.build()

# Creating our index (in memory)
index = tantivy.Index(schema)

```

--------------------------------

### Indexing workflow with wait_merging_threads

Source: https://tantivy-py.readthedocs.io/en/latest/explanation.html

Pseudocode showing the steps to add documents to a tantivy index and wait for background merge threads to complete. Call writer.wait_merging_threads() as the final step after adding data; this consumes the writer and the identifier becomes unusable.

```text
schema = Schema(...)
index = Index(schema)
writer = index.writer()
for ... in data:
    document = Document(...)
    writer.add_document(...)
writer.commit()
writer.wait_merging_threads()

```

--------------------------------

### Create compound splitter with Filter.split_compounds

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Creates a compound splitter filter using the given list of compound words. The example notes that it will split 'butterfly' into ['butter', 'fly'] but will not split 'buttering' or 'buttercupfly'.

```python
compound_spliter = Filter.split_compounds(['butter', 'fly'])
```

--------------------------------

### explain

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Method to explain how a query matches a given document.

```APIDOC
## explain(searcher: Searcher, doc_address: DocAddress) -> Explanation

### Description
Explains how this query matches a given document.

### Parameters
- **searcher** (Searcher) - Required - The searcher used to perform the search.
- **doc_address** (DocAddress) - Required - The address of the document to explain.

### Returns
- **Explanation** - An object containing detailed information about how the document matched the query, with a `to_json()` method.
```

--------------------------------

### Create a Document with add_text

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Demonstrates creating a Document and adding text fields with add_text. The output shows the document's internal representation, which truncates long values in the display.

```python
>>> doc = tantivy.Document()
>>> doc.add_text("title", "The Old Man and the Sea")
>>> doc.add_text("body", ("He was an old man who fished alone in a "
...                       "skiff in the Gulf Stream and he had gone "
...                       "eighty-four days now without taking a fish."))
>>> doc
Document(body=[He was an ],title=[The Old Ma])
```

--------------------------------

### Escaping quotes: failing example

Source: https://tantivy-py.readthedocs.io/en/latest/reference.html

Attempts to parse the query 'sea\"' and expects a ValueError with message 'Syntax Error: sea\"'. This does not work because the query needs additional surrounding quotes.

```python
try:
    index.parse_query(r'sea\"', ["title", "body"])
except ValueError as e:
    assert str(e) == r'Syntax Error: sea\"'

```

--------------------------------

### from_dict

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Creates a Document from a dictionary.

```APIDOC
## from_dict(py_dict: dict, schema: Schema | None = None) -> Document

### Description
The type of the None singleton.
```

--------------------------------

### Escaping quotes: working example

Source: https://tantivy-py.readthedocs.io/en/latest/reference.html

Parses the query '"sea\""' successfully. Note that whether the escaped quote matches documents depends on the tokenizer; the default tokenizer discards punctuation.

```python
# Works!
index.parse_query(r'"sea\""', ["title", "body"])

```

--------------------------------

### ElasticSearch bool/dis_max query example

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

An ElasticSearch query that uses a bool query with a dis_max inside must. The dis_max contains two match queries: one on the title field with boost 2, and one on the body field with boost 1.5, and a tie_breaker of 0.3. This query cannot be expressed with the Tantivy query parser.

```json
{
    "query": {
        "bool": {
            "must": [
                {
                    "dis_max": {
                        "queries": [
                            {
                                "match": {
                                    "title": {
                                        "query": "fish",
                                        "boost": 2
                                    }
                                }
                            },
                            {
                                "match": {
                                    "body": {
                                        "query": "eighty-four days",
                                        "boost": 1.5
                                    }
                                }
                            }
                        ],
                        "tie_breaker": 0.3
                    }
                }
            ]
        }
    }
}
```

--------------------------------

### Test custom analyzer with .analyze()

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Tests the custom analyzer by calling its .analyze() method with sample text. The output is a list of tokens: ['this', 'website', 'might', 'exist'].

```python
# Will print: ['this', 'website', 'might', 'exist']
my_analyzer.analyze('www.this1website1might1exist.com')
```

--------------------------------

### Explanation.to_json

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Returns a JSON representation of the explanation.

```APIDOC
## to_json() -> str

### Description
Returns a JSON representation of the explanation.

### Method
to_json

### Returns
- **str** - A JSON string representation of the explanation.
```

--------------------------------

### Tokenizer

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Static factory methods for creating built-in Tantivy tokenizers.

```APIDOC
## def facet() -> Tokenizer

### Description
Creates a FacetTokenizer wrapper.

### Returns
- **Tokenizer** - A tokenizer for facets.

```

```APIDOC
## def ngram(min_gram: int = 2, max_gram: int = 3, prefix_only: bool = False) -> Tokenizer

### Description
Creates an NgramTokenizer with configurable n-gram lengths.

### Parameters
- **min_gram** (int) - Optional - Minimum character length of each ngram. Default: 2.
- **max_gram** (int) - Optional - Maximum character length of each ngram. Default: 3.
- **prefix_only** (bool) - Optional - If true, ngrams must count from the start of the word. Default: False.

### Returns
- **Tokenizer** - A tokenizer for n-grams.

```

```APIDOC
## def raw() -> Tokenizer

### Description
Creates a Raw Tokenizer wrapper.

### Returns
- **Tokenizer** - A raw tokenizer.

```

```APIDOC
## def regex(pattern: str) -> Tokenizer

### Description
Creates a Regex tokenizer using the given pattern.

### Parameters
- **pattern** (str) - Required - The regular expression pattern to use.

### Returns
- **Tokenizer** - A regex tokenizer.

```

```APIDOC
## def simple() -> Tokenizer

### Description
Creates a SimpleTokenizer wrapper.

### Returns
- **Tokenizer** - A simple tokenizer.

```

```APIDOC
## def whitespace() -> Tokenizer

### Description
Creates a WhitespaceTokenizer wrapper.

### Returns
- **Tokenizer** - A whitespace tokenizer.

```

--------------------------------

### Build Document with single constructor values

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Shows that a single value can be passed directly without a list, as syntactic sugar. This is legal when there is only one value for the field.

```python
>>> doc = tantivy.Document(title="The Old Man and the Sea", body="...")
```

--------------------------------

### Run specific tests with nox

Source: https://tantivy-py.readthedocs.io/en/latest/howto.html

Run only tests matching 'simple_search' on Python 3.11 using nox.

```bash
nox -s test-3.11 -- -k simple_search
```

--------------------------------

### Searcher.doc

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Fetches a document from Tantivy's store given a DocAddress.

```APIDOC
## Searcher.doc

### Description
Fetches a document from Tantivy's store given a DocAddress.

### Method
doc

### Signature
```python
def doc(self, doc_address: DocAddress) -> Document
```

### Parameters
- **doc_address** (DocAddress) - Required - The DocAddress that is associated with the document that we wish to fetch.

### Returns
Returns the Document, raises ValueError if the document can't be found.
```

--------------------------------

### Create Document from dict with schema

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Shows using Document.from_dict with a schema to provide explicit type information for numeric fields. The constructor alone cannot infer types, so from_dict is recommended for numeric fields.

```python
>>> schema = (
...     SchemaBuilder()
...         .add_unsigned_field("unsigned")
...         .add_integer_field("signed")
...         .add_float_field("float")
...         .build()
... )
>>> doc = tantivy.Document.from_dict(
...     {"unsigned": 1000, "signed": -5, "float": 0.4},
...     schema,
... )
```

--------------------------------

### Build a custom TextAnalyzer with TextAnalyzerBuilder

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Builds a custom text analyzer using TextAnalyzerBuilder with a regex tokenizer, a lowercase filter, and a custom stopword filter. The resulting TextAnalyzer is a pipeline of one tokenizer and any number of filters.

```python
from tantivy import (
    TextAnalyzer,
    TextAnalyzerBuilder,
    Tokenizer,
    Filter,
    Index,
    SchemaBuilder
)

my_analyzer: TextAnalyzer = (
    TextAnalyzerBuilder(
        # Create a `Tokenizer` instance.
        # It instructs the builder about which type of tokenizer
        # to create internally and with which arguments.
        Tokenizer.regex(r"(?i)([a-z]+)")
    )
    .filter(
        # Create a `Filter` instance.
        # Like `Tokenizer`, this object provides instructions
        # to the builder.
        Filter.lowercase()
    )
    .filter(
        # Define custom words.
        Filter.custom_stopword(["www", "com"])
    )
    # Finally, build a TextAnalyzer
    # chaining all tokenizer > [filter, ...] steps together.
    .build()
)
```

--------------------------------

### Create a regex tokenizer with Tokenizer.regex

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Creates a regex tokenizer using the pattern \w+ to split text into word tokens. This tokenizer can then be passed to TextAnalyzerBuilder.

```python
tokenizer = Tokenizer.regex(r"\w+")
```

--------------------------------

### phrase_prefix_query

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Constructs a Tantivy's PhrasePrefixQuery with custom offsets and slop.

```APIDOC
## phrase_prefix_query(schema: Schema, field_name: str, words: list[str | tuple[int, str]]) -> Query

### Description
Constructs a Tantivy's PhrasePrefixQuery with custom offsets and slop.

### Parameters
- **schema** (Schema) - Required - Schema of the target index.
- **field_name** (str) - Required - Field name to be searched.
- **words** (list[str | tuple[int, str]]) - Required - List of words or tuples of (offset, word) for the phrase prefix query.

### Returns
- **Query** - The constructed PhrasePrefixQuery.
```

--------------------------------

### SchemaBuilder.build

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Finalizes the creation of a Schema.

```APIDOC
## SchemaBuilder.build

### Description
Finalizes the creation of a Schema. Returns a Schema object. After this is called the SchemaBuilder cannot be used anymore.

### Method
build

### Signature
```python
def build(self) -> Schema
```

### Returns
Returns a Schema object.
```

--------------------------------

### to_dict

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Returns a dictionary with the different field values.

```APIDOC
## to_dict() -> dict[str, list[typing.Any]]

### Description
Returns a dictionary with the different field values. In tantivy, `Document` can hold multiple values for a single field. For this reason, the dictionary will associate a list of values for every field.

### Method
to_dict

### Returns
- **dict[str, list[Any]]** - A dictionary mapping field names to lists of values.
```

--------------------------------

### Cardinality Aggregation with Tantivy-py

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Shows how to use the cardinality aggregation in Tantivy-py to count unique values of a fast field. It creates an in-memory index, adds documents, and asserts that the cardinality of the 'rating' field is 2.0.

```python
import tantivy

# Create a schema with a numeric field
schema_builder = tantivy.SchemaBuilder()
schema_builder.add_integer_field("id", stored=True)
schema_builder.add_float_field("rating", stored=True, fast=True)
schema = schema_builder.build()

# Create an index in RAM
index = tantivy.Index(schema)

# Add some documents
writer = index.writer()
with writer:
    writer.add_document(tantivy.Document(id=1, rating=3.5))
    writer.add_document(tantivy.Document(id=2, rating=4.5))
    writer.add_document(tantivy.Document(id=3, rating=3.5))

# Reload the index to make the changes available for search
index.reload()

# Create a searcher
searcher = index.searcher()

# Create a query that matches all documents
query = tantivy.Query.all_query()

# Get the cardinality of the "rating" field
cardinality = searcher.cardinality(query, "rating")

assert cardinality == 2.0
```

--------------------------------

### is_empty

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Indicates whether the document is empty.

```APIDOC
## is_empty: bool

### Description
True if the document is empty, False otherwise.
```

--------------------------------

### extend

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Extends the document with a dictionary.

```APIDOC
## extend(py_dict: dict, schema: Schema | None) -> None

### Description
The type of the None singleton.
```

--------------------------------

### more_like_this_query

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Constructs a Tantivy's MoreLikeThisQuery from a document address.

```APIDOC
## more_like_this_query(doc_address: DocAddress, min_doc_frequency: int | None = 5, max_doc_frequency: int | None = None, min_term_frequency: int | None = 2, max_query_terms: int | None = 25, min_word_length: int | None = None, max_word_length: int | None = None, boost_factor: float | None = 1.0, stop_words: list[str] = []) -> Query

### Description
Constructs a Tantivy's MoreLikeThisQuery from a document address.

### Parameters
- **doc_address** (DocAddress) - Required - The address of the document to use as the basis for the query.
- **min_doc_frequency** (int | None) - Optional - Minimum document frequency. Default is 5.
- **max_doc_frequency** (int | None) - Optional - Maximum document frequency. Default is None.
- **min_term_frequency** (int | None) - Optional - Minimum term frequency. Default is 2.
- **max_query_terms** (int | None) - Optional - Maximum number of query terms. Default is 25.
- **min_word_length** (int | None) - Optional - Minimum word length. Default is None.
- **max_word_length** (int | None) - Optional - Maximum word length. Default is None.
- **boost_factor** (float | None) - Optional - Boost factor. Default is 1.0.
- **stop_words** (list[str]) - Optional - List of stop words. Default is empty list.

### Returns
- **Query** - The constructed MoreLikeThisQuery.
```

--------------------------------

### Build complex query with Query.boolean_query and disjunction_max_query

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Builds the equivalent of the ElasticSearch query programmatically using Tantivy's Query objects. It uses Query.boolean_query with Occur.Must, Query.disjunction_max_query, and Query.boost_query. The index.parse_query method is used to parse the query strings; by default, only the query parser will analyze the query string.

```python
from tantivy import Query, Occur, Index

...

complex_query = Query.boolean_query(
    [
        (
            Occur.Must,
            Query.disjunction_max_query(
                [
                    Query.boost_query(
                        # by default, only the query parser will analyze
                        # your query string
                        index.parse_query("fish", ["title"]), 
                        2.0
                    ),
                    Query.boost_query(
                        index.parse_query("eighty-four days", ["body"]), 
                        1.5
                    ),
                ],
                0.3,
            ),
        )
    ]
)
```

--------------------------------

### Add a document and commit with writer

Source: https://tantivy-py.readthedocs.io/en/latest/tutorials.html

Adds a document to the index using a writer, then commits and waits for merging threads. Note that wait_merging_threads() must be called last because the writer object is not usable after this call.

```python
writer = index.writer()
writer.add_document(tantivy.Document(
    doc_id=1,
    title=["The Old Man and the Sea"],
    body=["""He was an old man who fished alone in a skiff in the Gulf Stream and he had gone eighty-four days now without taking a fish."""],
))
# ... and committing
writer.commit()
writer.wait_merging_threads()

```

--------------------------------

### Delete documents by query with IndexWriter.delete_documents_by_query

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Shows how to delete documents matching a query using IndexWriter.delete_documents_by_query. It builds a schema with a fast text field, adds a JSON document, commits, then parses a query and deletes matching documents, committing again.

```python
schema_builder = SchemaBuilder()
schema_builder.add_text_field("title", fast=True)
schema = schema_builder.build()
index = Index(schema)
writer = index.writer()
source_doc = {
    "title": "Here is some text"
}
writer.add_json(json.dumps(source_doc))
writer.commit()
writer.wait_merging_threads()

query = index.parse_query("title:text")
writer = index.writer()
writer.delete_documents_by_query(query)
writer.commit()
writer.wait_merging_threads()
```

--------------------------------

### SnippetGenerator.create

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Creates a SnippetGenerator instance.

```APIDOC
## create(searcher, query, schema, field_name)

### Description
Creates a SnippetGenerator instance. The type of the None singleton.

### Method
create

### Parameters
- **searcher** (Searcher) - Required - The searcher to use.
- **query** (Query) - Required - The query to use.
- **schema** (Schema) - Required - The schema to use.
- **field_name** (str) - Required - The field name to use.

### Returns
SnippetGenerator - A new SnippetGenerator instance.
```

--------------------------------

### TextAnalyzerBuilder

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Methods for constructing and configuring a TextAnalyzer using a tokenizer and filters.

```APIDOC
## TextAnalyzerBuilder(tokenizer: Tokenizer)

### Description
Builds a TextAnalyzer by combining a tokenizer with filters. The builder is not mutated in-place; each filter call returns a new instance.

### Method
Constructor

### Parameters
- **tokenizer** (Tokenizer) - Required - The tokenizer to use.

### Returns
- **TextAnalyzerBuilder** - A new builder instance.

```

```APIDOC
## def build(self) -> TextAnalyzer

### Description
Builds the final TextAnalyzer object with the tokenizer and filters baked in.

### Method
build

### Returns
- **TextAnalyzer** - The configured analyzer. Use its `analyze(text)` method to test the analyzer.

```

```APIDOC
## def filter(self, filter: Filter) -> TextAnalyzerBuilder

### Description
Adds a filter to the builder. Returns a new builder instance; the original is not modified.

### Parameters
- **filter** (Filter) - Required - A Filter object to add.

### Returns
- **TextAnalyzerBuilder** - A new builder instance with the filter added.

```

--------------------------------

### phrase_query

Source: https://tantivy-py.readthedocs.io/en/latest/api/tantivy/tantivy.html

Constructs a Tantivy PhraseQuery with custom offsets and slop.

```APIDOC
## phrase_query

### Description
Construct a Tantivy's PhraseQuery with custom offsets and slop.

### Signature
```python
def phrase_query(schema: Schema, field_name: str, words: list[str | tuple[int, str]], slop: int = 0) -> Query
```

### Parameters
- **schema** (Schema) - Required - Schema of the target index.
- **field_name** (str) - Required - Field name to be searched.
- **words** (list[str | tuple[int, str]]) - Required - Word list that constructs the phrase. A word can be a term text or a pair of term text and its offset in the phrase.
- **slop** (int) - Optional - The number of gaps permitted between the words in the query phrase. Default is 0.

### Returns
- **Query** - The constructed PhraseQuery.
```
Tokens:
Raw