Go Faiss

https://pkg.go.dev/github.com/blevesearch/go-faiss
Admin
Go Faiss provides Go bindings for Faiss, a library for efficient vector similarity search and...
Tokens:13,968
Snippets:164
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install go-faiss Module

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

After successfully installing the Faiss library, install the go-faiss Go module using 'go get'.

```bash
go get github.com/blevesearch/go-faiss

```

--------------------------------

### Build and Install Faiss (Linux/macOS)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Follow these steps to build and install the Faiss library with C API enabled. Ensure Faiss is installed in a system-accessible path like /usr/local/lib.

```bash
git clone https://github.com/blevesearch/faiss.git
cd faiss
cmake -B build -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_C_API=ON -DBUILD_SHARED_LIBS=ON .
make -C build
sudo make -C build install

```

```bash
sudo cp build/c_api/libfaiss_c.so /usr/local/lib

```

--------------------------------

### Build and Install Faiss (macOS ARM64)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Adjustments for macOS ARM64, including setting LDFLAGS, CPPFLAGS, and CXX/CC to clang. Ensure FAISS_ENABLE_PYTHON is set to OFF in CMakeLists.txt.

```bash
LDFLAGS="-L/opt/homebrew/opt/llvm/lib" CPPFLAGS="-I/opt/homebrew/opt/llvm/include" CXX=/opt/homebrew/opt/llvm/bin/clang++ CC=/opt/homebrew/opt/llvm/bin/clang cmake -B build -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_C_API=ON -DBUILD_SHARED_LIBS=ON .
// set FAISS_ENABLE_PYTHON to OFF in CMakeLists.txt to ignore libpython dylib
make -C build
sudo make -C build install

```

--------------------------------

### IDSelector Get Method

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Retrieves the underlying C.FaissIDSelector.

```go
func (s *IDSelector) Get() *C.FaissIDSelector
```

--------------------------------

### Get Limits for Labels/Distances

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Lims returns a slice containing start and end indices for queries in the distances and labels slices returned by Labels.

```go
func (r *RangeSearchResult) Lims() []int
```

--------------------------------

### Get Number of Queries

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Nq returns the number of queries performed in the range search.

```go
func (r *RangeSearchResult) Nq() int
```

--------------------------------

### Get IndexFlat Vectors

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Retrieves the slice of vectors stored within an `IndexFlat`. The returned slice is invalidated by subsequent add or remove operations.

```go
func (idx *IndexFlat) Xb() []float32

```

--------------------------------

### Get Labels and Distances from Range Search

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Labels returns the unsorted IDs and respective distances for each query. The result for query i is labels[lims[i]:lims[i+1]].

```go
func (r *RangeSearchResult) Labels() (labels []int64, distances []float32)
```

--------------------------------

### RangeSearchResult.Lims

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns a slice containing start and end indices for queries in the distances and labels slices.

```APIDOC
## (*RangeSearchResult).Lims

### Description
Returns a slice containing start and end indices for queries in the distances and labels slices returned by Labels.

### Signature
```go
func (r *RangeSearchResult) Lims() []int
```

### Returns
- **[]int** - A slice of integers representing the limits.
```

--------------------------------

### Create New Flat Index (Inner Product)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Constructs a new `IndexFlat` using the inner product metric. Requires the vector dimension `d`.

```go
func NewIndexFlatIP(d int) (*IndexFlat, error)

```

--------------------------------

### GPUIndexImpl Add Method

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Adds data to the GPU index.

```go
func (g *GPUIndexImpl) Add(x []float32) error
```

--------------------------------

### Clone Index to CPU (GPU Build Tag Required)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

CloneToCPU is not available without the gpu build tag.

```go
func CloneToCPU(_ *GPUIndexImpl) (*IndexImpl, error)
```

--------------------------------

### Create New Flat Index (Generic)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new `IndexFlat` with a specified dimension `d` and metric type. The `metric` parameter should be one of the predefined Faiss metric constants.

```go
func NewIndexFlat(d int, metric int) (*IndexFlat, error)

```

--------------------------------

### Create New Flat Index (L2 Metric)

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Constructs a new `IndexFlat` using the L2 (Euclidean) metric. Requires the vector dimension `d`.

```go
func NewIndexFlatL2(d int) (*IndexFlat, error)

```

--------------------------------

### Clone Index to GPU

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

CloneToGPU is not available without the gpu build tag.

```go
func CloneToGPU(_ *IndexImpl) (*GPUIndexImpl, error)
```

--------------------------------

### Create New Parameter Space

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewParameterSpace creates a new ParameterSpace.

```go
func NewParameterSpace() (*ParameterSpace, error)
```

--------------------------------

### BinaryIndexFactory

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new binary index with the specified dimensions and description. This is the primary way to instantiate a new index.

```APIDOC
## BinaryIndexFactory

### Description
Creates a new binary index with the specified dimensions and description.

### Signature
```go
func BinaryIndexFactory(dims int, description string) (*BinaryIndexImpl, error)
```
```

--------------------------------

### GPUIndexImpl Search Method

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Searches the GPU index for the k nearest neighbors.

```go
func (g *GPUIndexImpl) Search(x []float32, k int64) ([]float32, []int64, error)
```

--------------------------------

### BinaryIndexFactory

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new binary index using a factory pattern. Requires dimensions and a description string.

```APIDOC
## BinaryIndexFactory

### Description
Creates a new binary index.

### Signature
func BinaryIndexFactory(dims int, description string) (*BinaryIndexImpl, error)
```

--------------------------------

### GPUIndexImpl Train Method

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Trains the GPU index with the provided data.

```go
func (g *GPUIndexImpl) Train(x []float32) error
```

--------------------------------

### NewParameterSpace

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new ParameterSpace.

```APIDOC
## NewParameterSpace

### Description
Creates a new ParameterSpace.

### Signature
```go
func NewParameterSpace() (*ParameterSpace, error)
```

### Returns
- **(*ParameterSpace, error)** - A pointer to the new ParameterSpace and an error if any occurred.
```

--------------------------------

### Index Interface Methods

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

This section details the methods available on the Index interface for interacting with Faiss indexes.

```APIDOC
## Index Interface

An `Index` is a Faiss index. Note that some index implementations do not support all methods. Check the Faiss wiki to see what operations an index supports.

### Methods

- **D() int**: Returns the dimension of the indexed vectors.
- **IsTrained() bool**: Returns true if the index has been trained or does not require training.
- **Ntotal() int64**: Returns the number of indexed vectors.
- **SetDirectMap(maptype int) error**: Sets the direct map type for IVF indexes (0 for No Map, 1 for Array, 2 for Hash).
- **SetNProbe(nprobe int32)**: Sets the number of probes for IVF indexes.
- **MetricType() int**: Returns the metric type of the index.
- **Train(x []float32) error**: Trains the index on a representative set of vectors.
- **Add(x []float32) error**: Adds vectors to the index.
- **AddWithIDs(x []float32, xids []int64) error**: Adds vectors to the index with specified IDs.
- **IsIVFIndex() bool**: Returns true if the index is an IVF index.
- **IsSQIndex() bool**: Returns true if the index is a scalar quantization (SQ) index.
- **HasRaBitQ() bool**: Returns true if the index has RaBitQ.
- **IVFParams() (nprobe, nlist int)**: Returns the IVF parameters nprobe and nlist for IVF indexes.
- **ObtainClusterVectorCountsFromIVFIndex(include Selector, nlist int) ([]int64, error)**: Applicable only to IVF indexes. Returns a slice where each index represents a cluster (list) ID and the value is the count of selected vectors belonging to that cluster.
- **ObtainClustersWithDistancesFromIVFIndex(x []float32, centroids Selector, numCentroids int64) ([]int64, []float32, error)**: Applicable only to IVF indexes. Returns the centroid IDs in the selector in decreasing order of proximity to query 'x' and their distance from 'x'.
- **ObtainKCentroidCardinalitiesFromIVFIndex(limit int, descending bool) ([]uint64, [][]float32, error)**: Applicable only to IVF indexes. Returns the top k centroid cardinalities and their vectors in chosen order.
- **Nlist() int**: Fetches the centroid count.
- **Search(x []float32, k int64) ([]float32, []int64, error)**: Queries the index with the vectors in x. Returns the IDs of the k nearest neighbors and their distances.
- **SearchWithOptions(x []float32, k int64, sel Selector, params json.RawMessage) ([]float32, []int64, error)**: Performs a search with additional optional constraints.
- **SearchClustersFromIVFIndex(eligibleCentroidIDs []int64, centroidDis []float32, centroidsToProbe int, x []float32, k int64, include Selector, params json.RawMessage) ([]float32, []int64, error)**: Applicable only to IVF indexes. Searches clusters whose IDs are in eligibleCentroidIDs.
- **Reconstruct(key int64) ([]float32, error)**: Reconstructs a vector by its key.
- **ReconstructBatch(keys []int64, recons []float32) ([]float32, error)**: Reconstructs multiple vectors by their keys.
- **MergeFrom(other Index, add_id int64) error**: Merges another index into this one.
- **RangeSearch(x []float32, radius float32) (*RangeSearchResult, error)**: Queries the index with vectors in x. Returns all vectors with distance < radius.
- **DistCompute(x []float32, labels []int64) ([]float32, error)**: Computes the distance between the query vector and specified vectors.
- **Reset() error**: Removes all vectors from the index.
- **RemoveIDs(sel *IDSelector) (int, error)**: Removes vectors specified by sel from the index.
- **Close()**: Frees the memory used by the index.
- **Size() uint64**: Consults the C++ side to get the size of the index.
- **SetQuantizers(source Index) error**: Sets the quantizers from a source index into this index, applicable only for IVF indexes.
```

--------------------------------

### CloneToCPU

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Clones a GPU index back to a CPU-based index. Requires a pointer to a GPUIndexImpl.

```APIDOC
## CloneToCPU

### Description
Clones a GPU index to a CPU-based index.

### Signature
func CloneToCPU(_ *GPUIndexImpl) (*IndexImpl, error)
```

--------------------------------

### IndexFlat.Xb

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns the internal data buffer of the IndexFlat.

```APIDOC
## IndexFlat.Xb

### Description
Returns the internal data buffer of the IndexFlat.

### Signature
func (idx *IndexFlat) Xb() []float32
```

--------------------------------

### Create Composite Index

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

IndexFactory builds a composite index. description is a comma-separated list of components.

```go
func IndexFactory(d int, description string, metric int) (*IndexImpl, error)
```

--------------------------------

### CloneToGPU

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Clones an existing index to a GPU-enabled index. Requires a pointer to an IndexImpl.

```APIDOC
## CloneToGPU

### Description
Clones an index to a GPU-enabled index.

### Signature
func CloneToGPU(_ *IndexImpl) (*GPUIndexImpl, error)
```

--------------------------------

### GPUIndexImpl Methods

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Methods available for the GPUIndexImpl type, used for GPU-accelerated operations. Note that GPUIndexImpl is an opaque type when not built with GPU support, and CloneToGPU is not available without the gpu build tag.

```APIDOC
## GPUIndexImpl

### Description
GPUIndexImpl is an opaque type when not built with GPU support.

### Methods

#### func (*GPUIndexImpl) Add(x []float32) error
Adds data to the GPU index.

#### func (*GPUIndexImpl) Close()
Closes the GPU index and releases resources.

#### func (*GPUIndexImpl) Search(x []float32, k int64) ([]float32, []int64, error)
Searches the GPU index for the nearest neighbors.

#### func (*GPUIndexImpl) Train(x []float32) error
Trains the GPU index with the provided data.
```

--------------------------------

### IndexFactory

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Builds a composite index. The description is a comma-separated list of components.

```APIDOC
## IndexFactory

### Description
Builds a composite index. The description is a comma-separated list of components.

### Signature
```go
func IndexFactory(d int, description string, metric int) (*IndexImpl, error)
```

### Parameters
#### Path Parameters
- **d** (int) - The dimension of the index.
- **description** (string) - A comma-separated list of index components.
- **metric** (int) - The distance metric to use.

### Returns
- **(*IndexImpl, error)** - A pointer to the created IndexImpl and an error if any occurred.
```

--------------------------------

### CloneToGPU Function

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Function to clone an IndexImpl to a GPUIndexImpl. This function is not available without the gpu build tag.

```APIDOC
## CloneToGPU

### Description
Clones an IndexImpl to a GPUIndexImpl. This function is not available without the gpu build tag.

### Signature
```go
func CloneToGPU(_ *IndexImpl) (*GPUIndexImpl, error)
```
```

--------------------------------

### BinaryIndexFactory Function Signature

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Provides the signature for BinaryIndexFactory, a constructor function for creating new BinaryIndexImpl instances. It takes the vector dimension and a description string as input.

```go
func BinaryIndexFactory(dims int, description string) (*BinaryIndexImpl, error)
```

--------------------------------

### Create Binary Search Parameters

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewBinarySearchParams creates a SearchParams object for binary indexes.

```go
func NewBinarySearchParams(idx BinaryIndex, params json.RawMessage, selector Selector,
	defaultParams *defaultSearchParamsIVF) (*SearchParams, error)
```

--------------------------------

### IndexFactory

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new index using a factory pattern. Requires dimensions, description, and metric.

```APIDOC
## IndexFactory

### Description
Creates a new index.

### Signature
func IndexFactory(d int, description string, metric int) (*IndexImpl, error)
```

--------------------------------

### Define Parameter Space Structure

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

ParameterSpace is used for managing index parameters.

```go
type ParameterSpace struct {
	// contains filtered or unexported fields
}
```

--------------------------------

### Define Abstract Index Structure

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

IndexImpl is an abstract structure for an index.

```go
type IndexImpl struct {
	Index
}
```

--------------------------------

### GPUIndexImpl Close Method

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Closes and frees resources associated with the GPU index.

```go
func (g *GPUIndexImpl) Close()
```

--------------------------------

### Read Index from File

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

ReadIndex reads an index from a file.

```go
func ReadIndex(filename string, ioflags int) (*IndexImpl, error)
```

--------------------------------

### NewIndexFlat

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new IndexFlat, a basic index type. Requires dimensions and a metric.

```APIDOC
## NewIndexFlat

### Description
Creates a new IndexFlat.

### Signature
func NewIndexFlat(d int, metric int) (*IndexFlat, error)
```

--------------------------------

### Define Search Parameters Structure

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

SearchParams holds parameters for controlling search behavior.

```go
type SearchParams struct {
	// contains filtered or unexported fields
}
```

--------------------------------

### Read Index from Buffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads an index from a byte buffer.

```go
func ReadIndexFromBuffer(buf []byte, ioflags int) (*IndexImpl, error)
```

--------------------------------

### BinaryIndex Interface Definition

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines the BinaryIndex interface, which represents a FAISS index that can be serialized. It includes methods for accessing index properties, training, adding vectors, merging, searching, and managing memory.

```go
type BinaryIndex interface {
	// D returns the dimension of the indexed vectors.
	D() int

	// MetricType returns the metric type of the index.
	MetricType() int

	// Ntotal returns the total number of vectors currently stored in the index.
	Ntotal() int64

	// set the direct map type for IVF indexes.
	// 0 for No Map
	// 1 for Array
	// 2 for Hash
	SetDirectMap(maptype int) error

	// set the number of probes for IVF indexes
	SetNProbe(nprobe int32)

	// returns true if the underlying index is an IVF index
	IsIVFIndex() bool

	// IVFParams returns the nlist and nprobe parameters for IVF indexes
	IVFParams() (nprobe int, nlist int)

	// trains the index on a representative set of vectors
	Train(xb []uint8) error

	// adds vectors to the index
	Add(xb []uint8) error

	// sets the qunatizers from the source index, supposed to be used only for
	// BIVF indexes and returns error otherwise
	SetQuantizers(srcIndex BinaryIndex) error

	// merges another binary index into this one, currently applicable only for
	// IVF indexes returns an error
	MergeFrom(other BinaryIndex, add_id int64) error

	// queries the index with the vectors in xb
	// returns the IDs of the k nearest neighbors for each query vector and
	// their corresponding distances
	Search(xb []uint8, k int64) (distances []int32, labels []int64, err error)

	// SearchWithOptions performs a search with additional optional constraints.
	// - Selector can be used to restrict the search to a subset of the indexed vectors based on their IDs.
	// - params is a JSON object that can contain additional search parameters specific to the index type, such as IVF search parameters.
	SearchWithOptions(xb []uint8, k int64, sel Selector, params json.RawMessage) (distances []int32, labels []int64, err error)

	// returns a slice where each index corresponds to a cluster in an IVF
	// index, and the value at each index is the count of vectors in that
	// cluster, considering only the vectors specified in the include selector.
	ObtainClusterVectorCountsFromIVFIndex(include Selector, nlist int) (
		[]int64, error)

	// returns the IDs and distances of the closest numCentroids centroids to
	// the query vector xb, considering only the centroids specified in the
	// includedCentroids selector.
	ObtainClustersWithDistancesFromIVFIndex(xb []uint8, includedCentroids Selector,
		numCentroids int64) ([]int64, []int32, error)

	// Applicable only to IVF indexes: Returns the top k centroid cardinalities and
	// their vectors in chosen order (descending or ascending)
	ObtainKCentroidCardinalitiesFromIVFIndex(limit int, descending bool) ([]uint64, [][]uint8, error)

	// searches the specified clusters in an IVF index for the k nearest neighbors
	// of the query vector xb, considering only the vectors specified in the include selector
	// and additional search parameters passed as a JSON object.
	SearchClustersFromIVFIndex(eligibleCentroidIDs []int64, centroidDis []int32,
		centroidsToProbe int, xb []uint8, k int64, include Selector,
		params json.RawMessage) ([]int32, []int64, error)

	// returns the total size of the index in bytes
	Size() uint64

	// frees the memory associated with the index
	Close()
	// contains filtered or unexported methods
}
```

--------------------------------

### NewIDSelectorBatch

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new batch selector.

```APIDOC
## NewIDSelectorBatch

### Description
Creates a new batch selector.

### Signature
```go
func NewIDSelectorBatch(indices []int64) (Selector, error)
```

### Parameters
#### Path Parameters
- **indices** ([]int64) - A slice of IDs to include in the batch selector.

### Returns
- **(Selector, error)** - A Selector object and an error if any occurred.
```

--------------------------------

### NewIDSelectorBitmap

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a selector using a bitset.

```APIDOC
## NewIDSelectorBitmap

### Description
Creates a selector using a bitset, where each bit indicates whether the corresponding ID is to be selected. NOTE: This function assumes that len(bitmap)*8 covers the full range of IDs in the index, and only works when we have vector IDs ranging from 0 to N-1, where N is the number of vectors in the index. The length of the bitmap should be at least ceil(N/8).

### Signature
```go
func NewIDSelectorBitmap(bitmap []byte) (Selector, error)
```

### Parameters
#### Path Parameters
- **bitmap** ([]byte) - The byte slice representing the bitset.

### Returns
- **(Selector, error)** - A Selector object and an error if any occurred.
```

--------------------------------

### GPUIndexImpl.Search

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Performs a similarity search on the GPU index. Returns distances, labels, and potential errors.

```APIDOC
## GPUIndexImpl.Search

### Description
Performs a similarity search on the GPU index.

### Signature
func (g *GPUIndexImpl) Search(x []float32, k int64) ([]float32, []int64, error)
```

--------------------------------

### NewIndexFlatIP

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new IndexFlat using inner product as the metric.

```APIDOC
## NewIndexFlatIP

### Description
Creates a new IndexFlat using inner product metric.

### Signature
func NewIndexFlatIP(d int) (*IndexFlat, error)
```

--------------------------------

### Define Selector Interface

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Selector interface for filtering during search. Note: currently we have only one implementation, but we keep the interface for future extensibility.

```go
type Selector interface {
	ExcludeFilter() bool
	Get() *C.FaissIDSelector
	Delete()
}
```

--------------------------------

### GPUIndexImpl.Add

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Adds vectors to a GPU-accelerated index. Requires a slice of float32 vectors.

```APIDOC
## GPUIndexImpl.Add

### Description
Adds vectors to a GPU-accelerated index.

### Signature
func (g *GPUIndexImpl) Add(x []float32) error
```

--------------------------------

### NewParameterSpace

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new ParameterSpace, used for managing index parameters.

```APIDOC
## NewParameterSpace

### Description
Creates a new ParameterSpace.

### Signature
func NewParameterSpace() (*ParameterSpace, error)
```

--------------------------------

### BinaryIndex Interface

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

The BinaryIndex interface defines the core operations for interacting with FAISS indices, including training, adding vectors, searching, and managing index properties.

```APIDOC
## BinaryIndex Interface

### Description
Represents a FAISS index that can be trained, populated, and searched.

### Methods

- **D() int**: Returns the dimension of the indexed vectors.
- **MetricType() int**: Returns the metric type of the index.
- **Ntotal() int64**: Returns the total number of vectors currently stored in the index.
- **SetDirectMap(maptype int) error**: Sets the direct map type for IVF indexes (0: No Map, 1: Array, 2: Hash).
- **SetNProbe(nprobe int32)**: Sets the number of probes for IVF indexes.
- **IsIVFIndex() bool**: Returns true if the underlying index is an IVF index.
- **IVFParams() (nprobe int, nlist int)**: Returns the nprobe and nlist parameters for IVF indexes.
- **Train(xb []uint8) error**: Trains the index on a representative set of vectors.
- **Add(xb []uint8) error**: Adds vectors to the index.
- **SetQuantizers(srcIndex BinaryIndex) error**: Sets the quantizers from a source index, applicable to BIVF indexes.
- **MergeFrom(other BinaryIndex, add_id int64) error**: Merges another binary index into this one, applicable to IVF indexes.
- **Search(xb []uint8, k int64) (distances []int32, labels []int64, err error)**: Queries the index with the vectors in xb, returning IDs and distances of k nearest neighbors.
- **SearchWithOptions(xb []uint8, k int64, sel Selector, params json.RawMessage) (distances []int32, labels []int64, err error)**: Performs a search with additional optional constraints like vector selection and index-specific parameters.
- **ObtainClusterVectorCountsFromIVFIndex(include Selector, nlist int) ([]int64, error)**: Returns counts of vectors in each cluster for IVF indexes, considering a selector.
- **ObtainClustersWithDistancesFromIVFIndex(xb []uint8, includedCentroids Selector, numCentroids int64) ([]int64, []int32, error)**: Returns IDs and distances of closest centroids to a query vector.
- **ObtainKCentroidCardinalitiesFromIVFIndex(limit int, descending bool) ([]uint64, [][]uint8, error)**: Returns top k centroid cardinalities and their vectors for IVF indexes.
- **SearchClustersFromIVFIndex(eligibleCentroidIDs []int64, centroidDis []int32, centroidsToProbe int, xb []uint8, k int64, include Selector, params json.RawMessage) ([]int32, []int64, error)**: Searches specified clusters in an IVF index.
- **Size() uint64**: Returns the total size of the index in bytes.
- **Close()**: Frees the memory associated with the index.
```

--------------------------------

### Go Faiss Index Interface Definition

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines the core `Index` interface for Faiss indexes in Go. Implementations must provide methods for vector operations, training, searching, and index management. Note that not all methods are supported by all index types.

```go
type Index interface {
	// D returns the dimension of the indexed vectors.
	D() int

	// IsTrained returns true if the index has been trained or does not require
	// training.
	IsTrained() bool

	// Ntotal returns the number of indexed vectors.
	Ntotal() int64

	// set the direct map type for IVF indexes.
	// 0 for No Map
	// 1 for Array
	// 2 for Hash
	SetDirectMap(maptype int) error

	// set the number of probes for IVF indexes
	SetNProbe(nprobe int32)

	// MetricType returns the metric type of the index.
	MetricType() int

	// Train trains the index on a representative set of vectors.
	Train(x []float32) error

	// Add adds vectors to the index.
	Add(x []float32) error

	// AddWithIDs is like Add, but stores xids instead of sequential IDs.
	AddWithIDs(x []float32, xids []int64) error

	// Returns true if the index is an IVF index.
	IsIVFIndex() bool

	// Returns true if the index is a scalar quantization (SQ) index.
	IsSQIndex() bool

	// Returns true if the index has RaBitQ
	HasRaBitQ() bool

	// Returns the IVF parameters nprobe and nlist for IVF indexes.
	IVFParams() (nprobe, nlist int)

	// Applicable only to IVF indexes: Returns a slice where each index represents
	// a cluster (list) ID and the value is the count of selected vectors belonging
	// to that cluster. Only vectors specified by the given Selector are considered.
	ObtainClusterVectorCountsFromIVFIndex(include Selector, nlist int) ([]int64, error)

	// Applicable only to IVF indexes: Returns the centroid IDs in the selector in
	// decreasing order of proximity to query 'x' and their distance from 'x'
	ObtainClustersWithDistancesFromIVFIndex(x []float32, centroids Selector, numCentroids int64) (
		[]int64, []float32, error)

	// Applicable only to IVF indexes: Returns the top k centroid cardinalities and
	// their vectors in chosen order (descending or ascending)
	ObtainKCentroidCardinalitiesFromIVFIndex(limit int, descending bool) ([]uint64, [][]float32, error)

	// fetch centroid count
	Nlist() int

	// Search queries the index with the vectors in x.
	// Returns the IDs of the k nearest neighbors for each query vector and the
	// corresponding distances.
	Search(x []float32, k int64) (distances []float32, labels []int64, err error)

	// SearchWithOptions performs a search with additional optional constraints.
	// - Selector can be used to restrict the search to a subset of the indexed vectors based on their IDs.
	// - params is a JSON object that can contain additional search parameters specific to the index type, such as IVF search parameters.
	SearchWithOptions(x []float32, k int64, sel Selector, params json.RawMessage) (distances []float32, labels []int64, err error)

	// Applicable only to IVF indexes: Search clusters whose IDs are in eligibleCentroidIDs
	SearchClustersFromIVFIndex(eligibleCentroidIDs []int64, centroidDis []float32, centroidsToProbe int,
		x []float32, k int64, include Selector, params json.RawMessage) ([]float32, []int64, error)

	Reconstruct(key int64) ([]float32, error)

	ReconstructBatch(keys []int64, recons []float32) ([]float32, error)

	MergeFrom(other Index, add_id int64) error

	// RangeSearch queries the index with the vectors in x.
	// Returns all vectors with distance < radius.
	RangeSearch(x []float32, radius float32) (*RangeSearchResult, error)

	// DistCompute computes the distance between the query vector and the vectors specified by ids.
	DistCompute(x []float32, labels []int64) ([]float32, error)

	// Reset removes all vectors from the index.
	Reset() error

	// RemoveIDs removes the vectors specified by sel from the index.
	// Returns the number of elements removed and error.
	RemoveIDs(sel *IDSelector) (int, error)

	// Close frees the memory used by the index.
	Close()

	// consults the C++ side to get the size of the index
	Size() uint64

	// set the quantizers from a source index into this index, applicable only
	// for IVF indexes
	SetQuantizers(source Index) error
	// contains filtered or unexported methods
}

```

--------------------------------

### NewBinarySearchParams

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates new search parameters for a binary index. Requires index, parameters, and selector.

```APIDOC
## NewBinarySearchParams

### Description
Creates new search parameters for a binary index.

### Signature
func NewBinarySearchParams(idx BinaryIndex, params json.RawMessage, selector Selector, ...) (*SearchParams, error)
```

--------------------------------

### GPUIndexImpl.Train

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Trains the GPU index with provided vectors. This is necessary for some index types.

```APIDOC
## GPUIndexImpl.Train

### Description
Trains the GPU index.

### Signature
func (g *GPUIndexImpl) Train(x []float32) error
```

--------------------------------

### Create Bitmap ID Selector

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewIDSelectorBitmap creates a selector using a bitset. NOTE: This function assumes that len(bitmap)*8 covers the full range of IDs in the index, and only works when we have vector IDs ranging from 0 to N-1, where N is the number of vectors in the index. The length of the bitmap should be at least ceil(N/8).

```go
func NewIDSelectorBitmap(bitmap []byte) (Selector, error)
```

--------------------------------

### IDSelector.Get

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Retrieves the underlying C.FaissIDSelector.

```APIDOC
## IDSelector.Get

### Description
Retrieves the underlying C.FaissIDSelector.

### Signature
func (s *IDSelector) Get() *C.FaissIDSelector
```

--------------------------------

### Define GPUIndexImpl Type

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

GPUIndexImpl is an opaque type when not built with GPU support.

```go
type GPUIndexImpl struct{}
```

--------------------------------

### WriteIndex Function Signature

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines the signature for the WriteIndex function, which persists an Index to a specified file. This is the standard way to save a trained FAISS index to disk.

```go
func WriteIndex(idx Index, filename string) error
```

--------------------------------

### NewIndexFlatIP

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new flat index with the inner product metric type.

```APIDOC
## func NewIndexFlatIP
```go
func NewIndexFlatIP(d int) (*IndexFlat, error)
```

NewIndexFlatIP creates a new flat index with the inner product metric type.
```

--------------------------------

### WriteBinaryIndexIntoBuffer Function Signature

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Provides the signature for WriteBinaryIndexIntoBuffer, which serializes a BinaryIndex into a byte slice. This is useful for saving index state or transferring it between processes.

```go
func WriteBinaryIndexIntoBuffer(idx BinaryIndex) ([]byte, error)
```

--------------------------------

### ReadBinaryIndexFromBuffer Function Signature

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines the signature for ReadBinaryIndexFromBuffer, which reconstructs a BinaryIndexImpl from a byte slice using specified I/O flags. This is used for loading a previously saved index.

```go
func ReadBinaryIndexFromBuffer(buf []byte, ioflags int) (*BinaryIndexImpl, error)
```

--------------------------------

### WriteIndex

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Writes an index to a specified file. This allows for persistent storage of the index.

```APIDOC
## WriteIndex

### Description
Writes an index to a specified file.

### Signature
func WriteIndex(idx Index, filename string) error
```

--------------------------------

### ReadBinaryIndexFromBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads a binary index from a byte buffer. Requires I/O flags for reading.

```APIDOC
## ReadBinaryIndexFromBuffer

### Description
Reads a binary index from a byte buffer.

### Signature
func ReadBinaryIndexFromBuffer(buf []byte, ioflags int) (*BinaryIndexImpl, error)
```

--------------------------------

### NewIndexFlatL2

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new IndexFlat using L2 distance as the metric.

```APIDOC
## NewIndexFlatL2

### Description
Creates a new IndexFlat using L2 distance metric.

### Signature
func NewIndexFlatL2(d int) (*IndexFlat, error)
```

--------------------------------

### Create Generic Search Parameters

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewSearchParams returns a valid SearchParams object, configured according to the provided parameters and selector. The returned SearchParams object is allocated, thus caller must clean up the object by invoking Delete() method.

```go
func NewSearchParams(idx Index, params json.RawMessage, selector Selector,
	defaultParams *defaultSearchParamsIVF) (*SearchParams, error)
```

--------------------------------

### FAISS I/O Flag Constants in Go

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines constants for I/O flags used in FAISS operations, including memory mapping (IOFlagMmap), read-only access (IOFlagReadOnly), read-only memory mapping with on-disk IVF (IOFlagReadMmap), and skipping prefetching (IOFlagSkipPrefetch). These flags control how index data is accessed and managed.

```go
const (
	IOFlagMmap         = C.FAISS_IO_FLAG_MMAP
	IOFlagReadOnly     = C.FAISS_IO_FLAG_READ_ONLY
	IOFlagReadMmap     = C.FAISS_IO_FLAG_READ_MMAP | C.FAISS_IO_FLAG_ONDISK_IVF
	IOFlagSkipPrefetch = C.FAISS_IO_FLAG_SKIP_PREFETCH
)
```

--------------------------------

### Create Batch ID Selector

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewIDSelectorBatch creates a new batch selector for specifying IDs.

```go
func NewIDSelectorBatch(indices []int64) (Selector, error)
```

--------------------------------

### WriteIndexIntoBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Writes an index into a byte buffer. Similar to WriteBinaryIndexIntoBuffer but for general Index types.

```APIDOC
## WriteIndexIntoBuffer

### Description
Writes an index into a byte buffer.

### Signature
func WriteIndexIntoBuffer(idx Index) ([]byte, error)
```

--------------------------------

### Create NOT ID Selector with Bitmap

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a NOT selector using a bitset. Assumes vector IDs range from 0 to N-1. The bitmap length should be at least ceil(N/8).

```go
func NewIDSelectorBitmapNot(bitmap []byte) (Selector, error)
```

--------------------------------

### WriteBinaryIndexIntoBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Writes a binary index into a byte buffer. Useful for serializing and deserializing indexes.

```APIDOC
## WriteBinaryIndexIntoBuffer

### Description
Writes a binary index into a byte buffer.

### Signature
func WriteBinaryIndexIntoBuffer(idx BinaryIndex) ([]byte, error)
```

--------------------------------

### IndexFlat.Xb

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns the index's vectors. The returned slice becomes invalid after any add or remove operation.

```APIDOC
## func (*IndexFlat) Xb
```go
func (idx *IndexFlat) Xb() []float32
```

Xb returns the index's vectors. The returned slice becomes invalid after any add or remove operation.
```

--------------------------------

### IndexFlat Type

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

IndexFlat is an index that stores the full vectors and performs exhaustive search.

```APIDOC
#### type IndexFlat
```go
type IndexFlat struct {
	Index
}
```

IndexFlat is an index that stores the full vectors and performs exhaustive search.
```

--------------------------------

### WriteBinaryIndexIntoBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Serializes a binary index into a byte slice. This is useful for saving the index to memory or transferring it between processes.

```APIDOC
## WriteBinaryIndexIntoBuffer

### Description
Serializes a binary index into a byte slice.

### Signature
```go
func WriteBinaryIndexIntoBuffer(idx BinaryIndex) ([]byte, error)
```
```

--------------------------------

### GPUIndexImpl.Close

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Closes the GPU index, releasing associated resources.

```APIDOC
## GPUIndexImpl.Close

### Description
Closes the GPU index.

### Signature
func (g *GPUIndexImpl) Close()
```

--------------------------------

### NewIndexFlat

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new flat index with the specified dimension and metric type.

```APIDOC
## func NewIndexFlat
```go
func NewIndexFlat(d int, metric int) (*IndexFlat, error)
```

NewIndexFlat creates a new flat index.
```

--------------------------------

### Set Index Parameter

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

SetIndexParameter sets one of the parameters for an index within the ParameterSpace.

```go
func (p *ParameterSpace) SetIndexParameter(idx Index, name string, val float64) error
```

--------------------------------

### WriteIndexIntoBuffer Function Signature

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Provides the signature for WriteIndexIntoBuffer, which serializes an Index into a byte slice. This is useful for in-memory storage or network transmission of index data.

```go
func WriteIndexIntoBuffer(idx Index) ([]byte, error)
```

--------------------------------

### FAISS Metric Constants in Go

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines constants for various metric types used in FAISS, such as InnerProduct, L2, L1, Linf, Lp, Canberra, BrayCurtis, and JensenShannon. These are essential for specifying the distance calculation method during index creation or operations.

```go
const (
	MetricInnerProduct  = C.METRIC_INNER_PRODUCT
	MetricL2            = C.METRIC_L2
	MetricL1            = C.METRIC_L1
	MetricLinf          = C.METRIC_Linf
	MetricLp            = C.METRIC_Lp
	MetricCanberra      = C.METRIC_Canberra
	MetricBrayCurtis    = C.METRIC_BrayCurtis
	MetricJensenShannon = C.METRIC_JensenShannon
)
```

--------------------------------

### Create Standard Search Parameters

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

NewStandardSearchParams returns a standard SearchParams object without any special settings with the provided selector. The returned SearchParams object is allocated, thus caller must clean up the object by invoking Delete() method.

```go
func NewStandardSearchParams(selector Selector) (*SearchParams, error)
```

--------------------------------

### Create Range ID Selector

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a selector that removes IDs within the specified range [imin, imax).

```go
func NewIDSelectorRange(imin, imax int64) (Selector, error)
```

--------------------------------

### NewSearchParams

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates new search parameters for a general index. Requires index, parameters, and selector.

```APIDOC
## NewSearchParams

### Description
Creates new search parameters for a general index.

### Signature
func NewSearchParams(idx Index, params json.RawMessage, selector Selector, ...) (*SearchParams, error)
```

--------------------------------

### ReadIndex

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads an index from a specified file. Requires I/O flags for reading.

```APIDOC
## ReadIndex

### Description
Reads an index from a specified file.

### Signature
func ReadIndex(filename string, ioflags int) (*IndexImpl, error)
```

--------------------------------

### BinaryIndexImpl Struct Definition

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Defines the BinaryIndexImpl struct, which embeds the BinaryIndex interface. This is likely a concrete implementation of the BinaryIndex interface.

```go
type BinaryIndexImpl struct {
	BinaryIndex
}
```

--------------------------------

### NewIndexFlatL2

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new flat index with the L2 metric type.

```APIDOC
## func NewIndexFlatL2
```go
func NewIndexFlatL2(d int) (*IndexFlat, error)
```

NewIndexFlatL2 creates a new flat index with the L2 metric type.
```

--------------------------------

### ReadIndexFromBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads an index from a byte buffer. Requires I/O flags for reading.

```APIDOC
## ReadIndexFromBuffer

### Description
Reads an index from a byte buffer.

### Signature
func ReadIndexFromBuffer(buf []byte, ioflags int) (*IndexImpl, error)
```

--------------------------------

### IndexFlat Struct Definition

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Represents a flat index in Faiss, which stores all vectors directly and performs exhaustive searches. It embeds the base `Index` interface.

```go
type IndexFlat struct {
	Index
}

```

--------------------------------

### NewIDSelectorBitmapNot

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a NOT selector using a bitset. Each bit indicates whether the corresponding ID is NOT to be selected. This function assumes that len(bitmap)*8 covers the full range of IDs in the index and only works when vector IDs range from 0 to N-1. The length of the bitmap should be at least ceil(N/8).

```APIDOC
## func NewIDSelectorBitmapNot

### Description
Creates a NOT selector using a bitset, where each bit indicates whether the corresponding ID is NOT to be selected. NOTE: This function assumes that len(bitmap)*8 covers the full range of IDs in the index, and only works when we have vector IDs ranging from 0 to N-1, where N is the number of vectors in the index. The length of the bitmap should be at least ceil(N/8).

### Signature
```go
func NewIDSelectorBitmapNot(bitmap []byte) (Selector, error)
```
```

--------------------------------

### NewIDSelectorBatch

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new ID selector that selects a batch of specific IDs.

```APIDOC
## NewIDSelectorBatch

### Description
Creates a new ID selector for a batch of IDs.

### Signature
func NewIDSelectorBatch(indices []int64) (Selector, error)
```

--------------------------------

### ReadIndex

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads an index from a file.

```APIDOC
## ReadIndex

### Description
Reads an index from a file.

### Signature
```go
func ReadIndex(filename string, ioflags int) (*IndexImpl, error)
```

### Parameters
#### Path Parameters
- **filename** (string) - The path to the index file.
- **ioflags** (int) - Flags to control file reading behavior.

### Returns
- **(*IndexImpl, error)** - A pointer to the loaded IndexImpl and an error if any occurred.
```

--------------------------------

### NewBinarySearchParams

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new SearchParams object for binary search.

```APIDOC
## NewBinarySearchParams

### Description
Creates a new SearchParams object for binary search.

### Signature
```go
func NewBinarySearchParams(idx BinaryIndex, params json.RawMessage, selector Selector, defaultParams *defaultSearchParamsIVF) (*SearchParams, error)
```

### Parameters
#### Path Parameters
- **idx** (BinaryIndex) - The binary index to search.
- **params** (json.RawMessage) - JSON raw message containing search parameters.
- **selector** (Selector) - The selector to use for filtering.
- **defaultParams** (*defaultSearchParamsIVF) - Default parameters for IVF search.

### Returns
- **(*SearchParams, error)** - A pointer to the created SearchParams object and an error if any occurred.
```

--------------------------------

### ReadIndexFromBuffer

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Reads an index from a byte buffer.

```APIDOC
## ReadIndexFromBuffer

### Description
Reads an index from a byte buffer.

### Signature
```go
func ReadIndexFromBuffer(buf []byte, ioflags int) (*IndexImpl, error)
```

### Parameters
#### Path Parameters
- **buf** ([]byte) - The byte buffer containing the index data.
- **ioflags** (int) - Flags to control buffer reading behavior.

### Returns
- **(*IndexImpl, error)** - A pointer to the loaded IndexImpl and an error if any occurred.
```

--------------------------------

### NewStandardSearchParams

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates standard search parameters with a given selector.

```APIDOC
## NewStandardSearchParams

### Description
Creates standard search parameters.

### Signature
func NewStandardSearchParams(selector Selector) (*SearchParams, error)
```

--------------------------------

### NewIDSelectorBitmap

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Creates a new ID selector using a bitmap representation.

```APIDOC
## NewIDSelectorBitmap

### Description
Creates a new ID selector using a bitmap.

### Signature
func NewIDSelectorBitmap(bitmap []byte) (Selector, error)
```

--------------------------------

### RangeSearchResult.Labels

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns the unsorted IDs and respective distances for each query.

```APIDOC
## (*RangeSearchResult).Labels

### Description
Returns the unsorted IDs and respective distances for each query. The result for query i is labels[lims[i]:lims[i+1]].

### Signature
```go
func (r *RangeSearchResult) Labels() (labels []int64, distances []float32)
```

### Returns
- **labels** ([]int64) - A slice of labels (IDs).
- **distances** ([]float32) - A slice of corresponding distances.
```

--------------------------------

### RangeSearchResult.Nq

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns the number of queries.

```APIDOC
## (*RangeSearchResult).Nq

### Description
Returns the number of queries.

### Signature
```go
func (r *RangeSearchResult) Nq() int
```

### Returns
- **int** - The number of queries.
```

--------------------------------

### RangeSearchResult.Labels

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Returns the labels and distances of the search results.

```APIDOC
## RangeSearchResult.Labels

### Description
Returns the labels and distances of the search results.

### Signature
func (r *RangeSearchResult) Labels() (labels []int64, distances []float32)
```

--------------------------------

### Delete Parameter Space

Source: https://pkg.go.dev/github.com/blevesearch/go-faiss

Delete frees the memory associated with the ParameterSpace.

```go
func (p *ParameterSpace) Delete()
```
Tokens:
Raw