### Install Libraries with Pip Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md Installs the Sentence Transformers and requests libraries required for interacting with SemaDB and generating embeddings. Ensure you have Python and pip installed. ```bash # Install the required libraries pip install -U sentence-transformers requests ``` -------------------------------- ### Run SemaDB from Source Source: https://github.com/semafind/semadb/blob/main/README.md Instructions on how to run the SemaDB server from its source code. This method requires Go to be installed and uses a YAML configuration file. The command starts a single server instance. ```bash SEMADB_CONFIG=./config/singleServer.yaml go run ./ ``` -------------------------------- ### Python Example: Interacting with SemaDB API Source: https://github.com/semafind/semadb/blob/main/README.md This Python code snippet demonstrates how to interact with the SemaDB API using the `requests` library. It shows how to set up the base URL, headers including API keys and host information, and provides examples for both RapidAPI hosted and self-hosted instances. This code is a starting point for making requests to the database. ```python import requests base_url = "https://semadb.p.rapidapi.com" # Or use an appropriate base_url if you are using a self-hosted instance, e.g. # base_url = "http://localhost:8081/v2" headers = { "content-type": "application/json", "X-RapidAPI-Key": "", "X-RapidAPI-Host": "semadb.p.rapidapi.com" # Or if self-hosting # "X-User-Id": "", # "X-User-Plan": "BASIC" } ``` -------------------------------- ### Compile and Run SemaDB Standalone (Bash) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/deployment.md This snippet demonstrates how to clone the SemaDB repository, compile the Go source code into a standalone executable, and run it using a specified configuration file. It assumes Go is installed and is suitable for development or small deployments. ```bash git clone --depth=1 https://github.com/Semafind/semadb.git cd semadb # Assuming you have Go installed to compile the code, or you can download the # pre-built binaries. go build # Specify your own configuration file SEMADB_CONFIG=config.yaml ./semadb ``` -------------------------------- ### Setup Message Pack RPC Server in Go Source: https://github.com/semafind/semadb/blob/main/cluster/mrpc/README.md Demonstrates how to set up a Message Pack RPC server using the mrpc package. It involves creating a new RPC server, registering an object, and starting an HTTP server to listen for incoming requests. Assumes 'myobject' is an exported struct or type that can be registered. ```go rpcMainServer := rpc.NewServer() rpcMainServer.Register(myobject) rpcServer := mrpc.NewHTTPServer(":9898", rpcMainServer) rpcServer.ListenAndServe() ``` -------------------------------- ### Search API - Kitchen Sink Example Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/search/kitchen.md Demonstrates a complex search query combining vector search and text search with various filters, weights, and sorting criteria. This example searches for products similar to a given vector and in stock with a price less than £100, or products containing 'summer dress' in the description and in stock. It returns the top 5 results sorted by price. ```APIDOC ## POST /query ### Description Executes a complex search query with hybrid search capabilities, allowing for combined vector and text-based searches with intricate filtering and sorting options. ### Method POST ### Endpoint /query ### Parameters #### Request Body - **query** (object) - Required - The main query object defining the search criteria. - **property** (string) - Required - Specifies the type of query operation (e.g., `_or`, `productEmbedding`, `_and`). - **_or** (array) - Optional - An array of query objects to be combined with a logical OR. - **productEmbedding** (object) - Optional - Defines a vector search operation. - **vector** (array) - Required - The vector to search for. - **operator** (string) - Required - The vector search operator (e.g., `near`). - **searchSize** (integer) - Optional - The number of initial candidates to consider. - **limit** (integer) - Optional - The maximum number of results to return from this specific search. - **filter** (object) - Optional - A filter to apply to the vector search results. - **property** (string) - Required - Specifies the type of filter operation (e.g., `_and`). - **_and** (array) - Optional - An array of filter objects to be combined with a logical AND. - **weight** (float) - Optional - The weight assigned to this part of the hybrid search. - **text** (object) - Optional - Defines a text search operation. - **value** (string) - Required - The text value to search for. - **operator** (string) - Required - The text search operator (e.g., `containsAll`). - **limit** (integer) - Optional - The maximum number of results to return from this specific search. - **weight** (float) - Optional - The weight assigned to this part of the hybrid search. - **stock** (object) - Optional - Defines a filter for the stock property. - **integer** (object) - Required - Specifies the integer filter parameters. - **operator** (string) - Required - The integer comparison operator (e.g., `greaterThan`). - **value** (integer) - Required - The value to compare against. - **price** (object) - Optional - Defines a filter for the price property. - **float** (object) - Required - Specifies the float filter parameters. - **operator** (string) - Required - The float comparison operator (e.g., `lessThan`). - **value** (float) - Required - The value to compare against. - **select** (array) - Required - An array of fields to be returned in the results. - **sort** (array) - Optional - An array of sorting criteria. - **field** (string) - Required - The field to sort by. - **descending** (boolean) - Optional - Whether to sort in descending order (default is false). - **limit** (integer) - Optional - The maximum number of results to return for the entire query. ### Request Example ```json { "query": { "property": "_or", "_or": [ { "property": "productEmbedding", "vectorVamana": { "vector": [1, 2], "operator": "near", "searchSize": 75, "limit": 10, "filter": { "property": "_and", "_and": [ { "property": "stock", "integer": { "operator": "greaterThan", "value": 0 } }, { "property": "price", "float": { "operator": "lessThan", "value": 100 } } ] }, "weight": 0.3 } }, { "property": "_and", "_and": [ { "property": "description", "text": { "value": "summer dress", "operator": "containsAll", "limit": 10, "weight": 0.7 } }, { "property": "stock", "integer": { "operator": "greaterThan", "value": 0 } } ] } ] }, "select": ["title", "description", "price"], "sort": [ { "field": "price", "descending": false } ], "limit": 5 } ``` ### Response #### Success Response (200) - **title** (string) - The title of the product. - **description** (string) - The description of the product. - **price** (float) - The price of the product. #### Response Example ```json { "results": [ { "title": "Summer Maxi Dress", "description": "A light and airy dress perfect for summer.", "price": 49.99 }, { "title": "Floral Summer Dress", "description": "Beautiful floral print summer dress.", "price": 59.99 } ] } ``` ``` -------------------------------- ### SemaDB Configuration Example (YAML) Source: https://context7.com/semafind/semadb/llms.txt An example configuration file for SemaDB using YAML format. This configuration covers cluster node settings, HTTP API parameters including metrics port, and user plan definitions with their respective limits. ```yaml # config/singleServer.yaml debug: true prettyLogOutput: true clusterNode: rootDir: ./data servers: - localhost:11001 rpcHost: localhost rpcPort: 11001 rpcTimeout: 300 maxShardSize: 2147483648 # 2GiB maxShardPointCount: 100000 maxSearchLimit: 75 shardManager: rootDir: ./data shardTimeout: 300 maxCacheSize: 1073741824 # 1GiB httpApi: httpHost: "" httpPort: 8081 enableMetrics: true metricsHttpPort: 8091 proxySecret: "" whiteListIPs: - "*" userPlans: BASIC: name: Basic Plan maxCollections: 1 maxCollectionPointCount: 1000000 maxPointSize: 1024 shardBackupFrequency: 3600 shardBackupCount: 2 ``` -------------------------------- ### Configure SemaDB API Headers with Python Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md Sets up the necessary headers for making requests to the SemaDB API using Python's requests library. This includes the content type, API key, and host, which are crucial for authentication and routing requests. Adjust `base_url` for self-hosted instances. ```python import requests base_url = "https://semadb.p.rapidapi.com" # Or use an appropriate base_url if you are using a self-hosted instance, e.g. # base_url = "http://localhost:8081/v2" headers = { "content-type": "application/json", "X-RapidAPI-Key": "", "X-RapidAPI-Host": "semadb.p.rapidapi.com" # Or if self-hosting # "X-User-Id": "", # "X-User-Plan": "BASIC" } ``` -------------------------------- ### Create a Collection Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md This endpoint allows you to create a new collection in SemaDB. You can specify the collection ID and the index schema, including vector parameters like size and distance metric. ```APIDOC ## POST /collections ### Description Creates a new collection to store vector data and their associated metadata. ### Method POST ### Endpoint /collections ### Parameters #### Request Body - **id** (string) - Required - The unique identifier for the collection. - **indexSchema** (object) - Required - Defines the schema for indexing data within the collection. - **vector** (object) - Required - Schema for the vector index. - **type** (string) - Required - The type of vector index (e.g., "vectorVamana"). - **vectorVamana** (object) - Optional - Configuration specific to the Vamana index. - **vectorSize** (integer) - Required - The dimensionality of the vectors. - **distanceMetric** (string) - Required - The distance metric to use (e.g., "cosine", "euclidean"). - **searchSize** (integer) - Optional - Controls the exhaustiveness of the search. - **degreeBound** (integer) - Optional - Controls the density of the graph. - **alpha** (number) - Optional - Controls the preference for longer edges in the graph. ### Request Example ```json { "id": "mycollection", "indexSchema": { "vector": { "type": "vectorVamana", "vectorVamana": { "vectorSize": 384, "distanceMetric": "cosine", "searchSize": 75, "degreeBound": 64, "alpha": 1.2 } } } } ``` ### Response #### Success Response (200) - **message** (string) - Indicates the success of the operation (e.g., "success"). - **failedRanges** (array) - An array of any failed ranges during the operation (usually empty on success). #### Response Example ```json { "message": "success", "failedRanges": [] } ``` ``` -------------------------------- ### SemaDB Cluster Mode Configuration Example (YAML) Source: https://context7.com/semafind/semadb/llms.txt An example YAML configuration snippet for running SemaDB in cluster mode. It specifies multiple server addresses for the cluster and defines the RPC host and port for the current node. ```yaml # Cluster mode with multiple servers # config/clusterServer.yaml clusterNode: servers: - server1.example.com:11001 - server2.example.com:11001 - server3.example.com:11001 rpcHost: server1.example.com rpcPort: 11001 ``` -------------------------------- ### Create a Collection with Vamana Index in SemaDB Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md Defines and creates a SemaDB collection named 'mycollection' using the Vamana index type. This snippet specifies parameters like vector size, distance metric, and search exhaustiveness. Ensure the `vectorSize` matches your embedding model's output. ```python payload = { "id": "mycollection", "indexSchema": { "vector": { "type": "vectorVamana", "vectorVamana": { "vectorSize": 384, # Sentence transformers give embeddings of size 384 "distanceMetric": "cosine", "searchSize": 75, # How exhaustive the search should be? "degreeBound": 64, # How dense the graph should be? "alpha": 1.2, # How much longer edges should be preferred? } } } } response = requests.post(base_url + "/collections", json=payload, headers=headers) print(response.json()) ``` -------------------------------- ### Encode Data with MessagePack for SemaDB Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md This snippet demonstrates how to use MessagePack to encode data for faster insertion into SemaDB collections. It requires the 'msgpack' and 'requests' libraries. The function takes a payload and headers, changing the content type to 'application/msgpack' before sending a POST request. Responses are returned in JSON. ```python import msgpack # pip install msgpack payload = { "points": points } # Same as before headers["content-type"] = "application/msgpack" # Change the content type to msgpack response = requests.post(base_url+"/collections/mycollection/points", data=msgpack.dumps(payload), headers=headers) # Responses are always in JSON print(response.json()) ``` -------------------------------- ### Run SemaDB with Docker (Bash) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/deployment.md This command deploys SemaDB as a Docker container. It maps local directories for configuration and data, exposes the HTTP port, and specifies the image to use. Ensure to replace 'main' with a specific release tag for stable deployments. ```bash docker run -it --rm -v ./config:/config -e SEMADB_CONFIG=/config/semadb-config.yaml -v ./data:/data -p 8081:8081 ghcr.io/semafind/semadb:main ``` -------------------------------- ### Basic Search Query Example (JSON) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/search/basic.md Demonstrates a basic search query in SemaDB using a JSON object. This query filters documents based on a 'price' property within a specified range and limits the number of results. It's a fundamental example for understanding SemaDB's query structure. ```json { "query": { "property": "price", "float": { "value": 100, "operator": "inRange", "endValue": 200 } }, "limit": 10 } ``` -------------------------------- ### Go Multi-line Comment Example for Locking in Greedy Search Source: https://github.com/semafind/semadb/blob/main/CONTRIBUTING.md This Go code snippet illustrates the use of multi-line comments (`/* */`) to explain the necessity of locking a node during neighbor distance calculations in a greedy search algorithm. It highlights potential concurrency issues and the trade-offs with approximate search. ```go /* We have to lock the point here because while we are calculating the * distance of its neighbours (edges in the graph) we can't have another * goroutine changing them. The case we aren't covering is after we have * calculated, they may change so the search we are doing is not * deterministic. With approximate search this is not a major problem. */ node.edgesMu.RLock() searchSet.AddWithLimit(node.neighbours...) node.edgesMu.RUnlock() ``` -------------------------------- ### Go Single-line Comment Example for Encoding/JSON Edge Case Source: https://github.com/semafind/semadb/blob/main/CONTRIBUTING.md This Go code snippet demonstrates the use of single-line comments (`//`) to explain a specific edge case related to the `encoding/json` package, which decodes numbers as float64. It shows how to handle such cases by converting them to int64. ```go // Floating point cases are here because encoding/json decodes // any number as float64. So you give it say 42, and it gives // you back float64(42) case float32: m[k] = int64(v) ``` -------------------------------- ### Example Point Data for E-commerce Products in JSON Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/concepts/indexing.md This JSON represents an example data point for an e-commerce product, conforming to a defined index schema. It includes fields such as SKU, currency, description embedding, description text, category, labels, size, price, and dates. This structure allows the point to be indexed and searched effectively within SemaDB. ```json { "SKU": "1234", "currency": "GBP", "descriptionEmbedding": [0.1, 0.2, 0.3, ...], "description": "This is a product description", "category": "electronics", "labels": ["new", "sale"], "size": 10, "price": 100.0, "dates": { "created": "2021-01-01", "updated": "2021-01-02" } } ``` -------------------------------- ### Pre-filter Example (Vector Search with Stock Filter) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/search/filtered.md Demonstrates how to use a pre-filter to include only in-stock products during a vector search. ```APIDOC ## Pre-filter Example (Vector Search with Stock Filter) ### Description This example shows a pre-filter applied to a vector search query. The `filter` parameter within `vectorVamana` specifies a condition that must be met (e.g., `stock` greater than 0) before the main search is performed. ### Method POST ### Endpoint `/search` (Assumed endpoint for search operations) ### Parameters #### Request Body - **query** (object) - Required - The main query object. - **property** (string) - Required - The property to query on (e.g., `productEmbedding`). - **vectorVamana** (object) - Required - Configuration for Vamana vector search. - **vector** (array) - Required - The vector to search with. - **operator** (string) - Required - The vector search operator (e.g., `near`). - **searchSize** (integer) - Required - The number of neighbors to search for. - **limit** (integer) - Required - The maximum number of results to return. - **filter** (object) - Optional - The pre-filter query. - **property** (string) - Required - The property to filter on (e.g., `stock`). - **integer** (object) - Required - Integer filter configuration. - **operator** (string) - Required - The integer operator (e.g., `greaterThan`). - **value** (integer) - Required - The integer value for comparison. - **limit** (integer) - Required - The overall limit for the search results. ### Request Example ```json { "query": { "property": "productEmbedding", "vectorVamana": { "vector": [1, 2], "operator": "near", "searchSize": 75, "limit": 10, "filter": { "property": "stock", "integer": { "operator": "greaterThan", "value": 0 } } } }, "limit": 10 } ``` ### Response #### Success Response (200) - **results** (array) - Description of the search results. #### Response Example ```json { "results": [ { "id": "doc1", "score": 0.95, "document": { ... } }, { "id": "doc3", "score": 0.92, "document": { ... } } ] } ``` ``` -------------------------------- ### Post-filter Example (Vector Search with Stock Filter) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/search/filtered.md Illustrates how to achieve post-filtering by combining a vector search query with a stock check using a composite AND query. ```APIDOC ## Post-filter Example (Vector Search with Stock Filter) ### Description This example demonstrates post-filtering using a composite `_and` query. The vector search is performed first, and then the results are filtered to include only items where the `stock` property is greater than 0. ### Method POST ### Endpoint `/search` (Assumed endpoint for search operations) ### Parameters #### Request Body - **query** (object) - Required - The main query object. - **property** (string) - Required - Must be `_and` for composite queries. - **_and** (array) - Required - An array of queries to be combined with a logical AND. - **Query 1** (object) - The primary search query. - **property** (string) - Required - The property for the primary search (e.g., `productEmbedding`). - **vectorVamana** (object) - Required - Configuration for Vamana vector search. - **vector** (array) - Required - The vector to search with. - **operator** (string) - Required - The vector search operator (e.g., `near`). - **searchSize** (integer) - Required - The number of neighbors to search for. - **limit** (integer) - Required - The maximum number of results to return from this part of the query. - **Query 2** (object) - The filter query. - **property** (string) - Required - The property to filter on (e.g., `stock`). - **integer** (object) - Required - Integer filter configuration. - **operator** (string) - Required - The integer operator (e.g., `greaterThan`). - **value** (integer) - Required - The integer value for comparison. - **limit** (integer) - Required - The overall limit for the search results. ### Request Example ```json { "query": { "property": "_and", "_and": [ { "property": "productEmbedding", "vectorVamana": { "vector": [1, 2], "operator": "near", "searchSize": 75, "limit": 10 } }, { "property": "stock", "integer": { "operator": "greaterThan", "value": 0 } } ] }, "limit": 10 } ``` ### Response #### Success Response (200) - **results** (array) - Description of the search results. #### Response Example ```json { "results": [ { "id": "doc1", "score": 0.95, "document": { ... } }, { "id": "doc3", "score": 0.92, "document": { ... } } ] } ``` ``` -------------------------------- ### Example SemaDB Point / Document Structure (JSON) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/concepts/point.md Demonstrates the structure of a typical point or document in SemaDB, showcasing various data types for fields like strings, integers, arrays, and nested objects. This structure serves as the fundamental unit of data storage. ```json { "city": "Edinburgh", "country": "Scotland", "areaCode": 0131, "population": 500000, "embedding": [0.1, 0.2, 0.3, 0.4, 0.5], "tags": ["capital", "historic", "tourist"], "link": "https://en.wikipedia.org/wiki/Edinburgh", "location": { "coordinates": [55.953251, -3.188267], "map": "https://www.openstreetmap.org/?mlat=55.953251&mlon=-3.188267" } } ``` -------------------------------- ### Insert Data into Collection Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md This endpoint allows you to insert vector points into a specified collection. Each point can include a vector and additional metadata fields. ```APIDOC ## POST /collections/{collectionId}/points ### Description Inserts one or more vector points into a specified collection. ### Method POST ### Endpoint /collections/{collectionId}/points ### Parameters #### Path Parameters - **collectionId** (string) - Required - The ID of the collection to insert points into. #### Request Body - **points** (array) - Required - An array of point objects to insert. - Each point object should contain: - **vector** (array of numbers) - Required - The vector embedding for the point. - **field_name** (any) - Optional - Any additional fields to store with the point. ### Request Example ```json { "points": [ {"vector": [0.1, 0.2, ...], "myfield": 0}, {"vector": [0.3, 0.4, ...], "anotherfield": "value"} ] } ``` ### Response #### Success Response (200) - **message** (string) - Indicates the success of the operation (e.g., "success"). - **failedRanges** (array) - An array of any failed ranges during the operation (usually empty on success). #### Response Example ```json { "message": "success", "failedRanges": [] } ``` ``` -------------------------------- ### Delete SemaDB Collection Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md This snippet shows how to delete a collection from SemaDB. It sends a DELETE request to the '/collections/{collectionName}' endpoint. This action removes the specified collection and all points contained within it. ```python # Clean up by deleting the collection response = requests.delete(base_url+"/collections/mycollection", headers=headers) print(response.json()) ``` -------------------------------- ### Generate Embeddings with Sentence Transformers Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md Generates vector embeddings for a list of sentences using the 'all-MiniLM-L6-v2' model from Sentence Transformers. It then normalizes these embeddings, which is important for cosine distance calculations in SemaDB. This requires the `sentence-transformers` and `numpy` libraries. ```python import numpy as np from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') sentences = [ "This flowing floral maxi dress is perfect for a summer day.", "These ripped denim jeans are a must-have for any fashion-forward wardrobe.", "A classic black turtleneck sweater is a versatile piece.", ] embeddings = model.encode(sentences) # We normalise the embeddings because for cosine distance SemaDB expects # normalised vectors. We can skip this step for example if we use euclidean # distance. embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True) print(embeddings.shape) ``` -------------------------------- ### Build and Run SemaDB Docker Image Locally Source: https://github.com/semafind/semadb/blob/main/README.md Instructions for building a SemaDB Docker image locally from the source code and then running it. This involves using `docker build` followed by `docker run`, with similar commands provided for Podman users. ```bash docker build -t semadb ./ docker run -it --rm -v ./config:/config -e SEMADB_CONFIG=/config/singleServer.yaml -v ./data:/data -p 8081:8081 semadb # If using podman podman build -t semadb ./ # The :Z argument relabels to access: see https://github.com/containers/podman/issues/3683 podman run -it --rm -v ./config:/config:Z -e SEMADB_CONFIG=/config/singleServer.yaml -v ./data:/data:Z -p 8081:8081 semadb ``` -------------------------------- ### Insert Data into SemaDB Collection Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md Inserts generated vector embeddings and associated data into the 'mycollection' in SemaDB. Each point includes a 'vector' field (the embedding) and an optional 'myfield'. This function requires previously generated embeddings and the `requests` library. ```python # Insert the points into the collection for searching points = [] for i in range(embeddings.shape[0]): # Here the "vector" field is the one that is indexed as per the indexSchema # definition. But "myfield" is completely optional and can be any field you # want to store. points.append({'vector': embeddings[i].tolist(), "myfield": i}) payload = { "points": points } response = requests.post(base_url+"/collections/mycollection/points", json=payload, headers=headers) print(response.json()) ``` -------------------------------- ### Run SemaDB with Docker Source: https://context7.com/semafind/semadb/llms.txt This command demonstrates how to run SemaDB using a Docker container. It maps local configuration and data directories and exposes the necessary API ports. Ensure the Docker image `ghcr.io/semafind/semadb:main` is available. ```bash # Run with Docker docker run -it --rm \ -v ./config:/config \ -e SEMADB_CONFIG=/config/singleServer.yaml \ -v ./data:/data \ -p 8081:8081 \ -p 8091:8091 \ ghcr.io/semafind/semadb:main ``` -------------------------------- ### Running All Go Tests in the Repository Source: https://github.com/semafind/semadb/blob/main/CONTRIBUTING.md This command executes all unit and integration tests within the SemaDB repository using the Go testing framework. It's useful for verifying component integrity and identifying potential issues across the codebase. ```bash go test ./... ``` -------------------------------- ### Run SemaDB using Docker Source: https://github.com/semafind/semadb/blob/main/README.md This command demonstrates how to run the latest version of SemaDB using a Docker container. It mounts local configuration and data directories and exposes the default port. Instructions for Podman are also provided. ```bash docker run -it --rm -v ./config:/config -e SEMADB_CONFIG=/config/singleServer.yaml -v ./data:/data -p 8081:8081 ghcr.io/semafind/semadb:main # If using podman podman run -it --rm -v ./config:/config:Z -e SEMADB_CONFIG=/config/singleServer.yaml -v ./data:/data:Z -p 8081:8081 ghcr.io/semafind/semadb:main ``` -------------------------------- ### Perform Vector Search with SemaDB Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/getting-started.md This code performs a vector search against a SemaDB collection. It first embeds a search sentence using a provided model and normalizes the resulting vector. Then, it constructs a search payload with a 'near' operator and sends a POST request to the '/points/search' endpoint. The results, including distance and hybrid scores, are printed. ```python # Create a search string and embed it using the same method search_sentence = "What can I wear on a hot day?" search_vector = model.encode(search_sentence) # Normalise the search vector for cosine distance, can be ignored if using # euclidean distance search_vector = search_vector / np.linalg.norm(search_vector) print(search_vector.shape) # Ask SemaDB to perform vector search import pprint payload = { "query": { "property": "vector", "vectorVamana": { "vector": search_vector.tolist(), "operator": "near", "searchSize": 75, "limit": 3 } }, # Restrict what is returned "select": ["sentence"], "limit": 3 } response = requests.post(base_url+"/collections/mycollection/points/search", json=payload, headers=headers) pprint.pprint(response.json()) ``` -------------------------------- ### Run SemaDB with Podman Source: https://context7.com/semafind/semadb/llms.txt Similar to the Docker command, this shows how to run SemaDB using Podman, including SELinux relabeling for persistent volumes. This is suitable for environments where Podman is preferred over Docker. ```bash # Run with Podman (with SELinux relabeling) podman run -it --rm \ -v ./config:/config:Z \ -e SEMADB_CONFIG=/config/singleServer.yaml \ -v ./data:/data:Z \ -p 8081:8081 \ -p 8091:8091 \ ghcr.io/semafind/semadb:main ``` -------------------------------- ### Running Load Random Script for Bulk Data Insertion Source: https://github.com/semafind/semadb/blob/main/CONTRIBUTING.md This command executes a script to insert random vectors into SemaDB for performance testing. Users should adjust configuration for maximum point size and number of points as needed. It relies on the Go runtime. ```bash go run ./internal/loadrand ``` -------------------------------- ### String Array Search Query Example (JSON) Source: https://github.com/semafind/semadb/blob/main/docs/content/docs/search/basic.md Illustrates a search query for a string array property in SemaDB. This example uses the 'containsAll' operator to find documents where the 'tags' property includes all specified tags. This is useful for filtering based on multiple criteria like categories or labels. ```json { "query": { "property": "tags", "stringArray": { "value": ["tag1", "tag2"], "operator": "containsAll" } }, "limit": 10 } ```