### Install ColiVara SDK Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Installs the ColiVara SDK for Python or Typescript using pip or npm. ```bash pip install colivara-py ``` ```bash npm install colivara-ts ``` -------------------------------- ### Colivara Local Setup Environment Variables Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Configuration for the local development environment of Colivara. These variables define the connection details for the embeddings service and AWS S3 storage. ```bash EMBEDDINGS_URL="the serverless embeddings service url" # for local setup use http://host.docker.internal:8000/runsync EMBEDDINGS_URL_TOKEN="the serverless embeddings service token" # for local setup use any string will do. AWS_S3_ACCESS_KEY_ID="an S3 or compatible storage access key" AWS_S3_SECRET_ACCESS_KEY="an S3 or compatible storage secret key" AWS_STORAGE_BUCKET_NAME="an S3 or compatible storage bucket name" ``` -------------------------------- ### ColiVara SDK Example (Python) Source: https://github.com/tjmlabs/colivara/blob/main/readme.md An example of how to use the ColiVara Python SDK to interact with the API for document retrieval. This demonstrates uploading documents, performing searches, and retrieving results. ```python from colivara import ColiVara client = ColiVara(api_key="YOUR_API_KEY") # Upload a document client.upload_document(file_path="./my_document.pdf", collection_id="my_collection") # Search for documents results = client.search(query="What is RAG?", collection_id="my_collection") for result in results: print(f"Document: {result['document_id']}, Score: {result['score']}") ``` -------------------------------- ### Colivara Docker Compose Commands Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Commands to manage the Colivara application using Docker Compose. This includes starting services, applying database migrations, and creating a superuser for authentication. ```bash docker-compose up -d --build docker-compose exec web python manage.py migrate docker-compose exec web python manage.py createsuperuser ``` -------------------------------- ### Retrieve Superuser Token Source: https://github.com/tjmlabs/colivara/blob/main/readme.md This Python snippet retrieves the authentication token for the superuser created during the setup process. This token is used for authorization. ```python from accounts.models import CustomUser user = CustomUser.objects.first().token print(user) ``` -------------------------------- ### Clone Colivara Repository Source: https://github.com/tjmlabs/colivara/blob/main/readme.md This command clones the Colivara project repository from GitHub. It is the first step in setting up the project locally. ```bash git clone https://github.com/tjmlabs/ColiVara ``` -------------------------------- ### Run Colivara Tests Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Executes the test suite for the Colivara project using Docker Compose. The project boasts 100% test coverage. ```bash docker-compose exec web pytest ``` -------------------------------- ### Colivara Type Checking with Mypy Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Performs static type checking on the Colivara project using Mypy within the Docker environment to ensure code quality and identify potential type errors. ```bash docker-compose exec web mypy . ``` -------------------------------- ### Index Document with ColiVara Typescript SDK Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Indexes a document using the ColiVara Typescript SDK. Supports document URLs, file paths, or base64 encoded files. Allows specifying metadata and collection names, with an option to wait for indexing. ```typescript import { ColiVara } from 'colivara-ts'; // Initialize the client const client = new ColiVara('your-api-key'); // Upload a document const document = await client.upsertDocument({ name: 'sample_document', // optional - specify a collection collection_name: 'user_1_collection', // You can use a file path, base64 encoded file, or a URL document_url: 'https://example.com/sample.pdf', // optional - wait for the document to index. Webhooks are also supported. wait: true, // optional - add metadata metadata: { author: 'John Doe' } }); ``` -------------------------------- ### Index Document with ColiVara Python SDK Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Indexes a document using the ColiVara Python SDK. Supports document URLs, file paths, or base64 encoded files. Allows specifying metadata and collection names, with an option to wait for indexing. ```python from colivara_py import ColiVara client = ColiVara(api_key=os.environ.get("COLIVARA_API_KEY"), # default and can be omitted base_url="https://api.colivara.com" # default and can be omitted ) # Upload a document to the default_collection document = client.upsert_document( name="sample_document", document_url="https://example.com/sample.pdf", # You can use a file path, base64 encoded file, or a URL metadata={"author": "John Doe"}, # optional - add metadata collection_name="user_1_collection", # optional - specify a collection wait=True # optional - wait for indexing (supports Webhooks) ) ``` -------------------------------- ### Search Documents with ColiVara Python SDK Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Searches for documents using the ColiVara Python SDK. Supports simple text queries, filtering by collection name, and filtering by document or collection metadata. ```python # Simple search results = client.search("what is 1+1?") # search with a specific collection results = client.search("what is 1+1?", collection_name="user_1_collection") # Search with a filter on document metadata results = client.search( "what is 1+1?", query_filter={ "on": "document", "key": "author", "value": "John Doe", "lookup": "key_lookup", # or contains }, ) # Search with a filter on collection metadata results = client.search( "what is 1+1?", query_filter={ "on": "collection", "key": ["tag1", "tag2"], "lookup": "has_any_keys", }, ) # top 3 pages with the most relevant information print(results) ``` -------------------------------- ### ColiVara REST API - Document Upload Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Demonstrates how to upload a document to ColiVara using the REST API. This involves sending a POST request with the document file and specifying the collection it belongs to. ```shell curl -X POST https://api.colivara.ai/v1/documents \ -H "Authorization: Bearer YOUR_API_KEY" \ -F "file=@./my_document.pdf" \ -F "collection_id=my_collection" ``` -------------------------------- ### ColiVara REST API - Search Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Shows how to perform a search query against ColiVara using the REST API. This involves sending a POST request with the query and the target collection ID. ```shell curl -X POST https://api.colivara.ai/v1/search \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" \ -d '{ "query": "What is RAG?", "collection_id": "my_collection" }' ``` -------------------------------- ### Search Documents with ColiVara Typescript SDK Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Searches for documents using the ColiVara Typescript SDK. Supports simple text queries, filtering by collection name, and filtering by document or collection metadata. ```typescript // Simple search const results = await client.search({query: "what is 1+1?"}) // search with a specific collection const results = await client.search({query: "what is 1+1?", collection_name: "user_1_collection"}) // Search with a filter on document metadata const results = await client.search({ query: "what is 1+1?", query_filter: { on: "document", key: "author", value: "John Doe", lookup: "key_lookup" } }) // search with a filter on collection metadata const results = await client.search({ query: "what is 1+1?", query_filter: { on: "collection", key: ["tag1", "tag2"], lookup: "has_any_keys" } }) // top 3 pages with the most relevant information console.log(results) ``` -------------------------------- ### ColiVara REST API - Filtering Source: https://github.com/tjmlabs/colivara/blob/main/readme.md Illustrates how to filter search results based on metadata fields using the ColiVara REST API. This allows for more targeted retrieval by specifying criteria like author or year. ```shell curl -X POST https://api.colivara.ai/v1/search \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_API_KEY" \ -d '{ "query": "What is RAG?", "collection_id": "my_collection", "filter": { "author": "John Doe", "year": 2023 } }' ``` -------------------------------- ### ColiVara Embedding Endpoint Source: https://github.com/tjmlabs/colivara/blob/main/readme.md This endpoint generates embeddings for documents using vision models without storing them. It's useful for users who want to manage their own vector storage and leverage ColiVara's advanced embedding capabilities, such as late-interaction and multi-vectors. ```shell curl -X POST https://api.colivara.ai/v1/embed \ -H "Content-Type: application/json" \ -d '{ "documents": [ { "id": "doc1", "content": "This is the content of the first document." }, { "id": "doc2", "content": "This is the content of the second document." } ] }' ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.