### Install Development Dependencies Source: https://github.com/milvus-io/pymilvus/blob/master/tests/benchmark/README.md Installs the necessary development dependencies for the project using uv. ```bash uv sync --group dev ``` -------------------------------- ### Database Operations Example Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/06-connection-and-orm.md Demonstrates creating, listing, and dropping databases using the db module. ```python from pymilvus.orm import db # Create database db.create_database("analytics") # List databases databases = db.list_databases() # Drop database db.drop_database("analytics") ``` -------------------------------- ### Install Pre-commit Hooks Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Install pre-commit and set up git hooks to automatically check and fix coding styles before each commit. ```shell # Install pre-commit (if not already installed) $ uv tool install pre-commit # Install the git hook scripts $ pre-commit install ``` -------------------------------- ### Inline Setup vs. Fixture Usage in Pytest Source: https://github.com/milvus-io/pymilvus/blob/master/tests/unit/README.md Demonstrates the incorrect approach of inline setup for tests, which leads to verbose code, and the correct approach using `conftest.py` fixtures for cleaner test definitions. ```python # WRONG: inline setup (5+ lines per test) def test_something(self): handler = MagicMock() handler.get_server_type.return_value = "milvus" handler._wait_for_channel_ready = MagicMock() with patch("pymilvus.client.grpc_handler.GrpcHandler", return_value=handler): client = MilvusClient() ... ``` ```python # RIGHT: use conftest fixture (0 setup lines) def test_something(self, mock_milvus_client): client, handler = mock_milvus_client ... ``` -------------------------------- ### Install from Test PyPI Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Install a development version of PyMilvus from Test PyPI using the --extra-index-url flag. ```shell python3 -m pip install --extra-index-url https://test.pypi.org/simple/ pymilvus==2.1.0.dev66 ``` -------------------------------- ### Install Local PyMilvus Repository Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Install the local PyMilvus repository for use with a Milvus server. ```shell make install ``` -------------------------------- ### Install PyMilvus Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Install PyMilvus using pip. Install with optional dependencies for milvus-model or bulk_writer. ```shell pip3 install pymilvus pip3 install pymilvus[model] # for milvus-model pip3 install pymilvus[bulk_writer] # for bulk_writer ``` -------------------------------- ### Example: Using MetricType in Search and Index Creation Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/07-exceptions-and-types.md Illustrates how to specify the metric type for search queries and index creation. ```python # In search results = client.search( collection_name="movies", data=vector, param={"metric_type": "COSINE"} ) # In index creation params.add_index( field_name="embedding", metric_type="L2" ) ``` -------------------------------- ### URI Parsing Examples Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Demonstrates various URI formats and their resulting connection configurations, including standard HTTP/S, Milvus-lite, and Unix domain sockets. ```text | URI | Result | |--------|--------| | `http://localhost:19530` | `address=localhost:19530, token=""` | | `https://host:19530/mydb` | `address=host:19530, db_name=mydb, secure=True` | | `https://user:pass@host:19530` | `address=host:19530, token=user:pass, secure=True` | | `./local.db` | milvus-lite: starts local server, rewrites to gRPC URI | | `unix:/var/run/milvus.sock` | `address=unix:/var/run/milvus.sock` (raw pass-through) | ``` -------------------------------- ### MilvusClient Constructor Examples Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/01-milvus-client.md Instantiate MilvusClient with various connection and authentication options. Ensure the URI and authentication details are correct for your Milvus instance. ```python from pymilvus import MilvusClient # Basic connection client = MilvusClient(uri="http://localhost:19530") # With authentication client = MilvusClient( uri="https://example.com:19538", user="admin", password="secret" ) # With token client = MilvusClient(uri="http://localhost:19530", token="admin:secret") # With timeout client = MilvusClient(uri="http://localhost:19530", timeout=30.0) ``` -------------------------------- ### Complete Workflow: Benchmarking and Profiling Source: https://github.com/milvus-io/pymilvus/blob/master/tests/benchmark/README.md A comprehensive workflow combining dependency installation, timing benchmarks, CPU profiling, and memory profiling to identify and fix performance bottlenecks. ```bash # Step 1: Install dependencies uv sync --group dev # Step 2: Run timing benchmarks (fast, ~minutes) pytest tests/benchmark/ --benchmark-only # Step 3: Identify slow tests from benchmark results # Step 4: CPU profile specific slow tests py-spy record -o cpu_slow_test.svg -- pytest tests/benchmark/test_search_bench.py::test_slow_one -v # Step 5: Memory profile tests with large results memray run -o mem_large.bin pytest tests/benchmark/test_search_bench.py::test_large_results -v memray flamegraph mem_large.bin # Step 6: Analyze results and fix bottlenecks # Step 7: Re-run benchmarks and compare with baseline pytest tests/benchmark/ --benchmark-only --benchmark-compare=baseline ``` -------------------------------- ### Accessing Aggregation Bucket Key Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Example showing how to access and print the key of the first aggregation bucket. ```python bucket = agg.buckets[0] print(f"Group key: {bucket.key}") # e.g., "2023" or 100 ``` -------------------------------- ### FunctionChain Builder Examples Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-003-function-chain-api-design.md Demonstrates how to build a FunctionChain using method chaining. It shows the resulting operation, output, and expression for 'map', 'sort', and 'limit' operations, preserving the order of operations. ```python map emits op="map", output, and expr. sort(col("$score")) emits op="sort", input, and typed column/desc params. limit(2) emits op="limit" and typed limit param. method chaining preserves operation order. ``` -------------------------------- ### Iterating Through TopHits Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Example demonstrating how to iterate over the top hits in a bucket and print their ID and distance. ```python top_hits = bucket.top_hits for hit in top_hits: print(f"ID: {hit['id']}, Distance: {hit['distance']}") ``` -------------------------------- ### Search Results with Pagination Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Demonstrates how to paginate through search results by using the 'offset' parameter. This example shows fetching the first page and then fetching the next page. ```python # Pagination through results results = collection.search( data=query, anns_field="embedding", param={"metric_type": "COSINE"}, limit=5, offset=0 ) result = results[0] print(f"Page 1: {result.ids}") # Next page results = collection.search( data=query, anns_field="embedding", param={"metric_type": "COSINE"}, limit=5, offset=5 ) ``` -------------------------------- ### RRFRanker Usage Examples Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/08-search-rankers-and-requests.md Demonstrates how to instantiate RRFRanker with default and custom 'k' values. The default 'k' is 60, while a custom value like 100 can be used to emphasize top results more. ```python from pymilvus import RRFRanker # Default RRF with k=60 ranker = RRFRanker() # Custom k value (emphasize top results more) ranker = RRFRanker(k=100) # Larger k = smaller impact of rank differences ``` -------------------------------- ### Iterating Through Aggregation Buckets Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Example demonstrating how to iterate through the aggregation buckets obtained from a search result and print the key and count for each bucket. ```python agg = result.agg_results for bucket in agg.buckets: print(f"Group: {bucket.key}, Count: {bucket.count}") ``` -------------------------------- ### Example: Adding an Index with IndexType Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/07-exceptions-and-types.md Shows how to add an index to a collection using either a string representation or the IndexType enum. ```python from pymilvus import IndexParams params = IndexParams() params.add_index( field_name="embedding", index_type="HNSW", # Can use string or IndexType.HNSW metric_type="COSINE" ) ``` -------------------------------- ### Create Multi-Field Indexes Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/11-index-management.md This example shows how to create indexes for multiple fields, including vector fields with different index types and metric types, and a scalar field for filtering. It uses the IndexParams class to define all indexes before applying them. ```python params = IndexParams() # Vector field index params.add_index( field_name="text_embedding", index_type="HNSW", metric_type="COSINE", M=30, efConstruction=200 ) # Second vector field params.add_index( field_name="image_embedding", index_type="HNSW", metric_type="L2", M=20, efConstruction=100 ) # Scalar field index for filtering params.add_index( field_name="category", index_type="AUTOINDEX" ) # Create all indexes client.create_index("products", params) ``` -------------------------------- ### Initialize IndexParams Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/11-index-management.md Create an empty IndexParams instance to hold multiple index configurations. This is the starting point for defining indexes for different fields. ```python from pymilvus.milvus_client.index import IndexParams params = IndexParams() ``` -------------------------------- ### Iterating Through Top Hits in a Bucket Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Example demonstrating how to access the top hits within an aggregation bucket and iterate through them. ```python top = bucket.top_hits for hit in top: print(hit) ``` -------------------------------- ### Example: Using DataType Enum for Field Definitions Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/07-exceptions-and-types.md Demonstrates how to use the DataType enum when defining field schemas for collections. ```python from pymilvus import DataType, FieldSchema # Use DataType enum for field definitions field = FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=384) field = FieldSchema("name", DataType.VARCHAR, max_length=100) field = FieldSchema("count", DataType.INT64) ``` -------------------------------- ### Create and Use a Partition Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Demonstrates creating a new partition, loading it into memory, and performing searches within that specific partition. Ensure the collection exists before creating a partition. ```python from pymilvus import Collection, Partition collection = Collection("movies") # Create new partition partition = Partition(collection, name="recent_movies", description="Movies from 2024") # Load partition partition.load() # Operate on partition results = collection.search( data=query, partition_names=["recent_movies"], anns_field="embedding", param={"metric_type": "COSINE"}, limit=10 ) ``` -------------------------------- ### Install Specific PyMilvus Version Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Install a specific version of PyMilvus using pip. ```shell pip3 install pymilvus==2.4.10 ``` -------------------------------- ### Partition.create() Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Creates the partition on the Milvus server. ```APIDOC ## create Creates the partition on the server. ### Method ```python def create(self, timeout: Optional[float] = None, **kwargs) -> None ``` ### Parameters #### Query Parameters - **timeout** (Optional[float]) - Optional - Operation timeout - **kwargs** (dict) - Optional - Additional options Returns: None Raises: `PartitionAlreadyExistException` if already exists ``` -------------------------------- ### col Helper Function Examples Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-003-function-chain-api-design.md Illustrates the usage of the 'col' helper function to create column arguments for Function Chain expressions. It shows how to reference regular columns, system-name columns like '$score', and rejects invalid column names. ```python col("ts") creates a column arg. col("$score") creates a system-name column arg. empty or non-string column names are rejected. ``` -------------------------------- ### fn Factory Function Examples Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-003-function-chain-api-design.md Showcases the 'fn' factory functions for creating common Function Chain expressions, including numerical combinations, decay functions, rounding, and reranking models. It also highlights the rejection of bare string positional arguments. ```python fn.num_combine(col("$score"), col("ts"), mode="sum") fn.num_combine(col("a"), col("b"), mode="weighted", weights=[0.7, 0.3]) fn.decay(col("ts"), function="linear", origin=now, scale=86400) fn.round_decimal(col("$score"), decimal=3) fn.rerank_model(col("doc"), queries=[...], provider="...") bare string positional arg rejection ``` -------------------------------- ### Create Tenant-Specific Partitions and Query Data Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/INDEX.md Demonstrates creating partitions for different tenants within a collection and querying data specific to a tenant. This is useful for multi-tenant applications. ```python from pymilvus import Collection, Partition collection = Collection("multi_tenant") for tenant_id in ["customer_a", "customer_b"]: Partition(collection, tenant_id).create() # Query tenant-specific data tenant_partition = Partition(collection, "customer_a") results = tenant_partition.search(data=[...]) ``` -------------------------------- ### alias Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Gets all aliases for the collection. ```APIDOC ## alias ### Description Gets all aliases for the collection. ### Method Signature ```python @property def alias(self) -> List[str] ``` ### Returns * **List[str]** - A list of aliases for the collection. ``` -------------------------------- ### partitions Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Gets all partitions in the collection. ```APIDOC ## partitions ### Description Gets all partitions in the collection. ### Method Signature ```python @property def partitions(self) -> List[Partition] ``` ### Returns * **List[Partition]** - A list of Partition objects. ``` -------------------------------- ### name property Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/04-schema-and-fields.md Gets the name of the field. ```APIDOC ### Properties #### name ```python @property def name(self) -> str ``` Gets field name. ``` -------------------------------- ### Using Database Context Manager Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/06-connection-and-orm.md Shows how to use the `using_database` context manager to temporarily switch to a specific database for operations. ```python from pymilvus.orm import db with db.using_database("analytics"): collection = Collection("sales_data") # Operations here use "analytics" database ``` -------------------------------- ### dtype property Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/04-schema-and-fields.md Gets the data type of the field. ```APIDOC #### dtype ```python @property def dtype(self) -> DataType ``` Gets data type. ``` -------------------------------- ### Create and Load Collection using ORM Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Demonstrates how to create a new collection with a defined schema or load an existing collection using the Collection constructor. Ensure a connection is established before creating or loading. ```python from pymilvus import Collection, CollectionSchema, FieldSchema, DataType, connections # First establish a connection connections.connect("default", host="localhost", port=19530) # Create new collection with schema fields = [ FieldSchema("movie_id", DataType.INT64, is_primary=True), FieldSchema("title", DataType.VARCHAR, max_length=200), FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=384) ] schema = CollectionSchema(fields=fields, description="Movie database") collection = Collection("movies", schema=schema) # Load existing collection collection = Collection("movies") ``` -------------------------------- ### Get Collection Description - Python Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves the description of the collection. ```python @property def description(self) -> str ``` -------------------------------- ### Get Collection Name - Python Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves the name of the collection. ```python @property def name(self) -> str ``` -------------------------------- ### Basic Vector Search with MilvusClient Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/README.md Demonstrates creating a collection, inserting data, loading the collection, and performing a basic vector search. Ensure Milvus is running and accessible at the specified address. ```python from pymilvus import MilvusClient client = MilvusClient("http://localhost:19530") client.create_collection("movies", dimension=384) client.insert("movies", [{"id": 1, "vector": [...], "title": "Inception"}]) client.load_collection("movies") results = client.search("movies", data=[...], limit=10) ``` -------------------------------- ### Index Field Name Property Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/11-index-management.md Gets the name of the field that is indexed. ```APIDOC ## Index Field Name Property ### Description Gets the indexed field name. ### Signature ```python field_name(self) -> str ``` ### Returns - **str**: The name of the indexed field. ``` -------------------------------- ### Initialize Milvus Client with Global Support Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-001-global-client-design.md Demonstrates how to initialize the MilvusClient for both standard and global cluster connections. The global connection uses the same API as the standard connection, ensuring transparency. ```python client = MilvusClient(uri="https://in01-xxx.zilliz.com", token="...") client = MilvusClient(uri="https://glo-xxx.global-cluster.vectordb.zilliz.com", token="...") ``` -------------------------------- ### Get Aliases Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves a list of all aliases assigned to the collection. This is a property access. ```python @property def alias(self) -> List[str] ``` -------------------------------- ### AsyncMilvusClient Initialization Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Initializes the AsyncMilvusClient, storing configuration but deferring actual connection to the first use. Suitable for asynchronous operations. ```python class AsyncMilvusClient: def __init__(self, uri, token, *, dedicated=False, **kwargs): self._config = ConnectionConfig.from_uri(uri, token=token, **kwargs) self._manager = None self._handler = None ``` -------------------------------- ### Get Partitions Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves a list of all partitions associated with the collection. This is a property access. ```python @property def partitions(self) -> List[Partition] ``` -------------------------------- ### Get Collection Schema - Python Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves the schema object associated with the collection. ```python @property def schema(self) -> CollectionSchema ``` -------------------------------- ### Create Customer/Tenant Partitions Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Ideal for SaaS multi-tenancy to ensure data isolation and enable tenant-specific operations. ```python collection = Collection("shared_app") for customer_id in customer_ids: Partition(collection, f"customer_{customer_id}").create() ``` -------------------------------- ### Get Field Name Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/04-schema-and-fields.md Access the `name` property to retrieve the name of the field. ```python @property def name(self) -> str: ``` -------------------------------- ### MilvusClient Initialization and Connection Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Initializes the MilvusClient and establishes a connection via the ConnectionManager. Use this for synchronous operations. ```python class MilvusClient: def __init__(self, uri, token, *, dedicated=False, **kwargs): self._manager = ConnectionManager.get_instance() self._handler = self._manager.get_or_create(config, dedicated, client=self) ``` -------------------------------- ### Get Number of Entities - Python Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves the total count of entities currently stored in the collection. ```python @property def num_entities(self) -> int ``` -------------------------------- ### Create Database Async Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Use this method to create a new database in Milvus. Specify the database name and optional timeout. ```python async def create_db( self, db_name: str, timeout: Optional[float] = None, **kwargs ) -> None ``` -------------------------------- ### Get Field Data Type Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/04-schema-and-fields.md Access the `dtype` property to retrieve the data type of the field. ```python @property def dtype(self) -> DataType: ``` -------------------------------- ### Convert FieldSchema to Dictionary Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/04-schema-and-fields.md Use the `to_dict` method to get a dictionary representation of the FieldSchema object. ```python def to_dict(self) -> Dict: ``` -------------------------------- ### Bulk Data Import using LocalBulkWriter Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/README.md Demonstrates how to prepare and write data in bulk for import into Milvus using `LocalBulkWriter`. This method is efficient for large datasets. ```python from pymilvus.bulk_writer import LocalBulkWriter, BulkFileType schema = CollectionSchema([ FieldSchema("id", DataType.INT64, is_primary=True), FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=384), ]) writer = LocalBulkWriter(schema, "/tmp/import", chunk_size=50_000) for row in data: writer.append_row(row) writer.commit() ``` -------------------------------- ### Get the Number of Search Results Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Returns the total number of results contained within the SearchResult object. ```python def __len__(self) -> int: ``` ```python num_results = len(result) ``` -------------------------------- ### Get Git Submodules Source: https://github.com/milvus-io/pymilvus/blob/master/README.md Fetch git submodules, which may include protos required for Milvus development. ```shell git submodule update --init ``` -------------------------------- ### Create AsyncMilvusClient Instance Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Instantiates an asynchronous Milvus client. Use the context manager for recommended resource management or manual connection and disconnection. ```python import asyncio from pymilvus import AsyncMilvusClient async def main(): # Option 1: Context manager (recommended) async with AsyncMilvusClient(uri="http://localhost:19530") as client: await client.create_collection("movies", dimension=384) # Option 2: Manual connection client = AsyncMilvusClient(uri="http://localhost:19530") await client._connect() try: await client.create_collection("movies", dimension=384) finally: await client.close() asyncio.run(main()) ``` -------------------------------- ### FieldOp.replace() Examples Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/12-field-operations.md FieldOp.replace is used to replace the entire value of a field. It is the default operation for non-array fields. ```python from pymilvus import FieldOp # Replace string field value op = FieldOp.replace() # Replace numeric field op = FieldOp.replace() ``` -------------------------------- ### Get Primary Field Schema - Python Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Retrieves the schema definition for the collection's primary key field. ```python @property def primary_field(self) -> FieldSchema ``` -------------------------------- ### Connect, Create DB/Collection, List, Disconnect Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/06-connection-and-orm.md Connect to Milvus, create a new database, define and create a collection within that database, list available databases and collections, and finally disconnect. ```python from pymilvus import connections, Collection, CollectionSchema, FieldSchema, DataType from pymilvus.orm import db, utility # Connect to Milvus connections.connect( alias="default", host="localhost", port=19530, user="admin", password="secret" ) # Create database db.create_database("analytics") # Create collection in database with db.using_database("analytics"): fields = [ FieldSchema("id", DataType.INT64, is_primary=True), FieldSchema("vector", DataType.FLOAT_VECTOR, dim=384) ] schema = CollectionSchema(fields=fields) collection = Collection("embeddings", schema=schema) # List databases databases = db.list_databases() print(f"Databases: {databases}") # List collections in database collections = utility.list_collections() print(f"Collections: {collections}") # Disconnect connections.disconnect() ``` -------------------------------- ### Establish Milvus Connections Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/06-connection-and-orm.md Use the `connect` method to establish connections to Milvus. You can specify connection details like host, port, user, password, or use a URI. Multiple named connections can be managed for different environments or purposes. ```python from pymilvus import connections # Basic connection connections.connect() # Uses default localhost:19530 # Specific host/port connections.connect( alias="prod", host="milvus.example.com", port=19530, user="admin", password="secret" ) # Using URI connections.connect( alias="cloud", uri="https://user:pass@cloud.example.com:19538" ) # Multiple connections connections.connect(alias="local", host="localhost", port=19530) connections.connect(alias="remote", host="remote.example.com", port=19530) ``` -------------------------------- ### SearchRequest Preparation with Function Chains Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-003-function-chain-api-design.md Illustrates how Function Chains are integrated into SearchRequests. It covers accepting single or multiple FunctionChains, populating the request, and rejecting invalid combinations like ranker with function_chains. ```python single FunctionChain is accepted. list of FunctionChain is accepted. function chains populate request.function_chains. ranker plus function_chains is rejected. invalid function_chains input is rejected. ``` -------------------------------- ### Get Collection Statistics Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/01-milvus-client.md Use this to fetch statistical information about a collection, such as the total number of rows. The result is returned as a dictionary. ```python stats = client.get_collection_stats("movies") print(f"Total rows: {stats['row_count']}") ``` -------------------------------- ### Create Partitions for Tenant Isolation Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Create partitions for a collection to isolate data for different tenants. Each partition can hold data for a specific tenant. ```python from pymilvus import Collection, Partition collection = Collection("multi_tenant_app") # Create partition per tenant for tenant_id in ["tenant_a", "tenant_b", "tenant_c"]: partition = Partition( collection, name=tenant_id, description=f"Data for {tenant_id}" ) partition.create() # Query tenant-specific data tenant_partition = Partition(collection, "tenant_a") results = tenant_partition.search( data=query, anns_field="embedding", param={"metric_type": "COSINE"} ) ``` -------------------------------- ### Create a Partition Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Creates a partition on the Milvus server. This method should be called after initializing a Partition object. ```python partition = Partition(collection, "2024_movies") partition.create() ``` -------------------------------- ### Handle MilvusUnavailableException Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/07-exceptions-and-types.md Example of catching MilvusUnavailableException when a Milvus client cannot connect to the server. Implement retry logic or fallback mechanisms. ```python try: client = MilvusClient("http://unreachable-host:19530") client.search(...) except MilvusUnavailableException as e: print(f"Server unavailable: {e.message}") # Implement retry logic or fallback ``` -------------------------------- ### Define Schema and Create Collection Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/INDEX.md Shows how to define a collection schema with various field types, including primary keys, VARCHAR, and FLOAT_VECTOR, and then create the collection using MilvusClient. Dynamic fields can be enabled. ```python from pymilvus import CollectionSchema, FieldSchema, DataType, MilvusClient fields = [ FieldSchema("id", DataType.INT64, is_primary=True), FieldSchema("title", DataType.VARCHAR, max_length=200), FieldSchema("embedding", DataType.FLOAT_VECTOR, dim=384), ] schema = CollectionSchema(fields, enable_dynamic_field=True) client = MilvusClient() client.create_collection("movies", schema=schema) ``` -------------------------------- ### Async Get Load State Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Asynchronously check the current loading state of a collection. This can be useful to determine if a collection is ready for operations. ```python async def get_load_state( self, collection_name: str, timeout: Optional[float] = None, **kwargs ) -> str ``` -------------------------------- ### Async Create Partition Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Asynchronously create a new partition within a specified collection. Partitions help in organizing and managing data. ```python async def create_partition( self, collection_name: str, partition_name: str, timeout: Optional[float] = None, **kwargs ) -> None ``` -------------------------------- ### URI Component Mapping to ConnectionConfig Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Maps components of a connection URI to their corresponding attributes in the `ConnectionConfig` object, providing examples for clarity. ```text | URI Component | Maps To | Example | |---------------|---------|---------| | `user:pass@` | `token` | `user:pass@` → `token="user:pass"` | | `host:port` | `address` | `host:19530` | | `/path` | `db_name` | `/mydb` → `db_name="mydb"` | ``` -------------------------------- ### FieldOp.array_append() Examples Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/12-field-operations.md Use FieldOp.array_append to add items to an existing array field. This method can append a single item or a list of items. ```python from pymilvus import FieldOp # Append single item to array op = FieldOp.array_append(5) # Append multiple items op = FieldOp.array_append([1, 2, 3]) # Append strings op = FieldOp.array_append(["tag1", "tag2"]) ``` -------------------------------- ### Creating Unique MilvusClient Connections with Aliases (Python) Source: https://github.com/milvus-io/pymilvus/blob/master/examples/manage_milvus_client/how_to_manage_milvus_client.md Demonstrates how to establish multiple, independent connections to Milvus servers by assigning unique aliases to `MilvusClient` instances. This is useful when single connection performance is insufficient or when managing distinct server endpoints. ```python def advanced_unique_connections(): c1 = MilvusClient(uri=URI, alias="c1-alias") c2 = MilvusClient(uri=URI, alias="c2-alias") ``` -------------------------------- ### PyMilvus Design Document Header Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/README.md Every design document must start with this header, including title, creation/update dates, and author. ```markdown # [Title] - **Created:** YYYY-MM-DD - **Updated:** YYYY-MM-DD - **Author(s):** @github-handle ``` -------------------------------- ### Load a Partition with Replicas Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Loads a partition into memory for searching, specifying the number of replicas for high availability. Ensure the partition is created before loading. ```python partition.load(replica_number=2) ``` -------------------------------- ### Create Partition Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/03-collection-orm.md Creates a new partition within a collection. Requires partition name and optionally accepts a description and timeout. ```python def create_partition( self, partition_name: str, description: str = "", timeout: Optional[float] = None, **kwargs ) -> Partition ``` -------------------------------- ### Get Server Type (Synchronous) Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Returns the type of the Milvus server. Note that this is a synchronous method and should not be awaited. It may raise MilvusException if not connected. ```python def get_server_type(self) -> str ``` -------------------------------- ### AsyncMilvusClient Constructor Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/02-async-milvus-client.md Creates an asynchronous Milvus client. Requires async context or explicit connection. Supports various connection parameters and additional options. ```APIDOC ## AsyncMilvusClient Constructor ### Description Creates an asynchronous Milvus client. Requires async context or explicit connection. ### Signature ```python AsyncMilvusClient( uri: str = "http://localhost:19530", user: str = "", password: str = "", db_name: str = "", token: str = "", timeout: Optional[float] = None, **kwargs ) -> AsyncMilvusClient ``` ### Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | uri | str | No | "http://localhost:19530" | Connection URI | | user | str | No | "" | Username for authentication | | password | str | No | "" | Password for authentication | | db_name | str | No | "" | Target database name | | token | str | No | "" | Authentication token | | timeout | Optional[float] | No | None | Request timeout in seconds | | **kwargs | dict | No | | Additional options | ### Returns AsyncMilvusClient instance ### Example ```python import asyncio from pymilvus import AsyncMilvusClient async def main(): # Option 1: Context manager (recommended) async with AsyncMilvusClient(uri="http://localhost:19530") as client: await client.create_collection("movies", dimension=384) # Option 2: Manual connection client = AsyncMilvusClient(uri="http://localhost:19530") await client._connect() try: await client.create_collection("movies", dimension=384) finally: await client.close() asyncio.run(main()) ``` ``` -------------------------------- ### AsyncMilvusClient Establish Connection Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Establishes the asynchronous connection to Milvus. This method is idempotent and ensures connection is made only once. ```python async def _connect(self): """Establish the async connection (idempotent).""" if self._handler is not None: return self._manager = AsyncConnectionManager.get_instance() self._handler = await self._manager.get_or_create( self._config, dedicated, client=self ) ``` -------------------------------- ### Solve zsh: no matches found Error Source: https://github.com/milvus-io/pymilvus/blob/master/README.md When encountering the 'zsh: no matches found' error with package extras, install using quoted package names. ```shell pip install "pymilvus[model]" ``` -------------------------------- ### Initialize Role Object Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Instantiate a Role object with a name and an optional connection alias. ```python from pymilvus.orm.role import Role # Create role object role = Role("data_analyst") ``` -------------------------------- ### AsyncMilvusClient Get Connection Handler Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Retrieves the connection handler, automatically establishing a connection if one does not already exist. Ensures connection before returning the handler. ```python async def _get_connection(self): """Return handler, auto-connecting if needed.""" if self._handler is None: await self._connect() return self._handler ``` -------------------------------- ### MilvusClient Constructor Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/01-milvus-client.md Creates a new MilvusClient instance that connects to a Milvus server. Supports various connection parameters including URI, user credentials, database name, authentication token, and request timeout. ```APIDOC ## MilvusClient Constructor ### Description Creates a new MilvusClient instance that connects to a Milvus server. ### Signature ```python MilvusClient( uri: str = "http://localhost:19530", user: str = "", password: str = "", db_name: str = "", token: str = "", timeout: Optional[float] = None, **kwargs ) -> MilvusClient ``` ### Parameters #### Path Parameters * **uri** (str) - Optional - Connection URI. Format: `scheme://[user:password@]host[:port]`. Examples: "http://localhost:19530", "https://user:pass@example.com:19538" * **user** (str) - Optional - Username for authentication * **password** (str) - Optional - Password for authentication * **db_name** (str) - Optional - Target database name * **token** (str) - Optional - Authentication token (overrides user/password if provided) * **timeout** (Optional[float]) - Optional - Request timeout in seconds * **kwargs** (dict) - Optional - Additional options (dedicated, cluster_id, etc.) ### Returns MilvusClient instance ### Example ```python from pymilvus import MilvusClient # Basic connection client = MilvusClient(uri="http://localhost:19530") # With authentication client = MilvusClient( uri="https://example.com:19538", user="admin", password="secret" ) # With token client = MilvusClient(uri="http://localhost:19530", token="admin:secret") # With timeout client = MilvusClient(uri="http://localhost:19530", timeout=30.0) ``` ``` -------------------------------- ### Create Collection with MilvusClient (Python) Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/README.md Use MilvusClient for modern API interactions. Ensure Milvus is running at the specified address. ```python from pymilvus import MilvusClient client = MilvusClient("http://localhost:19530") client.create_collection("movies", dimension=384) ``` -------------------------------- ### Create Index using ORM Style Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/11-index-management.md This snippet demonstrates index creation using the ORM style with the Collection object. It defines index parameters within a dictionary and creates the index, which then builds in the background. It also includes waiting for the index building to complete. ```python from pymilvus import Collection collection = Collection("movies") # Create HNSW index index = collection.create_index( field_name="embedding", index_params={ "index_type": "HNSW", "metric_type": "COSINE", "params": { "M": 30, "efConstruction": 200 } } ) # Index is created and building in background # Wait for completion from pymilvus import utility utility.wait_for_index_building_complete("movies") ``` -------------------------------- ### FieldOp.array_remove() Examples Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/12-field-operations.md Use FieldOp.array_remove to remove items from an array field. This method supports removing a single item or multiple items specified in a list. ```python from pymilvus import FieldOp # Remove single item op = FieldOp.array_remove(5) # Remove multiple items op = FieldOp.array_remove([1, 2, 3]) ``` -------------------------------- ### Get a Single Search Result by Index Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/05-search-results.md Retrieve a specific search result by its 0-based index. Returns a dictionary containing the entity data and its distance score. ```python def get(self, index: int) -> dict: ``` ```python first_result = result.get(0) print(f"ID: {first_result['id']}, Distance: {first_result['distance']}") ``` -------------------------------- ### Create Collection with Defaults Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/01-milvus-client.md Use this when you need a simple collection with auto-generated defaults. Ensure the dimension is provided. ```python client.create_collection( collection_name="movies", dimension=384 ) ``` -------------------------------- ### Partition.load() Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/13-partition-and-role.md Loads the partition into memory, making it available for search operations. ```APIDOC ## load Loads partition into memory for searching. ### Method ```python def load( self, replica_number: int = 1, timeout: Optional[float] = None, **kwargs ) -> None ``` ### Parameters #### Query Parameters - **replica_number** (int) - Optional - Number of replicas. Default is 1. - **timeout** (Optional[float]) - Optional - Operation timeout - **kwargs** (dict) - Optional - Additional options Returns: None ``` -------------------------------- ### Get Milvus Collection Load State Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/01-milvus-client.md Checks the current memory load state of a Milvus collection. Possible states include 'loaded', 'loading', 'not_loaded', or 'not_exist'. ```python client.get_load_state("movies") ``` -------------------------------- ### Asynchronous Operations with AsyncMilvusClient Source: https://github.com/milvus-io/pymilvus/blob/master/_autodocs/README.md Shows how to perform Milvus operations asynchronously using `AsyncMilvusClient`. This is beneficial for I/O-bound tasks to improve application responsiveness. ```python import asyncio from pymilvus import AsyncMilvusClient async def main(): async with AsyncMilvusClient() as client: await client.create_collection("movies", dimension=384) result = await client.insert("movies", data) results = await client.search("movies", data=[...]) asyncio.run(main()) ``` -------------------------------- ### PyMilvus Client Directory Structure Source: https://github.com/milvus-io/pymilvus/blob/master/docs/plans/pymilvus-002-connection-manager-design.md Overview of the file structure within the pymilvus/client directory, highlighting key modules related to connection management and handlers. ```text pymilvus/client/ ├── connection_manager.py │ ├── ConnectionConfig │ ├── ManagedConnection │ ├── ConnectionStrategy (ABC) │ ├── RegularStrategy │ ├── _GlobalStrategyMixin │ ├── GlobalStrategy │ ├── ConnectionManager (sync) │ ├── AsyncRegularStrategy │ ├── AsyncGlobalStrategy │ └── AsyncConnectionManager │ ├── global_topology.py # Topology helpers (fetch, refresh, data classes) ├── grpc_handler.py # Simplified: no global detection └── async_grpc_handler.py ```