https://aitoolchain.io/tools/milvus New releases and features in Milvus, tracked by The AI Toolchain. en Wed, 29 Jul 2026 14:27:06 GMT https://github.com/milvus-io/milvus/releases/tag/v3.0.0 https://github.com/milvus-io/milvus/releases/tag/v3.0.0 Wed, 29 Jul 2026 14:27:06 GMT Milvus 3.0.0 adds FAISS passthrough, TEXT long-text fields, SINDI sparse index, StructArray search, faceted search, composable reranking, and Woodpecker standalone deployment. • Adds `FAISS` index type accepting arbitrary Faiss index-factory strings via the `faiss_index_name` parameter (e.g., `IVF64,Flat`, `HNSW16,Flat`, `OPQ16,IVF64,PQ16x4`), with search parameters passed through so Faiss recipes reproduce directly on Milvus. • Adds `milvus-table` external format that treats Milvus Snapshot metadata and Storage V3 manifests as an external source, enabling a collection snapshot to be served as an external table for shared, manifest-backed access across batch and serving systems. • Introduces TEXT fields for long text with storage-side length limits removed, supporting `text_match`, `phrase_match`, and BM25; values under 64 KB stay inline, larger values go to partition-level LOB files in Vortex format storing only (file_id, offset) references. • Adds native XGBoost scoring for L0 reranking using UBJ models registered as FileResources, plus Hugging Face Inference Providers for server-managed text embedding and sentence-similarity reranking via the Function Chain API. • Overhauled sparse vector index with SINDI, Block-Max WAND, and Block-Max MaxScore algorithms, inverted-list compression, and configurable quantization; SINDI becomes the default for sparse IP search and MaxScore for BM25 once new index versions are enabled — compressed BM25 index is ~3x smaller than the 2.6 sparse index and SINDI reaches ~10x the QPS of MaxScore on learned sparse embeddings. • Supports online schema evolution: columns can be added, backfilled (external or inner/kernel-derived), and dropped on live collections without downtime, including BM25 and MinHash function output fields computed over existing data automatically. • Extends External Collection to support function output fields (BM25 sparse vectors, MinHash signatures, text embeddings) and additive schema refresh — new columns in the external table patch affected segments instead of rebuilding the collection. • Adds faceted search on the search path: specify a facet field at search time and Milvus returns top facet values with their best-matching ANN member and aggregates such as COUNT and AVG in a single request. • Expands StructArray with null value support, bitmap indexes, dynamic field addition on live collections, partial upsert of struct fields, REST and bulk-import coverage, and element-level hybrid search with per-entity collapse (`max` / `sum` / `avg` / top-k variants), range search, group-by, `element_filter` predicates, `MATCH_ANY` / `MATCH_ALL` / `MATCH_LEAST` / `MATCH_MOST` / `MATCH_EXACT` quantifiers, positional sub-field access such as `tags[0][name]`, and array_length() on the struct column. • Adds composable reranking through the Function Chain API, executing an ordered typed pipeline within a single search request combining L0 rescoring on QueryNode and L2 post-reduction reranking on Proxy, with score transformation, model-based reranking, sorting, and candidate trimming. • Adds Lance alongside Parquet for open-ecosystem storage interchange, and introduces Vortex as the next-generation internal columnar format with adaptive encodings (dictionary, RLE, bit-packing, float-specific compression) and zero-copy decompression. • Woodpecker WAL can now be deployed as an independent standalone service for independent scaling, fault isolation, and observability on large clusters and high-write workloads. Breaking changes: • GPU images now require CUDA 12.9 and no longer support Ubuntu 20.04 GPU compatibility. • Once Storage V3 is enabled or used (e.g., via `common.storage.useLoonFFI`), rollback from 3.0 to 2.6 is no longer possible, even though a 3.0 deployment that has not used Storage V3 features can be rolled back. https://github.com/milvus-io/milvus/releases/tag/v3.0.0 https://github.com/milvus-io/milvus/releases/tag/v3.0.0 Wed, 29 Jul 2026 14:27:06 GMT Milvus 3.0.0 adds FAISS passthrough, TEXT fields, SINDI sparse index, StructArray hybrid search, faceted search, Function Chain reranking, and Woodpecker standalone deployment. • Adds `FAISS` index type accepting arbitrary Faiss index-factory strings via the `faiss_index_name` parameter — e.g. `IVF64,Flat`, `HNSW16,Flat`, `OPQ16,IVF64,PQ16x4` — with search parameters passed through so Faiss recipes run directly on Milvus. • New index algorithms (SINDI, Block-Max WAND, Block-Max MaxScore) activate by setting `dataCoord.targetVecIndexVersion` to 10 and `dataCoord.targetScalarIndexVersion` to 4; SINDI becomes the default for sparse IP search and MaxScore for BM25 once enabled. • Storage V3 (Loon) is opt-in via `common.storage.useLoonFFI`; enabling it unlocks Snapshot and TEXT fields. • New `milvus-table` external format lets a collection snapshot be served as an external table, giving batch and serving systems a shared, manifest-backed view of the same data. • Adds TEXT field type for long text with no storage-side length limit, supporting `text_match`, `phrase_match`, and BM25; values under 64 KB stay inline, larger values go to partition-level LOB files in Vortex format storing (file_id, offset) references. • Introduces Function Chain API for composable reranking: an ordered, typed pipeline combining L0 rescoring on QueryNode with L2 post-reduction reranking on Proxy, supporting score transformation, model-based reranking, sorting, and candidate trimming in a single search request. • Adds native XGBoost scoring for L0 reranking using UBJ models registered as FileResources, plus Hugging Face Inference Providers for server-managed text embedding and sentence-similarity reranking. • Faceted search: specify a facet field at search time to receive top facet values annotated with COUNT and AVG aggregates alongside ANN ranking results in a single request. • Sparse index overhaul delivers a compressed BM25 index roughly 3x smaller than the 2.6 index at comparable recall, and SINDI reaches up to ~10x the QPS of MaxScore on learned sparse embeddings, with inverted-list compression and configurable quantization. • External Collection now supports function output fields (BM25 sparse vectors, MinHash signatures, text embeddings) fed from external fields, and additive schema refresh patches affected segments instead of rebuilding when the external table gains new columns. • Online schema evolution: columns can be added, backfilled (external backfill for offline-computed values; inner backfill for BM25/MinHash kernel-derived values), and dropped while the collection continues serving. • StructArray gains null value support, bitmap indexes, dynamic field addition on live collections, partial upsert of struct fields, REST and bulk-import coverage, and element-level hybrid search across vector sub-fields with configurable per-entity collapse (`max` / `sum` / `avg` / top-k variants). • StructArray nested filtering adds `element_filter` predicates, `MATCH_ANY` / `MATCH_ALL` / `MATCH_LEAST` / `MATCH_MOST` / `MATCH_EXACT` quantifiers, positional sub-field access such as `tags[0][name]`, and array_length() on the struct column. • Adds Vortex as a next-generation internal columnar format (adaptive encodings: dictionary, RLE, bit-packing, float-specific compression; zero-copy decompression) and Lance format alongside Parquet for open-ecosystem interchange. • Woodpecker WAL can now be deployed as a standalone independent service for separate scaling, fault isolation, and observability. Breaking changes: • GPU images move to CUDA 12.9 and no longer support Ubuntu 20.04 GPU compatibility. • Once Storage V3 features that change the serialized data format are enabled or used, rollback from 3.0 to 2.6 is no longer possible. https://github.com/milvus-io/milvus/releases/tag/v2.6.20 https://github.com/milvus-io/milvus/releases/tag/v2.6.20 Tue, 14 Jul 2026 06:44:48 GMT Milvus v2.6.20 improves query scheduling, batching, index loading, filter performance, and streaming rebalancing with observability upgrades. • Adds named C++ thread-pool activity metrics with Grafana monitoring support. • Enables the channel-level score balancer by default and introduces a safer default threshold for channel-exclusive mode, improving streaming rebalancing. • Improves QueryCoord scheduling by decoupling task dispatch from distribution polling to allow independent scheduling intervals. • Improves QueryNode query batching by increasing the default NQ grouping limits for larger merged query batches. • Improves filter execution performance by skipping null-bitmap processing for all-valid results. • Triggers streaming rebalancing immediately when the primary resource group configuration changes. • Improves index-loading resilience by safely completing pending range reads after partial failures. • Optimizes VARCHAR primary-key population when loading sealed segments. https://github.com/milvus-io/milvus/releases/tag/v2.6.19 https://github.com/milvus-io/milvus/releases/tag/v2.6.19 Fri, 26 Jun 2026 11:47:18 GMT Milvus v2.6.19 adds RBAC role descriptions, configurable array capacity, zero-copy search result paths, and JSON shredding by default. • Adds RBAC role description support across clients, APIs, and role metadata. • Adds a configuration option for maximum array capacity. • Adds an optional zero-copy path for passing search results, reducing search result serialization overhead. • Enables JSON shredding by default for improved JSON field handling. • Adds configurable concurrency for function runner text tokenization. • Improves S3 `PutObject` compatibility with OpenSSL FIPS mode by forcing CRC32C checksums. • Upgrades GPU Docker images to CUDA 12.9.1 for Ubuntu 22.04 builds and runtime. https://github.com/milvus-io/milvus/releases/tag/v2.6.18 https://github.com/milvus-io/milvus/releases/tag/v2.6.18 Fri, 05 Jun 2026 04:03:06 GMT Milvus v2.6.18 adds nullable vector fields, element-level Struct Array search, and HTTP/2 for the proxy REST server. • Adds HTTP/2 support for the proxy REST server, including h2c and ALPN-based TLS listeners. • Adds support for importing Arrow `FixedSizeList` data from Parquet into non-nullable array and dense vector fields. • Extends Arrow IO thread pool configuration to DataNode to improve compaction and import throughput. • Introduces nullable vector fields: vector fields can now be declared nullable so entities with missing or not-yet-generated embeddings can be inserted without a placeholder; NULL vectors consume no extra storage and are skipped automatically during search. • Enables element-level vector search on Struct Array fields, returning each result's matched element offset within the array rather than scoring the whole row. • Improves QueryNode read-task scheduling under heavy load with deadline-aware admission, cleanup, grouping, and metrics. • Limits QueryNode delegator post-load concurrency to reduce CPU spikes during segment loading. • Optimizes QueryCoord collection filtering in ChannelDistManager and reduces temporary allocations in distribution lookups. • Optimizes ReplicaManager locking to reduce cross-collection contention in QueryCoord. • Optimizes garbage collection for dropped segment index files and metadata. • Improves REST timeout handling to safely discard late handler writes after request timeouts. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.5 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.5 Tue, 26 May 2026 02:44:44 GMT Milvus Go SDK v2.6.5 adds nullable vector columns for all vector types and Array field partial update helpers for upsert requests. • Adds nullable column constructors `NewNullableColumnFloatVector`, `NewNullableColumnBinaryVector`, `NewNullableColumnFloat16Vector`, `NewNullableColumnBFloat16Vector`, `NewNullableColumnInt8Vector`, and `NewNullableColumnSparseFloatVector` for dense, binary, float16, bfloat16, int8, and sparse vector types; each accepts compact non-null vector values plus a `validData` mask and validates that values length matches the valid row count. • Extends `FieldDataColumn` to decode nullable vector field data using `ValidData` for `FloatVector`, `BinaryVector`, `Float16Vector`, `BFloat16Vector`, `SparseFloatVector`, and `Int8Vector`, preserving null rows while expanding compact wire-format payloads back to row-aligned columns. • Adds upsert helpers WithArrayAppend(fieldName), WithArrayRemove(fieldName), and WithFieldPartialOp(fieldName, op) for Array field partial updates; non-`REPLACE` ops are serialized into `UpsertRequest.FieldOps` and automatically enable `partial_update` for both column-based and row-based upserts. • Validates that vector fields passed to `AddCollectionField` are marked `nullable=true` before sending the RPC, rejecting invalid schema changes at the client. https://github.com/milvus-io/milvus/releases/tag/v2.6.17 https://github.com/milvus-io/milvus/releases/tag/v2.6.17 Fri, 22 May 2026 03:15:16 GMT Milvus v2.6.17 adds ARRAY_APPEND and ARRAY_REMOVE partial update operators for Array fields via gRPC and REST APIs. • Adds `ARRAY_APPEND` and `ARRAY_REMOVE` partial update operators for Array fields, exposed through both gRPC and REST upsert APIs. • Improves load/search isolation by using separate C++ executor pools and converting SegmentLoad and ReopenSegment to async futures with proper context cancellation. https://github.com/milvus-io/milvus/releases/tag/v2.6.16 https://github.com/milvus-io/milvus/releases/tag/v2.6.16 Thu, 14 May 2026 04:15:09 GMT Milvus v2.6.16 adds streaming node resource group isolation, a new REST config endpoint, SIMD-accelerated IN queries, and Go SDK struct-array support. • Adds `$partial_update` field to the proxy access log for Upsert requests, exposing both explicit and implicitly promoted partial-update flags. • Introduces streaming node resource group isolation with a new RESTful config inspection endpoint, allowing replicas to be assigned strictly within their configured resource groups. • Exposes Arrow IO thread pool capacity as a refreshable paramtable knob to relieve HIGH-pool stalls under heavy storage v2 read load. • Bumps Go SDK to v2.6.4 with full struct-array support (vector sub-fields, EmbeddingList search, schema validation), gRPC authority configuration, and preserved default gRPC dial options when custom `DialOptions` are provided. • Accelerates `TermExpr IN` evaluation with a SIMD (AVX2/AVX512) batch filter, significantly improving query performance for `IN` predicates. • Increases the default L0 compaction deltalog max count from 30 to 1000 to reduce compaction backlog under high-delete workloads. • Adds fast-fail retry capping and delegator stall detection so proxy queries failover to a healthy QueryNode immediately instead of exhausting the full backoff budget on a dead node. • Parallelizes text match index loading on QueryNode to speed up segment load for collections with text indexes. • Bypasses Knowhere search-pool scheduling for vector iterators to reduce per-Next overhead in iterator-heavy group-by search paths. • Reduces proxy tail latency and memory pressure during traffic storms by fast-failing Enqueue before TSO/ID allocation and using a non-blocking edge-triggered task notifier. • Rewrites sync manager key lock dispatcher with per-key FIFO queues and semaphore backpressure for non-blocking submission and graceful shutdown. • Allows simultaneous pchannel increase and cluster/topology changes in replication config validation. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.4 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.4 Fri, 08 May 2026 02:40:41 GMT Milvus Go SDK v2.6.4 adds struct-array vector sub-field columns, EmbeddingList/MAX_SIM search types, and gRPC authority routing. • Adds `ColumnFloatVectorArray`, `ColumnFloat16VectorArray`, `ColumnBFloat16VectorArray`, `ColumnBinaryVectorArray`, and `ColumnInt8VectorArray` column types for struct-array vector sub-fields supporting `ArrayOfVector` sub-fields inside struct arrays. • Adds `entity.FloatVectorArray`, `entity.Float16VectorArray`, `entity.BFloat16VectorArray`, `entity.BinaryVectorArray`, and `entity.Int8VectorArray` search vector types, dispatched to matching `PlaceholderType_EmbList*` values, enabling MAX_SIM / EmbeddingList search against struct-array vector sub-fields such as `clips[clip_emb]`. • Adds WithStructArrayColumn(name, structSchema, rows) helper to column-based insert/upsert options, accepting `[]map[string]any` row data and inferring scalar/vector sub-column types from the provided struct schema. • Adds ClientConfig.WithGrpcAuthority(authority) to set the gRPC `:authority` header for proxy-based routing. https://github.com/milvus-io/milvus/releases/tag/v2.6.15 https://github.com/milvus-io/milvus/releases/tag/v2.6.15 Fri, 17 Apr 2026 02:29:29 GMT Milvus v2.6.15 adds sub-millisecond latency metrics, a requery policy parameter, delegator-side segment pruning, and a querycoord load-priority override. • Adds `common.searchRequeryPolicy` parameter to control the requery stage in regular search, enabling fine-grained tuning of search result refresh behavior. • Adds a querycoord configuration to force-override segment load task priority, giving operators direct control over load scheduling. • Adds delegator-side segment pruning via PK predicate hints, reducing unnecessary data scanned during search. • Improves latency metrics precision by switching from milliseconds to microseconds for insert, delete, query, search, and upsert paths, preserving sub-millisecond resolution. • Separates metrics for internal upsert/delete requery operations from user-initiated queries, enabling more accurate monitoring of user-facing query performance. • Reduces remote IO syscalls during BM25 IDF preload by buffering the storage reader in `streamOneFile`. • Removes the unused SuffixSnapshot time-travel layer from RootCoord metadata storage, simplifying the catalog and eliminating orphaned snapshot keys. https://github.com/milvus-io/milvus/releases/tag/v2.6.14 https://github.com/milvus-io/milvus/releases/tag/v2.6.14 Tue, 07 Apr 2026 02:08:37 GMT Milvus v2.6.14 adds configurable thread pool sizing, force-failover DR via `UpdateReplicateConfiguration`, and data salvage for streaming-node failures. • Adds `common.threadCoreCoefficient.maxThreadsSize` config key to make thread pool max thread count configurable with dynamic (no-restart) update support. • Adds force-promote support for primary-secondary disaster-recovery failover via the `UpdateReplicateConfiguration` API. • Adds data salvage capability for force failover, recovering unpersisted data from failed streaming nodes. • Improves query filter performance with type-aware bidirectional rewriting between `in` and `==` expressions. • Optimizes unfiltered search on sealed segments using MVCC fast path, hardware `popcnt`, and redundant bitset operation elimination. • Switches import retry strategy from allowlist to denylist, improving resilience against transient errors during data import. • Reduces MixCoord recovery time by parallelizing startup phases including sub-meta loading, index reload, and batch etcd operations. • Aligns error mapping across Azure, GCP, and MinIO for consistent retry and error-handling behavior. • Optimizes bool `IN`/`NOT IN` expressions with proper nullable field handling. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.3 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.3 Tue, 24 Mar 2026 06:33:20 GMT Milvus Go SDK v2.6.3 adds TruncateCollection, replication config API, nullable pointer fields, and per-cluster mTLS for CDC. • Adds `TruncateCollection` method to the Go SDK client, invoked via NewTruncateCollectionOption(collectionName), to clear all data from a collection without dropping and recreating it. • Adds `GetReplicateConfiguration` API for viewing replication topology (with tokens redacted) and a `force_promote` field to `UpdateReplicateConfigurationRequest`; supports strong consistency via the `WithFreshRead` option. • Adds `TLSConfig` field to `ClientConfig` and a `BuildTLSConfig` helper for mTLS support in CDC; `NewMilvusClient` can now read per-cluster TLS config by target cluster ID via GetClusterTLSConfig(clusterID). • Go pointer struct fields (`*string`, `*int32`, etc.) now represent nullable columns in the row-based data path — a `nil` pointer maps to NULL — across `ParseSchema`, `AnyToColumns`, `SetField`, `fillData`, and `fillPKEntry`. https://github.com/milvus-io/milvus/releases/tag/v2.6.13 https://github.com/milvus-io/milvus/releases/tag/v2.6.13 Mon, 23 Mar 2026 07:58:53 GMT Milvus v2.6.13 adds Google Gemini embedding support and a Go SDK TruncateCollection method. • Adds `TruncateCollection` method to the Go SDK client for clearing all data in a collection without dropping it. • Adds Google Gemini as a built-in text embedding function, configurable via a Gemini API key, supporting models including Gemini Embedding 2. • Adds query metrics for JSON-related filter expressions to improve observability of JSON field query performance. https://github.com/milvus-io/milvus/releases/tag/v2.6.12 https://github.com/milvus-io/milvus/releases/tag/v2.6.12 Fri, 13 Mar 2026 07:24:42 GMT Milvus v2.6.12 adds replication topology inspection API, configurable TLS minimum version for object storage, and CDC enhancements. • Adds `GetReplicateConfiguration` API for inspecting replication topology with redacted tokens. • Adds `minio.ssl.tlsMinVersion` config key for setting a minimum TLS version on object storage connections across all supported backends. • Adds a configurable skip list for replicate message types in CDC. • Adds per-cluster TLS configuration support for CDC outbound mTLS connections. • Supports configuring different replica numbers on secondary CDC clusters independently from the primary. • Allows pchannel count increase in CDC `ReplicateConfiguration` to support heterogeneous cluster topologies. • Adds user-specified warmup support for the RESTful API. • Adds automatic warmup for large tenant collections to reduce cold-start query latency. • Adds phase-level timing logs and metrics for sort compaction. • Includes text index memory cost in segment loading memory estimation for more accurate resource accounting. https://github.com/milvus-io/milvus/releases/tag/v2.6.11 https://github.com/milvus-io/milvus/releases/tag/v2.6.11 Thu, 12 Feb 2026 01:57:24 GMT Milvus 2.6.11 adds a truncate API, RESTful `search_by_pk`, sparse filtering in search, and Storage V2 I/O pipelining. • Adds a truncate API to remove all data in a collection more efficiently. • Adds RESTful `search_by_pk` endpoint to look up records by primary key over the REST API. • Adds sparse filtering support in vector search queries. • Adds `LoadWithStrategyAsync` to enable true I/O pipelining in Storage V2, improving throughput on large segment loads. • Adds support for user-specified warmup settings on index load. • Adds semantic highlighting support for dynamic fields. • Normalizes constant-folded boolean expressions to `AlwaysTrueExpr`/`AlwaysFalseExpr` during query plan rewriting for simpler execution plans. • Differentiates load priorities by scenario to improve scheduling behavior across mixed workloads. • Reduces memory usage by enabling multi-cell `DefaultValueChunk` layout for default-value columns. https://github.com/milvus-io/milvus/releases/tag/v2.6.10 https://github.com/milvus-io/milvus/releases/tag/v2.6.10 Thu, 05 Feb 2026 01:34:08 GMT Milvus 2.6.10 adds automatic FP32-to-FP16/BF16 search conversion, KMS key revocation WAL control, BM25 search-by-PK, and mmap page-fault reduction. • Adds the `map_populate` flag for `mmap` to reduce page faults during memory-mapped data access. • Adds loading timeout and cancellation support for better control of long-running segment load operations. • Supports alter_collection_field() updating a field's description. • Adds `BM25 search_by_pk` support, enabling primary-key lookups through the BM25 path. • Adds automatic FP32-to-FP16/BF16 conversion during search, reducing memory and compute overhead without manual type casting. • Updates default auto-index configuration for vector fields. • Disables storage-version upgrade compaction by default. • Limits segment load concurrency by submitting loads to a dedicated load pool. • Improves slow logs by recording average cost per NQ for latency diagnostics. • Adds `StoppingBalancer` and extracts assign policy from the balancer for finer query-node load-balancing control. Breaking changes: • Storage-version upgrade compaction is now disabled by default; deployments relying on automatic storage-version upgrade compaction must re-enable it explicitly. https://github.com/milvus-io/milvus/releases/tag/v2.5.26 https://github.com/milvus-io/milvus/releases/tag/v2.5.26 Fri, 23 Jan 2026 00:46:33 GMT Milvus 2.5.26 adds security controls for the `/expr` endpoint and gRPC metadata headers for client request time tracking. • Adds security controls for the internal `/expr` endpoint to prevent remote expression execution vulnerabilities. • Adds gRPC metadata header for client request time tracking. https://github.com/milvus-io/milvus/releases/tag/v2.6.9 https://github.com/milvus-io/milvus/releases/tag/v2.6.9 Mon, 19 Jan 2026 02:48:46 GMT Milvus 2.6.9 adds primary-key search, highlight scores, dimension params for embedding providers, and security controls on the `/expr` endpoint. • Adds security controls for the `/expr` endpoint to prevent unauthorized access. • Adds `dimension` parameter support for siliconflow and cohere embedding providers. • Supports searching by primary keys, with duplicate checking and automatic `anns_field` inference. • Adds a storage version upgrade compaction policy to facilitate version migrations. • Adds a storage version label metric for better observability. • Implements batch processing for ngram operations to improve efficiency. • Adds automatic retry mechanism for binlog write operations. • Adds glog sink to transfer CGO logs into the zap logger for unified logging. • Enforces storage V2 format usage and deprecates V1 writes. • Streaming service now remains enabled until the required streaming node count is reached. • QueryCoord supports segment reopen when manifest path changes or when data/schema changes occur. Breaking changes: • Storage V1 writes are deprecated and V2 format is now enforced; existing pipelines that rely on V1 write paths may require migration. https://github.com/milvus-io/milvus/releases/tag/v2.5.25 https://github.com/milvus-io/milvus/releases/tag/v2.5.25 Thu, 08 Jan 2026 08:43:29 GMT Milvus 2.5.25 adds configurable metadata batch processing and variable-length field size estimation options. • Improves reliability of object storage operations under high load with automatic retry on rate-limit errors. https://github.com/milvus-io/milvus/releases/tag/v2.6.8 https://github.com/milvus-io/milvus/releases/tag/v2.6.8 Sun, 04 Jan 2026 06:18:40 GMT Milvus 2.6.8 adds search result highlighting, collection-level GC pause, nullable dynamic fields, and QueryNode penalty policies. • Supports search with result highlighting via the Text Highlighter feature, enriching full-text retrieval responses with matched term context. • Supports pausing garbage collection at the collection level, giving operators fine-grained control over GC timing per collection. • Supports nullable dynamic fields with an empty JSON object as the default value, enabling more flexible schema designs. • Implements a penalty policy for QueryNodes to handle resource exhaustion, improving cluster stability under heavy load. • Supports DML and DQL forwarding in Proxy for RESTful v2, extending REST API coverage for data write and query operations. • Introduces a tolerance duration to delay collection drop operations, reducing risk of accidental immediate data loss. • Rejects duplicate primary keys in upsert batch requests, enforcing data integrity at ingestion time. • Adds metrics to monitor Jemalloc cached memory, improving observability of memory allocator behavior. • Enables concurrent execution of text index tasks for multiple fields, reducing indexing latency for multi-field text workloads. • Adds validation for embedding models and schema field types, catching mismatches earlier in the pipeline. • Moves query optimization logic to the Proxy layer to improve query performance. • Adds a fallback mechanism for write paths when accessing object storage without condition-write support, broadening compatible storage backends. • Adds retry mechanism for object storage reads on rate-limit errors, improving resilience against throttled storage backends. • Improves index task scheduling by estimating slots based on field size and type, reducing resource contention during index builds. Breaking changes: • The RootCoord default port has changed to a non-ephemeral port; deployments that rely on the previous default port number may need configuration updates. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.2 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.2 Mon, 29 Dec 2025 10:23:23 GMT Milvus Go SDK v2.6.2 adds QueryIterator for large result pagination and Struct Array field type support. • Adds client.QueryIterator(ctx, opt) method for PK-based pagination over large query result sets, supporting both Int64 and VarChar primary key types. • Adds `milvusclient.NewQueryIteratorOption` with WithBatchSize(int) (default: 1000), WithIteratorLimit(int64), WithFilter(string), WithOutputFields(...string), WithPartitions(...string), and WithConsistencyLevel(ConsistencyLevel) options to configure iteration behavior. • Adds iter.Next(ctx) on the returned iterator, returning `io.EOF` when exhausted, enabling a standard Go iteration loop over result pages. • Adds `columnStructArray` column type implementing the `Column` interface, with support for parsing `StructArrayField` from protobuf responses and schema construction via `entity.FieldTypeArray`. https://github.com/milvus-io/milvus/releases/tag/v2.5.24 https://github.com/milvus-io/milvus/releases/tag/v2.5.24 Thu, 11 Dec 2025 11:21:12 GMT Milvus 2.5.24 improves BM25 sparse vector search load performance and cluster load balancing stability. • Optimizes loading performance of BM25 statistics for sparse vector search, reducing startup latency for relevant workloads. • Refines cluster load balancing by excluding stopping nodes from resource groups, improving stability under node churn. https://github.com/milvus-io/milvus/releases/tag/v2.6.7 https://github.com/milvus-io/milvus/releases/tag/v2.6.7 Thu, 04 Dec 2025 09:40:47 GMT Milvus 2.6.7 adds a `/livez` Kubernetes liveness endpoint, GroupBy on `TIMESTAMPZ` fields, and mmap for JSON shredding indices. • Adds `/livez` HTTP endpoint to support Kubernetes native liveness probes for container orchestration health checks. • Supports **GroupBy** operations on `TIMESTAMPZ` fields, enabling time-series group aggregation queries. • Supports `mmap` for JSON shredding's shared key indices to reduce RAM footprint on query nodes. • Supports DML request forwarding in the Proxy to improve write availability and routing resilience. • Supports asynchronous write syncing for logs to prevent disk I/O blocking from affecting the main execution path. • Enforces Buffered I/O for high-priority load tasks to optimize OS page cache utilization and throughput. • Optimizes `mmap` strategy to map group chunks in a single system call, reducing kernel overhead during segment loading. • Adds granular cancellation checks in query operators to allow faster termination of aborted or timed-out queries. • Enhances WAL retention strategy to better balance disk usage with data recovery safety. https://github.com/milvus-io/milvus/releases/tag/v2.6.6 https://github.com/milvus-io/milvus/releases/tag/v2.6.6 Fri, 21 Nov 2025 08:38:23 GMT Milvus 2.6.6 adds Geometry and TIMESTAMPTZ data types, a Boost Ranker for rescoring, and JSONL/NDJSON bulk-insert support. • New `Geometry` data type stores OGC-compliant geometric objects (`POINT`, `LINESTRING`, `POLYGON`) with spatial relationship operators (`st_contains`, `st_intersects`, `st_within`, `st_dwithin`) and an `RTREE` spatial index for LBS and mapping workloads. • New `TIMESTAMPTZ` data type brings timezone-aware temporal storage; supports a `timezone` property on Databases and Collections for a default time context, expression-based filtering for time range queries, and a `timezone` parameter on query/search for on-the-fly timestamp conversion. • New Boost Ranker lets searches apply an optional filtering condition and a specified weight to promote or demote matched candidates in final results, independent of raw vector distance scores. • `STL_SORT` index now supports `VARCHAR` and `TIMESTAMPTZ` datatypes. • Dynamic field can now be enabled on an existing collection via `ALTER COLLECTION` without recreating it. • Supports `JSONL`/`NDJSON` files for bulk-insert (`bulkinsert`) operations. • `geometrycache` is now an optional configuration, allowing operators to tune memory usage for spatial workloads. • Adds `max_connection` config for remote storage connections. • Removes the maximum vector field number limit per collection. • Adds new config keys with dynamic update support. • Knowhere thread pool config is now refreshable at runtime. • RBAC support added for `updateReplicateConfiguration`. • Import job listings now show create time. • `nullable` support added for `Geometry` and `TIMESTAMPTZ` field types. https://github.com/milvus-io/milvus/releases/tag/v2.4.24 https://github.com/milvus-io/milvus/releases/tag/v2.4.24 Sat, 08 Nov 2025 07:33:51 GMT Milvus 2.4.24 adds DDL deny rules via database properties and new DDL rate quota configurations. • Adds DDL rate type and related quota configurations to control DDL operation throughput. • Supports denying DDL operations according to database property, enabling per-database DDL lockdown. • Makes segment prune config refreshable at runtime without restart. https://github.com/milvus-io/milvus/releases/tag/v2.5.21 https://github.com/milvus-io/milvus/releases/tag/v2.5.21 Sat, 08 Nov 2025 07:33:12 GMT Milvus 2.5.21 adds nullable BM25 input fields, sparse query filtering, and access log template-value length tracking. • Adds access log field for template value length info, expanding observability in `accesslog` output. • Supports nullable fields as BM25 function input fields, removing a previous schema restriction on full-text search. • Introduces sparse filter in query execution, enabling more efficient filtered vector searches. • Implements a rewatch mechanism for etcd failure scenarios, improving resilience under etcd connectivity issues. • Optimizes bitmap reverse lookup performance, accelerating scalar-filtered queries at scale. https://github.com/milvus-io/milvus/releases/tag/v2.6.4 https://github.com/milvus-io/milvus/releases/tag/v2.6.4 Tue, 21 Oct 2025 07:54:24 GMT Milvus 2.6.4 adds Struct-in-ARRAY for complex data modeling, enables JSON Shredding by default, and supports Qwen GTE-rerank-v2 via DashScope. • Introduces `DataType.ARRAY` of `Struct` elements, enabling Array of Vector storage where each row can contain multiple vectors for complex data modeling and search. • Enables JSON Shredding by default, improving query performance and efficiency across JSON fields. • Adds disk quota for loaded binlog size to prevent query node load failures under memory pressure. • Adds load parameters for vector index configuration. • Adds mmap support for struct array in MemVectorIndex. • Adds caching layer management for TextMatchIndex. • Adds `accesslog.$consistency_level` field to access logs representing the actual consistency level used. • Adds access log field for template value length information. • Supports the Qwen GTE-rerank-v2 reranking model via DashScope integration. • Allows overwriting the current index type during index build. • Optimizes bitmap reverse lookup performance. https://github.com/milvus-io/milvus/releases/tag/v2.5.19 https://github.com/milvus-io/milvus/releases/tag/v2.5.19 Mon, 20 Oct 2025 09:59:41 GMT Milvus 2.5.19 adds `common.requery.hybridSearchPolicy` config and granular flush targets for flushall operations. • Adds `common.requery.hybridSearchPolicy` configuration key to control the requery policy used during hybrid search. • Adds support for granular flush targets in the `flushall` operation, enabling more precise control over which data gets flushed. • Ensures `accesslog.$consistency_level` now reflects the actual consistency level value in use, improving observability of access log entries. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.1 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.1 Mon, 13 Oct 2025 02:32:45 GMT Milvus Go SDK v2.6.1 adds geospatial types, CDC replication, manual L0 compaction, and per-collection FlushAll targeting. • Adds Geospatial data type support with WKT/WKB data exchange, R-Tree index for geometry columns, and GIS query functions including `ST_EQUALS` in query expressions. • Adds CDC (Change Data Capture) service implementation enabling log-based cross-cluster replication for Milvus 2.6. • Adds manual L0 compaction triggering for finer control over compaction operations. • Enhances `FlushAll` to support targeting specific collections within a database, while maintaining backward-compatible database-level flushing. • Allows inserting primary key data even when AutoID is enabled, providing more flexibility in data insertion workflows. • Adds receiver parse result cache to reduce redundant parsing overhead during result deserialization. https://github.com/milvus-io/milvus/releases/tag/v2.6.3 https://github.com/milvus-io/milvus/releases/tag/v2.6.3 Fri, 10 Oct 2025 03:21:09 GMT Milvus 2.6.3 adds manual L0 compaction, gRPC tokenizer, sparse filters, and new autoindex options for int8 vectors. • Adds new configuration options for the `int8` vector type in autoindexing. • Adds parameter items to control hybrid search requery policy. • Enables `autoid`-enabled collections to accept explicit primary key values on insert, including via the Go SDK. • Adds manual compaction support for L0 segments. • Integrates a gRPC tokenizer for enhanced query flexibility. • Encodes cluster ID into auto-generated IDs. • Adds configuration options for batch processing in metadata. • Enables granular flush targets for `flushall` operations. • Introduces sparse filter support in queries. • Enables nullable fields as input for BM25 functions. • Adds Azure Blob Storage support in Woodpecker. • Enables random score functionality for boosting queries. • Adds support for controlling insertion of function output fields. • Adds configurable score merging to the decay function. • Adds storage resource usage tracking for scalar and vector searches, as well as delete/upsert/REST operations. • Various updates to enhance tiered index functionality. https://github.com/milvus-io/milvus/releases/tag/v2.5.18 https://github.com/milvus-io/milvus/releases/tag/v2.5.18 Mon, 22 Sep 2025 01:07:04 GMT Milvus 2.5.18 adds configurable delete snapshot size, expression result caching, and a configurable interim index build ratio. • Adds a param to modify delete snapshot size, giving operators control over memory trade-offs during delete-heavy workloads. • Adds a param item forcing all indices ready for a segment before it is served, improving query consistency guarantees. • Supports expression result cache, reducing redundant evaluation overhead for repeated filter expressions. • Makes the build ratio of the interim index configurable, letting operators tune indexing resource usage. • Removes the timeout for compaction tasks, preventing premature cancellation of long-running compaction jobs. • Returns collection metadata from cache, reducing metadata lookup latency. https://github.com/milvus-io/milvus/releases/tag/v2.6.2 https://github.com/milvus-io/milvus/releases/tag/v2.6.2 Fri, 19 Sep 2025 02:04:56 GMT Milvus 2.6.2 adds JSON Shredding, NGram indexing, partial upsert, Boost Function, and more flexible schema evolution. • Adds NGRAM Index to accelerate `LIKE` query operations on text fields. • Adds JSON Shredding to accelerate dynamic field filtering. • Adds partial field updates via the upsert API, allowing individual fields to be updated without supplying the full entity. • Adds Boost Function (Boost Ranker) for score boosting in hybrid search ranking. • Adds support for `GROUP BY` on JSON fields and dynamic fields. • Adds ability to enable dynamic schema on existing collections. • Adds ability to drop indexes without first releasing collections. • Adds StorageV2 configurable split policy with schema-based and size-based options, including configurations for size-based split policy. • Adds configuration to allow custom characters in collection/field names. • Adds configuration to modify dump snapshot batch size. • Adds StorageV2 Azure build support enabled by default. • Adds support for waiting for all indices to be ready before loading segments. • Adds internal core latency metric for rescore node. • Adds load resource estimation for tiered index. • Adds autoindex config for deduplication use cases. • Adds CachingLayer metrics and configurations. • Adds `cchannel` support for streaming service. https://github.com/milvus-io/milvus/releases/tag/v2.6.1 https://github.com/milvus-io/milvus/releases/tag/v2.6.1 Mon, 01 Sep 2025 06:05:31 GMT Milvus 2.6.1 adds model-based rerankers, POSIX remote storage support, and new configurable limits for indexes and disk writes. • Makes the build ratio for interim indexes configurable. • Adds a configurable write rate limit to the disk writer. • Adds a configurable switch to adjust consistency guarantees for higher availability. • Supports POSIX-compatible file systems for remote storage. • Introduces model-based rerankers. • Enables SegCore parameters to be updated dynamically without restarting the Milvus service. • Adds unified gRPC latency metrics for better observability. • Includes client request timestamps in gRPC headers to simplify debugging. • Supports trace log level for segcore. • Supports dynamic interval updates for ticker components. • Optimizes query performance by converting multiple `!=` conditions into a single `NOT IN` clause. • Accelerates text match by collecting doc_id from posting list directly. • Optimizes performance of comparison expressions on primary key fields. • Improves auto-detection of ARM SVE instruction sets for bitset operations. https://github.com/milvus-io/milvus/releases/tag/client/v2.6.0 https://github.com/milvus-io/milvus/releases/tag/client/v2.6.0 Wed, 27 Aug 2025 07:26:51 GMT Milvus Go client v2.6.0 adds search iterator v2, function reranker, IvfRabitQ & MinHashLSH indexes, and per-field analyzer support. • Adds search iterator v2 with a `limit` parameter and static param validation for the search iterator. • Supports `IvfRabitQ` and `MinHashLSH` index types. • Supports a function reranker for search results. • Enables running an analyzer scoped to a specific collection field. • Passes client request time via gRPC metadata. https://github.com/milvus-io/milvus/releases/tag/v2.5.17 https://github.com/milvus-io/milvus/releases/tag/v2.5.17 Thu, 21 Aug 2025 10:39:41 GMT Milvus 2.5.17 adds ARM SVE bitset acceleration, direct I/O disk writer, and CPU-scaled import concurrency. • Adds a disk file writer with direct I/O support, bypassing the OS page cache for more predictable write throughput. • Adds write rate limiting for the disk file writer to cap I/O pressure during heavy ingest. • Enables ARM SVE (Scalable Vector Extension) hardware acceleration for bitset operations, speeding up filtered searches on ARM deployments. • Scales import task concurrency automatically based on CPU count, making bulk imports self-tuning. • Supports skipping TSafe checks in specific scenarios for lower-latency query paths. • Reduces etcd calls during `ShowCollections` and `DescribeCollections` operations to lower metadata overhead at scale. https://github.com/milvus-io/milvus/releases/tag/client/v2.5.6 https://github.com/milvus-io/milvus/releases/tag/client/v2.5.6 Mon, 11 Aug 2025 06:32:49 GMT Milvus client v2.5.6 adds `SearchIteratorV2` for paginated vector search iteration. • Adds `SearchIteratorV2` to the Milvus client, enabling iterator-based traversal of large vector search result sets. https://github.com/milvus-io/milvus/releases/tag/v2.6.0 https://github.com/milvus-io/milvus/releases/tag/v2.6.0 Tue, 05 Aug 2025 11:48:44 GMT Milvus 2.6.0 ships Storage Format V2, JSON Flat Index, RaBitQ quantization, phrase matching, MinHash LSH, and embedding functions. • Introduces Storage Format V2, an adaptive columnar layout using a 'narrow column merging + wide column independence' strategy that delivers up to 100x performance gains over the previous Parquet format for point lookups and small-batch retrievals, reduces file count by up to 98%, cuts major compaction memory by 300%, and improves read I/O by up to 80% and write I/O by over 600%. • Adds JSON Flat Index (beta), which automatically discovers and indexes all nested structures under a given JSON path — including deeply nested fields like `metadata.version2.features.experimental` — by creating inverted index entries for every path-value pair without requiring pre-declared paths or types. • Adds RaBitQ 1-bit quantization with high recall for compressed vector storage and faster search. • Adds phrase matching for text search queries. • Adds MinHash LSH support for near-duplicate detection and deduplication workflows. • Adds time-aware ranking functions for search result ordering. • Adds embedding functions enabling a 'data-in, data-out' workflow that generates vectors at ingest and query time. • Supports online schema evolution, allowing schema changes without downtime. • Adds INT8 vector support. • Adds enhanced tokenizers for global language support. • Introduces a cache layer with lazy loading that enables processing datasets larger than available memory. • Graduates Streaming Node (WAL management) to GA, with native WAL powered by Woodpecker, removing the dependency on Kafka or Pulsar. • Merges coordinators into a unified MixCoord and consolidates IndexNode and DataNode to reduce component complexity. Breaking changes: • Direct upgrade from 2.6.0-RC1 is not supported due to architectural changes; use the official upgrade guide for all existing deployments. https://github.com/milvus-io/milvus/releases/tag/v2.5.14 https://github.com/milvus-io/milvus/releases/tag/v2.5.14 Wed, 02 Jul 2025 07:49:49 GMT Milvus 2.5.14 adds AUTOINDEX for JSON fields, a separate chunk cache pool, local BM25 stats cache, and a toggleable Web UI. • Adds support for `AUTOINDEX` on JSON fields, enabling automatic index selection for JSON-typed collection fields. • Makes the Web UI toggleable via configuration. • Adds a separate chunk cache pool to isolate chunk cache memory from the main pool. • Introduces a local cache for BM25 segment statistics, reducing repeated remote lookups during sparse/full-text search. • Enables running an analyzer scoped to a collection field. • Uses English name as language identifiers for all language types, standardizing language specification across the system. • Adds support for printing NQ and parameters in search and query logs, improving query observability. • Fills in `dbname` for `operateprivilegev2request` in the interceptor, correcting privilege request context. • Enables the Tantivy collector to set bitset directly, improving full-text search performance. • Adds a size interface to the file reader to eliminate `statobject` calls during reads, reducing object-store overhead. https://github.com/milvus-io/milvus/releases/tag/client/v2.5.4 https://github.com/milvus-io/milvus/releases/tag/client/v2.5.4 Mon, 16 Jun 2025 11:53:52 GMT Milvus client v2.5.4 exports `milvusclient.annRequest` and removes the default replica count. • Exports `milvusclient.annRequest`, making the ANN request type part of the public API surface for downstream Go clients. • Removes the default value for `replicaNum` on load, allowing callers to omit the field without an implicit replica count being applied. https://github.com/milvus-io/milvus/releases/tag/v2.5.13 https://github.com/milvus-io/milvus/releases/tag/v2.5.13 Mon, 09 Jun 2025 09:39:24 GMT Milvus 2.5.13 adds field property dropping, a `cast` function for JSON indexes, TTL expiry filtering, and expanded `DescribeIndex` REST responses. • Adds a `cast` function for use with JSON indexes, enabling type coercion in JSON index queries. • The `DescribeIndex` RESTful API now returns index parameters in its response. • Adds support for dropping properties from a field. • Adds support for filtering out expired data using TTL during compaction. • Access logs now capture hybrid search expressions and fields. • Sets the CAGRA GPU image as the default GPU index image. • Server side now automatically fills absent nullable fields, reducing client-side handling. • Slow query identification now considers `nq` (number of queries) as a factor. • Supports balancing multiple collections in a single trigger. • Increases the default import buffer size for faster bulk ingestion. • Enables running an analyzer scoped to a collection's field to reduce repeated analyzer creation and destruction. https://github.com/milvus-io/milvus/releases/tag/v2.5.12 https://github.com/milvus-io/milvus/releases/tag/v2.5.12 Mon, 19 May 2025 08:58:56 GMT Milvus 2.5.12 adds JSON index support for `contains` expressions, RESTful consistency levels, and CDC multi-DDL sync. • Adds JSON index support for JSON `contains` expressions, enabling indexed evaluation of containment queries on JSON fields. • The RESTful API now supports consistency levels for query/get operations. • The `DescribeCollection` API now includes the update timestamp in its results. • The `DescribeIndex` interface now outputs index version information. • Adds authorization checks for `DescribeCollection` and `DescribeDatabase` tasks. • Adds support for altering collection descriptions. • CDC now supports synchronizing multiple DDL APIs. • Adds stricter expiry compaction to clean deleted data without waiting for a large number of deletions. • Adds a timeout for message reception in `MQMsgStream`. • Adds parameters to ignore configuration type exceptions. • Disk quota checks are now skipped for L0 imports. • Sets worker `totalSlot` in standalone mode to half of that in cluster mode. https://github.com/milvus-io/milvus/releases/tag/v2.5.11 https://github.com/milvus-io/milvus/releases/tag/v2.5.11 Mon, 28 Apr 2025 09:16:32 GMT Milvus 2.5.11 adds multi-analyzer support, new tokenizers (Jieba, Lindera, ICU, Language Identifier), new text filters, and expanded JSON index capabilities. • Introduces a `run_analyzer` API for dry-run tokenization analysis, letting practitioners inspect how text is tokenized before committing to an analyzer configuration. • Adds a `remove_punct` filter to strip punctuation marks from tokenized text during analysis. • Adds a `regex` filter for pattern-based text filtering during analysis. • Adds support for configuring multiple analyzers per field and selecting the appropriate one based on input data language or instruction. • Adds support for the Lindera tokenizer for Japanese/Korean text analysis. • Adds support for the ICU tokenizer for Unicode-aware, locale-sensitive tokenization. • Adds a Language Identifier tokenizer for automatic language detection. • Adds support for customizing Jieba tokenizer parameters for Chinese text segmentation. • Expands language support for the built-in stop word filter. • Adds support for modifying the maximum capacity of array fields after collection creation. • Adds support for binary range expressions in JSON path indexes. • Adds support for infix and suffix match types in JSON stats. • Adds a configuration option to force rebuilding indexes to the latest version. • Enables dynamic updates to the segment loading thread pool size without restart. • Adds monitoring parameters for the expression filter ratio. https://github.com/milvus-io/milvus/releases/tag/v2.5.10 https://github.com/milvus-io/milvus/releases/tag/v2.5.10 Fri, 18 Apr 2025 06:10:15 GMT Milvus 2.5.10 adds configurable RESTful timeouts, SVE-accelerated FP16/BF16 metric computation, and faster LIKE and index load performance. • Optimizes LIKE expression performance and switches LIKE to scan mode even when an inverted index exists, improving query correctness and speed. • Optimizes index format for improved collection load performance. • Suppresses index metrics reporting for non-existent indexes, reducing noise in metrics output. https://github.com/milvus-io/milvus/releases/tag/v2.5.9 https://github.com/milvus-io/milvus/releases/tag/v2.5.9 Fri, 11 Apr 2025 12:52:38 GMT Milvus 2.5.9 adds weighted re-ranker score normalization control and faster batch JSON key stats building. • Supports skipping score normalization for the weighted re-ranker, giving practitioners direct control over re-ranking score output. • Improves JSON key statistics build performance by adding documents in batches, accelerating indexing on JSON-heavy collections. • Uses `int32` internally when creating array indexes for `int8`/`int16` element types, broadening index compatibility for small-integer arrays. • Aligns brute-force search results with JSON index behavior for the `exists` expression, ensuring consistent query results regardless of execution path. https://github.com/milvus-io/milvus/releases/tag/client/v2.5.2 https://github.com/milvus-io/milvus/releases/tag/client/v2.5.2 Fri, 11 Apr 2025 05:14:59 GMT Milvus Go SDK v2.5.2 adds JSON Path index support for the milvusclient package. • Adds JSON Path index support to the Go SDK `milvusclient` package, enabling index creation on nested JSON fields. https://github.com/milvus-io/milvus/releases/tag/v2.5.8 https://github.com/milvus-io/milvus/releases/tag/v2.5.8 Tue, 01 Apr 2025 09:24:23 GMT Milvus 2.5.8 adds JSON null/exists expressions, UTF-8 validation, sparse vector Parquet support, and detailed manual compaction criteria. • Adds a trigger interval configuration for auto-balancing, giving operators control over how frequently segment rebalancing fires. • Supports JSON `null` and `exists` expressions in query/filter conditions. • Supports parsing sparse vectors from Parquet structs during bulk inserts. • Adds UTF-8 string validation for all VARCHAR fields and during import operations. • Supports detailed manual compaction criteria, enabling finer-grained control over compaction runs. • Retains raw tokens for audit logging, improving auditability of authenticated requests. • Introduces batch subscriptions in `MsgDispatcher` to improve message fan-out throughput. • Converts multiple OR expressions to IN expressions automatically for more efficient query execution. • Balances collections with the largest row count first, improving load distribution across nodes. • Refines array views to reduce memory usage for array-type fields. https://github.com/milvus-io/milvus/releases/tag/v2.5.7 https://github.com/milvus-io/milvus/releases/tag/v2.5.7 Sun, 23 Mar 2025 10:31:07 GMT Milvus 2.5.7 introduces JSON Path Index for inverted indexes on dynamic and JSON columns to boost query performance. • Adds JSON Path Index, enabling inverted indexes on dynamic columns and specific JSON paths to bypass slower JSON load processes and significantly improve query performance. • Adds more config options for `interimindex` to support refined modes. • Makes segment prune config refreshable at runtime. • Supports retrieving segment binlogs via the new `GetSegmentsInfo` interface. • Adds a channel seal policy based on blocking L0. • Improves import error messages for better usability. • Reorders sub-expressions for conjunct expressions to improve query execution. • Removes unnecessary collection and partition labels from metrics to reduce cardinality. https://github.com/milvus-io/milvus/releases/tag/v2.4.23 https://github.com/milvus-io/milvus/releases/tag/v2.4.23 Fri, 28 Feb 2025 13:28:13 GMT Milvus 2.4.23 adds a QueryCoord balance status API, auto-balance trigger interval config, and collection descriptions. • Adds a management API to check QueryCoord balance status, enabling operators to programmatically inspect load-balancing state. • Adds a trigger interval configuration for auto-balancing, giving operators control over how frequently Milvus initiates rebalance operations. • Adds `GetVector` latency metrics for observability into vector retrieval performance. • Supports creating a collection with a description, enriching collection metadata. • Optimizes the result format of `GetQueryNodeDistribution` for clearer node distribution visibility. • Accelerates object listing during binlog import, improving large-scale data ingestion throughput. https://github.com/milvus-io/milvus/releases/tag/v2.5.5 https://github.com/milvus-io/milvus/releases/tag/v2.5.5 Thu, 27 Feb 2025 03:57:59 GMT Milvus 2.5.5 scales to 10K collections and 100K partitions, adds new metrics, management API, and interim index improvements. • Adds `proxy.maxTaskNum` config key (default synced to 1024) to control the proxy task queue depth. • Exports index request timeout interval via a new config entry to allow tuning without code changes. • Adds configs for compaction schedule, giving operators control over compaction timing behavior. • Adds a management API to check querycoord balance status, enabling operational visibility into query node distribution. • Adds `withEnableMatch` syntactic sugar to the Go SDK for simpler match-based search construction. • Adds monitor metrics for proxy queue, parse expression, get-vector latency, raw data retrieval, and write amplification, expanding Prometheus/observability surface. • Adds a DSL log field for hybrid search to improve query debugging and traceability. • Interim index now supports multiple index types and additional data types including FP16 and BF16. • Supports creating a collection with a description field. • Supports returning configurable properties when describing an index. • RESTful v2 search now returns top-k results, aligning REST response behavior with SDK behavior. • Scales single-cluster support to 10K collections and 100K partitions. • Accelerates listing objects during binlog import for faster bulk-load operations. • Decreases dump snapshot limit from 100K (10w) to 10K (1w) to reduce metadata overhead. https://github.com/milvus-io/milvus/releases/tag/v2.4.22 https://github.com/milvus-io/milvus/releases/tag/v2.4.22 Tue, 18 Feb 2025 07:17:29 GMT Milvus 2.4.22 adds configurable compaction intervals, `topks` in RESTful v2 search responses, and broad load/recovery performance improvements. • Returns `topks` field in RESTful v2 search responses. • Adds a secondary index for QueryNode segment manager to accelerate query execution. • Reads collection-level metadata concurrently to speed up failure recovery. • Increases metadata list batch size to speed up recovery. • Reduces locking in DataCoord to speed up load and insert operations. • Accelerates task generation, scheduling, and execution in QueryCoord to speed up data loading. • Removes unnecessary Bloom Filters in QueryNode and DataNode to reduce memory usage. • Uses `WalkWithPrefix` instead of `LoadWithPrefix` for etcd list operations to improve performance. • Decreases update frequency for rapidly refreshed metrics to accelerate recovery. • Skips generating the partition limiter when no partition is set, reducing overhead. • Improves collection-information fetch speed from RootCoord by eliminating unnecessary copies. https://github.com/milvus-io/milvus/releases/tag/v2.5.4 https://github.com/milvus-io/milvus/releases/tag/v2.5.4 Fri, 24 Jan 2025 02:30:17 GMT Milvus 2.5.4 adds PartitionKey isolation, Sparse Index DAAT MaxScore, `is_null` expressions, and scales to 10K collections and 1M partitions. • Adds `is_null` expression support for filtering queries on nullable fields. • Introduces PartitionKey isolation to improve query performance when multiple partition keys are in use. • Sparse Index now supports DAAT MaxScore algorithm for improved sparse vector search performance. • Root privileges can now be customized, enabling finer-grained access control configuration. • Scales to support 10,000 collections and 1 million partitions in a single cluster, unlocking large-scale multi-tenant deployments. • Adds primary field names in `SearchResult` and `QueryResults` responses. • Expands RESTful API surface with additional endpoint support. • Disk quota throttling now uses both binlog size and index size as combined standards. • Adds version control for scalar indexes. https://github.com/milvus-io/milvus/releases/tag/v2.4.21 https://github.com/milvus-io/milvus/releases/tag/v2.4.21 Wed, 22 Jan 2025 01:59:05 GMT Milvus v2.4.21 adds customizable root privileges and surfaces primary field names in search/query results. • Adds primary field names to `SearchResult` and `QueryResults` responses, making result parsing more explicit without requiring a separate lookup. • Root privileges can now be customized, giving administrators finer control over built-in RBAC permissions. • Accelerates bitset operations with SIMD, improving filter and expression evaluation throughput. https://github.com/milvus-io/milvus/releases/tag/v2.5.3 https://github.com/milvus-io/milvus/releases/tag/v2.5.3 Mon, 13 Jan 2025 10:47:52 GMT Milvus 2.5.3 adds a resource group API for the RESTful interface and boosts retrieve performance via bitset SIMD methods. • Adds a resource group API for the RESTful interface, enabling programmatic resource group management over HTTP. • Optimizes retrieve performance by leveraging bitset SIMD methods, unlocking faster query throughput at scale. • Adds missing delete metrics, improving observability for delete operations. • Uses MVCC timestamp as the guarantee timestamp when specified, enabling more precise consistency control. https://github.com/milvus-io/milvus/releases/tag/v2.5.2 https://github.com/milvus-io/milvus/releases/tag/v2.5.2 Fri, 03 Jan 2025 04:00:20 GMT Milvus 2.5.2 adds tunable maximum VARCHAR length and supports parameter type conversion in expressions. • Adds a parameter to tune the maximum VARCHAR column length, restoring the upper limit to 65,535 characters. • Supports automatic parameter type conversion for filter/query expressions, reducing the need for explicit casting. https://github.com/milvus-io/milvus/releases/tag/v2.4.20 https://github.com/milvus-io/milvus/releases/tag/v2.4.20 Thu, 02 Jan 2025 01:30:50 GMT Milvus 2.4.20 adds a YAML config param to tune the system maximum varchar length. • Adds a YAML configuration parameter to adjust the system limit for maximum varchar length, lifting a previously hard-coded constraint. https://github.com/milvus-io/milvus/releases/tag/v2.4.19 https://github.com/milvus-io/milvus/releases/tag/v2.4.19 Fri, 27 Dec 2024 01:14:37 GMT Milvus 2.4.19 adds expression templates for faster hybrid searches and new deletion monitoring metrics. • Introduces expression templates to accelerate hybrid searches. • Adds additional metrics for improved deletion monitoring. • Restricts L0 file generation to specific partitions for partitionKey deletion to reduce write amplification. https://github.com/milvus-io/milvus/releases/tag/v2.5.0 https://github.com/milvus-io/milvus/releases/tag/v2.5.0 Mon, 23 Dec 2024 03:45:57 GMT Milvus 2.5 adds full-text search (BM25), bitmap indexes, nullable fields, CSV import, clustering compaction, and a cluster management WebUI. • Supports `nullable=True` and default values for scalar fields, allowing null data to be omitted on insert without errors. • Adds CSV bulk import format alongside the existing JSON and Parquet support. • Introduces Clustering Compaction (Beta): specifying a scalar field as a clustering key redistributes data by range, enabling efficient pruning during queries when scalar filters are applied. • Introduces Full Text Search via built-in Sparse-BM25 (Tantivy-powered), with built-in analyzers and sparse vector extraction so collections can accept raw text instead of pre-computed vectors. • Adds Text Match capability using Tantivy-based analyzers and indexing for precise term-level natural language filtering within vector searches. • Adds Bitmap Index, a new scalar index type suited for low-cardinality fields that uses a bit-array representation to accelerate filtered searches. • Adds Streaming Node (Beta) component providing Write-Ahead Logging (WAL) services for channel read/write consensus; disabled by default in 2.5, planned for general availability in v3.0. • Introduces a built-in Cluster Management WebUI (Beta) for visualizing runtime environment info including segments, channels, node health, slow queries, and task status. • Supports multiple HNSW quantization methods via Faiss migration: SQ (Scalar Quantizers), PQ (Product Quantizer), and PRQ (Product Residual Quantizer). • Adds IPv6 support for expanded network connectivity. • Supports expression templates for query acceleration, improving parsing efficiency for complex filter expressions. • Expands GroupBy with customizable group size (number of entries returned per group) and hybrid GroupBy search across multiple vector columns. • Adds MVCC support for iterators so in-flight inserts and deletions do not affect ongoing iteration. • Adds persistent cursor for QueryIterator, enabling iteration to resume from the last position after a Milvus restart. • Adds TLS support for inter-node communication in enterprise and complex network environments. • Introduces Local Storage V2, a new local file format improving scalar data load/query efficiency and reducing memory overhead. • Aligns RESTful API functionality with other SDKs for consistency. • Score-based channel balancing policy dynamically redistributes loads across channels in large-scale deployments. • Compaction now removes maximum segment limits in mixed compaction and prioritizes smaller segments first. Breaking changes: • Upgrade to Pulsar 3.0.7 LTS is not compatible with previous Pulsar 2.x versions; existing deployments must upgrade ETCD and Pulsar before using new features (see Upgrade Pulsar from 2.x to 3.x guide). https://github.com/milvus-io/milvus/releases/tag/v2.4.18 https://github.com/milvus-io/milvus/releases/tag/v2.4.18 Fri, 20 Dec 2024 23:28:22 GMT Milvus 2.4.18 adds customizable RBAC privilege groups, a new Grant/Revoke API, and RESTful expression templates. • New Grant/Revoke API lets callers grant or revoke privileges without specifying an `ObjectType`, using privilege group names directly. • `alterindex` and `altercollection` now support modifying index and collection properties at runtime. • `alterdatabase` now supports deleting database properties. • Adds expression template support for RESTful API query/search requests, enabling parameterized expressions. • Adds database-scoped requests to the RESTful API. • Enables rate limiting for RESTful V1 endpoints. • Introduces customizable privilege groups — create, drop, list, and dynamically add/remove privileges within user-defined groups. • Adds nine built-in privilege groups for common operational scenarios: `ClusterReadOnly`, `ClusterReadWrite`, `ClusterAdmin` (cluster-level); `DatabaseReadOnly`, `DatabaseReadWrite`, `DatabaseAdmin` (database-level); `CollectionReadOnly`, `CollectionReadWrite`, `CollectionAdmin` (collection-level). • Adds mmap file usage metrics for observability. • Adds detailed replica counts per resource group. • Adds sparse search metrics counting non-zero values/tokens. • Adds collection ID dimension to search request count metrics. • Supports score-based balancing for channel policies. • Removes the per-round load task limit, allowing more parallel collection loading. • Removes the RPC layer of the coordinator when running in standalone or mixed mode, reducing overhead. • Supports automatic search retry when topk is reduced and results are insufficient. • Allows hyphens in partition names. • Sparse index performance improved ~10% via updated Knowhere version. https://github.com/milvus-io/milvus/releases/tag/v2.5.0-beta https://github.com/milvus-io/milvus/releases/tag/v2.5.0-beta Fri, 22 Nov 2024 15:22:18 GMT Milvus 2.5.0-beta adds full-text BM25 search, clustering compaction, bitmap indexes, nullable fields, CSV bulk import, and a cluster management WebUI. • Adds `nullable=True` property and default value support for scalar fields, letting inserters omit fields without errors. • Supports CSV format for bulk import alongside the existing JSON and Parquet formats. • Introduces expression templates to accelerate query expression parsing, especially for complex expressions. • Introduces full-text search via built-in Sparse-BM25 tokenization and sparse vector extraction, accepting raw text input directly through the API instead of requiring pre-generated vectors. • Adds a new BitMap index for low-cardinality scalar fields, using a bit-array equal in length to the row count to accelerate filtered searches. • Introduces Clustering Compaction (Beta), allowing a scalar field to be designated as a clustering key so data is redistributed by range for faster query pruning. • Adds Text Match capability using Tantivy analyzers and indexing for precise term-based natural language filtering. • Introduces a built-in Cluster Management WebUI (Beta) for visualizing runtime environment details including segments, channels, node health, tasks, and slow queries. • Migrates HNSW support from hnswlib to Faiss and adds SQ, PQ, and PRQ quantization methods on HNSW indexes. • Adds a Streaming Node component (Beta, disabled by default) providing Write-Ahead Logging (WAL) services for consensus before and after channel reads/writes. • Adds IPv6 support for expanded network connectivity. • GroupBy search now supports a customizable group size (number of entries returned per group) and hybrid GroupBy search across multiple vector columns. • QueryIterator gains MVCC support so iterators are unaffected by concurrent inserts/deletions, plus a persistent cursor enabling resumption from the last position after a restart. • Introduces Local Storage V2, a new local file format improving scalar data loading and query efficiency while reducing memory overhead. Breaking changes: • The upgrade to Pulsar 3.0.7 LTS is not compatible with previous Pulsar 2.x versions. https://github.com/milvus-io/milvus/releases/tag/v2.4.14 https://github.com/milvus-io/milvus/releases/tag/v2.4.14 Tue, 29 Oct 2024 13:01:48 GMT Milvus 2.4.14 adds memory-mode chunk cache, database-scoped bulk insert, and a dynamically tunable import concurrency config key. • Adds `datanode.import.maxconcurrenttasknum` as a dynamically adjustable configuration item to tune import concurrency at runtime without restarting. • Adds `queryNode.mmap.growingMmapEnabled` configuration key to control interim index behavior for growing segments. • Supports memory-mode chunk cache for faster in-memory data access. • Supports database (db) scoping for bulk insert operations. • Allows deleting data when disk quota is exhausted, preventing quota-induced write deadlocks. • Adds metrics for QueryNode delete buffer info and collection name labels to existing metrics for improved observability. • Adds middleware-based observability for RESTful v2 inbound/outbound RPC stats. • Enables parallel execution of L0 compactions to improve compaction throughput. • Adds prioritization of compaction tasks in DataCoord. • Upgrades Minio dependency to support EKS Pod Identities. • Limits the maximum number of concurrent import jobs. • Refactors `CreateCollection` in the RESTful API. • Updates Knowhere to version 2.3.12. https://github.com/milvus-io/milvus/releases/tag/v2.4.13 https://github.com/milvus-io/milvus/releases/tag/v2.4.13 Sat, 12 Oct 2024 09:00:01 GMT Milvus 2.4.13 adds dynamic replica adjustment, sparse vector MMAP in growing segments, and a new import-jobs REST endpoint. • Adds a new RESTful URL to describe import jobs, enabling programmatic monitoring of bulk-import progress. • Supports dynamic replica adjustment for loaded collections, letting users change replica counts without releasing and reloading the collection. • Enables MMAP for sparse vectors in growing segment types, reducing memory footprint for sparse workloads. • Enables load balancing across query nodes with varying memory capacities. • Enables manual compaction for collections that have no indexes. • Adds metrics to monitor import throughput and imported row count. • Adds a streaming forward policy switch for the delegator. • Adds a balance report log for the query coordinator balancer. https://github.com/milvus-io/milvus/releases/tag/v2.3.22 https://github.com/milvus-io/milvus/releases/tag/v2.3.22 Tue, 08 Oct 2024 07:25:52 GMT Milvus v2.3.22 adds trace ID propagation from client to server for distributed request tracing. • Propagates trace IDs from the client through the system, enabling end-to-end distributed tracing of requests. https://github.com/milvus-io/milvus/releases/tag/v2.4.12 https://github.com/milvus-io/milvus/releases/tag/v2.4.12 Wed, 25 Sep 2024 09:27:55 GMT Milvus 2.4.12 adds empty sparse row support, L0 remote loading, and delete-entry metrics for L0 segments. • Adds metrics for delete entries in L0 segments, enabling visibility into L0 segment deletion activity. • Implements L0 forward policy to support remote loading of L0 segments. • Enables empty sparse row support, allowing sparse vectors with no non-zero entries to be stored and queried. • Disallows reserved keywords as field names or dynamic field names, surfacing errors earlier at schema definition time. • Implements a stats handler for request/response size metrics, exposing payload size observability. • Adds ANN field loading check in proxy to validate approximate nearest-neighbor field state before search. https://github.com/milvus-io/milvus/releases/tag/v2.4.11 https://github.com/milvus-io/milvus/releases/tag/v2.4.11 Tue, 10 Sep 2024 07:31:57 GMT Milvus 2.4.11 adds static expression views and delete buffer quota logic for improved delete stability. • Implements delete buffer quota logic to govern delete throughput and improve delete stability under load. • Adds static view for filter expressions, enabling inspection of query expressions at rest. https://github.com/milvus-io/milvus/releases/tag/v2.4.10 https://github.com/milvus-io/milvus/releases/tag/v2.4.10 Fri, 30 Aug 2024 09:21:31 GMT Milvus 2.4.10 adds upsert with AutoID, partial field loading, RBAC backup/restore API, and expanded MMAP controls. • Adds a new API for backing up and restoring RBAC configurations, enabling disaster recovery and portability of access-control policies. • Introduces `readonly`, `readwrite`, and `admin` privilege groups to simplify the RBAC grant process. • Adds RBAC message support for Change Data Capture (CDC). • Adds new database properties to restrict read access to databases. • Supports upsert operations on collections with AutoID enabled. • Adds beta preview of partial field loading, allowing specific fields of a collection to be loaded rather than the full schema. • Expands MMAP configuration options with more general controls over MMAP behavior to optimize memory usage. • Adds support for HTTP v1/v2 throttling. • Adds hit segment number metrics for the query hook. • Enhanced tracing now encodes `traceid` and `spanid` as hex strings. https://github.com/milvus-io/milvus/releases/tag/v2.4.8 https://github.com/milvus-io/milvus/releases/tag/v2.4.8 Wed, 14 Aug 2024 11:10:41 GMT Milvus 2.4.8 adds clustering compaction, OTLP HTTP export, async CGO search, and expanded system metrics. • Adds `range_search_k` parameter support in Knowhere to speed up range searches. • Adds OTLP HTTP exporter support for observability and monitoring. • Introduces configurable database properties `max collections` and `disk quota`, dynamically modifiable at runtime. • Enables configuration of replica numbers and resource groups at the cluster, database, and collection levels with support for dynamic modifications. • Implements clustering compaction — data redistributed based on a designated clustering key to reduce scanned data and enhance query efficiency. • Separates compaction from the Shard DataNode, allowing any DataNode to perform compaction independently for improved fault tolerance and scalability. • Adds client pooling support in QueryNode within the proxy/delegator for improved performance. • Integrates Sonic for JSON marshaling/unmarshaling in Gin and RestfulV1 handlers to reduce CPU overhead. • Introduces an in-memory cache for authentication result retrieval. • Enables memory-mapped file support for growing segments. • Improves access logs with RESTful API support, consistency level logging, and distinction between system and user errors. • Applies blocked Bloom filters to accelerate filter construction and querying. • Expands system metrics to cover force-deny-writing state, queue latency, disk quota, task execution time, binlog size, insert rate, memory high water level, RESTful API metrics, and search latency. • Changes the default AutoIndex metric types for `FloatVector` and `BinaryVector` to `Cosine` and `Hamming` respectively for open-source users. • Introduces fixed versions for third-party C++ dependency libraries hosted on JFrog Cloud using Conan Recipe Revisions (RREV) to stabilize supply chain management. Breaking changes: • The AutoIndex metric types for `FloatVector` and `BinaryVector` are changed to `Cosine` and `Hamming` respectively; existing open-source deployments relying on the previous defaults will behave differently after upgrade. • Developers on operating systems other than Ubuntu 22.04 may need to upgrade their `glibc` version due to the newly fixed third-party dependency versions. https://github.com/milvus-io/milvus/releases/tag/v2.3.21 https://github.com/milvus-io/milvus/releases/tag/v2.3.21 Wed, 14 Aug 2024 11:07:13 GMT Milvus v2.3.21 adds disk quota, max insert rate, and query node memory metrics alongside a changed autoindex default metric type. • Adds metrics to track disk quota usage. • Adds metrics to record `maxinsertrate` and `querynodememoryhighwaterlevel`. • Adds tracking for the number of times Milvus enters a force-deny-writing state. • Changes the default metric type for autoindex. https://github.com/milvus-io/milvus/releases/tag/v2.3.20 https://github.com/milvus-io/milvus/releases/tag/v2.3.20 Wed, 24 Jul 2024 11:33:19 GMT Milvus v2.3.20 adds a proxy queue wait-time metric and upgrades Go to 1.21 for C++ pprof profiling. • Adds a new metric to record the duration that requests wait in the proxy queue, improving observability of request latency at the proxy layer. • Updates Knowhere to v2.2.7, allowing Milvus to be compiled against glibc versions ≤ 2.30. https://github.com/milvus-io/milvus/releases/tag/v2.4.5 https://github.com/milvus-io/milvus/releases/tag/v2.4.5 Thu, 20 Jun 2024 09:41:01 GMT Milvus 2.4.5 adds sparse HNSW indexing, disk index for binary vectors, sparse float vector support in RESTful v2, and RBAC on database APIs. • Adds RBAC support to the describe/alter database API. • Supports building the HNSW index for sparse vectors, enabling efficient high-dimensional sparse data search. • Supports building the Disk index for binary vectors. • Supports sparse vector type on RESTful v2. • Adds `mergeInterval`, `targetBufSize`, and `maxTolerantLag` of `msgdispatcher` to configurations. • Enables flush rate limiter at the collection level. • Supports importing delete data to L0 segment. • Adds auto-index mapping for binary and sparse data types, allowing auto-indexing without manual index selection. • Handles float16 and bfloat16 vectors similarly to BinaryVector in numpy bulk insert. • Improves `GetVectorByID` for sparse vectors. • Executes Bloom filter application in parallel to speed up segment prediction. • Speeds up loading of small collections. https://github.com/milvus-io/milvus/releases/tag/v2.3.18 https://github.com/milvus-io/milvus/releases/tag/v2.3.18 Wed, 19 Jun 2024 05:55:19 GMT Milvus v2.3.18 adds RESTful APIs to trigger component stop and improves delete rate limiting and Bloom filter throughput. • Adds RESTful APIs to trigger component stop, enabling programmatic lifecycle management of Milvus components. • Improves delete-by-expression rate limiting to measure against matched data size rather than delete request size, honoring rate limit configuration more accurately. • Accelerates Bloom filter processing via batch submission and parallel execution. • Enables parallel processing for `applydelete` at the segment level. • Logs the `UseDefaultConsistency` parameter in read requests for improved observability. https://github.com/milvus-io/milvus/releases/tag/v2.4.3 https://github.com/milvus-io/milvus/releases/tag/v2.4.3 Wed, 29 May 2024 07:35:28 GMT Milvus 2.4.3 adds sparse float vector bulk insert, dynamic balancer policy updates, and new observability config options. • Supports sparse float vector bulk insert for `binlog`, `json`, and `parquet` formats. • Adds a configuration option to control initialization of public role permissions. • Adds config to control initialization failure handling for plugins. • Adds score compute consistency config for knowhere. • Supports dynamic config updates for OpenTelemetry tracing. • Enables dynamic updating of balancer policy in QueryCoord at runtime. • Exposes `describedatabase` API in proxy. • Adds cost response metadata to REST API replies. • Changes default partition number to 16 when using partition key. • Enables channel meta table to write more than 200k segments. • Adds metrics for segment index file sizes. • Adds feature to track the size of data in memory for binlog. • Uses collection default consistency level for `restv2`. • Enables channel exclusive balance policy. • Enables batch uploading support. https://github.com/milvus-io/milvus/releases/tag/v2.3.15 https://github.com/milvus-io/milvus/releases/tag/v2.3.15 Sat, 11 May 2024 08:41:15 GMT Milvus v2.3.15 adds channel checkpoint info in flush responses and a config to validate IDs on autoID insert. • Adds a config option to check whether an ID is provided during data insertion when `autoID` is enabled, optimizing data migration workflows with Milvus-CDC. • Returns channel checkpoint info in flush responses, giving callers visibility into replication progress at flush time. https://github.com/milvus-io/milvus/releases/tag/v2.4.1 https://github.com/milvus-io/milvus/releases/tag/v2.4.1 Mon, 06 May 2024 12:00:08 GMT Milvus 2.4.1 adds Float16/BFloat16 bulk insert, sparse vector iterator search, and a declarative resource group API. • Adds a declarative resource group API for programmatic resource management. • Adds a configuration option to control the maximum amount of data that can be inserted in a single request. • Adds a configuration option to control whether to enforce activation of the `partitionKey` feature. • Adds `Float16` and `BFloat16` vector data type support in bulk insert. • Adds `db` label to metrics for delete and bulk insert operations, enabling per-database observability. • Enables sparse float vector to support brute-force iterator search and range search. • Adds `client_request_id` propagation: when provided by the client, it is used as the TraceID for distributed tracing. • Adds `WithBlock` option for etcd client creation. • Parallelizes the `applyDelete` operation at the segment level, accelerating Delete message processing by the Delegator. • Refines garbage collection to minimize list operations against object storage, reducing overhead at scale. • Enhances `milvus.yaml` management by auto-generating relevant configuration items through code. Breaking changes: • Delete operations with an empty filter expression are no longer supported and will fail on upgrade. https://github.com/milvus-io/milvus/releases/tag/v2.3.14 https://github.com/milvus-io/milvus/releases/tag/v2.3.14 Mon, 29 Apr 2024 11:14:22 GMT Milvus v2.3.14 adds a rolling-upgrade REST API, configurable GC scan intervals, and SDK-type access log tracking. • Adds a RESTful API for DevOps to execute rolling upgrades of Milvus clusters. • Supports configurable intervals for GC scan, allowing different scan intervals to be set. • Adds configuration items to skip Auto ID and Partition Key checks to improve check speed. • Allows users to disable search optimization via configuration. • Supports retrieving SDK type from the user agent in access logs. • Implements task-driven collection observation for QueryCoordV2. • Removes support for always-true expressions in delete expressions. Breaking changes: • Always-true expressions in delete `expr` are no longer supported and will be rejected. https://github.com/milvus-io/milvus/releases/tag/v2.4.0 https://github.com/milvus-io/milvus/releases/tag/v2.4.0 Tue, 16 Apr 2024 15:00:53 GMT Milvus 2.4.0 adds MinIO TLS, AutoIndex for scalar fields, new observability metrics, and import task improvements. • Adds new metrics for QueryCoord current target channel check point lag latency. • Adds new `db` label to common metrics, plus new metrics for deleted, indexed, and loaded entity counts with `collectionName` and `dbName` labels. • Supports AutoIndex for scalar fields, extending automatic index selection beyond vector fields. • Supports MinIO TLS connections for encrypted object-storage communication. • Import tasks now support waiting for data index completion before returning. • Adds enforced limits on imported file size and number, plus improved import task scheduling and compatibility. • Accelerates filtering operations through bitset and bitset_view refactoring. • Supports invalidating the database meta cache when dropping databases. • Improves error handling for mismatched vector types and unsupported index builds (raises error instead of crashing). • Hybrid search refactored for consistent execution paths with regular search. Breaking changes: • Grouping search on binary vectors is no longer supported. • Grouping search combined with hybrid search is no longer supported. • HNSW index on binary vectors is no longer supported. https://github.com/milvus-io/milvus/releases/tag/v2.3.13 https://github.com/milvus-io/milvus/releases/tag/v2.3.13 Sun, 07 Apr 2024 07:05:21 GMT Milvus v2.3.13 adds TLS for MinIO, new observability metrics, entity-stats tracking, and expanded RESTful APIs for Partition and Index operations. • Adds TLS support for MinIO connections, enabling encrypted object-storage communication. • Adds new metrics for QueryCoord current-target checkpoint lag, improving replication observability. • Adds new metrics for entities statistics, enabling per-collection entity count monitoring. • Expands RESTful interfaces with Partition and Index operation endpoints (see the RESTful API reference for v2.3.x). • Adds validation checks for field data type legality on ingestion. • Optimizes `DescribeIndex` performance via bulk index information retrieval. • Speeds up QueryCoord target recovery after restart by saving collection targets in batches. https://github.com/milvus-io/milvus/releases/tag/v2.4.0-rc.1 https://github.com/milvus-io/milvus/releases/tag/v2.4.0-rc.1 Wed, 20 Mar 2024 11:14:39 GMT Milvus v2.4.0-rc.1 adds GPU CAGRA index, sparse vectors, multi-vector hybrid search, inverted index, and grouping search. • Adds `group_by_field` argument to the search() operation for grouping search results by a scalar field, enabling document-level recall for RAG applications. • Adds Tantivy-based inverted index for all numeric and string scalar field types, reducing keyword query times by 10x with lower memory usage via data compression and MMap. • Adds fuzzy matching in scalar filtering using prefix, infix, and suffix patterns. • Adds sparse vector (`SPARSE_FLOAT_VECTOR`) field type supporting storage, indexing, and Maximum Inner Product Search (MIPS) for neural models such as SPLADEv2/BGE-M3 and BM25 (Beta). • Adds `Float16` and `BFloat16` half-precision vector data types for vector fields, reducing memory usage at a small accuracy tradeoff. • Adds multi-vector support, enabling multiple vector fields of different types (e.g., `FLOAT_VECTOR` and `SPARSE_FLOAT_VECTOR`) in a single collection with Reciprocal Rank Fusion (RRF) and Average Weighted Scoring reranking strategies for hybrid search. • Adds GPU CAGRA index, a graph-based GPU index offering 10x performance improvement for batch searches, including small-batch queries where CPU indices previously dominated. • Adds L0 Segment architecture for recording deletes separately, reducing flush overhead and improving delete and upsert performance. • Adds dynamic, fine-grained Memory-mapped Storage (MMap) control without requiring a Milvus restart, enabling up to 4x more data loaded on the same hardware with under 10% performance degradation for HNSW-indexed collections. • Adds bulk-insert via Milvus RESTful API, supporting multiple files in a single bulk-insert request. • Promotes Milvus-CDC (Change Data Capture) to general availability for incremental backup and disaster recovery between Milvus instances. • Expands `MilvusClient` interfaces to cover most ORM module functionality using a purely functional, gRPC-based approach. https://github.com/milvus-io/milvus/releases/tag/v2.3.11 https://github.com/milvus-io/milvus/releases/tag/v2.3.11 Fri, 08 Mar 2024 10:24:26 GMT Milvus v2.3.11 adds TLS for Kafka, varchar autoID in bulk insert, and collection-level flush rate limiting. • Adds TLS support for Kafka connections, securing message-bus traffic without requiring a separate proxy. • Adds support for varchar autoID in bulk insert operations, enabling string-typed primary keys to be auto-generated at ingest time. • Adds collection-level rate limiting for flush operations, giving operators fine-grained control over flush throughput per collection. • Adds a load memory factor for enhanced memory estimation during loading, reducing over-provisioning risk. • Enables Milvus containers to run as a non-root user, improving deployment security posture. • Optimizes JSON loading by eliminating one memory-copy pass, reducing peak memory during data ingestion. • Optimizes memory usage and loading speed for variable-length data fields. • Optimizes automatic balancing in QueryCoord for more even query-node load distribution. • Reduces DataNode memory consumption when handling multiple collections simultaneously. • Removes time-tick delay metrics for offline nodes from monitoring output, reducing noise in dashboards. https://github.com/milvus-io/milvus/releases/tag/v2.3.8 https://github.com/milvus-io/milvus/releases/tag/v2.3.8 Wed, 07 Feb 2024 08:12:10 GMT Milvus v2.3.8 expands BulkInsert to support auto-incrementing primary keys for VarChar types and adds build metadata to monitoring metrics. • Expands `BulkInsert` to support auto-incrementing primary keys for `VarChar` types. • Integrates Milvus build details — commit information and dependency identifiers — into monitoring metrics for improved observability. • Optimizes segment Binlog file loading strategy for improved performance. • Improves memory estimation algorithm during data loading to reduce out-of-memory (OOM) errors. • Upgrades to Knowhere version 2.2.4. • Improves error messaging for dimension mismatches in search vectors. https://github.com/milvus-io/milvus/releases/tag/v2.3.7 https://github.com/milvus-io/milvus/releases/tag/v2.3.7 Mon, 29 Jan 2024 01:49:07 GMT Milvus v2.3.7 adds array/JSON support in RESTful APIs, chunk cache pre-warming, and unified collection/partition/shard limits. • Adds array and JSON data type support to RESTful APIs, enabling richer document structures over HTTP without a native SDK. • Introduces chunk cache mechanism to pre-load vector data into query node local disk cache before it is needed, reducing cold-start retrieval latency. • Adds proactive pre-warming logic for ChunkCache to mitigate high latency when retrieving raw vectors during cold start queries. • Implements unified restrictions on the number of Collections, Partitions, and Shards across a Milvus instance (up to 65,536 collections). • Adds a counter metric for monitoring rate-limited requests, improving observability of throttling events. • Accelerates index loading through concurrent methods, reducing time-to-ready for large indexes. • Introduces privilege association logic to simplify the authorization process. • Optimizes load balancing algorithm by assigning weight to growing segments for more even query node distribution. https://github.com/milvus-io/milvus/releases/tag/v2.3.5 https://github.com/milvus-io/milvus/releases/tag/v2.3.5 Wed, 17 Jan 2024 12:27:12 GMT Milvus v2.3.5 adds RBAC for aliases, restores MVCC, and improves GPU task pooling and cgroupv2 metrics support. • Adds RBAC controls to aliases, allowing role-based permissions to govern alias operations. • Authorizes users to query grant information scoped to their own roles. • Supports reading hardware metrics for cgroupv2 environments. • Supports access log printing with a cluster prefix. • Adds GPU pool usage for GPU tasks to improve GPU workload performance. • Adds concurrency to DataCoord segment garbage collection to increase GC throughput. • Reduces proxy CPU usage by caching collection schema attributes. • Reads Azure files without ReadAll to reduce peak memory consumption. https://github.com/milvus-io/milvus/releases/tag/v2.3.4 https://github.com/milvus-io/milvus/releases/tag/v2.3.4 Tue, 02 Jan 2024 09:56:50 GMT Milvus 2.3.4 adds access logs, Parquet bulk import, and binlog indexes on growing segments for faster search. • Adds access logs for monitoring external gRPC interfaces, recording method names, user requests, response times, and error codes. • Adds Parquet file import support for bulk ingestion, including arrays and JSON data types, superseding the prior JSON and NumPy-only limitation. • Introduces binlog index on growing segments, enabling advanced index types (IVF, Fast Scann) and up to 10x faster searches on growing segments. • Expands cluster support to 10,000 collections/partitions, benefiting multi-tenant environments via timetick mechanism and goroutine management improvements. • Adds MMap support for index loading. • Adds partition-level privileges. • Implements balance channel in `querycoord` for improved query shard management. Breaking changes: • Regular expression searches in partitions are discontinued by default to reduce resource consumption; the feature can be re-enabled via configuration. https://github.com/milvus-io/milvus/releases/tag/v2.2.16 https://github.com/milvus-io/milvus/releases/tag/v2.2.16 Mon, 27 Nov 2023 11:34:34 GMT Milvus v2.2.16 makes etcdkv request timeout configurable and accelerates DiskAnn index loading via Knowhere 1.3.20. • Makes etcdkv request timeout configurable via a new configuration option. • Updates Knowhere to version 1.3.20, accelerating DiskAnn index loading times. • Increases the QueryCoord gRPC probe timeout for query nodes to 2 seconds. https://github.com/milvus-io/milvus/releases/tag/v2.2.15 https://github.com/milvus-io/milvus/releases/tag/v2.2.15 Tue, 14 Nov 2023 10:46:35 GMT Milvus 2.2.15 adds `bulkinsert` support for `partitionkey` and pure-list JSON format, and removes MySQL metastore. • Enables `bulkinsert` of binlog data with `partitionkey`, allowing bulk imports into partition-key-enabled collections. • Adds support for `bulkinsert` with pure list JSON format, expanding the accepted input formats for bulk data ingestion. Breaking changes: • MySQL metastore support has been removed; deployments using MySQL as the Milvus metastore will break on upgrade. https://github.com/milvus-io/milvus/releases/tag/v2.3.3 https://github.com/milvus-io/milvus/releases/tag/v2.3.3 Fri, 10 Nov 2023 12:33:12 GMT Milvus v2.3.3 adds pure list JSON support in bulk insert and improves rolling upgrade reliability. • Supports pure list JSON format in bulk insert operations. https://github.com/milvus-io/milvus/releases/tag/v2.3.2 https://github.com/milvus-io/milvus/releases/tag/v2.3.2 Thu, 26 Oct 2023 11:02:57 GMT Milvus 2.3.2 adds array datatypes, complex delete expressions, float16 vectors, TiKV meta, mmap indexes, and a new upsert REST API. • Adds a new upsert interface to the REST API. • Adds `ChunkCache` to fetch raw vectors directly from storage, with a configurable read-ahead policy. • Supports array datatype for collection fields. • Supports `float16` vector type (index support coming in a future release). • Introduces complex delete expressions, enabling richer filter-based deletes. • Reintroduces binary metric types `SUBSTRUCTURE` and `SUPERSTRUCTURE`. • Enables memory-mapped (`mmap`) vector indexes to reduce memory pressure. • Integrates TiKV as a distributed metadata store alternative. • CDC: Adds capability to replicate message-queue (MQ) messages. • Supports renaming the database associated with a collection. • Enables bulk insert of binlog data with partition keys. • Enhances support for multiple index engines. • Integrates gRPC compression for inter-component communication. • Displays index details when calling `GetSegmentInfo`. • Improves rolling upgrade stability to minimize service disruptions during updates. Breaking changes: • TimeTravel support in the compactor has been removed; workflows relying on compactor-level TimeTravel will break. • The MySQL metastore backend has been removed; clusters using MySQL as a metastore must migrate before upgrading. https://github.com/milvus-io/milvus/releases/tag/v2.3.1 https://github.com/milvus-io/milvus/releases/tag/v2.3.1 Fri, 22 Sep 2023 13:37:36 GMT Milvus 2.3.1 restores SUBSTRUCTURE/SUPERSTRUCTURE binary metric support and surfaces index info in GetSegmentInfo. • Restores support for SUBSTRUCTURE and SUPERSTRUCTURE binary metric types, previously unavailable. • Exposes index information in `GetSegmentInfo` responses, giving operators visibility into per-segment index state. • Improves `Flush`, `FlushAll`, and `GetFlushAllState` API operations for more reliable data syncing with object storage. • Adds delayed connection reset for `Canceled` or `DeadlineExceeded` gRPC codes, reducing unnecessary reconnects under transient errors. • Enhances RPC client with a retry mechanism for improved resilience against transient failures. • Significantly improves VARCHAR bulk insert throughput via batch processing reads. • Improves MMap efficiency and capacity, enabling larger in-memory datasets without additional hardware. • Reduces unnecessary data copies during segment loading, improving overall load performance. https://github.com/milvus-io/milvus/releases/tag/v2.3.0 https://github.com/milvus-io/milvus/releases/tag/v2.3.0 Wed, 23 Aug 2023 15:38:04 GMT Milvus 2.3.0 adds GPU support, upsert API, range search, cosine metrics, ScaNN index, iterators, JSON_CONTAINS, CDC, and NATS message queue. • Adds `radius` and `range_filter` parameters to `search_params` in the search API to retrieve vectors within a specific distance range (max 16,384 results). • Adds `JSON_CONTAINS` filter expression to query entities whose JSON field values contain one or more specified elements. • Adds native support for the `upsert` API, combining search, delete, and insert operations for ambiguous insert/update scenarios. • Adds the `count` statement as an alternative to `num_entities` for counting entities in a collection without triggering flushes. • Introduces iterator support in PyMilvus for retrieving more than 16,384 entities in search or range search operations, similar to Elasticsearch's scroll API. • Adds support for the ScaNN index (based on FAISS FastScan), delivering ~20% higher QPS than HNSW and ~7x improvement over IVF-FLAT in benchmarks. • Adds native Cosine Metrics support, eliminating the need to quantize vectors for IP (Inner Product) comparisons. • Enables raw vectors to be included in search results for supported metrics (HNSW, IVF_FLAT); not supported for IVF_PQ or IVF_SQ8. • Introduces MMap-based capacity expansion, mapping disk space to memory to increase single-machine data capacity with ~20% performance trade-off. • Adds Change Data Capture (CDC) support for active/standby synchronization, incremental backup, and data migration scenarios. • Adds GPU-accelerated query support via NVIDIA RAFT algorithm integration, achieving up to 10x QPS improvement on certain datasets. • Adds Arm64 (aarch64) Docker images, enabling native support on Arm-based cloud instances and Apple Silicon (MacOS) systems. • Introduces an experimental NATS-based built-in message queue as an alternative to Pulsar and Kafka. • Merges IndexCoord and DataCoord into a single component, simplifying Milvus deployment topology. • Adds a growing index for streaming (unindexed) data segments to avoid brute-force search degradation during ingestion. • Improves scalar query performance in hybrid searches by optimizing HNSW data-filtering policies. Breaking changes: • The time-travel feature has been removed; queries relying on time-travel will no longer work. • CentOS-based images are no longer provided; deployments using CentOS images must migrate to Amazonlinux or Ubuntu images. • The ANNOY and RHNSW index types for float vectors have been removed; collections using these indexes must be rebuilt with a supported index type. • The TANIMOTO index type for binary vectors has been removed; collections using this index must be rebuilt. • The Superstructure and Substructure metric types have been removed; queries using these metrics will break. https://github.com/milvus-io/milvus/releases/tag/v2.2.14 https://github.com/milvus-io/milvus/releases/tag/v2.2.14 Wed, 23 Aug 2023 14:27:18 GMT Milvus v2.2.14 adds disk metrics, configurable Pulsar timeout, and a default MALLOC_CONF for memory release after collection drops. • Adds a default `MALLOC_CONF` environment variable to release memory back to the OS after dropping a collection. • Makes Pulsar request timeout configurable. • Adds disk metric information to monitoring output. • Prohibits setting a partition name on a collection that already has a partition key, surfacing a clear error instead of silent misbehavior. https://github.com/milvus-io/milvus/releases/tag/v2.2.13 https://github.com/milvus-io/milvus/releases/tag/v2.2.13 Wed, 09 Aug 2023 08:16:18 GMT Milvus 2.2.13 extends object storage support to AliyunOSS and adds a configurable HTTP proxy port. • Adds `proxy.http.port` configuration item to control the HTTP proxy port. • Adds `region` and `virtual host` config options to the segcore chunk manager, enabling object storage backends beyond S3 and MinIO (including AliyunOSS). https://github.com/milvus-io/milvus/releases/tag/v2.2.12 https://github.com/milvus-io/milvus/releases/tag/v2.2.12 Tue, 25 Jul 2023 02:58:58 GMT Milvus 2.2.12 adds high-level RESTful APIs, vector retrieval by ID, `json_contains` filtering, and GCS/OSS access-key support. • Adds a high-level RESTful API that listens on the same port as gRPC, simplifying client-side operations (note: a token must be set even when authentication is disabled). • Adds `json_contains` expression support for filtering on JSON fields in searches and queries. • Enables bulk-insert to support partition keys. • Enables the chunk manager to use GCS and OSS object storage with an access key. • Adds `minCPUParallelTaskNumRatio` config to improve parallelism when a single task's estimated CPU usage exceeds total CPU capacity. • Supports setting the vector field as an output field in ANN searches and queries against HNSW-, DiskANN-, or IVF-FLAT-indexed collections. • Makes compaction RPC timeout and maximum parallelism configurable. • Writes cache files to the `cacheStorage.rootpath` directory. • Adds a PK index for string data types, improving query performance on string primary keys. • Changes default log level to `info`. https://github.com/milvus-io/milvus/releases/tag/v2.2.10 https://github.com/milvus-io/milvus/releases/tag/v2.2.10 Wed, 14 Jun 2023 11:28:10 GMT Milvus 2.2.10 extends RBAC to the Database API and FlushAll, and enables default params for HNSW & DISKANN indexes. • Adds RBAC enforcement for the `FlushAll` API, extending access-control protections to that operation. • Adds RBAC enforcement for the Database API, covering database-level operations under role-based access control. • Enables default parameter support for HNSW and DISKANN indexes, removing the requirement to supply all index params explicitly. • Enforces metric-type validation before search execution, catching mismatches earlier in the request lifecycle. • Enables a max result window limit to cap query result sizes. • Optimizes the rate-limit penalty mechanism for smoother behaviour under sustained high load. • Replaces the CGO payload writer with a Go payload writer in data nodes, significantly reducing memory usage. https://github.com/milvus-io/milvus/releases/tag/v2.2.9 https://github.com/milvus-io/milvus/releases/tag/v2.2.9 Fri, 02 Jun 2023 10:55:34 GMT Milvus 2.2.9 adds JSON fields, dynamic schemas, partition keys, database-scoped RBAC, Alibaba Cloud OSS, and a connect API. • Adds `consistency_level` field to search/query requests, letting callers control read consistency per operation. • Introduces a native JSON data type for collection fields, with expression support and bulk-insert compatibility for JSON data. • Adds dynamic schema support, allowing fields to be added to a collection without a fixed schema definition, including in bulk insert operations. • Introduces partition keys to route data within a single collection across logical partitions — enabling multi-tenant data separation without separate collections. • Extends RBAC to the database level, scoping roles and privileges to specific databases for stronger multi-tenancy control. • Implements a `connect` API for explicit connection management, including existence checks for the target database at connect time. • Adds Alibaba Cloud OSS as a supported object storage backend, configurable via access key (AK) or IAM authentication. • Implements `AutoIndex` to automatically select index parameters without requiring manual tuning. • Adds a configurable scheduling policy for query nodes, with user-level schedule policy support. • Implements rate limiting based on growing segment size to protect write throughput. • Adds SIMD-accelerated evaluation for several filtering expressions, improving filtered-search performance. • Adds support for single-quoted string literals inside filter expressions. Breaking changes: • Milvus standalone deployments using MinIO standalone have incompatible data layouts; manual data migration to a new MinIO instance is required before upgrading (see https://min.io/docs/minio/linux/operations/install-deploy-manage/migrate-fs-gateway.html). https://github.com/milvus-io/milvus/releases/tag/v2.2.7 https://github.com/milvus-io/milvus/releases/tag/v2.2.7 Fri, 28 Apr 2023 14:27:46 GMT Milvus v2.2.7 adds QueryNode plugin support for dynamic shared-library loading, replica-granularity load balancing, and a score-based balancing strategy. • Adds plugin logic to QueryNode to support dynamic loading of shared library files. • Supports load balancing with replica granularity. • Releases a score-based load-balancing strategy. • Improves search grouping algorithm to enhance query throughput. • Improves compaction algorithm to drive segment sizes toward an ideal distribution. • Adds a coroutine pool to limit concurrency of cgo calls triggered by delete operations. • Reduces peak memory consumption during collection loading. • Changes the default shard number to 1. Breaking changes: • The default shard number is changed to 1; collections created without an explicit shard count will now have 1 shard instead of the previous default. https://github.com/milvus-io/milvus/releases/tag/v2.2.6 https://github.com/milvus-io/milvus/releases/tag/v2.2.6 Tue, 18 Apr 2023 06:40:09 GMT Milvus v2.2.6 adds slow query/search logging for operations with latency of 5 seconds or more. • Adds slow logging for query and search operations when latency is not less than 5 seconds, surfacing performance outliers in production. https://github.com/milvus-io/milvus/releases/tag/v2.2.4 https://github.com/milvus-io/milvus/releases/tag/v2.2.4 Fri, 17 Mar 2023 10:47:43 GMT Milvus 2.2.4 adds resource grouping for QueryNodes, collection renaming, Google Cloud Storage support, and a new search/query performance option. • Adds a new option to the search() and query() APIs to skip searching all growing segments, trading data freshness for better search performance under insertion load. • Adds RBAC controls for the `GetLoadingProgress` and `GetLoadState` APIs. • Introduces namespace-based resource grouping: QueryNodes in a cluster can be assigned to isolated resource groups with fully separated access to physical resources. • Adds a collection-renaming API (currently available in PyMilvus; other SDK support in progress). • Adds Google Cloud Storage as a supported object storage backend. • Compaction is no longer restricted to indexed segments only, expanding when compaction can run. https://github.com/milvus-io/milvus/releases/tag/v2.2.3 https://github.com/milvus-io/milvus/releases/tag/v2.2.3 Fri, 10 Feb 2023 11:29:35 GMT Milvus 2.2.3 adds rolling upgrades via Helm, coordinator active-standby HA, and new `GetLoadState` and `WalkWithPrefix` APIs. • Adds `GetLoadState` API to query the load state of a collection or partition. • Adds `WalkWithPrefix` API to the `MetaKv` interface for prefix-based metadata traversal. • Adds a `refresh` option to `LoadCollection` and `LoadPartition` APIs to support reloading after bulk-insert. • Adds a new segment metric counting the number of binlog files per segment to Milvus monitoring. • Adds a segment seal policy triggered by the number of binlog files. • Adds GC for snapshot KV based on time-travel to reclaim stale metadata storage. • Introduces rolling upgrade support for Milvus clusters installed via Helm charts, allowing the cluster to serve requests during upgrades without downtime. • Introduces Coordinator HA (active-standby mode) for RootCoord and QueryCoord, ensuring failure recovery within 30 seconds. • Adds bulk-insert progress reporting so operators can track import status in real time. • Reduces memory usage during scalar field indexing and bulk-insert operations, and adds stream-reading for NumPy data imports. • Surfaces explicit denial reasons when Milvus rejects read/write requests. • Improves `ListSegments` performance for DataCoord catalog and `LoadWithPrefix` performance for `SuffixSnapshot`. • Upgrades Knowhere to 1.3.8 for improved ANN search performance. Breaking changes: • The maximum number of fields allowed in a collection is reduced from 256 to 64. https://github.com/milvus-io/milvus/releases/tag/v2.2.1 https://github.com/milvus-io/milvus/releases/tag/v2.2.1 Thu, 15 Dec 2022 14:40:05 GMT Milvus 2.2.1 adds Pulsar tenant auth, TLS for etcd, configurable TopK limits, and 30%+ search performance gains. • Adds `MaxWatchDuration` as a configurable setting to prevent shards with large data loads from timing out. • Makes TopK limit configurable to control maximum search result size. • Supports Pulsar tenant and authentication for message queue connections. • Supports TLS in etcd config source for encrypted coordination-plane communication. • Improves search performance by over 30% via an upgraded Knowhere vector engine and revised parallelism strategy. Breaking changes: • Each RPC is now hard-limited to 64 MB; requests exceeding this size will be rejected, which can break inserts or queries that previously sent large message packs. https://github.com/milvus-io/milvus/releases/tag/v2.2.0 https://github.com/milvus-io/milvus/releases/tag/v2.2.0 Fri, 18 Nov 2022 16:23:04 GMT Milvus 2.2.0 adds RBAC, disk-based ANN search, bulk file insertion, query pagination, quota limits, and collection-level TTL. • Adds `common.retentionDuration` parameter to manually enable Time Travel, which is now disabled by default to save disk usage. • Introduces bulk insertion APIs to load entities from JSON files directly into Milvus collections at scale. • Adds Role-Based Access Control (RBAC) to manage users, roles, and privileges — similar to traditional database access management. • Introduces quota limitation mechanisms to cap ingestion rate, search rate, and other traffic, protecting against OOM and crash conditions — configurable via Quota and Limitation Configurations. • Supports configuring Time to Live (TTL) at the collection level when creating or modifying a collection, replacing the previous cluster-only TTL. • Adds DiskANN, an SSD-resident Vamana graph-based ANNS algorithm (Beta), enabling large-scale vector search without loading the full index into memory — saving up to 10x memory. • Introduces `milvus-backup` tool (Beta) for data backup and restore, usable as a command-line tool or API server. • Supports `offset` and keyword filtering on search and query results to paginate large result sets across RPCs. • Exposes the flush() API to let users force-seal a growing segment and sync it to object storage. • Adds collection-level latency metrics for search, query, insertion, and deletion operations. • Further optimizes BirdWatcher debug tool to connect to Milvus meta storage and inspect internal system state. • Enforces that a collection can only be loaded if an index has been created on it, and indexes cannot be created after a collection is loaded. Breaking changes: • Milvus 2.2.0 is not fully compatible with earlier releases due to refined metadata storage and normalized API usage — migration from 2.1.x requires following the official upgrade guides. • Time Travel is disabled by default; to re-enable it, `common.retentionDuration` must be configured manually. • A collection can only be loaded with an index already created on it; loading without an index is no longer allowed. • Indexes cannot be created after a collection is loaded. • A loaded collection must be released before dropping its index. • Auto-flush is no longer triggered by SDK APIs such as num_entities() or create_index(). https://github.com/milvus-io/milvus/releases/tag/v2.1.2 https://github.com/milvus-io/milvus/releases/tag/v2.1.2 Fri, 16 Sep 2022 10:07:33 GMT Milvus 2.1.2 adds configurable SASL mechanism support for Kafka. • Supports configurable SASL mechanism for Kafka broker connections. https://github.com/milvus-io/milvus/releases/tag/v2.1.1 https://github.com/milvus-io/milvus/releases/tag/v2.1.1 Mon, 15 Aug 2022 01:20:34 GMT Milvus v2.1.1 adds dynamic HTTP-based log level control as its sole new capability. • Supports dynamic change of log level at runtime through an HTTP endpoint, enabling operators to adjust verbosity without restarting the service. https://github.com/milvus-io/milvus/releases/tag/v2.1.0 https://github.com/milvus-io/milvus/releases/tag/v2.1.0 Wed, 27 Jul 2022 11:33:05 GMT Milvus 2.1.0 adds VARCHAR type, in-memory replicas, embedded install, Kafka support, RESTful API, and TLS/auth security. • Adds `VARCHAR` scalar data type with MARISA-trie-based inverted index for prefix query and exact match, usable as an output field or for attribute filtering. • Adds in-memory replicas, enabling data to be loaded across multiple query nodes to scale read throughput without changing the dataset. • Adds RESTful API (Beta) for Milvus, built on the GIN web framework, enabling PHP and Ruby applications to interact with Milvus over HTTP. • Adds Apache Kafka support (Beta) as a message storage backend, configurable via Milvus configuration. • Adds username, password, and TLS connection support, plus secure connections to S3, Kafka, and etcd dependencies. • Adds Ansible deployment support for cluster installations in non-Kubernetes environments. • Adds Embedded Milvus, installable via `pip install milvus`, enabling quick demos and Python scripts on macOS including M1. • Adds search QPS and latency metrics to the Milvus monitoring dashboard. • Search performance improved approximately 3.2x through search combination and merge logic; ZSTD compression enabled for Pulsar. https://github.com/milvus-io/milvus/releases/tag/v2.0.1 https://github.com/milvus-io/milvus/releases/tag/v2.0.1 Wed, 23 Feb 2022 10:39:40 GMT Milvus 2.0.1 adds automatic item expiration on compaction, mixed compaction logic, and Mac platform support. • Adds `collectionID` to the return of `SearchResults` and `QueryResults`. • Implements automatic item expiration on compaction. • Implements mixed compaction logic. • Supports compiling and running Milvus on Mac. https://github.com/milvus-io/milvus/releases/tag/v1.1.1 https://github.com/milvus-io/milvus/releases/tag/v1.1.1 Wed, 16 Jun 2021 06:50:46 GMT Milvus v1.1.1 adds S3 storage support, GPU index persistence, and raises CPU search topk limit to 1M. • Enables S3 storage backend support for Milvus deployments. • Supports keeping indexes resident in GPU memory to avoid reloading overhead between queries. • Raises the CPU search `topk` limit from 16,384 to 1,000,000, unlocking large-scale nearest-neighbor retrieval. • Raises the `index_file_size` limit from 4 GB to 128 GB, supporting much larger index files. https://github.com/milvus-io/milvus/releases/tag/v1.1.0 https://github.com/milvus-io/milvus/releases/tag/v1.1.0 Sat, 08 May 2021 15:07:26 GMT Milvus v1.1.0 adds partition-scoped entity lookup and deletion, plus a new method to unload collections from cache. • Adds partition targeting to get_entity_by_id(), allowing entity lookups to be scoped to a specific partition. • Adds partition targeting to delete_entity_by_id(), allowing entity deletions to be scoped to a specific partition. • Adds new release_collection() method to explicitly unload a specific collection from cache. https://github.com/milvus-io/milvus/releases/tag/v1.0.0 https://github.com/milvus-io/milvus/releases/tag/v1.0.0 Wed, 10 Mar 2021 10:20:14 GMT Milvus v1.0.0 adds stdout log output support. • Supports writing log output to stdout. https://github.com/milvus-io/milvus/releases/tag/v0.10.6 https://github.com/milvus-io/milvus/releases/tag/v0.10.6 Tue, 23 Feb 2021 02:50:59 GMT Milvus v0.10.6 adds `nbits` parameter for IVF_PQ indexing and Prometheus label configuration support. • Adds optional `nbits` argument to the create_index() method for the IVF_PQ index, enabling finer control over product quantization bit-width. • Supports configuring Prometheus labels `cluster_label` and `instance_label` under the `metric` config section. • Improves FLAT search performance on binary vectors using the AVX2 instruction set. https://github.com/milvus-io/milvus/releases/tag/v0.10.5 https://github.com/milvus-io/milvus/releases/tag/v0.10.5 Thu, 07 Jan 2021 11:11:35 GMT Milvus v0.10.5 lets load_collection() preload specific partitions instead of entire collections. • Extends load_collection() to accept specified partitions, enabling targeted preloading instead of loading an entire collection into memory. https://github.com/milvus-io/milvus/releases/tag/v0.10.3 https://github.com/milvus-io/milvus/releases/tag/v0.10.3 Mon, 21 Sep 2020 02:03:10 GMT Milvus v0.10.3 expands IVF search limits and unlocks per-search metric_type on FLAT indexes • Allows specifying `metric_type` at search time for collections using a FLAT index, enabling per-query distance metric selection without re-indexing. • Expands valid values for IVF_PQ indexing parameter `m` to any value evenly divisible by `dim` (the number of vector dimensions), unlocking more tuning options. • Expands the valid range of `nprobe` to [1, 16384] for all IVF indexes, allowing finer recall/latency tradeoffs at scale. • Expands the valid range of `top_k` in embedding searches to [1, 16384], supporting much larger result sets per query. https://github.com/milvus-io/milvus/releases/tag/v0.9.0 https://github.com/milvus-io/milvus/releases/tag/v0.9.0 Fri, 15 May 2020 11:36:25 GMT Milvus v0.9.0 adds partition existence checks, log rotation, and search-priority index suspension across all SDKs. • Adds `HasPartition` / `has_partition` / `hasPartition` API method across C++, Python, Java, and Go SDKs to check whether a partition exists. • Renames `DescribeCollection` to `GetCollectionInfo` (`get_collection_info` / `getCollectionInfo`) across all SDKs. • Renames `CountCollection` to `CountEntities` (`count_entities` / `countEntities`) across all SDKs. • Renames `ShowCollections` to `ListCollections` (`list_collections` / `listCollections`) across all SDKs. • Renames `ShowCollectionInfo` to `GetCollectionStats` (`get_collection_stats` / `getCollectionStats`) across all SDKs. • Renames `DescribeIndex` to `GetIndexInfo` (`get_index_info` / `getIndexInfo`) across all SDKs. • Renames `ShowPartitions` to `ListPartitions` (`list_partitions` / `listPartitions`) across all SDKs. • Renames `GetEntitiesByID` to `GetEntityByID` (`get_entity_by_id` / `getEntityByID`) across all SDKs. • Renames `GetIDsInSegment` to `ListIDInSegment` (`list_id_in_segment` / `listIDInSegment`) across all SDKs. • Renames `DeleteByID` to `DeleteEntityByID` (`delete_entity_by_id` / `deleteEntityByID`) across all SDKs. • Renames `PreloadCollection` to `LoadCollection` (`load_collection` / `loadCollection`) across all SDKs. • Renames Python `search_in_files` to `search_in_segment`; removes the Java `searchInFiles` method entirely. • Supports log file rotating to prevent unbounded log growth. • Suspends index building when a search request arrives, prioritising query latency over background indexing. Breaking changes: • `DescribeCollection` is renamed to `GetCollectionInfo` (`get_collection_info` / `getCollectionInfo`) in all SDKs; callers must update method names. • `CountCollection` is renamed to `CountEntities` (`count_entities` / `countEntities`) in all SDKs; callers must update method names. • `ShowCollections` is renamed to `ListCollections` (`list_collections` / `listCollections`) in all SDKs; callers must update method names. • `ShowCollectionInfo` is renamed to `GetCollectionStats` (`get_collection_stats` / `getCollectionStats`) in all SDKs; callers must update method names. • `DescribeIndex` is renamed to `GetIndexInfo` (`get_index_info` / `getIndexInfo`) in all SDKs; callers must update method names. • `ShowPartitions` is renamed to `ListPartitions` (`list_partitions` / `listPartitions`) in all SDKs; callers must update method names. • `GetEntitiesByID` is renamed to `GetEntityByID` (`get_entity_by_id` / `getEntityByID`) in all SDKs; callers must update method names. • `GetIDsInSegment` is renamed to `ListIDInSegment` (`list_id_in_segment` / `listIDInSegment`) in all SDKs; callers must update method names. • `DeleteByID` is renamed to `DeleteEntityByID` (`delete_entity_by_id` / `deleteEntityByID`) in all SDKs; callers must update method names. • `PreloadCollection` is renamed to `LoadCollection` (`load_collection` / `loadCollection`) in all SDKs; callers must update method names. • The C++ `FlushCollection` and `FlushCollection`/`Flush` pair is consolidated into a single `Flush`; C++ callers using `FlushCollection` must switch. • The C++ `CompactCollection`/`Compact` pair is consolidated into a single `Compact`; C++ callers using `CompactCollection` must switch. • The Python `connect`, `connected`, and `disconnect` methods are deleted; Python callers must remove those calls. • The Java `searchInFiles` method is deleted; Java callers must migrate to `search_in_segment` (Python) or the equivalent new surface. https://github.com/milvus-io/milvus/releases/tag/v0.8.0 https://github.com/milvus-io/milvus/releases/tag/v0.8.0 Wed, 15 Apr 2020 09:48:34 GMT Milvus v0.8.0 adds ANNOY index support, vector deletion across 8 index types, and new HTTP similarity metrics. • Adds `ANNOY` index type support for approximate nearest-neighbor search. • Adds vector deletion for one or multiple vectors across `Flat`, `IVFlat`, `IVFPQ`, `IVFSQ8`, `IVFSQ8H`, `NSG`, `HNSW`, and `ANNOY` index types. • Adds `SuperStructure` and `SubStructure` similarity metrics to the HTTP module. https://github.com/milvus-io/milvus/releases/tag/v0.7.1 https://github.com/milvus-io/milvus/releases/tag/v0.7.1 Wed, 01 Apr 2020 13:10:39 GMT Milvus v0.7.1 adds substructure and superstructure distance metrics for chemical structure search on FLAT indexes. • Adds substructure and superstructure distance metrics to the `FLAT` index type, enabling similarity search over chemical structures. • Combines identical concurrent search requests from multiple clients into a single execution, significantly improving search throughput. https://github.com/milvus-io/milvus/releases/tag/v0.7.0 https://github.com/milvus-io/milvus/releases/tag/v0.7.0 Wed, 11 Mar 2020 09:51:50 GMT Milvus v0.7.0 adds vector deletion, RESTful API, Go SDK, HNSW index, WAL, and Jaccard/Hamming/Tanimoto distance support. • Adds WAL (Write-Ahead Logging) configurable via `server_config.yaml` to significantly improve reliability of data operations. • Adds RESTful API for interacting with Milvus over HTTP. • Adds Go SDK, available at github.com/milvus-io/milvus-sdk-go. • Adds support for HNSW index type for approximate nearest neighbor search. • Adds vector deletion support for one or multiple vectors (available with FLAT, IVFLAT, IVFSQ8, and related index types). • Adds `get vector by ID` capability to retrieve vector data directly by its ID. • Adds flush and compaction support — configurable at an interval or triggered manually via Milvus clients to release space from deleted vectors. • Adds runtime configuration updates: Milvus server parameters can now be changed via Milvus clients without restarting the server. • Adds support for Jaccard, Hamming, and Tanimoto distance metrics for vector search. • Adds Pushgateway support for Prometheus, enabling short-lived and batch metrics collection. • Adds support for AVX 512 instruction set, broadening compatible CPU hardware. Breaking changes: • Milvus client parameters for index creation and vector search now use JSON strings as values instead of previous parameter formats. • The Milvus server configuration file `server_config.yaml` is updated to version 0.2 with parameter changes — existing configuration files must be migrated. • The entity previously called 'Table' is now named 'Collection' throughout Milvus — any client code or tooling referencing 'Table' by name will need to be updated. https://github.com/milvus-io/milvus/releases/tag/v0.6.0 https://github.com/milvus-io/milvus/releases/tag/v0.6.0 Sat, 07 Dec 2019 09:05:01 GMT Milvus v0.6.0 adds table partitioning, Mishards sharding middleware, new index types, and multi-GPU index building via `build_index_resources`. • Adds `build_index_resources` configuration parameter to enable multi-GPU index building, reducing index build and overall query time. • Adds table partitioning APIs across Python, Java, and C++ SDKs — supporting partition creation, vector insertion into a specified partition, and partition-scoped queries. • Introduces experimental index types `SPTAG-KDT`, `SPTAG-BKT`, `RNSG`, and `IVFPQ` for broader ANN search coverage. • Introduces Mishards, an experimental Milvus sharding middleware enabling distributed deployment with request forwarding, read/write splitting, horizontal scalability, and dynamic extension. • Publishes performance test reports for `IVFFLAT`, `IVFSQ8`, and `IVFSQ8H` indexes. • Open-sources Milvus internal FAISS with deep optimizations, adding support for the `IVFSQ8H` index type. • Provides Docker images for both CPU-only and GPU-enabled Milvus, with Docker compilation supported on machines with or without a GPU. https://github.com/milvus-io/milvus/releases/tag/v0.5.3 https://github.com/milvus-io/milvus/releases/tag/v0.5.3 Wed, 13 Nov 2019 11:44:48 GMT Milvus v0.5.3 doubles search-result transmission speed and adds new SDK APIs for id/distance retrieval across Python, Java, and C++. • Adds `results.id_array[i][j]` and `results.distance_array[i][j]` accessors in the Python SDK to retrieve a specific target vector's id and distance from search results. • Splits search result ids and distances into separate arrays (`results.id_array`, `results.distance_array`) in the Python SDK, reducing API response time and enabling faster looping when `nq` and `topk` are large. • Adds `getResultIdsList` and `getResultDistancesList` methods to the Java SDK for retrieving search result ids and distances separately with better performance, alongside the existing `getQueryResultsList` returning `QueryResult` objects. • Adds `keepalive` and `idleTimeout` settings in the Java SDK when connecting to the Milvus server. • Doubles the transmission speed of search results to client applications via gRPC message optimizations and API changes. • Enhances search performance of `IVF_SQ8H` index type. • C++ SDK now ships as a shared library. Breaking changes: • The gRPC generated-code API has changed in v0.5.3; clients relying on the previous generated API will need to update their code. https://github.com/milvus-io/milvus/releases/tag/v0.5.1 https://github.com/milvus-io/milvus/releases/tag/v0.5.1 Mon, 04 Nov 2019 07:05:06 GMT Milvus v0.5.1 adds GPU searching threshold control and pure GPU mode for IVFSQ8 and IVFFlat indexes. • Adds a GPU searching threshold to control the trigger point for GPU execution during vector search. • Supports pure GPU mode for IVFSQ8 and IVFFlat index types. • Enables CPU-based index building as an option. https://github.com/milvus-io/milvus/releases/tag/v0.5.0 https://github.com/milvus-io/milvus/releases/tag/v0.5.0 Mon, 21 Oct 2019 16:28:49 GMT Milvus v0.5.0 adds IVFSQ8 Hybrid index, startup data preloading, and a new Java SDK • Adds new `IVFSQ8 Hybrid` index type for approximate nearest-neighbor search. • Adds preloading of data during system startup phase to reduce cold-start query latency. • Adds a new Java SDK for interacting with Milvus. • Improves result merge performance. Breaking changes: • The `DeleteByDate` API has been removed.