Backfill issue -418 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 2 tools matched

AI & LLM Tooling

◆ AI Agent Frameworks

deepset Haystack

v0.7.0 NOTES

Haystack v0.7.0 adds summarization pipelines, a demo UI, batch/generator document streaming, and filter support for DensePassageRetriever.

└──▷ GET THIS VERSION

$ git clone --branch v0.7.0 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout v0.7.0

└──▷ USE IT

Run a retrieve-then-summarize pipeline to display document summaries as search result previews.

python

from haystack.pipeline import SearchSummarizationPipeline
from haystack.summarizer import TransformersSummarizer

summarizer = TransformersSummarizer(model_name_or_path="google/pegasus-xsum")
pipe = SearchSummarizationPipeline(summarizer=summarizer, retriever=retriever)
results = pipe.run(query="What caused the California wildfires?")

Update embeddings on a million-document corpus without exhausting RAM by processing in chunks.

python

document_store.update_embeddings(retriever=retriever, batch_size=10000)

›Adds batch_size parameters to most DocumentStore methods (write_documents(), update_embeddings(), get_all_documents()) to load documents in chunks and reduce memory footprint on large datasets.
›Adds get_all_documents_generator() method to stream documents one-by-one from a document store, enabling low-memory iteration over datasets exceeding 1 million documents.
›Adds TransformersSummarizer class supporting models like PEGASUS, usable standalone via summarizer.predict(documents=docs, generate_single_summary=False) or as a pipeline node.
›Adds SearchSummarizationPipeline predefined pipeline that chains retrieval and summarization in a single pipe.run() call.
›Adds a simple demo UI for interactively testing search pipelines, inspecting API responses, and adjusting basic config params.

+2 moreshow less

›Adds filter support for DensePassageRetriever combined with InMemoryDocumentStore.
›Adds support for a custom embedding field in InMemoryDocumentStore.

└──▷ BREAKING ON UPGRADE

!The index_buffer_size argument is removed from FAISSDocumentStore.__init__(); replace it with the new batch_size argument on methods like write_documents(), update_embeddings(), and get_all_documents().
!The PreProcessor argument split_stride is renamed to split_overlap; any code passing split_stride=N must be updated to split_overlap=N.

Was this useful?

Other / Uncategorized

◆ VECTOR DB RAG

Milvus

v0.10.5 NOTES

Milvus v0.10.5 lets load_collection() preload specific partitions instead of entire collections.

└──▷ GET THIS VERSION

$ git clone --branch v0.10.5 https://github.com/milvus-io/milvus.git
# already have the repo? check out this version:
$ git checkout v0.10.5

›Extends load_collection() to accept specified partitions, enabling targeted preloading instead of loading an entire collection into memory.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · January 21, 2021
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -418, January 21, 2021

deepset Haystack

Milvus