Backfill issue -394 — The AI Toolchain

EDITIONS tail grep head diff uniq

The daily firehose — everything the toolchain shipped today, already filtered.

// HOW THIS ISSUE IS MADE

We read every release from the 174 tools on our watchlist at the source — GitHub and GitLab release notes, vendor release pages and changelogs, project blogs and feeds, vendor press releases, and the source code behind the tag. Bug-fix-only releases and non-product newsroom noise are dropped; what's left is summarized down to the new capability, how to try it, and any screenshots or videos the release itself published. Every entry links to the sources it was built from.

VIEW

ISSUE VIEW full issue

Do you prefer this view?

$ tct list   # 4 tools matched

AI & LLM Tooling

◆ AI Agent Frameworks

deepset Haystack

v1.13.0 NOTES

Haystack v1.13 adds stop words for PromptNode, ImageToText and CsvTextConverter nodes, tiktoken support, and HA for Weaviate.

└──▷ GET THIS VERSION

$ git clone --branch v1.13.0 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout v1.13.0

└──▷ USE IT

Stop LLM output at a sentinel phrase to keep answers concise when using PromptNode.

python

from haystack.nodes import PromptNode

pn = PromptNode(
    model_name_or_path='text-davinci-003',
    stop_words=['\nHuman:', 'END']
)
result = pn.run(prompt='Summarize the following document: ...')

›Adds stop_words list parameter to PromptNode to halt LLM text generation when any stop word is encountered; stop words are excluded from the response.
›Adds index parameter to TfidfRetriever to specify which index to query.
›Adds knn_engine parameter to SearchEngineDocumentStore to make score_script a first-class citizen for KNN search.
›New ImageToText node generates captions from image files and produces Haystack Document objects from them.
›New CsvTextConverter node loads CSV files of FAQ question-answer pairs and sends them to a DocumentStore for FAQ matching pipelines.

+10 moreshow less

›Adds retry with exponential back-off to PromptNode's OpenAI model integrations.
›Supports cl100k_base tokenization via OpenAI's tiktoken library for dramatically faster tokenization of GPT models; falls back to HuggingFace tokenizers on unsupported platforms (Python < 3.8, arm64, macOS).
›Adds high-availability (HA) support for the Weaviate DocumentStore.
›Enables text-embedding-ada-002 model for EmbeddingRetriever.
›Updates Cohere embedding models support and adds use of Cohere's truncate option in Cohere.embed.
›Stores id_hash_keys in Document objects to make documents clonable.
›Adds async functionality support for Ray Serve pipelines.
›Makes new sklearn models the default in QueryClassifier.
›Adds PromptModel, PromptNode, and PromptTemplate to expand LLM support.
›Raises a warning in Preprocessor when a document's length exceeds the configured threshold.

└──▷ BREAKING ON UPGRADE

!Native PyTorch AMP replaces the previous AMP integration; existing code relying on the old AMP behaviour will break.
!invocation_context is moved from meta to its own pipeline variable; code reading meta['invocation_context'] will break.
!The batch_size parameter names in distillation are renamed for consistency; existing calls using the old names will break.

Was this useful?

LangChain

v0.0.75 10 RELEASES · 2023-01-16 → 2023-01-31 NOTES STABLE

LangChain v0.0.75 adds MMR search, TensorFlow embeddings, pinnable LangChainHub deps, and SQL intermediate steps.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.75 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.75

›Enables MMR (maximal marginal relevance) search on vector stores for more diverse retrieval results.
›Adds TensorFlow embeddings support as a new embedding provider.
›Exposes memory key name configuration, allowing callers to control the key used for memory in chains.
›Returns intermediate SQL steps from the SQL agent, giving visibility into query construction and execution.
›Centralizes LangChainHub loading logic and adds the ability to pin dependency versions when loading from the Hub.

+1 moreshow less

›Passes kwargs from initialize_agent into the agent classmethod, enabling custom agent parameters at initialization time.

9 more releases in this issue · 2023-01-16 → 2023-01-31

v0.0.74 NOTES STABLE

LangChain v0.0.74 adds a tool decorator, HuggingFace pipeline LLM, sample rows in SQLDatabase info, and Cohere stop-token kwargs.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.74 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.74

›Adds a @tool decorator for defining custom agent tools from plain Python functions.
›Adds a HuggingFace pipeline integration, enabling local HF pipelines as LLM backends.
›Adds model_kwargs support to the Cohere LLM to pass stop tokens and other provider-specific parameters.
›Includes sample rows from each table in SQLDatabase table info, giving agents richer schema context.
›Increases the context-size limit for text-davinci-003 to 4097 tokens.

+1 moreshow less

›Improves SQL prompt construction for better agent query generation.

v0.0.72 NOTES STABLE

LangChain v0.0.72 adds hub loading for chains and agents, full serialization support, and agent iteration limits.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.72 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.72

›Adds max_iterations upper bound parameter to agents to cap runaway iteration loops.
›Adds dynamic k reduction parameter to retrieval to stay within token limits at query time.
›Enables loading chains directly from the LangChain Hub.
›Enables loading agents directly from the LangChain Hub.
›Adds serialization support for agents, chains, output parsers, LLMs, and tools.

+1 moreshow less

›Adds prompt type tagging to prompt objects.

v0.0.71 NOTES STABLE

LangChain v0.0.71 adds tracing support for chain and agent execution.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.71 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.71

›Adds tracing support to LangChain, enabling instrumentation of chain and agent runs for observability.

v0.0.70 NOTES STABLE

LangChain v0.0.70 adds LLM chain serialization, stop sequences for streaming, and moves HyDE into chains.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.70 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.70

›Adds namespace parameter to Pinecone.from_index for scoped vector store queries.
›Adds stop parameter support to the streaming interface via add stop to stream.
›Enables serialization of LLM chains, allowing chains to be saved and reloaded.
›Moves HyDE (Hypothetical Document Embeddings) into the chains module for more consistent access.

v0.0.68 NOTES STABLE

LangChain v0.0.68 adds a verbose flag, OpenAI callback, extra SerpAPI tools, and a common prompt load method.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.68 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.68

›Adds a verbose flag for tracing and debugging chain/agent execution.
›Adds an OpenAI callback for tracking and handling OpenAI API interactions.
›Forwards model_kwargs through HuggingFacePipeline so arbitrary model parameters can be passed at pipeline construction time.
›Adds a common prompt load method for loading prompts via a shared interface.
›Adds extra SerpAPI tools beyond the base search wrapper.

v0.0.67 NOTES STABLE

LangChain v0.0.67 adds ConversationEntityMemory, FAISS local save/load, and kwargs passthrough for tools.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.67 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.67

›Adds ConversationEntityMemory, a chain that performs entity extraction and summarization to maintain per-entity context across conversation turns.
›Adds local saving and loading support for FAISS vector stores, enabling persistent index storage without a remote vector database.
›Adds kwargs passthrough support to load_tools, allowing callers to pass additional keyword arguments through to individual tool constructors.
›Adds support for loading few-shot prompt templates from YAML files.

v0.0.66 NOTES STABLE

LangChain v0.0.66 adds Bing search wrapper, Qdrant vector store integration, and search_kwargs support across vector DB chains.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.66 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.66

›Adds search_kwargs option to VectorDBQAWithSourcesChain to pass additional parameters to the underlying vector store search.
›Adds ids parameter to Pinecone's from_texts and add_texts methods, enabling caller-specified document IDs on upsert.
›New Bing search wrapper integration for use as a tool or retriever.
›New Qdrant vector store integration.

v0.0.65 NOTES STABLE

LangChain v0.0.65 adds an SQL database chain and experimental Cohere support.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.65 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.65

›Adds a new SQL database chain for querying SQL databases via natural language.
›Adds experimental Cohere integration support.

v0.0.64 NOTES STABLE

LangChain v0.0.64 adds a new API chain and more complex SQL chain support.

└──▷ GET THIS VERSION

$ git clone --branch v0.0.64 https://github.com/langchain-ai/langchain.git
# already have the repo? check out this version:
$ git checkout v0.0.64

›Adds a new API chain for building LLM-powered workflows that interact with external APIs.
›Extends the SQL chain to support more complex query generation scenarios.

Was this useful?

LlamaIndex

v0.3.1 3 RELEASES · 2023-01-29 → 2023-01-31 NOTES STABLE

LlamaIndex v0.3.1 adds ObsidianReader for parsing Markdown vaults with automatic hyperlink and image removal.

└──▷ GET THIS VERSION

$ git clone --branch v0.3.1 https://github.com/run-llama/llama_index.git
# already have the repo? check out this version:
$ git checkout v0.3.1

›Adds ObsidianReader to parse a directory of Markdown files (Obsidian vaults or any Markdown collection), automatically stripping hyperlinks and images.

2 more releases in this issue · 2023-01-29 → 2023-01-31

v0.3.0 NOTES STABLE

LlamaIndex v0.3.0 improves the composability interface for building composite indices.

└──▷ GET THIS VERSION

$ git clone --branch v0.3.0 https://github.com/run-llama/llama_index.git
# already have the repo? check out this version:
$ git checkout v0.3.0

›Improves the composability interface for combining multiple indices into composite query structures.

└──▷ BREAKING ON UPGRADE

!The composability interface has changed; existing code using composable indices must be updated — see the ComposableIndices notebook for migration details.

v0.2.17 NOTES STABLE

LlamaIndex v0.2.17 adds file-level control to SimpleDirectoryReader and recursive Notion child-page reading.

└──▷ GET THIS VERSION

$ git clone --branch v0.2.17 https://github.com/run-llama/llama_index.git
# already have the repo? check out this version:
$ git checkout v0.2.17

└──▷ USE IT

Load only a specific subset of files from a mixed directory — useful in CI pipelines where you want to index only changed documents.

python

from llama_index import SimpleDirectoryReader

reader = SimpleDirectoryReader(file_paths=['docs/overview.pdf', 'docs/changelog.md'])
documents = reader.load_data()

›Adds file_paths argument to SimpleDirectoryReader, letting callers specify an explicit list of files instead of an entire directory.
›Upgrades the Notion reader to recursively read child pages in addition to top-level pages.

Was this useful?

◆ AI Coding Agents

Zed

v0.70.0 4 RELEASES · 2023-01-04 → 2023-01-25 NOTES STABLE

Zed v0.70.0 adds vim replace mode and macOS 'Open With' context menu support.

└──▷ GET THIS VERSION

$ git clone --branch v0.70.0 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.70.0

›Adds replace mode to vim keybindings.
›Enables opening files in Zed via the macOS right-click context menu 'Open With' option.

3 more releases in this issue · 2023-01-04 → 2023-01-25

v0.69.2 NOTES STABLE

Zed v0.69.2 adds vim f/t bindings, scroll-position restore, resilient collaboration, and diagnostic opt-out settings.

└──▷ GET THIS VERSION

$ git clone --branch v0.69.2 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.69.2

›Adds settings to opt out of diagnostic and metric reporting.
›Adds f and t bindings to vim mode.
›Makes collaboration significantly more resilient to temporary disconnects.
›Following into the terminal and search results now automatically opens screensharing (when permitted).
›Improves sorting and filtering of LSP suggestions, particularly for TypeScript.

v0.68.1 NOTES STABLE

Zed v0.68.1 adds git gutters in multi buffers, a recent projects picker, and new workspace commands.

└──▷ GET THIS VERSION

$ git clone --branch v0.68.1 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.68.1

›Adds Close clean Items, Close all items, Request a feature, File a bug, and Copy system specs commands to the workspace command palette.
›Adds a recent projects picker menu to the workspace.
›Adds git gutter support in multi buffers.
›Adds source information to TypeScript autocomplete suggestions.
›Adds Home and End key support in the editor.

+1 moreshow less

›Adds a discoverable way to close the editor toolbar.

v0.67.5 NOTES STABLE

Zed v0.67.5 adds workspace restoration, Vim scroll commands, and syntax highlighting for Scheme and Racket.

└──▷ GET THIS VERSION

$ git clone --branch v0.67.5 https://github.com/zed-industries/zed.git
# already have the repo? check out this version:
$ git checkout v0.67.5

›Adds workspace restoration: reopening a project restores the state of panes, tabs, and terminals.
›Adds scroll commands to Vim mode.
›Adds syntax highlighting for Scheme and Racket.
›Enables following collaborators into 'Find All References' and 'Refactor' multi-buffers.
›Redesigned app icon to respect macOS conventions.

Was this useful?

// END OF TRANSMISSION

The AI Toolchain · curated daily · January 31, 2023
Releases & features from the open-source and commercial AI tools that practitioners actually run.

The AI Toolchain — issue -394, January 31, 2023

deepset Haystack

LangChain

LlamaIndex

Zed