Haystack v1.2.0 adds brownfield Elasticsearch import, scored Tapas QA, MongoDB-style metadata filters, and new pipeline/REST capabilities.
$ git clone --branch v1.2.0 https://github.com/deepset-ai/haystack.git # already have the repo? check out this version: $ git checkout v1.2.0
from haystack.document_stores import InMemoryDocumentStore
from haystack.utils import es_index_to_document_store
document_store = es_index_to_document_store(
document_store=InMemoryDocumentStore(),
original_index_name="existing_index",
original_content_field="content",
original_name_field="name",
included_metadata_fields=["date_field"],
index="new_index",
) from haystack.nodes import TableReader reader = TableReader(model_name_or_path="deepset/tapas-large-nq-reader", max_seq_len=512)
- ›Adds
es_index_to_document_storefunction to import existing Elasticsearch indices into any HaystackDocumentStoreby converting records to Document objects, accepting parametersoriginal_index_name,original_content_field,original_name_field,included_metadata_fields, andindex. - ›Adds
top_k_joinparameter toJoinDocuments.runto control how many documents are returned by the join node. - ›Adds
DELETE /feedbackREST API endpoint for clearing feedback/labels during testing, with label IDs now generated server-side. - ›Adds pipeline.save_to_deepset_cloud() method to push pipelines to Deepset Cloud.
- ›Adds pipeline.to_code() method to generate Python code from a pipeline definition.
+15 moreshow less
- ›Adds JSON Schema autogeneration for Pipeline YAML files, including a schema index for Schemastore.
- ›Adds YAML versioning support for Pipeline configuration files.
- ›Extends metadata filter syntax across document stores to support MongoDB-style nested boolean (
$and,$or,$not) and comparison ($eq,$in,$gt,$gte,$lt,$lte) operators; defaults to$and/$eqwhen operators are omitted, keeping existing filter expressions valid. - ›Adds
TapasForScoredQAmodel class enablingTableReaderto load Tapas models that return confidence scores (e.g.deepset/tapas-large-nq-reader,deepset/tapas-large-nq-hn-reader); answers are auto-sorted by table score then answer span score. - ›Adds reciprocal rank fusion as an additional merging method in the join node.
- ›Adds highlighting support in
ElasticsearchDocumentStore. - ›Adds
dot_productOpenSearch Script Scoring support inOpenSearchDocumentStore, includingdot_productsimilarity via HNSW. - ›Introduces read-only
DCDocumentStore(without labels support) for Deepset Cloud. - ›Adds pipeline.load_from_deepset_cloud() and pipeline listing via the Deepset Cloud SDK.
- ›Autogenerates OpenAPI specs file (
openapi.json) for the REST API, formatted as multiline for diff readability. - ›Introduces optional dependency groups for installation (e.g.
farm-haystack,farm-haystack[colab,faiss],farm-haystack[all],farm-haystack[dev]) so only required packages are installed; pip 22+ recommended. - ›Adds extended metadata filtering support to
WeaviateDocumentStorealong with more supported data types. - ›Adds extended metadata filtering support to
InMemoryDocumentStoreandSQLDocumentStore. - ›Makes
FileTypeClassifiermore flexible for routing documents by file type in pipelines. - ›Distributes intermediate layer distillation loss calculation across multiple GPUs.
- !Dependency management was restructured (
farm-haystacknow installs only a minimal subset by default); setups that relied on the previous all-inclusive install may be missing packages after upgrade. - !
uiandrestare now proper packages; imports or references assuming their previous module structure will break. - !
aiorwlockwas added to therayextra and maximum versions for some dependencies were pinned; environments using therayextra may need to update their dependency pins.