THE AI TOOLCHAIN Haystack v1.5.0 adds Generative Pseudo Labeling, batch pipeline querying, advanced eval label scopes, and DeBERTa support.
└──▷ GET THIS VERSION
$ git clone --branch v1.5.0 https://github.com/deepset-ai/haystack.git # already have the repo? check out this version: $ git checkout v1.5.0
└──▷ USE IT
Generate pseudo labels from an unlabeled domain corpus and fine-tune an EmbeddingRetriever in one workflow — no human annotation required.
from haystack.nodes.retriever import EmbeddingRetriever
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes.question_generator.question_generator import QuestionGenerator
from haystack.nodes.label_generator.pseudo_label_generator import PseudoLabelGenerator
document_store = InMemoryDocumentStore()
document_store.write_documents([...])
retriever = EmbeddingRetriever(
document_store=document_store,
embedding_model="sentence-transformers/msmarco-distilbert-base-tas-b",
max_seq_len=200
)
document_store.update_embeddings(retriever)
qg = QuestionGenerator(model_name_or_path="doc2query/msmarco-t5-base-v1", max_length=64, split_length=200, batch_size=12)
psg = PseudoLabelGenerator(qg, retriever)
output, _ = psg.run(documents=document_store.get_all_documents())
retriever.train(output["gpl_labels"]) Run multiple queries through an ExtractiveQAPipeline in a single call to reduce overhead in batch evaluation or CI pipelines.
from haystack.pipelines import ExtractiveQAPipeline
pipe = ExtractiveQAPipeline(reader, retriever)
predictions = pipe.pipeline.run_batch(
queries=["Who is the father of Arya Stark?", "Who is the mother of Arya Stark?"],
params={"Retriever": {"top_k": 10}, "Reader": {"top_k": 5}}
) Score pipeline evaluation only when a predicted answer appears within the correct surrounding context, not just as a string match.
eval_result = pipeline.eval(labels=eval_labels, params={"Retriever": {"top_k": 5}})
metrics = eval_result.calculate_metrics(answer_scope="context")
print(f'Reader - F1-Score: {metrics["Reader"]["f1"]}') - ›Adds
PseudoLabelGeneratorclass inhaystack.nodes.label_generator.pseudo_label_generatorthat automatically generates pseudo labels for dense retriever fine-tuning using aQuestionGeneratorand a cross-encoder, enabling unsupervised domain adaptation without manual annotation. - ›Adds run_batch() method to every query pipeline and node (e.g. Pipeline.run_batch(), FARMReader.predict_batch()), accepting a list of queries and single or nested lists of documents to process multiple queries in one call.
- ›Adds
answer_scopeanddocument_scopeparameters to EvaluationResult.calculate_metrics(), enabling fine-grained correctness definitions such asanswer_scope='context'for context-window-bounded answer matching. - ›Adds a
sortargument toJoinAnswersnode for controlling answer ordering. - ›Adds support for DeBERTa models (e.g.
'microsoft/deberta-v3-base','microsoft/deberta-v3-large') in FARMReader, delivering F1-score improvements up to ~92% on SQuAD 2.0.
+2 moreshow less
- ›Adds training checkpoint support in the retriever trainer.
- ›Includes document metadata when computing embeddings in
EmbeddingRetriever.
└──▷ BREAKING ON UPGRADE
- !Validation is now enforced for Ray pipelines, which may reject previously accepted but invalid pipeline configurations.
- !Context matching support added to pipeline.eval() changes evaluation behaviour — existing eval workflows may see different metric results.
Was this useful?