The AI Toolchain — issue -424, July 30, 2020

Haystack 0.3.0 adds Dense Passage Retrieval, pipeline evaluation, PDF/DOCX indexing, ONNXRuntime support, and a file-upload REST endpoint.

$ git clone --branch 0.3.0 https://github.com/deepset-ai/haystack.git
# already have the repo? check out this version:
$ git checkout 0.3.0

from haystack.retriever.dense import DensePassageRetriever

retriever = DensePassageRetriever(
    document_store=document_store,
    embedding_model="dpr-bert-base-nq",
    do_lower_case=True,
    use_gpu=True
)
results = retriever.retrieve(query="What is cosine similarity?")

document_store.add_eval_data("../data/nq/nq_dev_subset_v2.json")

retriever.eval(top_k=10)
reader.eval(document_store=document_store, device=device)
finder.eval(top_k_retriever=10, top_k_reader=10)

from haystack.indexing.file_converters.pdf import PDFToTextConverter

converter = PDFToTextConverter(
    remove_header_footer=True,
    remove_numeric_tables=True,
    valid_languages=["de", "en"]
)
pages = converter.extract_pages(file_path="report.pdf")

• ›Adds DensePassageRetriever class with embedding_model, do_lower_case, and use_gpu arguments, enabling dual-encoder BERT-based retrieval that outperforms token-overlap methods when query and passage vocabulary differ.
• ›Adds eval() methods to retriever, reader, and finder (via finder.eval(top_k_retriever=..., top_k_reader=...)) for end-to-end pipeline evaluation of recall, precision, and speed.
• ›Adds document_store.add_eval_data() to load evaluation datasets (e.g. NQ-format JSON) directly into a DocumentStore for retriever and reader benchmarking.
• ›Adds PDFToTextConverter (from haystack.indexing.file_converters.pdf) with remove_header_footer, remove_numeric_tables, and valid_languages arguments, plus DocxToTextConverter (from haystack.indexing.file_converters.docx), both exposing extract_pages(file_path=...) for ingesting PDF and DOCX documents.
• ›Adds BaseConverter class with shared cleaning functions (header/footer removal, numeric table stripping) as a foundation for file-format-specific converters.

ONNX Runtime v1.4.0 adds Python IOBinding APIs, new quantization ops, AMD MIGraphX and ARM NN execution providers, and CUDA Einsum support.

$ git clone --branch v1.4.0 https://github.com/microsoft/onnxruntime.git
# already have the repo? check out this version:
$ git checkout v1.4.0

import onnxruntime as ort

sess = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider'])
io_binding = sess.io_binding()
io_binding.bind_cpu_input('input', input_array)
io_binding.bind_output('output')
sess.run_with_iobinding(io_binding)
outputs = io_binding.copy_outputs_to_cpu()

• ›Adds Python IOBinding methods bind_cpu_input, bind_output, and copy_outputs_to_cpu to simplify benchmarking on non-CPU devices without manual allocation or copy via third-party allocators.
• ›Adds quantization operators QLinearAdd and QAttention to the quantization operator set.
• ›Introduces pre-packing for constant Matrix B in DynamicQuantizeMatMul and QAttention to improve CPU transformer quantization performance.
• ›Adds CUDA execution support for the Einsum operator (opset 12).
• ›Adds preview-tier AMD MIGraphX Execution Provider for AMD GPU inference.

Triton v2.1.0 adds automatic FP16 TensorFlow optimization and TorchVision support in the PyTorch backend.

$ git clone --branch v2.1.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v2.1.0

• ›Adds a TensorFlow optimization option that enables automatic FP16 optimization of models served by the server.
• ›Adds TorchVision operations support to the PyTorch backend.

$ git clone --branch v1.15.0 https://github.com/triton-inference-server/server.git
# already have the repo? check out this version:
$ git checkout v1.15.0

Weaviate 0.22.13 adds _semanticPath GraphQL property to trace concept paths between search terms and results.

$ git clone --branch 0.22.13 https://github.com/weaviate/weaviate.git
# already have the repo? check out this version:
$ git checkout 0.22.13

• ›Adds _semanticPath underscore property to Get{} GraphQL queries with explore: {} set, returning the concept chain (e.g. ['iphone', 'apple', 'company', 'microsoft']) between the search term and each result; maximum limit for requests including _semanticPath is 25; requires contextionary v0.4.14 or later.