Trungdaik/Visual_information_retrieval

GDZ Scientific Document Retrieval Benchmark

A needle‑in‑a‑haystack benchmark for scientific document retrieval, built from historical volumes of the Göttinger Digitalisierungszentrum (GDZ). This dataset explicitly adapts the IRPAPERS methodology onto a real‑world, multilingual corpus to evaluate both text-based and visual document retrieval models.

Dataset Structure

The dataset is divided into two operational configurations:

1. queries

Contains the curated evaluation question set.

• Fields: pdf_id, page_number, question, answer
• Size: 180 needle-in-the-haystack questions.

Query Generation

The 180 needle-in-the-haystack questions were generated with Gemini Flash 3.5 using a prompt similar to IRPAPPERS designed to produce highly specific, self-contained, single-page-answerable questions. The model was shown each page image and instructed as follows:

I need your help simulating information seeking questions that information retrieval (IR) scientists would ask about the images attached to this message.

Could you please write a question that each page of the attached images uniquely answers? Further, these questions will be used to find this particular page amongst a large collection of other research papers, therefore, please try to make the question highly specific to this particular paper. Don't make the question too complicated, it should have a relatively short answer, not more than one or two sentences.

Please further explain why this question is only answerable by the cited page and not others, and why this question is only answerable by this particular paper and not other IR papers. Please also provide the answer to the question.

Please follow these instructions when generating these questions, answers, and explanations:

• Avoid phrases such as "this study" — make sure the questions are self-contained, as if searching through a large collection of similar papers for the answers. Use the specific name of the paper if needed, but generally avoid such references.
• Avoid "the researchers" — reference the study by the paper's name if needed.
• Do not reference the study as "the paper" — always use the paper's name. Questions must be self-contained.
• Avoid mathematical equations in answers.
• Keep answers short — no more than one or two sentences.

Each generated question was manually filtered for quality and page-uniqueness before inclusion in the final query set.

2. docs

Contains the complete target corpus text and image database.

• Fields: dataset_id, pdf_id_x, pdf_name, pdf_title, page_number, base64_str, base64_bytes, transcription, transcription_input_tokens, transcription_output_tokens, pdf_id_y, error
• Size: ~3,000 document pages.
• Format: Tab-separated value format compressed via Gzip (.csv.gz), ensuring raw text alignment and completely unquoted ("") Base64 image strings.

How It Was Built

1. Corpus Selection: Selected historical geophysics & mathematics volumes from the GDZ containing complex scientific text, tables, formulas, and diagrams (English + German).
2. Visual Processing: Extracted high-resolution page images (300 DPI) and encoded them into raw Base64 layout feature matrices (base64_str).
3. Text Extraction: Processed all pages through Tesseract OCR to build a parallel searchable text layer (transcription).
4. Query Generation: Generated and manually filtered 180 highly specific verification questions targeting fine-grained methodological parameters and equations buried across the ~3,000 pages.

Usage

You can load either configuration directly into your evaluation pipeline using the Hugging Face datasets library:

from datasets import load_dataset

# 1. Load the 180 question evaluation set
queries_dataset = load_dataset("Trungdaik/Visual_information_retrieval", "queries", split="train")
print("Sample Query:", queries_dataset[0])

# 2. Load the ~3,000 target document pages corpus
docs_corpus = load_dataset("Trungdaik/Visual_information_retrieval", "docs", split="train")
print("Sample Target Page:", docs_corpus[0]["dataset_id"])