SciEx: Squeezing Scientific Papers into a Glass of Structured Data

Have you ever wondered if it’s possible to use an LLM to automatically extract key information from a stack of scientific papers and organize it into a structured format like a spreadsheet or JSON file?

Imagine columns for things like virus names, temperature, humidity, experimental results, all cleanly laid out. The tricky part is, these papers don’t just contain text. They’re packed with tables, figures, and all kinds of formatting. Can a model really read through that and turn it into usable data?

This post might offer an insightful perspective.

Why Extracting Data from Scientific Papers Is So Difficult

Extracting structured information from research papers is harder than it sounds.

Many scientific papers are long-context documents and often exceed model context windows, which makes extraction difficult.

And the important details are scattered across different sections, sometimes buried deep in the supplementary materials. The needed information might appear in the text, but is often spread across tables, figures, or plots, sometimes made harder to extract by poor PDF quality or low-resolution scans.

To make things more complicated, the same concept can appear in many different forms. One paper might say “temp.” while another spells out “temperature.” Some use Celsius, others use Fahrenheit, so extraction pipelines often need unit normalization during aggregation.

And even when the data is there, the format you need it in can change depending on the project. One day it’s just temperature and pressure, the next you need virus strain, incubation time, and pH levels. For rule/template-based extractors or fine-tuned systems, schema changes often require re-architecting the pipeline or re-training/fine-tuning, which doesn’t scale well.

Furthermore, complex table structures like merged cells and nested headers make data alignment extremely difficult.

Extraction also requires complex cross-modal reasoning, such as correctly linking textual descriptions to data points in figures.

SciEx

SciEx is a processing pipeline for research papers. It is a prompt-driven, RAG-based framework that transforms research PDFs into structured knowledge.

It breaks down PDFs, stores the pieces in a contextualized database (with text embeddings in a vector DB and figures indexed via metadata), and extracts the fields you ask for. Under the hood, it loops through a cycle of REV (Retrieve–Extract–Verify), then compiles the results from multiple papers into one unified format.

The workflow can be viewed like map-reduce: per-paper extraction (map) plus cross-paper aggregation (reduce).

Figure 1: Overall framework design and workflow of SciEx. SciEx pre-processes PDFs using a PDF Extractor, segmenting text, scientific figures and page-level images. Scientific figures are linked with captions, and all data points structured into JSON and stored in a contextualized database with all extracted contents. Given a researcher’s request (explicit schema or LLM-generated), the REV module iteratively retrieves, extracts, and verifies information. An Aggregator then gathers data from multiple PDFs, canonicalizes variant terms referring to the same entity, and outputs a unified, structured JSON. [Source].

You can think of it as a small factory for turning scientific papers into clean, structured data. It runs through four main stages:

A. PDF Breakdown: Turning Papers into Processable Parts

The first step is to disassemble the paper into usable components.

• The text gets segmented into digestible chunks
• Charts/diagrams/tables are extracted as individual PNGs (preserving layout), and SciEx also stores full-page images for structural/visual context.
• A VLM identifies scientific figures, pairs them with captions (extracted or generated), and tries to extract axes, labels, and even data points into JSON, which are then stored and later used as retrievable evidence across modalities.

All of this goes into a multimodal database, one that combines text, tables, and images in one place.

In the implementation, SciEx uses Docling for fine-grained layout analysis and structural recognition when parsing PDFs. It supports DSPy for prompt optimization.

B. Schema Translation: What Exactly Are You Looking For?

There are two ways to define what data you need:

• Provide a clear list of fields (an explicit schema), or
• Describe your goal in natural language, and let the system infer the schema automatically

C. The REV Loop: Retrieve, Extract, Verify

REV stands for Retrieval, Extraction, and Verification. It runs until missing fields are resolved, a confidence threshold is met, or a predefined number of rounds is reached.

• Retrieval: Use the schema as a query blueprint and performs vector-based semantic search to retrieve top-k evidence across modalities (text chunks, table entries, figure JSON).
• Extraction: Use a language model to fill in the values for each field, along with their source for traceability.
• Verification: If anything is missing or uncertain, go back and retrieve again. Repeat the cycle until the output is good enough.

D. Aggregation: Merging Results Across Papers

Once multiple papers are processed, the results are aggregated into a unified, schema-conforming JSON representation (which can later be converted to tables for analysis).

• Group entries by entity or condition
• Normalize units (like converting Fahrenheit to Celsius)
• Resolve different wordings of the same concept into a canonical form

If sources conflict, a hierarchical strategy using statistical voting and cross-model ensembling helps decide.

And if information is genuinely absent from all retrieved evidence, the field is explicitly labeled as null (not guessed).

Case Study

Figure 2: Processing of a single PDF through the SciEx pipeline. Initial row data is extracted from relevant graphs in the PDF and formatted according to a specific schema. Missing information is identified and searched for through vector DB queries and passed examples to the LLM. [Source].

As shown in Figure 2, here’s a step-by-step walkthrough of how a research paper gets turned into rows of data.

The process is split into three stages, top to bottom:

1. Reading the chart like a data extraction task

It starts with a curve plot from the paper, shown in the upper-left corner. Lots of points, multiple lines.

Next to it is a prompt asking the model to convert relevant plots/graphs into structured JSON records (an initial draft).

The output on the right is a raw block of JSON. This is the first version of the data, directly “read” from the figure.

2. Converting raw data into a clean, structured row

The next step is formatting. The JSON from earlier is unstructured.

Now a query asks the model to organize the data according to a defined schema, for example, columns like virus type, surface material, humidity, exposure time, and so on.

The model takes the raw data and fits it into this schema, turning it into a single structured row in a table, shown in the middle-right of the figure.

3. Searching the paper to fill in missing fields

The key idea here: it’s fine if the first pass isn’t perfect, some fields might be missing, say, the surface material wasn’t found in the plot. The system can go back, look things up, and refine the output, just like a human doing a second round of review.

That’s where the system performs vector DB queries over a contextualized database. This database is built from sliced-up chunks of the paper: text blocks, images, tables, all indexed and searchable.

At this stage, the system looks for clues. The system retrieves relevant context via vector search and passes phrase patterns and example values to the model to improve filling missing fields. There’s even a hint list in the figure showing terms like “polystyrene” or “steel” as likely values.

The final step feeds the retrieved context and hints back into the model, helping it fill in the missing field. The result: a more complete, structured row of data.

Evaluation

A benchmark is built by manually annotating 143 papers from medical and environmental sciences (annotated by PhD students), and evaluate on its component datasets/subsets.

Two models were tested for extraction: Gemini-2.5-Flash and GPT-4o.

Each round retrieves the top-5 most relevant evidence chunks from the contextualized database as REV context; evidence may come from text, tables, or figure-related content.

Figure 3: SciEx’s performance using (a) Gemini-2.5-flash and (b) GPT-4o across three datasets. [Source].

Here’s the rough takeaway from Figure 3:

• GPT-4o consistently outperformed Gemini-2.5-Flash across the board, especially on visually dominated datasets, reflecting its stronger multi-modal capabilities.
• That said, both models showed low precision and F1 scores overall, especially on more complex tasks like CFS. Furthermore, recall generally exceeds precision, indicating the models tend to extract unwanted, noisy information.
• The conclusion is clear: even with prompt tuning, retrieval augmentation, and modular pipelines, the current performance still falls short of what’s needed for reliable, large-scale deployment.

Thoughts

The challenge of using LLMs to extract scientific data isn’t really about whether they can understand a sentence. It lies in maintaining coherent, verifiable alignment across a paper’s complex structure and modalities. The harder part is aligning information that spans across pages, figures, and even multiple papers.

The models are still far from stable. As it points out, there’s plenty of room for progress, especially in areas like cross-modal reasoning, domain generalization, and evaluation benchmarks.

After reading this study, my impression is that locating the right evidence matters more than extracting the data itself. Once the model is pointed to the correct segment, field-level extraction tends to be reasonably accurate, typically in the 0.5 to 0.6 range. The overall pattern shows higher recall than precision.

This suggests a shift in priorities for real-world deployment. Instead of focusing only on scale, it is effective to combine strong multimodal reasoning with better semantic retrieval, and active iterative verification. A verified evidence pool can often do more than another round of model upgrades.

Reference: Exploring LLMs for Scientific Information Extraction using the SciEx Framework.