Beyond “Vibes”: How OPEN NOVELTY is Automating Verifiable Innovation in the Age of AI Paper Inflation

The AI research community is currently drowning in its own success. With hundreds of papers hitting arXiv every day and major conferences receiving thousands of submissions, the traditional peer-review system is reaching a breaking point.

Even the most dedicated reviewers can’t keep up. The result? Shallow reviews, missed prior art, and — increasingly — the problematic use of LLMs to generate “confident nonsense” reviews that sound authoritative but lack depth.

To tackle this, the NLP team at Fudan University, in collaboration with WisPaper.AI, has released OPEN NOVELTY: a system designed to transform academic novelty assessment from a subjective “gut feeling” into an evidence-driven, verifiable science.

The Problem: The Reviewer’s Blind Spot

We’ve all seen it. Reviewers are often forced to judge a paper’s “novelty” based on a quick skim. They may lack the time or the specific domain memory to trace every obscure reference. Worse, when LLMs are used to assist, they often rely on internal “knowledge” that can lead to hallucinations, incorrectly claiming a work is unoriginal without providing proof.

The Solution: Evidence-Driven Analysis

The core philosophy of OPEN NOVELTY is simple: Innovation must be verifiable. It moves away from subjective impressions and moves toward a strict, four-stage analytical pipeline:

1. Extraction & Query Generation: Using Claude 3.5 Sonnet, the system deconstructs a paper to “digitize” its core tasks and specific contributions, generating a precise set of search queries.
2. Massive Retrieval & Filtering: The WisPaper engine scans millions of documents. It applies multi-layer filters — filtering by date and stripping out self-citations — to narrow the field to the most relevant candidate papers.
3. Contribution-Level Comparison: Instead of just looking at abstracts, the system performs a “Contribution-level” full-text comparison. It builds a hierarchical classification to see where your work sits in the existing ecosystem, even detecting un-cited text overlaps.
4. Verifiable Verdicts: The output is a Markdown or PDF report containing a three-tier judgment:

• can_refute: Evidence shows the contribution is not original.
• cannot_refute: No evidence found to disprove the novelty.
• unclear: Insufficient information to make a call.

A Design for Fairness: Preventing AI Hallucinations

What makes OPEN NOVELTY stand out is its refusal to rely on the “internal knowledge” of an LLM. To prevent hallucinations, the system must retrieve a real paper with a valid DOI or arXiv ID before making a claim. If the LLM “feels” a work isn’t novel but cannot produce a specific quote or evidence from a retrieved paper, the system automatically downgrades the judgment. This conservative, evidence-first strategy ensures that authors are treated fairly.

Real-World Impact

This isn’t just a theoretical project. The team has already deployed the system on over 500 submissions for ICLR 2026, and the reports are publicly available for the community to see.

For researchers, this is a powerful “pre-flight” tool. By running your paper through OPEN NOVELTY before submission, you can catch missing citations or “accidental reinventing of the wheel” before a reviewer does.

In an era of paper explosions, we don’t just need more reviews — we need verifiable ones. OPEN NOVELTY is a massive step in that direction.

| Find papers faster on arXivSub with AI summary (CVPR/ICCV/ICML/ICLR/NeurIPS/AAAI/MICCAI)