When a P-Value Is Not a Claim: Statistical Verifier Semantics in Constraint Generative Theory

A statistical number is not the same thing as a scientific claim.

A p-value, confidence interval, posterior distribution, conformal prediction set, benchmark score, or risk estimate may be mathematically correct inside its method. But that does not automatically tell us what the number is allowed to support. Was the analysis path fixed in advance? Was there adaptive selection? Was the stopping rule declared? Was calibration preserved? Does the benchmark population match the deployment population? Is the loss function known? Can the result be replayed from audit logs?

These questions are not secondary details. They decide what the statistical artifact means.

Statistical Verifier Semantics in Constraint Generative Theory addresses this problem by extending Constraint Generative Theory (CGT) into a meta-statistical layer. Its core idea is simple:

A statistical report is a projection of a richer effect profile. A verifier should not only check the reported number; it should check which claim status that number can actually support.

This is not a replacement for statistics. It is a disciplined way to read statistical outputs under declared constraints.

The Problem: Statistical Reports Hide Their Generating Conditions

Many statistical disputes are not really disputes about arithmetic. They are disputes about what a reported result is being used to claim.

Consider two studies that report the same p-value. In one case, the hypothesis, data-cleaning rule, stopping rule, and model specification were declared before the analysis, and the audit trail can replay the result. In the other case, the same p-value appears after many unreported analysis choices.

The displayed number is the same. The scientific status is not.

The same issue appears in many settings:

• a benchmark score used as evidence of real-world capability;
• a posterior used outside its modeling assumptions;
• a conformal set applied under distribution shift;
• an e-value or confidence sequence reported without its stopping or combination rule;
• a risk estimate transported to a deployment population with a different loss function;
• a model card that reports accuracy but not the intended action, cost, or subgroup weighting.

The paper’s main contribution is to make these hidden coordinates explicit. Sampling, selection, calibration, stopping, loss, deployment context, audit path, marker state, and continuation obligation are treated as typed effect coordinates, not informal side notes.

CGT’s View: A Report Is Not the Whole Effect Profile

In CGT, a report is a visible or declared output. But the report is not identical to the full system that generated it.

For statistical artifacts, this means a number can hide many important conditions:

• which analysis paths were available;
• which path was selected;
• which calibration domain was used;
• which data dependencies were read;
• which verifier checked the result;
• which uncertainty or failure markers remain active;
• which deployment purpose the result is being used for.

The paper formalizes this using a report lens. A report lens maps a richer statistical effect profile into a visible report. Everything outside the lens may be forgotten by the report, even if it matters for the claim.

The key diagnostic question becomes:

If two effect profiles produce the same report, do they also support the same scientific claim?

If not, the report alone is insufficient.

What Is a Statistical Verifier?

A statistical verifier is not merely a calculator that recomputes a statistic. It is a checker that asks what status should be attached to a claim under declared evidence.

For example, a verifier may say:

• the claim is exactly supported;
• the claim is only bounded;
• the result is replayable but not minimal;
• the result is probability-qualified;
• the approximation is valid only under a declared error envelope;
• the benchmark score is reportable, but deployment use is unsupported;
• the analysis has calibration debt;
• the correct output is abstention.

This is important because scientific claims often fail by over-promotion. A weak report gets treated as a strong conclusion. A diagnostic result gets treated as deployment evidence. An approximate frontier gets treated as an exact optimum. A timeout gets treated as absence of a problem.

The paper blocks those upgrades unless the verifier has the required support coordinates.

Same Statistic, Different Status

One of the clearest examples is the same p-value with different certificate status.

If the analysis path was sealed, the stopping rule was declared, calibration was checked, and the replay record is accepted by a verifier, the p-value may carry a strong fixed-path statistical status.

If the same p-value appears after an adaptive menu of unsealed analysis choices, it may carry open search status, selection markers, or calibration debt.

The number is identical. The status differs.

This is the core intuition of the paper. Statistical meaning is not only in the number. It is also in the constraint system that made the number available.

Benchmark Scores Are Not Deployment Claims

The same logic applies to AI benchmarks.

A model card may report a benchmark accuracy. That score can be a valid report. It may even be auditable. But using that score as evidence for deployment requires additional coordinates:

• target population;
• intended use;
• action set;
• loss function;
• subgroup weighting;
• distribution shift assumptions;
• failure and rollback conditions.

Without those coordinates, the benchmark score remains a report, not a full deployment claim.

This distinction matters for AI safety, AI governance, and research automation. AI systems are increasingly evaluated through compressed reports: benchmark scores, leaderboard ranks, automated experiment summaries, model cards, and audit logs. Statistical Verifier Semantics asks what those reports are actually allowed to support.

Purpose Matters

A central idea in the paper is purpose-indexed availability.

A statistical artifact does not have one universal meaning. Its availability depends on the purpose for which it is used.

A p-value used for exploratory screening reads different coordinates from a p-value used for confirmatory publication. A benchmark used for internal diagnosis reads different coordinates from a benchmark used for deployment approval. A risk estimate used for comparison reads different coordinates from a risk estimate used for safety-critical release.

The paper represents this through purpose requirement signatures and purpose morphisms. In plain terms:

A result can be available for one purpose and unavailable for a stronger purpose.

Moving from a weaker purpose to a stronger one requires a certificate that the needed support rows are preserved. If the bridge is missing, the claim cannot be silently promoted.

This gives a useful rule:

Stronger use requires stronger support.

Replay Is Not the Same as Minimality

The paper also separates two ideas that are often confused:

1. Can the result be replayed from the declared record?
2. Is the replay record minimal or optimal?

A replay certificate may show that a result can be reconstructed from audit logs, read nodes, failure branches, digests, resolver rules, and randomness policies. That is valuable.

But finding the smallest read set that still determines the result can be computationally hard. The paper formalizes an Audit Read-Set Minimization problem and shows that minimization can be NP-complete in finite fragments.

This distinction matters for audit design. A system may have a valid replay certificate while still lacking a minimality certificate. That is not a failure of replay. It is a different status coordinate.

In ordinary language:

A certificate can be easy to check even when the best certificate is hard to find.

Approximation Needs Its Own Ledger

Modern statistical and AI systems often use approximations: finite audit abstractions, surrogate kernels, numerical enclosures, bounded losses, total variation bounds, Wasserstein distances, integral probability metrics, or heuristic search.

The paper does not reject approximation. It gives approximation its own typed status.

An approximate risk claim is scientifically useful only when the approximation is connected to a declared discrepancy object, loss class, probability qualification, and composition rule. For example, total variation may control bounded losses, while Wasserstein distance requires appropriate Lipschitz conditions. These cannot be silently substituted.

If the loss is unbounded, if the probability event is not sealed, or if the composition rule is missing, the approximation remains marked, outer, bracketed, or abstained.

This is a strong anti-overclaiming rule:

Approximate evidence can support approximate claims, but it cannot become exact evidence without the required certificate.

Numerical Results Need Validated Enclosures

The paper also extends the discussion to represented spaces and effective replay. This is the implementation-facing layer.

A mathematical object may exist, but a verifier still needs finite, checker-readable evidence. For continuous or numerical settings, that means realizer traces, finite name prefixes, precision schedules, oracle policies, validated enclosures, interval bounds, rounding rows, residual rows, and digests.

A floating-point output without containment evidence is not automatically a scientific availability witness. It may be useful, but it carries rounding debt unless the enclosure and residual rows are declared.

This is especially relevant for automated research systems. AI scientists and research agents may generate numerical results quickly. Statistical Verifier Semantics gives a language for asking whether those results are replayable, bounded, approximate, probability-qualified, or merely heuristic.

Frontiers Instead of Fake Scalar Answers

The paper repeatedly avoids collapsing multi-dimensional uncertainty into a single score.

When several constraints matter at once — resource use, approximation error, risk, order robustness, probability qualification, continuation affordance, and valuation — there may be no single best answer unless a scalarization or valuation ledger is declared.

In that case, the correct output is a frontier.

For example, one analysis may use fewer resources but lose order robustness. Another may preserve order but require more computation. Another may keep the risk bound but close later follow-up paths. Without a declared valuation rule, the verifier should return a coupled frontier rather than inventing a scalar optimum.

This is useful for scientific integrity. It prevents systems from pretending that tradeoffs have been resolved when the valuation basis has not been declared.

Why This Matters for AI Research Agents

The framework is especially relevant to AI research agents and automated science systems.

Such agents may soon read papers, run experiments, generate hypotheses, produce benchmark reports, and suggest deployments at high speed. The bottleneck will not only be generation. It will be claim control.

A research agent needs to know:

• what a statistic supports;
• which hidden assumptions remain active;
• whether the result can be replayed;
• whether the approximation is certified;
• whether a benchmark transfers to a new purpose;
• whether deployment valuation is available;
• whether to emit a claim, a bound, a frontier, or abstention.

Statistical Verifier Semantics gives a vocabulary for this. It helps an agent avoid turning reported numbers into unsupported scientific claims.

What the Paper Does Not Claim

The paper is careful about its limits.

It does not replace statistical inference, causal inference, Blackwell theory, conformal prediction, safe testing, e-values, confidence sequences, computable analysis, validated numerics, provenance schemas, model cards, or multiobjective optimization.

It also does not require every valid statistical claim to be deterministic, nonadaptive, point-identified, marker-free, or decision-ready.

Instead, it asks for status discipline:

Whatever status the system emits should match the evidence coordinates the verifier can actually read.

Exact images, lower-upper envelopes, sensitivity regions, menu images, risk frontiers, replay certificates, and probability-qualified statuses can all be legitimate scientific outputs when their generating constraints are auditable.

The problem is not uncertainty. The problem is pretending that uncertainty has been resolved when it has only been hidden by the report lens.

The Main Insight

The main insight of Statistical Verifier Semantics in Constraint Generative Theory is that statistical artifacts need verifier semantics.

A p-value is not a full claim. A posterior is not a deployment certificate. A benchmark score is not a real-world capability guarantee. A numerical estimate is not automatically a validated enclosure. A replay certificate is not automatically a minimal read-set certificate. A frontier is not a scalar optimum unless a valuation rule is declared.

The paper gives a CGT-based language for keeping these distinctions explicit.

Its practical message is:

Do not ask only what number was reported. Ask what status the declared verifier is allowed to assign to the claim.

That status may be exact, bounded, outer, probability-qualified, approximate, frontier-certified, marked, or abstained. Each is more scientifically honest than promoting a report beyond its support.

For AI evaluation, automated research, statistical reporting, reproducible science, and scientific audit systems, this is a useful shift. It moves attention from isolated statistical outputs to the evidence structure that makes claims scientifically available.

Citation

Takahashi, K. (2026). Constraint Generative Theory: Typed Constraint Effects and Scientific Availability. Zenodo. https://doi.org/10.5281/zenodo.20360377 Supplement: Statistical Verifier Semantics in Constraint Generative Theory

CGT Series Overview: https://kadubon.github.io/github.io/constraint-generative-theory-index.html