← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better188 / 280 · beu-9fdfdfd4938926ce

White Paper on Light Sterile Neutrino Searches and Related Phenomenology

Astrophysics · 2203.07323v3

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

White Paper on Light Sterile Neutrino Searches and Related Phenomenology

3 Interpretations of the Anomalies > 3.4 Summary of Interpretations

…ns the indication in favor of a short-baseline disappearance of electron neutrinos, possibly due to neutrino oscillations. Figure 40 shows the 90% contours in the | U e ​ 4 | 2 |U_{e4}|^{2} – Δ ​ m 41 2 \Delta{m}^{2}_{41} plane obtained from the analysis of the measured and expected Ge 71 {}^{71}\rm{Ge} event rates, considering the neutrino survival probability Eq. ( 7 ), where one can see that the squared-mass difference Δ ​ m 41 2 \Delta{m}^{2}_{41} allowed values are ∼ 1 \sim 1 eV 2 or larger, for the different approaches to compute the ν e − Ga 71 \nu_{e}\,-\,{}^{71}\rm{Ga} cross section.

…+1 neutrino oscillation hypothesis with the Reactor Antineutrino Anomaly and with the inclusion of data from tritium experiments and from experiments measuring solar neutrinos [Giunti:2022btk]. Remarkably, it was found that the Gallium Anomaly is in strong tension with bounds obtained from the other data sets. In addition, when all data are combined, the corresponding parameter goodness of fit is below 0.042%, implying a tension of ∼ 5 ​ σ \sim 5\sigma , leading to the conclusion that it should be necessary to seek for alternative solutions to the short-baseline oscillations for this anomaly.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs provide coherent, relevant continuations that fit the academic context. However, Output B better serves the manuscript's argumentative arc. The draft explicitly states that the Gallium Anomaly is in 'strong tension' with other data sets, with a parameter goodness of fit below 0.042% and ~5σ tension, concluding that 'alternative solutions' are needed. Output B directly reinforces and extends this conclusion by explicitly contrasting the Gallium Anomaly with reactor/accelerator anomalies and stating that 'no single sterile neutrino parameter point simultaneously describes all short-baseline observations'—a natural synthesis that follows from the tension described. Output A, while factually accurate based on sources, shifts focus to the reactor antineutrino anomaly's resolution via flux model refinements. This introduces a somewhat parallel narrative rather than continuing the draft's specific point about the Gallium Anomaly's unique problematic status. The draft's paragraph is building toward explaining why the Gallium Anomaly is specially problematic; Output B continues that trajectory, while Output A pivots to discuss a different anomaly's resolution. Output B requires no repair and flows seamlessly; Output A is usable but slightly less fitted to the immediate argumentative need.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

The reactor antineutrino anomaly, by contrast, is largely alleviated once the measured reactor rates and fuel-evolution data are compared with the Kurchatov Institute conversion model and the Estienne-Fallot summation model, both of which are favored over the Huber-Mueller prediction by goodness-of-fit tests.

Passages supplied to the Evidence version

Card 1 · Reactor antineutrino anomaly in light of recent flux model refinements

We study the status of the reactor antineutrino anomaly in light of recent reactor flux models obtained with the conversion and summation methods. We present a new improved calculation of the IBD yields of the standard Huber-Mueller (HM) model and those of the new models. We show that the reactor ra…

Read full passage excerpt

We study the status of the reactor antineutrino anomaly in light of recent reactor flux models obtained with the conversion and summation methods. We present a new improved calculation of the IBD yields of the standard Huber-Mueller (HM) model and those of the new models. We show that the reactor rates and the fuel evolution data are consistent with the predictions of the Kurchatov Institute (KI) conversion model and with those of the Estienne-Fallot (EF) summation model, leading to a plausible robust demise of the reactor antineutrino anomaly. We show that the results of several goodness of fit tests favor the KI and EF models over other models that we considered. We also discuss the implications of the new reactor flux models for short-baseline neutrino oscillations due to active-sterile mixing. We show that reactor data give upper bounds on active-sterile neutrino mixing that are not very different for the reactor flux models under consideration and are in tension with the large mix …

Card 2 · 3+1 and 3+2 Sterile Neutrino Fits

In spite of the presence of a tension in the interpretation of the data, 3+1 neutrino mixing is attractive for its simplicity and for the natural correspondence of one new entity (a sterile neutrino) with a new effect (short-baseline oscillations). The allowed regions in the oscillation parameter sp…

Read full passage excerpt

In spite of the presence of a tension in the interpretation of the data, 3+1 neutrino mixing is attractive for its simplicity and for the natural correspondence of one new entity (a sterile neutrino) with a new effect (short-baseline oscillations). The allowed regions in the oscillation parameter space can be tested in near-future experiments. In the framework of 3+2 neutrino mixing there is less tension in the interpretation of the data, at the price of introducing a second sterile neutrino. Moreover, the improvement of the parameter goodness of fit is mainly a statistical effect due to an increase of the number of parameters.

Card 3 · BEST impact on sterile neutrino hypothesis

Recently, the Baksan Experiment on Sterile Transitions (BEST) has presented results [1] confirming the gallium anomaly---a lack of electron neutrinos ${\ensuremath{\nu}}_{e}$ at calibrations of SAGE [2,3] and GALLEX [4]---at the statistical significance exceeding $5\ensuremath{\sigma}$. This result…

Read full passage excerpt

Recently, the Baksan Experiment on Sterile Transitions (BEST) has presented results [1] confirming the gallium anomaly---a lack of electron neutrinos ${\ensuremath{\nu}}_{e}$ at calibrations of SAGE [2,3] and GALLEX [4]---at the statistical significance exceeding $5\ensuremath{\sigma}$. This result is consistent with explanation of the gallium anomaly as electron neutrino oscillations into a sterile neutrino, ${\ensuremath{\nu}}_{s}$. Within this explanation, the BEST experiment itself provides the strongest evidence for the sterile neutrino among all the previous anomalous results in the neutrino sector. We combine the results of gallium experiments with searches for sterile neutrinos in reactor antineutrino experiments (assuming $CPT$ conservation in the $3+1$ neutrino sector). While the ``gallium'' best-fit point in the model parameter space (sterile neutrino mass squared ${m}_{{\ensuremath{\nu}}_{s}}^{2}\ensuremath{\approx}1.25\text{ }\text{ }{\mathrm{eV}}^{2}$, sterile-electron ne …

Card 4 · KATRIN bound on 3+1 active-sterile neutrino mixing and the reactor antineutrino anomaly

Abstract We present the bounds on 3+1 active-sterile neutrino mixing obtained from the first results of the KATRIN experiment. We show that the KATRIN data extend the Mainz and Troitsk bound to smaller values of $$ \Delta {m}_{41}^2 $$ Δ m 41 2 for large mixing and improves the exclusion of the larg…

Read full passage excerpt

Abstract We present the bounds on 3+1 active-sterile neutrino mixing obtained from the first results of the KATRIN experiment. We show that the KATRIN data extend the Mainz and Troitsk bound to smaller values of $$ \Delta {m}_{41}^2 $$ Δ m 41 2 for large mixing and improves the exclusion of the large- $$ \Delta {m}_{41}^2 $$ Δ m 41 2 solution of the Huber-Muller reactor antineutrino anomaly. We also show that the combined bound of the Mainz, Troitsk, and KATRIN tritium experiments and the Bugey-3, NEOS, PROSPECT, and DANSS reactor spectral ratio measurements exclude most of the region in the (sin22ϑ ee , $$ \Delta {m}_{41}^2 $$ Δ m 41 2 ) plane allowed by the Huber-Muller reactor antineutrino anomaly. Considering two new calculations of the reactor antineutrino fluxes, we show that one, that predicts a lower 235U antineutrino flux, is in agreement with the tritium and reactor spectral ratio measurements, whereas the other leads to a larger tension than the Huber-Muller prediction. We a …

Card 5 · The PROSPECT physics program

Abstract The precision reactor oscillation and spectrum experiment, PROSPECT, is designed to make a precise measurement of the antineutrino spectrum from a highly-enriched uranium reactor and probe eV-scale sterile neutrinos by searching for neutrino oscillations over a distance of several meters. P…

Read full passage excerpt

Abstract The precision reactor oscillation and spectrum experiment, PROSPECT, is designed to make a precise measurement of the antineutrino spectrum from a highly-enriched uranium reactor and probe eV-scale sterile neutrinos by searching for neutrino oscillations over a distance of several meters. PROSPECT is conceived as a 2-phase experiment utilizing segmented 6 Li-doped liquid scintillator detectors for both efficient detection of reactor antineutrinos through the inverse beta decay reaction and excellent background discrimination. PROSPECT Phase I consists of a movable 3 ton antineutrino detector at distances of 7–12 m from the reactor core. It will probe the best-fit point of the ν e disappearance experiments at 4 σ in 1 year and the favored region of the sterile neutrino parameter space at > 3 σ in 3 years. With a second antineutrino detector at 15–19 m from the reactor, Phase II of PROSPECT can probe the entire allowed parameter space below 10 eV 2 at 5 σ in 3 additional years. …

Model-reported card use

Card 1 · fact

Affected output: largely alleviated once the measured reactor rates and fuel-evolution data are compared with the Kurchatov Institute conversion model and the Estienne-Fallot summation model

Model explanation: This row supplied the concrete result that reactor rates and fuel-evolution data agree with the KI and EF flux models, which is the content of the generated sentence.

Card 1 · fact

Affected output: both of which are favored over the Huber-Mueller prediction by goodness-of-fit tests

Model explanation: This row supplied the goodness-of-fit preference for the KI and EF models, which the generated sentence reports as favoring them over the Huber-Mueller prediction.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

The Gallium Anomaly thus stands apart from the reactor and accelerator anomalies: whereas the latter can be accommodated within a 3+1 framework, the gallium data are instead in strong tension with the bounds derived from other data sets, and no single sterile neutrino parameter point simultaneously describes all short-baseline observations.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the no-Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs provide coherent, grounded continuations, but B better serves the manuscript's argumentative arc. The draft explicitly states the Gallium Anomaly is in 'strong tension' with other data sets, with parameter goodness of fit below 0.042% and ~5σ tension, concluding that 'alternative solutions' to short-baseline oscillations are needed. Output B directly extends this conclusion by contrasting the Gallium Anomaly with reactor/accelerator anomalies and noting that no single sterile neutrino parameter point describes all observations—this is supported by the sources (e.g., the '3+1 and 3+2' source notes tension in 3+1 interpretation, the BEST source notes exclusion of the gallium best-fit point by reactor searches). Output A, while factually grounded in the first source about reactor flux models, shifts focus to the reactor antineutrino anomaly's resolution. This is somewhat tangential: the draft's preceding paragraph already discussed the 3+1 hypothesis combined with reactor anomaly data and tritium experiments, and the immediate context emphasizes the Gallium Anomaly's unique tension. Output A's contrastive 'by contrast' also creates a slight logical friction since the draft already noted the Gallium Anomaly's tension with 'other data sets'—Output A reads more like a new topic introduction than a continuation. Output B maintains thematic coherence by elaborating on the anomaly's isolated status, matching the draft's concluding thrust about seeking 'alternative solutions.'

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.