FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteAbout the same51 / 280 · beu-c8efee328aae8e6d

White Paper on Light Sterile Neutrino Searches and Related Phenomenology

Astrophysics · 2203.07323v3

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

White Paper on Light Sterile Neutrino Searches and Related Phenomenology

3 Interpretations of the Anomalies > 3.4 Summary of Interpretations

…ns the indication in favor of a short-baseline disappearance of electron neutrinos, possibly due to neutrino oscillations. Figure 40 shows the 90% contours in the | U e ​ 4 | 2 |U_{e4}|^{2} – Δ ​ m 41 2 \Delta{m}^{2}_{41} plane obtained from the analysis of the measured and expected Ge 71 {}^{71}\rm{Ge} event rates, considering the neutrino survival probability Eq. ( 7 ), where one can see that the squared-mass difference Δ ​ m 41 2 \Delta{m}^{2}_{41} allowed values are ∼ 1 \sim 1 eV 2 or larger, for the different approaches to compute the ν e − Ga 71 \nu_{e}\,-\,{}^{71}\rm{Ga} cross section.

…+1 neutrino oscillation hypothesis with the Reactor Antineutrino Anomaly and with the inclusion of data from tritium experiments and from experiments measuring solar neutrinos [Giunti:2022btk]. Remarkably, it was found that the Gallium Anomaly is in strong tension with bounds obtained from the other data sets. In addition, when all data are combined, the corresponding parameter goodness of fit is below 0.042%, implying a tension of ∼ 5 ​ σ \sim 5\sigma , leading to the conclusion that it should be necessary to seek for alternative solutions to the short-baseline oscillations for this anomaly.

A summary of the interpretations detailed in this section is provided in Tab. 10 . The columns, from left to

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: About the same

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

This rationale combines reviews with different display orders.

Order-swapped judges disagreed. Pass 1: Both outputs provide plausible, parallel completions for this table-description sentence. The draft states 'The columns, from left to' and both outputs continue with 'right, list...' followed by parallel structures of what the table contains. Output A mentions 'the anomaly, the experiments involved, the preferred interpretation, and the resulting tension with other data sets.' Output B mentions 'the relevant experiments, the interpretation of the anomaly, and the corresponding references.' Both are generic academic table descriptions that fit the neutrino physics context. Neither is materially better: A's 'tension with other data sets' is slightly more substantive given the sources discuss tensions between reactor data and gallium anomalies, but B's 'corresponding references' is a standard table element. Both are comparably usable as neutral, appropriate completions. Neither makes unsupported concrete claims. The 'complete' action is appropriate here as the sentence is describing a table structure, which is a self-contained descriptive move. Pass 2: Both outputs attempt to complete the sentence describing what Table 10's columns contain. Output A is preferable because 'relevant experiments, the interpretation of the anomaly, and the corresponding references' follows a more natural logical order for a table summary: first what was measured (experiments), then what was found (interpretation), then where to find more (references). This matches standard scientific table conventions. Output B's 'anomaly, the experiments involved, the preferred interpretation, and the resulting tension with other data sets' is also coherent but less ideal: starting with 'the anomaly' is slightly abrupt when the anomaly hasn't been named in the visible draft, and ending with 'tension with other data sets' introduces a comparative element that may not fit a summary table's scope. The draft asks for 'a summary of the interpretations'—Output A's focus on references also better serves a summary table's function. Both are usable, but A has tighter fit to the sentence structure and table purpose.

Evidence

Same model and position · retrieved passages supplied · first output

right, list the anomaly, the experiments involved, the preferred interpretation, and the resulting tension with other data sets.

Passages supplied to the Evidence version

Card 1 · Reactor antineutrino anomaly in light of recent flux model refinements

We study the status of the reactor antineutrino anomaly in light of recent reactor flux models obtained with the conversion and summation methods. We present a new improved calculation of the IBD yields of the standard Huber-Mueller (HM) model and those of the new models. We show that the reactor ra…

Read full passage excerpt

We study the status of the reactor antineutrino anomaly in light of recent reactor flux models obtained with the conversion and summation methods. We present a new improved calculation of the IBD yields of the standard Huber-Mueller (HM) model and those of the new models. We show that the reactor rates and the fuel evolution data are consistent with the predictions of the Kurchatov Institute (KI) conversion model and with those of the Estienne-Fallot (EF) summation model, leading to a plausible robust demise of the reactor antineutrino anomaly. We show that the results of several goodness of fit tests favor the KI and EF models over other models that we considered. We also discuss the implications of the new reactor flux models for short-baseline neutrino oscillations due to active-sterile mixing. We show that reactor data give upper bounds on active-sterile neutrino mixing that are not very different for the reactor flux models under consideration and are in tension with the large mix …

Card 2 · 3+1 and 3+2 Sterile Neutrino Fits

In spite of the presence of a tension in the interpretation of the data, 3+1 neutrino mixing is attractive for its simplicity and for the natural correspondence of one new entity (a sterile neutrino) with a new effect (short-baseline oscillations). The allowed regions in the oscillation parameter sp…

Read full passage excerpt

In spite of the presence of a tension in the interpretation of the data, 3+1 neutrino mixing is attractive for its simplicity and for the natural correspondence of one new entity (a sterile neutrino) with a new effect (short-baseline oscillations). The allowed regions in the oscillation parameter space can be tested in near-future experiments. In the framework of 3+2 neutrino mixing there is less tension in the interpretation of the data, at the price of introducing a second sterile neutrino. Moreover, the improvement of the parameter goodness of fit is mainly a statistical effect due to an increase of the number of parameters.

Card 3 · BEST impact on sterile neutrino hypothesis

Recently, the Baksan Experiment on Sterile Transitions (BEST) has presented results [1] confirming the gallium anomaly---a lack of electron neutrinos ${\ensuremath{\nu}}_{e}$ at calibrations of SAGE [2,3] and GALLEX [4]---at the statistical significance exceeding $5\ensuremath{\sigma}$. This result…

Read full passage excerpt

Recently, the Baksan Experiment on Sterile Transitions (BEST) has presented results [1] confirming the gallium anomaly---a lack of electron neutrinos ${\ensuremath{\nu}}_{e}$ at calibrations of SAGE [2,3] and GALLEX [4]---at the statistical significance exceeding $5\ensuremath{\sigma}$. This result is consistent with explanation of the gallium anomaly as electron neutrino oscillations into a sterile neutrino, ${\ensuremath{\nu}}_{s}$. Within this explanation, the BEST experiment itself provides the strongest evidence for the sterile neutrino among all the previous anomalous results in the neutrino sector. We combine the results of gallium experiments with searches for sterile neutrinos in reactor antineutrino experiments (assuming $CPT$ conservation in the $3+1$ neutrino sector). While the ``gallium'' best-fit point in the model parameter space (sterile neutrino mass squared ${m}_{{\ensuremath{\nu}}_{s}}^{2}\ensuremath{\approx}1.25\text{ }\text{ }{\mathrm{eV}}^{2}$, sterile-electron ne …

Card 4 · KATRIN bound on 3+1 active-sterile neutrino mixing and the reactor antineutrino anomaly

Abstract We present the bounds on 3+1 active-sterile neutrino mixing obtained from the first results of the KATRIN experiment. We show that the KATRIN data extend the Mainz and Troitsk bound to smaller values of $$ \Delta {m}_{41}^2 $$ Δ m 41 2 for large mixing and improves the exclusion of the larg…

Read full passage excerpt

Abstract We present the bounds on 3+1 active-sterile neutrino mixing obtained from the first results of the KATRIN experiment. We show that the KATRIN data extend the Mainz and Troitsk bound to smaller values of $$ \Delta {m}_{41}^2 $$ Δ m 41 2 for large mixing and improves the exclusion of the large- $$ \Delta {m}_{41}^2 $$ Δ m 41 2 solution of the Huber-Muller reactor antineutrino anomaly. We also show that the combined bound of the Mainz, Troitsk, and KATRIN tritium experiments and the Bugey-3, NEOS, PROSPECT, and DANSS reactor spectral ratio measurements exclude most of the region in the (sin22ϑ ee , $$ \Delta {m}_{41}^2 $$ Δ m 41 2 ) plane allowed by the Huber-Muller reactor antineutrino anomaly. Considering two new calculations of the reactor antineutrino fluxes, we show that one, that predicts a lower 235U antineutrino flux, is in agreement with the tritium and reactor spectral ratio measurements, whereas the other leads to a larger tension than the Huber-Muller prediction. We a …

Card 5 · The PROSPECT physics program

Abstract The precision reactor oscillation and spectrum experiment, PROSPECT, is designed to make a precise measurement of the antineutrino spectrum from a highly-enriched uranium reactor and probe eV-scale sterile neutrinos by searching for neutrino oscillations over a distance of several meters. P…

Read full passage excerpt

Abstract The precision reactor oscillation and spectrum experiment, PROSPECT, is designed to make a precise measurement of the antineutrino spectrum from a highly-enriched uranium reactor and probe eV-scale sterile neutrinos by searching for neutrino oscillations over a distance of several meters. PROSPECT is conceived as a 2-phase experiment utilizing segmented 6 Li-doped liquid scintillator detectors for both efficient detection of reactor antineutrinos through the inverse beta decay reaction and excellent background discrimination. PROSPECT Phase I consists of a movable 3 ton antineutrino detector at distances of 7–12 m from the reactor core. It will probe the best-fit point of the ν e disappearance experiments at 4 σ in 1 year and the favored region of the sterile neutrino parameter space at > 3 σ in 3 years. With a second antineutrino detector at 15–19 m from the reactor, Phase II of PROSPECT can probe the entire allowed parameter space below 10 eV 2 at 5 σ in 3 additional years. …

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

right, list the relevant experiments, the interpretation of the anomaly, and the corresponding references.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.