FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better48 / 280 · beu-dcd05a5c9ce07fb5

The halo model for cosmology: a pedagogical review

Astrophysics · 2303.08752v2

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

The halo model for cosmology: a pedagogical review

1 Introduction

…pecific tracer (e.g., galaxies), such that the desired clustering properties can be directly measured. While the analytical approach is quicker and more flexible, the simulation-based approach is potentially more accurate, but it is slower and requires N N -body simulations. This can become a problem in cosmological analyses where a wide range of parameters and/or cosmological models need to be explored. Once the analytical approach is adjusted to reach a desired accuracy, it can be extended more readily to other cosmological parameters and/or models compared to the simulation based approach.

…d et al., 2015a ; Mead et al., 2021 ) which have their roots in the halo-model approach (see section 3.7 ) form the basis of all primary analysis of recent cosmic shear data: Canada France Hawaii Telescope lensing Survey ( Heymans et al., 2013 ; Joudaki et al., 2017 , CFHTLenS,) , Deep Lens Survey ( Jee et al., 2013 , DLS,) , Kilo Degree Survey ( Hildebrandt et al., 2017 ; Hildebrandt et al., 2020 ; Asgari et al., 2021 , KiDS,) , Dark Energy Survey ( Troxel et al., 2018 ; Amon et al., 2022 ; Secco et al., 2022 , DES,) and Hyper Suprime-Cam ( Hikage et al., 2019 ; Hamana et al., 2020 , HSC,) .

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

The draft ends with a list of cosmic shear surveys (CFHTLenS, DLS, KiDS, DES, HSC) and the preceding text establishes that analytical approaches based on the halo model (Mead et al. 2015a, 2021) form the basis of primary analysis for these surveys. Output B provides a natural continuation that synthesizes this information: it connects the halo-model-based predictions to their role as standard tools for these specific surveys, completing the thought that was left hanging. The paragraph flows logically from 'these approaches form the basis of analysis for [list of surveys]' to 'have thus become the standard tool for extracting cosmological constraints from these surveys.' Output A's abstention is not appropriate here because the draft clearly needs a next move—the list of surveys ends abruptly without a concluding statement about their significance or current status. The paragraph is incomplete and requires continuation to tie together the halo model approach with the surveys just enumerated.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

The halo-model-based predictions of the nonlinear matter power spectrum have thus become the standard tool for extracting cosmological constraints from these surveys, and they continue to underpin the interpretation of cosmic shear measurements from current and forthcoming weak-lensing data sets.

Passages supplied to the Evidence version

Card 1 · Dark Energy Survey Year 3 results: Cosmology from cosmic shear and robustness to modeling uncertainty

This work and its companion paper, Amon et al. [Phys. Rev. D 105, 023514 (2022)], present cosmic shear measurements and cosmological constraints from over 100 million source galaxies in the Dark Energy Survey (DES) Year 3 data. We constrain the lensing amplitude parameter ${S}_{8}\ensuremath{\equiv}…

Read full passage excerpt

This work and its companion paper, Amon et al. [Phys. Rev. D 105, 023514 (2022)], present cosmic shear measurements and cosmological constraints from over 100 million source galaxies in the Dark Energy Survey (DES) Year 3 data. We constrain the lensing amplitude parameter ${S}_{8}\ensuremath{\equiv}{\ensuremath{\sigma}}_{8}\sqrt{{\mathrm{\ensuremath{\Omega}}}_{\mathrm{m}}/0.3}$ at the 3% level in $\mathrm{\ensuremath{\Lambda}}\mathrm{CDM}$: ${S}_{8}=0.75{9}_{\ensuremath{-}0.023}^{+0.025}$ (68% CL). Our constraint is at the 2% level when using angular scale cuts that are optimized for the $\mathrm{\ensuremath{\Lambda}}\mathrm{CDM}$ analysis: ${S}_{8}=0.77{2}_{\ensuremath{-}0.017}^{+0.018}$ (68% CL). With cosmic shear alone, we find no statistically significant constraint on the dark energy equation-of-state parameter at our present statistical power. We carry out our analysis blind, and compare our measurement with constraints from two other contemporary weak lensing experiments: the Ki …

Card 2 · Consistency of cosmic shear analyses in harmonic and real space

Motivated by discrepancies found in the literature on the parameter S 8 S_{8} by the HSC Y1 ( Hamana et al., 2020 ; Hikage et al., 2019 ) and KiDS-450 ( Hildebrandt et al., 2017 ; Köhlinger et al., 2017 ) collaborations between their analyses of two-point statistics in harmonic and real space, we ex…

Read full passage excerpt

Motivated by discrepancies found in the literature on the parameter S 8 S_{8} by the HSC Y1 ( Hamana et al., 2020 ; Hikage et al., 2019 ) and KiDS-450 ( Hildebrandt et al., 2017 ; Köhlinger et al., 2017 ) collaborations between their analyses of two-point statistics in harmonic and real space, we explore the expected consistency of Year 3 cosmic shear data from the Dark Energy Survey and how similar discrepancies can arise. We suggest several scale cuts and a method to test them, which we apply to the forthcoming analysis of DES Y3. To do so, we generate 500 mocks of a DES Y3-like survey from Gaussian simulations, which we analyze using a fast importance sampling method with various scale cuts, in order to measure the discrepancies that can be expected from pure statistical fluctuations in parameter space versus that originating in systematic effects.

Card 3 · A data compression and optimal galaxy weights scheme for Dark Energy Spectroscopic Instrument and weak lensing data sets

ABSTRACT Combining different observational probes, such as galaxy clustering and weak lensing, is a promising technique for unveiling the physics of the Universe with upcoming dark energy experiments. The galaxy redshift sample from the Dark Energy Spectroscopic Instrument (DESI) will have a signifi…

Read full passage excerpt

ABSTRACT Combining different observational probes, such as galaxy clustering and weak lensing, is a promising technique for unveiling the physics of the Universe with upcoming dark energy experiments. The galaxy redshift sample from the Dark Energy Spectroscopic Instrument (DESI) will have a significant overlap with major ongoing imaging surveys specifically designed for weak lensing measurements: the Kilo-Degree Survey (KiDS), the Dark Energy Survey (DES), and the Hyper Suprime-Cam (HSC) survey. In this work, we analyse simulated redshift and lensing catalogues to establish a new strategy for combining high-quality cosmological imaging and spectroscopic data, in view of the first-year data assembly analysis of DESI. In a test case fitting for a reduced parameter set, we employ an optimal data compression scheme able to identify those aspects of the data that are most sensitive to cosmological information and amplify them with respect to other aspects of the data. We find this optimal …

Card 4 · KiDS+VIKING-450 and DES-Y1 combined: Cosmology with cosmic shear

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift…

Read full passage excerpt

We present a combined tomographic weak gravitational lensing analysis of the Kilo Degree Survey (KV450) and the Dark Energy Survey (DES-Y1). We homogenize the analysis of these two public cosmic shear datasets by adopting consistent priors and modeling of nonlinear scales, and determine new redshift distributions for DES-Y1 based on deep public spectroscopic surveys. Adopting these revised redshifts results in a 0.8σreduction in the DES-inferred value forS​8, which decreases to a 0.5σreduction when including a systematic redshift calibration error model from mock DES data based on the MICE2 simulation. The combined KV450+DES-Y1 constraint onS8= 0.762−0.024+0.025is in tension with thePlanck2018 constraint from the cosmic microwave background at the level of 2.5σ. This result highlights the importance of developing methods to provide accurate redshift calibration for current and future weak-lensing surveys.

Card 5 · Consistency of cosmic shear analyses in harmonic and real space

Motivated by discrepancies found in the literature on the parameter S 8 S_{8} by the HSC Y1 ( Hamana et al., 2020 ; Hikage et al., 2019 ) and KiDS-450 ( Hildebrandt et al., 2017 ; Köhlinger et al., 2017 ) collaborations between their analyses of two-point statistics in harmonic and real space, we ex…

Read full passage excerpt

Motivated by discrepancies found in the literature on the parameter S 8 S_{8} by the HSC Y1 ( Hamana et al., 2020 ; Hikage et al., 2019 ) and KiDS-450 ( Hildebrandt et al., 2017 ; Köhlinger et al., 2017 ) collaborations between their analyses of two-point statistics in harmonic and real space, we explore the expected consistency of Year 3 cosmic shear data from the Dark Energy Survey and how similar discrepancies can arise. We suggest several scale cuts and a method to test them, which we apply to the forthcoming analysis of DES Y3. To do so, we generate 500 mocks of a DES Y3-like survey from Gaussian simulations, which we analyze using a fast importance sampling method with various scale cuts, in order to measure the discrepancies that can be expected from pure statistical fluctuations in parameter space versus that originating in systematic effects.

Model-reported card use

Card 1 · fact

Affected output: The halo-model-based predictions of the nonlinear matter power spectrum have thus become the standard tool for extracting cosmological constraints from these surveys

Model explanation: The DES Y3 cosmic shear analysis provides a concrete example of a survey using halo-model-based predictions to extract cosmological constraints, grounding the claim that these predictions are the standard tool.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

No continuation text was produced.

Model action：complete

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The with-card arm produced text and reported card use, but the no-card arm produced no text. This counts as reported use, without entering the two-text preference comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.