FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better53 / 280 · beu-6f0265caee722ce1

A brief review on cosmological analysis of galaxy surveys with multiple tracers

Astrophysics · 2009.03862v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

A brief review on cosmological analysis of galaxy surveys with multiple tracers

I Introduction

Mapping the Universe with massive galaxy surveys provides critical cosmological information, which is key to reveal the physics governing the evolution of the Universe. In particular, galaxy surveys play a crucial role for understanding the origin of the cosmic acceleration discovered in the late 1990s [ 72 , 70 ] , which may be due to the existence of Dark Energy (see Huterer & Shafer 50 for a recent review), an energy component with a negative pressure, or to the extension of General Relativity (GR), as reviewed in [ 54 ] .

…on acoustic oscillations (BAO) and the redshift space distortions (RSD), which are specific clustering patterns of biased tracers [ 68 , 66 , 36 , 26 , 69 , 15 , 19 , 9 , 64 , 76 , 6 ] . Being complementary to other probes including the supernovae (SNIa) [ 72 , 70 , 73 , 83 , 84 , 14 , 79 , 29 ] and the cosmic microwave background (CMB) [ 12 , 47 , 71 ] , the BAO, as a standard ruler, maps the expansion history of the Universe, thus is essential to constrain dark energy, while the RSD, caused by peculiar motions of galaxies, is a natural probe for the nature of gravity on cosmological scales.

Great progress on galaxy surveys has been made since last century. Started in the 1970’s, the Harvard-Smithsonian Center for Astrophysics

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs correctly identify the CfA redshift survey and its significance for large-scale structure mapping. However, Output B is more precise and accurate: (1) 'three-dimensional distribution of galaxies' is more specific than 'distances and peculiar velocities'—the CfA survey primarily measured redshifts (which give distances/radial positions) to map 3D structure, not peculiar velocities directly; peculiar velocities were a separate analysis. (2) 'first clear evidence for large-scale structure' is more historically accurate than 'first maps of large-scale structure'—the CfA survey famously revealed the 'stickman' figure and voids, providing compelling evidence for structure, but earlier surveys had already begun mapping the field. Output B's phrasing better captures the survey's actual contribution without overstating it. Both fit grammatically and contextually, but B's greater factual precision makes it more useful.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

(CfA) redshift survey measured the distances and peculiar velocities of galaxies, providing the first maps of large-scale structure.

Passages supplied to the Evidence version

Card 1 · Comparison of recent SnIa datasets

We rank the six latest Type Ia supernova (SnIa) datasets (Constitution (C), Union (U), ESSENCE (Davis) (E), Gold06 (G), SNLS 1yr (S) and SDSS-II (D)) in the context of the Chevalier-Polarski-Linder (CPL) parametrization w ( a ) = w 0 + w 1 (1− a ), according to their Figure of Merit (FoM), their con…

Read full passage excerpt

We rank the six latest Type Ia supernova (SnIa) datasets (Constitution (C), Union (U), ESSENCE (Davis) (E), Gold06 (G), SNLS 1yr (S) and SDSS-II (D)) in the context of the Chevalier-Polarski-Linder (CPL) parametrization w ( a ) = w 0 + w 1 (1− a ), according to their Figure of Merit (FoM), their consistency with the cosmological constant (ΛCDM), their consistency with standard rulers (Cosmic Microwave Background (CMB) and Baryon Acoustic Oscillations (BAO)) and their mutual consistency. We find a significant improvement of the FoM (defined as the inverse area of the 95.4% parameter contour) with the number of SnIa of these datasets ((C) highest FoM, (U), (G), (D), (E), (S) lowest FoM). Standard rulers (CMB+BAO) have a better FoM by about a factor of 3, compared to the highest FoM SnIa dataset (C). We also find that the ranking sequence based on consistency with ΛCDM is identical with the corresponding ranking based on consistency with standard rulers ((S) most consistent, (D), (C), (E) …

Card 2 · Constraints on modified gravity from the observed X-ray luminosity function of galaxy clusters

These results are consistent with and complementary to those based on measurements of the cosmic expansion history, as deduced from distances to type Ia supernovae (SNIa) ( Riess et al., 1998 ; Perlmutter et al., 1999 ; Davis et al., 2007 ; Kowalski et al., 2008 ) , the gas mass fraction in galaxy c…

Read full passage excerpt

These results are consistent with and complementary to those based on measurements of the cosmic expansion history, as deduced from distances to type Ia supernovae (SNIa) ( Riess et al., 1998 ; Perlmutter et al., 1999 ; Davis et al., 2007 ; Kowalski et al., 2008 ) , the gas mass fraction in galaxy clusters ( Allen et al., 2004 ; Rapetti et al., 2005 ; Rapetti et al., 2007 ; Allen et al., 2008 ) , and Baryon Acoustic Oscillations in the distribution of galaxies ( Cole et al., 2005 ; Eisenstein et al., 2005 ) . In combination with independent measurements of the cosmic microwave background (CMB) ( Spergel et al., 2003 ; Spergel et al., 2007 ; Komatsu et al., 2008 ) , such measurements argue for a universe that is spatially flat, with most matter being cold and dark, and with the energy density being currently dominated by a cosmological constant Λ \Lambda .

Card 3 · Testing dark energy using pairs of galaxies in redshift space

The distribution of angles subtended between pairs of galaxies and the line of sight, which is uniform in real space, is distorted by their peculiar motions, and has been proposed as a probe of cosmic expansion. We test this idea using N-body simulations of structure formation in a cold dark matter…

Read full passage excerpt

The distribution of angles subtended between pairs of galaxies and the line of sight, which is uniform in real space, is distorted by their peculiar motions, and has been proposed as a probe of cosmic expansion. We test this idea using N-body simulations of structure formation in a cold dark matter universe with a cosmological constant and in two variant cosmologies with different dark energy models. We find that the distortion of the distribution of angles is sensitive to the nature of dark energy. However, for the first time, our simulations also reveal dependences of the normalization of the distribution on both redshift and cosmology that have been neglected in previous work. This introduces systematics that severely limit the usefulness of the original method. Guided by our simulations, we devise a new, improved test of the nature of dark energy. We demonstrate that this test does not require prior knowledge of the background cosmology and that it can even distinguish between mode …

Card 4 · Determination of dark energy by the Einstein Telescope: Comparing with CMB, BAO, and SNIa observations

A design study is currently in progress for a third-generation gravitational-wave (GW) detector called the Einstein Telescope (ET). An important kind of source for ET will be the inspiral and merger of binary neutron stars up to $z\ensuremath{\sim}2$. If binary neutron star mergers are the progenito…

Read full passage excerpt

A design study is currently in progress for a third-generation gravitational-wave (GW) detector called the Einstein Telescope (ET). An important kind of source for ET will be the inspiral and merger of binary neutron stars up to $z\ensuremath{\sim}2$. If binary neutron star mergers are the progenitors of short-hard $\ensuremath{\gamma}$-ray bursts, then some fraction of them will be seen both electromagnetically and through GW, so that the luminosity distance and the redshift of the source can be determined separately. An important property of these ``standard sirens'' is that they are self-calibrating: the luminosity distance can be inferred directly from the GW signal, with no need for a cosmic distance ladder. Thus, standard sirens will provide a powerful independent check of the $\ensuremath{\Lambda}\mathrm{CDM}$ model. In previous work, estimates were made of how well ET would be able to measure a subset of the cosmological parameters (such as the dark energy parameter ${w}_{0}$) …

Card 5 · Constraints on modified gravity from the observed X-ray luminosity function of galaxy clusters

These results are consistent with and complementary to those based on measurements of the cosmic expansion history, as deduced from distances to type Ia supernovae (SNIa) ( Riess et al., 1998 ; Perlmutter et al., 1999 ; Davis et al., 2007 ; Kowalski et al., 2008 ) , the gas mass fraction in galaxy c…

Read full passage excerpt

These results are consistent with and complementary to those based on measurements of the cosmic expansion history, as deduced from distances to type Ia supernovae (SNIa) ( Riess et al., 1998 ; Perlmutter et al., 1999 ; Davis et al., 2007 ; Kowalski et al., 2008 ) , the gas mass fraction in galaxy clusters ( Allen et al., 2004 ; Rapetti et al., 2005 ; Rapetti et al., 2007 ; Allen et al., 2008 ) , and Baryon Acoustic Oscillations in the distribution of galaxies ( Cole et al., 2005 ; Eisenstein et al., 2005 ) . In combination with independent measurements of the cosmic microwave background (CMB) ( Spergel et al., 2003 ; Spergel et al., 2007 ; Komatsu et al., 2008 ) , such measurements argue for a universe that is spatially flat, with most matter being cold and dark, and with the energy density being currently dominated by a cosmological constant Λ \Lambda .

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

(CfA) redshift survey measured the three-dimensional distribution of galaxies and provided the first clear evidence for large-scale structure.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.