FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better46 / 280 · beu-67bdf3d876d61ed5

A brief review on cosmological analysis of galaxy surveys with multiple tracers

Astrophysics · 2009.03862v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

A brief review on cosmological analysis of galaxy surveys with multiple tracers

I Introduction

…t and redshift range. One possibility to create such ‘multi-tracer’ samples from a single-tracer survey is to split the sample into subsamples by luminosity or color [ 20 , 75 ] , but this may be subject to a limited relative galaxy bias, as samples in a single-tracer survey usually do not differ much in the galaxy bias. Alternatively, one can combine tracers observed by different surveys [ 59 , 16 ] . However, this approach could be limited by the small overlapping area, as most galaxy surveys are designed to be complementary to each other, in terms of the sky coverage and/or redshift range.

… with a large overlap in the cosmic volume. Targeting at both the luminous red galaxies (LRGs, denoted as ‘Old Red Galaxies’ in Fig. 1 at 0.6 < z < 1.0 0.6<z<1.0 ), and emission line galaxies (ELGs, ‘Young Blue Galaxies’ in Fig. 1 at 0.6 < z < 1.1 0.6<z<1.1 ), eBOSS offers observations of the clustering of these two tracers in a large overlapping area (over 700 700 deg 2 ) in the same redshift range [ 31 ] . A multi-tracer analysis on the final eBOSS sample, which is tagged as the Data Release (DR)16 data sample, was preformed in both the configuration space [ 92 ] and Fourier space [ 101 ] .

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs attempt to continue, but the draft already forms a complete argumentative move: it introduces a problem (limited overlap for multi-tracer analyses), presents eBOSS as a solution with specific tracers and overlapping area, and cites the actual multi-tracer analyses performed (configuration space [92] and Fourier space [101]). The paragraph is self-contained and complete. However, since both outputs provide text, I must evaluate their utility as continuations. Output A introduces DESI as a successor/extension, which is a natural forward-looking continuation but contains an unsupported concrete claim: '[DESI Collaboration et al. 2024]' — this citation format and specific reference is invented/not verified in the sources. The sources provided are all eBOSS DR16 papers from 2020 or earlier; there is no DESI 2024 source. This is a material unsupported citation that affects usability. Output B provides additional detail about the eBOSS ELG sample (173,736 galaxies, ~730 deg², NGC/SGC regions, growth rate of structure measurement). These numbers and claims are directly supported by the first source snippet about the eBOSS DR16 ELG sample. However, this information is largely redundant with what the draft already states (the draft already mentions 'over 700 deg²' and the redshift range), and the draft already cited [31] for the sample description and [92], [101] for the analyses. Output B essentially unpacks details that the draft has already summarized and cited, making it somewhat repetitive rather than advancing the argument. Between the two, Output B is more accurate and grounded in the provided sources with no unsupported claims, while Output A has an invented citation. However, Output B is more redundant with existing draft content. Given that the draft is actually complete, and considering that Output A's unsupported citation is a concrete material flaw while Output B merely repeats verified information, Output B is the more usable choice — though neither is ideal since the paragraph needs no continuation.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

The eBOSS ELG sample alone comprises 173,736 galaxies with reliable redshifts over 0.6 < z < 1.1, spanning an effective area of roughly 730 deg^2 across the NGC and SGC regions, and its anisotropic clustering has been analysed in configuration space to constrain the growth rate of structure.

Passages supplied to the Evidence version

Card 1 · The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Growth rate of structure measurement from anisotropic clustering analysis in configuration space between redshift 0.6 and 1.1 for the Emission Line Galaxy sample

We performed a pure RSD analysis and a joint RSD+BAO analysis in configuration space for the eBOSS DR16 Emission Line Galaxies sample described in Raichoor et al. (2020) . This sample is composed of 173,736 galaxies with a reliable redshift in the range 0.6 < z < 1.1 0.6<z<1.1 , covering an effectiv…

Read full passage excerpt

We performed a pure RSD analysis and a joint RSD+BAO analysis in configuration space for the eBOSS DR16 Emission Line Galaxies sample described in Raichoor et al. (2020) . This sample is composed of 173,736 galaxies with a reliable redshift in the range 0.6 < z < 1.1 0.6<z<1.1 , covering an effective area of ∼ \sim 730 deg 2 over the two NGC and SGC regions. The post-reconstruction BAO measurement in configuration space of this sample is analysed in Raichoor et al. (2020) . The BAO and RSD measurements in Fourier space and a consensus of our results for the eBOSS ELG sample are presented in de Mattia et al. (2020) .

Card 2 · The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: 1000 multi-tracer mock catalogues with redshift evolution and systematics for galaxies and quasars of the final data release

ABSTRACT We produce 1000 realizations of synthetic clustering catalogues for each type of the tracers used for the baryon acoustic oscillation and redshift space distortion analysis of the Sloan Digital Sky Surveys-iv extended Baryon Oscillation Spectroscopic Survey final data release (eBOSS DR16),…

Read full passage excerpt

ABSTRACT We produce 1000 realizations of synthetic clustering catalogues for each type of the tracers used for the baryon acoustic oscillation and redshift space distortion analysis of the Sloan Digital Sky Surveys-iv extended Baryon Oscillation Spectroscopic Survey final data release (eBOSS DR16), covering the redshift range from 0.6 to 2.2, to provide reliable estimates of covariance matrices and test the robustness of the analysis pipeline with respect to observational systematics. By extending the Zel’dovich approximation density field with an effective tracer bias model calibrated with the clustering measurements from the observational data, we accurately reproduce the two- and three-point clustering statistics of the eBOSS DR16 tracers, including their cross-correlations in redshift space with very low computational costs. In addition, we include the gravitational evolution of structures and sample selection biases at different redshifts, as well as various photometric and spectr …

Card 3 · The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: BAO and RSD measurements from anisotropic clustering analysis of the quasar sample in configuration space between redshift 0.8 and 2.2

ABSTRACT We measure the anisotropic clustering of the quasar sample from Data Release 16 (DR16) of the Sloan Digital Sky Survey IV extended Baryon Oscillation Spectroscopic Survey (eBOSS). A sample of 343 708 spectroscopically confirmed quasars between redshift 0.8 < z < 2.2 are used as tracers of t…

Read full passage excerpt

ABSTRACT We measure the anisotropic clustering of the quasar sample from Data Release 16 (DR16) of the Sloan Digital Sky Survey IV extended Baryon Oscillation Spectroscopic Survey (eBOSS). A sample of 343 708 spectroscopically confirmed quasars between redshift 0.8 < z < 2.2 are used as tracers of the underlying dark matter field. In comparison with DR14 sample, the final sample doubles the number of objects as well as the survey area. In this paper, we present the analysis in configuration space by measuring the two-point correlation function and decomposing it using the Legendre polynomials. For the full-shape analysis of the Legendre multipole moments, we measure the baryon acoustic oscillation (BAO) distance and the growth rate of the cosmic structure. At an effective redshift of zeff = 1.48, we measure the comoving angular diameter distance DM(zeff)/rdrag = 30.66 ± 0.88, the Hubble distance DH(zeff)/rdrag = 13.11 ± 0.52, and the product of the linear growth rate and the rms linear …

Card 4 · The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: N-body mock challenge for the eBOSS emission line galaxy sample

ABSTRACT Cosmological growth can be measured in the redshift space clustering of galaxies targeted by spectroscopic surveys. Accurate prediction of clustering of galaxies will require understanding galaxy physics, which is a very hard and highly non-linear problem. Approximate models of redshift spa…

Read full passage excerpt

ABSTRACT Cosmological growth can be measured in the redshift space clustering of galaxies targeted by spectroscopic surveys. Accurate prediction of clustering of galaxies will require understanding galaxy physics, which is a very hard and highly non-linear problem. Approximate models of redshift space distortion (RSD) take a perturbative approach to solve the evolution of dark matter and galaxies in the universe. In this paper, we focus on extended Baryon Oscillation Spectroscopic (eBOSS) emission line galaxies (ELGs) that live in intermediate mass haloes. We create a series of mock catalogues using haloes from the Multidark and outer rim dark matter only N-body simulations. Our mock catalogues include various effects inspired by baryonic physics such as assembly bias and the characteristics of satellite galaxies kinematics, dynamics, and statistics deviating from dark matter particles. We analyse these mocks using the TNS RSD model in Fourier space and the convolution Lagrangian pertu …

Card 5 · The Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Growth rate of structure measurement from anisotropic clustering analysis in configuration space between redshift 0.6 and 1.1 for the Emission Line Galaxy sample

We performed a pure RSD analysis and a joint RSD+BAO analysis in configuration space for the eBOSS DR16 Emission Line Galaxies sample described in Raichoor et al. (2020) . This sample is composed of 173,736 galaxies with a reliable redshift in the range 0.6 < z < 1.1 0.6<z<1.1 , covering an effectiv…

Read full passage excerpt

We performed a pure RSD analysis and a joint RSD+BAO analysis in configuration space for the eBOSS DR16 Emission Line Galaxies sample described in Raichoor et al. (2020) . This sample is composed of 173,736 galaxies with a reliable redshift in the range 0.6 < z < 1.1 0.6<z<1.1 , covering an effective area of ∼ \sim 730 deg 2 over the two NGC and SGC regions. The post-reconstruction BAO measurement in configuration space of this sample is analysed in Raichoor et al. (2020) . The BAO and RSD measurements in Fourier space and a consensus of our results for the eBOSS ELG sample are presented in de Mattia et al. (2020) .

Model-reported card use

Card 1 · fact

Affected output: comprises 173,736 galaxies with reliable redshifts over 0.6 < z < 1.1, spanning an effective area of roughly 730 deg^2 across the NGC and SGC regions

Model explanation: The row supplied the concrete sample size, redshift interval, and survey area that the sentence uses to characterize the eBOSS ELG sample.

Card 1 · fact

Affected output: its anisotropic clustering has been analysed in configuration space to constrain the growth rate of structure

Model explanation: The row established that the ELG sample's clustering was analysed in configuration space, which the sentence reports as the analysis performed.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

These multi-tracer measurements have since been extended to the larger samples of the Dark Energy Spectroscopic Instrument (DESI), which observes both luminous red galaxies and emission line galaxies over a wide redshift range [DESI Collaboration et al. 2024].

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Output B is grounded in the supplied sources: the eBOSS ELG sample size (173,736 galaxies), redshift range (0.6 < z < 1.1), effective area (~730 deg^2), and configuration-space analysis for RSD/BAO are all directly supported by the first and fifth sources. Output A introduces DESI as a follow-up with larger samples, but no source mentions DESI at all. The claim that multi-tracer measurements 'have since been extended' to DESI is an unsupported factual claim about chronology and another survey's capabilities. While Output A's DESI claim might be plausible field knowledge, this is a closed-book audit where plausibility does not count as support. Output B provides a coherent continuation that elaborates on the eBOSS ELG sample details already established in the draft, fitting the paragraph's focus on eBOSS multi-tracer analysis.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.