FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better190 / 280 · beu-1128ca93aefe7ee7

Long-term modulation of solar cycles

Astrophysics · 2302.14845v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Long-term modulation of solar cycles

7 Summary

…on, Pipin (1999) found the Gleissberg cycle as a result of the re-establishment of differential rotation after the magnetic feedback on the angular momentum transport. Cameron and Schüssler (2017) modelled the overall power spectrum of solar activity using a generic normal form model for a noisy and weakly nonlinear limit cycle, and Cameron and Schüssler (2019) showed that the long-term modulations beyond the 11-year cycle are consistent with the realization noise, thus casting doubt whether secular and supersecular modulations are connected to the intrinsic periodicities of the solar dynamo.

…ear quasi-periodic Schwabe cycles, which is however variable per se in both magnitudes, duration and phase. While the direct telescopic observations of the Sun cover roughly four centuries since 1610 and cover a full range of solar-activity levels from the Maunder minimum in the 17th century to the Modern grand maximum in the late 20th century, the quality of the sunspot-number dataset is inhomogeneous and greatly degrades back in time, being quite imprecise before ≈ \approx 1820s. Moreover, it is too short to study the statistical properties of the solar-cycle modulation on a long timescale.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs are usable completions, but B is significantly better positioned for this specific cursor location. The draft ends by establishing two problems with the telescopic record: (1) inhomogeneous quality/degradation before ~1820s, and (2) too short for long-timescale statistical properties. Output B directly continues this argument by identifying the logical consequence: the need for cosmogenic radionuclide series extending over millennia. This flows naturally from the draft's setup and bridges to the likely next topic (proxy records). Output A, while not incorrect in content, performs a broad survey-style conclusion about dynamo mechanisms that doesn't follow from the immediate local context. The draft had just shifted from discussing dynamo models to discussing observational limitations; Output A jumps back to summarizing dynamo mechanisms, creating a disjunction. Output B maintains the argumentative thread about data limitations and their solution, making it the more coherent continuation.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

The telescopic record, though spanning the full range of solar variability from the Maunder minimum to the Modern grand maximum, is too short to characterise the statistical properties of long-term solar-cycle modulation, which motivates the use of cosmogenic radionuclide series extending over several millennia.

Passages supplied to the Evidence version

Card 1 · Extreme Value Theory Applied to the Millennial Sunspot Number Series

Abstract In this work, we use two decadal sunspot number series reconstructed from cosmogenic radionuclide data (14C in tree trunks, SN 14C, and 10Be in polar ice, SN 10Be) and the extreme value theory to study variability of solar activity during the last nine millennia. The peaks-over-threshold te…

Read full passage excerpt

Abstract In this work, we use two decadal sunspot number series reconstructed from cosmogenic radionuclide data (14C in tree trunks, SN 14C, and 10Be in polar ice, SN 10Be) and the extreme value theory to study variability of solar activity during the last nine millennia. The peaks-over-threshold technique was used to compute, in particular, the shape parameter of the generalized Pareto distribution for different thresholds. Its negative value implies an upper bound of the extreme SN 10Be and SN 14C timeseries. The return level for 1000 and 10,000 years were estimated leading to values lower than the maximum observed values, expected for the 1000 year, but not for the 10,000 year return levels, for both series. A comparison of these results with those obtained using the observed sunspot numbers from telescopic observations during the last four centuries suggests that the main characteristics of solar activity have already been recorded in the telescopic period (from 1610 to nowadays) w …

Card 2 · Long-term forcing of the Sun’s coronal field, open flux, and cosmic ray modulation potential during grand minima, maxima, and regular activity phases by the solar dynamo mechanism

ABSTRACT Magnetic fields generated in the Sun’s interior by the dynamo mechanism drive solar activity over a range of time-scales. Direct sunspot observations exist for a few centuries; reconstructed variations based on cosmogenic isotopes in the solar open flux and cosmic ray flux exist over thousa…

Read full passage excerpt

ABSTRACT Magnetic fields generated in the Sun’s interior by the dynamo mechanism drive solar activity over a range of time-scales. Direct sunspot observations exist for a few centuries; reconstructed variations based on cosmogenic isotopes in the solar open flux and cosmic ray flux exist over thousands of years. While such reconstructions indicate the presence of extreme solar activity fluctuations in the past, causal links between millennia scale dynamo activity, consequent coronal field, solar wind, open flux and cosmic ray flux variations remain elusive; a lack of coronal field observations compounds this issue. By utilizing a stochastically forced solar dynamo model and potential field source surface extrapolation, we perform long-term simulations to illuminate how dynamo generated magnetic fields govern the structure of the solar corona and the state of the heliosphere – as indicated by variations in the open flux and cosmic ray modulation potential. We establish differences in th …

Card 3 · Calibration of the Sunspot and Group Numbers Using the Waldmeier Effect

The Waldmeier Effect is the observation that the rise time of a sunspot cycle varies inversely with the cycle amplitude: strong cycles rise to their maximum faster than weak cycles. The shape of the cycle and thus the rise time does not depend on the scale factor of the sunspot number and can thus b…

Read full passage excerpt

The Waldmeier Effect is the observation that the rise time of a sunspot cycle varies inversely with the cycle amplitude: strong cycles rise to their maximum faster than weak cycles. The shape of the cycle and thus the rise time does not depend on the scale factor of the sunspot number and can thus be used to verify the constancy of the scale factor with time as already noted by Wolfer (1902) and Waldmeier (1978). We extend their analysis until the present using the new SILSO sunspot number (version 2) and group number and confirm that the scale factors have not varied significantly the past 250 years. The effect is also found in sunspot areas, in an EUV (and F10.7) proxy (the daily range of a geomagnetic variation), and in Cosmic Ray Modulation. The result is that solar activity reached similar high values in every one of the (17th?) 18th, 19th, and 20th centuries, supporting the finding that there has been no modern Grand Maximum.

Card 4 · Long-term evolution of the heliospheric magnetic field inferred from cosmogenic 44Ti activity in meteorites

Typical reconstructions of historic heliospheric magnetic field (HMF) BHMF are based on the analysis of the sunspot activity, geomagnetic data or on measurement of cosmogenic isotopes stored in terrestrial reservoirs like trees (14C) and ice cores (10Be). The various reconstructions of BHMF are howe…

Read full passage excerpt

Typical reconstructions of historic heliospheric magnetic field (HMF) BHMF are based on the analysis of the sunspot activity, geomagnetic data or on measurement of cosmogenic isotopes stored in terrestrial reservoirs like trees (14C) and ice cores (10Be). The various reconstructions of BHMF are however discordant both in strength and trend. Cosmogenic isotopes, which are produced by galactic cosmic rays impacting on meteoroids and whose production rate is modulated by the varying HMF convected outward by the solar wind, may offer an alternative tool for the investigation of the HMF in the past centuries. In this work, we aim to evaluate the long-term evolution of BHMF over a period covering the past twenty-two solar cycles by using measurements of the cosmogenic 44Ti activity (τ1∕2 = 59.2 ± 0.6 yr) measured in 20 meteorites which fell between 1766 and 2001. Within the given uncertainties, our result is compatible with a HMF increase from 4.87-0.30+0.24 nT in 1766 to 6.83-0.11+0.13 nT i …

Card 5 · Multiple dynamo modes as a mechanism for long-term solar activity variations

Context. Solar magnetic activity shows both smooth secular changes, such as the modern Grand Maximum, and quite abrupt drops that are denoted as grand minima, such as the Maunder Minimum. Direct numerical simulations (DNS) of convection-driven dynamos offer one way of examining the mechanisms behind…

Read full passage excerpt

Context. Solar magnetic activity shows both smooth secular changes, such as the modern Grand Maximum, and quite abrupt drops that are denoted as grand minima, such as the Maunder Minimum. Direct numerical simulations (DNS) of convection-driven dynamos offer one way of examining the mechanisms behind these events.

Model-reported card use

Card 1 · fact

Affected output: spanning the full range of solar variability from the Maunder minimum to the Modern grand maximum

Model explanation: This row supplied the specific fact that the telescopic period covers the full range of solar variability from a Grand minimum to a Grand maximum, which the generated sentence uses to characterise the telescopic record.

Card 2 · fact

Affected output: is too short to characterise the statistical properties of long-term solar-cycle modulation, which motivates the use of cosmogenic radionuclide series extending over several millennia

Model explanation: This row supplied the contrast between the few-century span of direct sunspot observations and the multi-millennial span of cosmogenic-isotope reconstructions, which the generated sentence uses to justify turning to radionuclide series.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

This review has surveyed the principal mechanisms proposed to account for long-term modulation of solar cycles, ranging from stochastic fluctuations in a noisy dynamo to deterministic nonlinear interactions and magnetic feedback on the differential rotation. While each of these frameworks captures some aspects of the observed variability, no single mechanism has yet been shown to account for the full range of secular and supersecular modulation evident in the proxy record.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Output A is a coherent continuation grounded in the draft and sources. It directly addresses the gap identified in the draft's final sentence—that the telescopic record is too short for long-term statistical studies—by introducing cosmogenic radionuclide series as the natural solution. This is supported by Source 1 (Extreme Value Theory Applied to the Millennial Sunspot Number Series), which discusses using 'decadal sunspot number series reconstructed from cosmogenic radionuclide data' and notes that 'the main characteristics of solar activity have already been recorded in the telescopic period,' consistent with the draft's framing. The continuation preserves the draft's meaning and scope without adding unsupported claims. Output B introduces multiple unsupported claims. It characterizes the preceding text as 'this review' surveying 'principal mechanisms proposed to account for long-term modulation'—but the draft is not a review, does not survey mechanisms systematically, and focuses specifically on cycle variability and data limitations rather than comparing frameworks. B claims 'no single mechanism has yet been shown to account for the full range of secular and supersecular modulation evident in the proxy record'—but the draft does not discuss proxy records in this evaluative way, and the sources do not support this synthetic conclusion about mechanism inadequacy. B also introduces 'deterministic nonlinear interactions' as a surveyed mechanism, which is not mentioned in the draft. The phrase 'proxy record' and the evaluative comparison of frameworks against it are unsupported extrapolations.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.