← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better44 / 280 · beu-212c6745c7a2d22c

Long-term modulation of solar cycles

Astrophysics · 2302.14845v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Long-term modulation of solar cycles

6 Implications for the dynamo theory > 6.1 Introduction to the solar dynamo

The solar magnetic cycle is maintained by a dynamo process, operating in the solar convection zone (SCZ). Thus, it is natural to expect that the variations in the solar cycle are caused by some mechanisms in the solar dynamo. Here we identify the causes of the variations in the solar cycle and demonstrate them by presenting results from some illustrative models. Let us first summarise the mechanism of the solar dynamo.

…so-called Babcock–Leighton process is clearly identified in the observed magnetic field data on the solar surface ( Mordvinov et al, 2022 , e.g.,) . The observed correlation between the polar field (or its proxy) at the solar minima and the amplitude of the next cycle ( Wang and Sheeley, 2009 ; Kitchatinov and Olemskoy, 2011 ; Muñoz-Jaramillo et al, 2013 ; Priyal et al, 2014 ) and the flux budgets of the observed and the generated poloidal and toroidal fields ( Cameron and Schüssler, 2015 ) suggest that the Babcock–Leighton process is possibly the main source of the poloidal field in the Sun.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs provide accurate, coherent continuations that explain the Babcock–Leighton mechanism. However, Output B is more useful at this position because: (1) It includes the critical role of meridional circulation in transporting poloidal flux to the bottom of the convection zone, which is explicitly mentioned in the sources and is essential for completing the dynamo loop. The draft text preceding the cursor emphasizes identifying 'causes of variations in the solar cycle' and presenting 'illustrative models'—the transport mechanism is a key component that makes the dynamo a closed system. (2) Output A stops at describing poloidal and toroidal field generation but omits the transport step, leaving the dynamo description incomplete. The sources highlight that 'a mechanism transporting the new polar surface magnetic flux generated by the Babcock-Leighton mechanism to the bottom of the convection zone is necessary' and that 'initially a meridional circulation flow was assumed.' Output B's inclusion of this element better fits the draft's goal of summarizing the mechanism comprehensively before proceeding to discuss variations. Both outputs are factually accurate and could be inserted without repair, but B provides more complete coverage of the mechanism at this introductory position.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

In the Babcock–Leighton framework, the poloidal field is generated near the solar surface through the decay and dispersal of tilted bipolar active regions, while the toroidal field is generated by differential rotation acting on that poloidal field. A meridional circulation is then required to transport the surface poloidal flux to the bottom of the convection zone, where it is sheared to regenerate the toroidal field.

Passages supplied to the Evidence version

Card 1 · The hemispheric asymmetry of solar activity during the last century and the solar dynamo

We believe the Babcock--Leighton process of poloidal field generation to be the main source of irregularity in the solar cycle. The random nature of this process may make the poloidal field in one hemisphere stronger than that in the other hemisphere at the end of a cycle. We expect this to induce a…

Read full passage excerpt

We believe the Babcock--Leighton process of poloidal field generation to be the main source of irregularity in the solar cycle. The random nature of this process may make the poloidal field in one hemisphere stronger than that in the other hemisphere at the end of a cycle. We expect this to induce an asymmetry in the next sunspot cycle. We look for evidence of this in the observational data and then model it theoretically with our dynamo code. Since actual polar field measurements exist only from 1970s, we use the polar faculae number data recorded by Sheeley (1991) as a proxy of the polar field and estimate the hemispheric asymmetry of the polar field in different solar minima during the major part of the twentieth century. This asymmetry is found to have a reasonable correlation with the asymmetry of the next cycle. We then run our dynamo code by feeding information about this asymmetry at the successive minima and compare with observational data. We find that the theoretically compu …

Card 2 · Double Peaks of the Solar Cycle: An Explanation from a Dynamo Model

Abstract One peculiar feature of the solar cycle that is yet to be understood properly is the frequent occurrence of double peaks (also known as the Gnevyshev peaks). The double peaks, and also multiple peaks and spikes, are often observed in any phase of the cycle. We propose that these peaks and s…

Read full passage excerpt

Abstract One peculiar feature of the solar cycle that is yet to be understood properly is the frequent occurrence of double peaks (also known as the Gnevyshev peaks). The double peaks, and also multiple peaks and spikes, are often observed in any phase of the cycle. We propose that these peaks and spikes are generated due to fluctuations in the Babcock–Leighton process (the poloidal field generation from tilted bipolar magnetic regions). When the polar field develops, large negative fluctuations in the Babcock–Leighton process can reduce the net polar field abruptly. As these fluctuations in the polar field are propagated to the new toroidal field, these can promote double peaks in the next solar cycle. When fluctuations in the polar field occur outside the solar maximum, we observe their effects as spikes or dips in the following sunspot cycle. Using an axisymmetric Babcock–Leighton dynamo model, we first demonstrate this idea. Later, we perform a long simulation by including random s …

Card 3 · BABCOCK–LEIGHTON SOLAR DYNAMO: THE ROLE OF DOWNWARD PUMPING AND THE EQUATORWARD PROPAGATION OF ACTIVITY

ABSTRACT The key elements of the Babcock–Leighton dynamos are the generation of poloidal field through decay and the dispersal of tilted bipolar active regions and the generation of toroidal field through the observed differential rotation. These models are traditionally known as flux transport dyna…

Read full passage excerpt

ABSTRACT The key elements of the Babcock–Leighton dynamos are the generation of poloidal field through decay and the dispersal of tilted bipolar active regions and the generation of toroidal field through the observed differential rotation. These models are traditionally known as flux transport dynamo models as the equatorward propagations of the butterfly wings in these models are produced due to an equatorward flow at the bottom of the convection zone. Here we investigate the role of downward magnetic pumping near the surface using a kinematic Babcock–Leighton model. We find that the pumping causes the poloidal field to become predominately radial in the near-surface shear layer, which allows the negative radial shear to effectively act on the radial field to produce a toroidal field. We observe a clear equatorward migration of the toroidal field at low latitudes as a consequence of the dynamo wave even when there is no meridional flow in the deep convection zone. Both the dynamo wav …

Card 4 · Recovery from Maunder-like Grand Minima in a Babcock–Leighton Solar Dynamo Model

Abstract The Sun occasionally goes through Maunder-like extended grand minima when its magnetic activity drops considerably from the normal activity level for several decades. Many possible theories have been proposed to explain the origin of these minima. However, how the Sun managed to recover fro…

Read full passage excerpt

Abstract The Sun occasionally goes through Maunder-like extended grand minima when its magnetic activity drops considerably from the normal activity level for several decades. Many possible theories have been proposed to explain the origin of these minima. However, how the Sun managed to recover from such inactive phases every time is even more enigmatic. The Babcock–Leighton type dynamos, which are successful in explaining many features of the solar cycle remarkably well, are not expected to operate during grand minima due to the lack of a sufficient number of sunspots. In this Letter, we explore the question of how the Sun could recover from grand minima through the Babcock–Leighton dynamo. In our three-dimensional dynamo model, grand minima are produced spontaneously as a result of random variations in the tilt angle of emerging active regions. We find that the Babcock–Leighton process can still operate during grand minima with only a minimal number of sunspots, and that the model c …

Card 5 · A mean-field Babcock-Leighton solar dynamo model with long-term variability

At the same time, the magnetic field of the following spot will also decay and connect with the polar magnetic field, which has opposite polarity. This process will at some point annihilate the flux in the polar region, causing a poloidal polarity reversal. In this case the poloidal and toroidal fie…

Read full passage excerpt

At the same time, the magnetic field of the following spot will also decay and connect with the polar magnetic field, which has opposite polarity. This process will at some point annihilate the flux in the polar region, causing a poloidal polarity reversal. In this case the poloidal and toroidal fields regeneration processes are spatially separated (recall Figure 1 h to Figure 1 j); a mechanism transporting the new polar surface magnetic flux generated by the Babcock-Leighton mechanism to the bottom of the convection zone is necessary. Facing this requirement, initially a meridional circulation flow was assumed in Babcock-Leighton dynamo modeling. As a matter of fact, a poleward meridional flow is indeed observed at the solar surface ( Duvall Jr,, 1979 ) .

Model-reported card use

Card 3 · fact

Affected output: the poloidal field is generated near the solar surface through the decay and dispersal of tilted bipolar active regions, while the toroidal field is generated by differential rotation acting on that poloidal field

Model explanation: Supplied the two-step generation mechanism (poloidal from tilted active regions, toroidal from differential rotation) that the opening sentence of the new paragraph states.

Card 5 · fact

Affected output: A meridional circulation is then required to transport the surface poloidal flux to the bottom of the convection zone

Model explanation: Supplied the necessity of transporting the surface poloidal flux downward to the base of the convection zone, which the sentence assigns to meridional circulation.

Card 1 · terminology

Affected output: the poloidal field is generated near the solar surface

Model explanation: Reinforced the field-native phrasing of poloidal field generation by the Babcock–Leighton process used in the opening sentence.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

In the Babcock–Leighton framework, the decay and dispersal of tilted bipolar active regions near the surface provide the poloidal field that seeds the next cycle, while differential rotation in the SCZ shears this poloidal field into the toroidal field that emerges as sunspots.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs provide coherent, grounded continuations of the Babcock–Leighton dynamo mechanism. Output A covers the basic poloidal-to-toroidal field generation process. Output B adds the critical meridional circulation transport step, which is explicitly supported by Source 5 ('a mechanism transporting the new polar surface magnetic flux generated by the Babcock-Leighton mechanism to the bottom of the convection zone is necessary...initially a meridional circulation flow was assumed') and is essential for completing the dynamo loop. The draft itself mentions 'the mechanism of the solar dynamo' and the sources provide this transport as a key element. Output B's additional sentence about meridional circulation makes it a more complete and useful paragraph that better prepares for the subsequent discussion of variations in the solar cycle. Both are factually correct and supported, but B provides more comprehensive mechanistic coverage with stronger source grounding for the full cycle description.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.