FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better87 / 280 · beu-204b10c756ab95a6

Reduction Methods in Climate Dynamics -- A Brief Review

Climate and environmental science · 2209.12240v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Reduction Methods in Climate Dynamics -- A Brief Review

1. Introduction > 1.1. Reduction of deterministic climate models

…in climate modelling, and to highlight others that may find use in future. To keep the review sufficiently brief, we focus on reduction methods for deterministic differential equations and only comment on reduction methods for stochastic differential equations in the outlook. Methods for model reduction have become so large and diverse in recent years that getting an overview of the main approaches for PDEs is already a major challenge when entering the field. Here, we hope to alleviate this issue by providing a relatively concise, yet sufficiently broad, entry point to the recent literature.

…ligned} \dot{x}&=f(x,y),\\ \dot{y}&=\varepsilon g(x,y),\end{aligned} (1-2) where z = ( x , y ) z=(x,y) is split into slow (climate) variables y y and the fast (weather) variables x x . The separation of timescales is indicated by a small parameter ε > 0 \varepsilon>0 . Some of the methods, such as slow manifold reductions, averaging or homogenization, try to make use of the special structure such as ( 1-2 ) with the aim of reducing to the climate variable y y only. Others, such as certain Galerkin approximations or asymptotic expansions, can be used to formally establish ( 1-2 ) from ( 1-1 ).

In Section 2 , we consider an abstract forced evolution equation and start with direct linear projection methods, which are among the most classical - yet quite coarse - reduction tools. There exist many methods

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

The draft is introducing reduction methods for evolution equations and states 'There exist many methods' — this clearly needs continuation to enumerate or characterize those methods. Output A directly follows through by naming concrete methods that 'refine this projection idea' (Galerkin approximations, balanced truncation, proper orthogonal decomposition), which naturally extends the 'direct linear projection methods' just mentioned and maintains the paper's apparent focus on model reduction techniques. Output B is more generic and vague ('their relative merits depend strongly on the structure of the underlying equations'); while grammatically correct, it adds little substantive content and reads as filler. Output A provides specific, useful information that advances the exposition and fits the academic register. Both are usable, but A is more informative and better scoped for this position.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

for reducing high-dimensional systems, and their relative merits depend strongly on the structure of the underlying equations.

Passages supplied to the Evidence version

Card 1 · Averaging for SDE-BSDE with null recurrent fast component Application to homogenization in a non periodic media

We establish an averaging principle for a family of solutions$(X^{\varepsilon}, Y^{\varepsilon})$ $ :=$ $(X^{1,\,\varepsilon},\,X^{2,\,\varepsilon},\, Y^{\varepsilon})$ of a system of SDE-BSDEwith a null recurrent fast component $X^{1,\,\varepsilon}$. Incontrast to the classical periodic case, we ca…

Read full passage excerpt

We establish an averaging principle for a family of solutions$(X^{\varepsilon}, Y^{\varepsilon})$ $ :=$ $(X^{1,\,\varepsilon},\,X^{2,\,\varepsilon},\, Y^{\varepsilon})$ of a system of SDE-BSDEwith a null recurrent fast component $X^{1,\,\varepsilon}$. Incontrast to the classical periodic case, we can not rely on aninvariant probability and the slow forward component$X^{2,\,\varepsilon}$ cannot be approximated by a diffusion process.On the other hand, we assume that the coefficients admit a limit in aCesaro sense. In such a case, the limit coefficients may havediscontinuity. We show that we can approximate the triplet$(X^{1,\,\varepsilon},\, X^{2,\,\varepsilon},\, Y^{\varepsilon})$ bya system of SDE-BSDE $(X^1, X^2, Y)$ where $X := (X^1, X^2)$ is aMarkov diffusion which is the unique (in law) weak solution of theaveraged forward component and $Y$ is the unique solution to the averaged backward component. This is done with a backward component whosegenerator depends on the variable $z$. …

Card 2 · Homogenization of the planar waveguide with frequently alternating boundary conditions

We consider the Laplacian in a planar strip with a Dirichlet boundary condition on the upper boundary and with a frequent alternation boundary condition on the lower boundary. The alternation is introduced by the periodic partition of the boundary into small segments on which Dirichlet and Neumann c…

Read full passage excerpt

We consider the Laplacian in a planar strip with a Dirichlet boundary condition on the upper boundary and with a frequent alternation boundary condition on the lower boundary. The alternation is introduced by the periodic partition of the boundary into small segments on which Dirichlet and Neumann conditions are imposed in turns. We show that under certain conditions the homogenized operator is the Dirichlet Laplacian and prove the uniform resolvent convergence. The spectrum of the perturbed operator consists of its essential part only and has a band structure. We construct the leading terms of the asymptotic expansions for the first band functions. We also construct the complete asymptotic expansion for the bottom of the spectrum.

Card 3 · Small-time asymptotics for fast mean-reverting stochastic volatility models

In this paper, we study stochastic volatility models in regimes where the maturity is small, but large compared to the mean-reversion time of the stochastic volatility factor. The problem falls in the class of averaging/homogenization problems for nonlinear HJB-type equations where the “fast variabl…

Read full passage excerpt

In this paper, we study stochastic volatility models in regimes where the maturity is small, but large compared to the mean-reversion time of the stochastic volatility factor. The problem falls in the class of averaging/homogenization problems for nonlinear HJB-type equations where the “fast variable” lives in a noncompact space. We develop a general argument based on viscosity solutions which we apply to the two regimes studied in the paper. We derive a large deviation principle, and we deduce asymptotic prices for out-of-the-money call and put options, and their corresponding implied volatilities. The results of this paper generalize the ones obtained in Feng, Forde and Fouque [SIAM J. Financial Math. 1 (2010) 126–141] by a moment generating function computation in the particular case of the Heston model.

Card 4 · Discrete-Time Statistical Inference for Multiscale Diffusions in the Averaging and Homogenization Regime

We study statistical inference for small-noise-perturbed multiscale dynamical systems under the assumption that we observe a single time series from the slow process only. We construct estimators for both averaging and homogenization regimes, based on an appropriate misspecified model motivated by a…

Read full passage excerpt

We study statistical inference for small-noise-perturbed multiscale dynamical systems under the assumption that we observe a single time series from the slow process only. We construct estimators for both averaging and homogenization regimes, based on an appropriate misspecified model motivated by a second-order stochastic Taylor expansion of the slow process with respect to a function of the time-scale separation parameter. In the case of a fixed number of observations, we establish consistency, asymptotic normality, and asymptotic statistical efficiency of a minimum contrast estimator (MCE), the limiting variance having been identified explicitly; we furthermore establish consistency and asymptotic normality of a simplified minimum constrast estimator (SMCE), which is however not in general efficient. These results are then extended to the case of high-frequency observations under a condition restricting the rate at which the number of observations may grow vis-a-vis the separation o …

Card 5 · Maximum Likelihood Drift Estimation for Multiscale Diffusions

The problem of parameter estimation for fast/slow systems of SDEs which admit a coarse-grained description in terms of an SDE for the slow variable was studied in this paper. It was shown that, when applied to the averaging problem, the maximum likelihood estimator (MLE) is asymptotically unbiased a…

Read full passage excerpt

The problem of parameter estimation for fast/slow systems of SDEs which admit a coarse-grained description in terms of an SDE for the slow variable was studied in this paper. It was shown that, when applied to the averaging problem, the maximum likelihood estimator (MLE) is asymptotically unbiased and we can use it to estimate accurately the parameters in the drift coefficient of the coarse-grained model using data from the slow variable in the fast/slow system. On the contrary, the MLE is asymptotically biased when applied to the homogenization problem and a systematic asymptotic error appears in the log-likelihood function, in the long time/infinite scale separation limit.

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

that refine this projection idea, including Galerkin approximations, balanced truncation, and proper orthogonal decomposition.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.