FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better12 / 280 · beu-5ded0d3c9dde7209

Generalized Modeling: A survey and guide

Quantitative biology · 2112.00360v2

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Generalized Modeling: A survey and guide

1 Introduction

In mathematics, working with unknown functions is the rule rather than the exception. For example, we do it every time we write the definition of a derivative. x ˙ = d d ​ t ​ x ​ ( t ) = lim δ → 0 x ⁡ ( t + δ ) − x ⁡ ( t ) δ \dot{x}=\frac{{\rm d}}{{\rm d}t}x(t)=\lim_{\delta\to 0}\frac{x(t+\delta)-x(t)}{\delta} (1) In models unspecified functions have been used to a lesser extent, but still the examples of such models are plentiful.

…hough the approach has been extended to many different contexts, GM has mostly been applied to explore systems of ordinary differential equations (ODEs). For this class of systems, GM can provide insights into the dynamics, identify conditions for the stability of steady states, explore the response of steady states to perturbations, identify parts of the system that are particularly vulnerable to perturbations, etc. Perhaps most importantly, GM generates these insights highly efficiently. It thus enables the exploration of large complex many-variable systems with comparatively little effort.

In the following we explain the basic idea of GM in Sec. 2. After this philosophical introduction, we provide a comprehensive review of previous works that used GM in

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

The draft introduces Sec. 2 as containing 'the basic idea of GM' and 'philosophical introduction.' Output B continues this structural logic by specifying that previous works will be reviewed in Sec. 3, with limitations discussed in Sec. 4—maintaining the paper's announced section sequence (2 → 3 → 4) and matching the academic convention of structure-previewing in introductions. Output A instead introduces 'ODE models' and 'mathematical foundations,' which is problematic: (1) it drifts toward specific mathematical content (ODEs) that isn't signaled in the draft's 'philosophical introduction' framing, and (2) 'mathematical foundations' competes with rather than follows from the 'basic idea' promised for Sec. 2, creating a vague, potentially redundant progression. Output B's scope is tighter, more predictable, and requires no repair to insert.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

the context of ODE models, before turning to the mathematical foundations that underpin the approach.

Passages supplied to the Evidence version

Card 1 · An Efficient Characterization of Complex-Balanced, Detailed-Balanced, and Weakly Reversible Systems

Very often, models in biology, chemistry, physics, and engineering are systems of polynomial or power-law ordinary differential equations, arising from a reaction network. Such dynamical systems can be generated by many different reaction networks. On the other hand, networks with special properties…

Read full passage excerpt

Very often, models in biology, chemistry, physics, and engineering are systems of polynomial or power-law ordinary differential equations, arising from a reaction network. Such dynamical systems can be generated by many different reaction networks. On the other hand, networks with special properties (such as reversibility or weak reversibility) are known or conjectured to give rise to dynamical systems that have special properties: existence of positive steady states, persistence, permanence, and (for well-chosen parameters) complex balancing or detailed balancing. These last two are related to thermodynamic equilibrium, and therefore the positive steady states are unique and stable. We describe a computationally efficient characterization of polynomial or power-law dynamical systems that can be obtained as complex-balanced, detailed-balanced, weakly reversible, and reversible mass-action systems.

Card 2 · A proof of unlimited multistability for phosphorylation cycles

Abstract The multiple futile cycle is a phosphorylation system in which a molecular substrate might be phosphorylated sequentially n times by means of an enzymatic mechanism. The system has been studied mathematically using reaction network theory and ordinary differential equations. It is known tha…

Read full passage excerpt

Abstract The multiple futile cycle is a phosphorylation system in which a molecular substrate might be phosphorylated sequentially n times by means of an enzymatic mechanism. The system has been studied mathematically using reaction network theory and ordinary differential equations. It is known that the system might have at least as many as 2 ⌊ n 2 ⌋ + 1 steady states (where ⌊ x ⌋ is the integer part of x ) for particular choices of parameters. Furthermore, for the simple and dual futile cycles ( n = 1, 2) the stability of the steady states has been determined in the sense that the only steady state of the simple futile cycle is globally stable, while there exist parameter values for which the dual futile cycle admits two asymptotically stable and one unstable steady state. For general n , evidence that the possible number of asymptotically stable steady states increases with n has been given, which has led to the conjecture that parameter values can be chosen such that ⌊ n 2 ⌋ + 1 ou …

Card 3 · A numerical framework for computing steady states of size-structured population models and their stability

Structured population models are a class of general evolution equations which are widely used in the study of biological systems. Many theoretical methods are available for establishing existence and stability of steady states of general evolution equations. However, except for very special cases, f…

Read full passage excerpt

Structured population models are a class of general evolution equations which are widely used in the study of biological systems. Many theoretical methods are available for establishing existence and stability of steady states of general evolution equations. However, except for very special cases, finding an analytical form of stationary solutions for evolution equations is a challenging task. In the present paper, we develop a numerical framework for computing approximations to stationary solutions of general evolution equations, which can also be used to produce existence and stability regions for steady states. In particular, we use the Trotter-Kato Theorem to approximate the infinitesimal generator of an evolution equation on a finite dimensional space, which in turn reduces the evolution equation into a system of ordinary differential equations. Consequently, we approximate and study the asymptotic behavior of stationary solutions. We illustrate the convergence of our numerical fr …

Card 4 · An Optimization Algorithm for Finding Parameters for Bistability

Motivation: Many biochemical pathways are known, but the numerous parameters required to correctly explore the dynamics of the pathways are not known. For this reason, algorithms that can make inferences by looking at the topology of a network are desirable. In this work, we are particular intereste…

Read full passage excerpt

Motivation: Many biochemical pathways are known, but the numerous parameters required to correctly explore the dynamics of the pathways are not known. For this reason, algorithms that can make inferences by looking at the topology of a network are desirable. In this work, we are particular interested in the question of whether a given pathway can potentially harbor multiple stable steady states. In other words, the challenge is to find the set of parameters such that the dynamical system defined by a set of ordinary differential equations will contain multiple stable steady states. Being able to find parameters that cause a network to be bistable may also be benefitial for engineering synthetic bistable systems where the engineer needs to know a working set of parameters. Result: We have developed an algorithm that optimizes the parameters of a dynamical system so that the system will contain at least one saddle or unstable point. The algorithm then looks at trajectories around this sa …

Card 5 · Local and global robustness at steady state

We study the robustness of the steady states of a class of systems of autonomous ordinary differential equations (ODEs), having as a central example those arising from (bio)chemical reaction networks. More precisely, we study under what conditions the steady states of the system are contained in a p…

Read full passage excerpt

We study the robustness of the steady states of a class of systems of autonomous ordinary differential equations (ODEs), having as a central example those arising from (bio)chemical reaction networks. More precisely, we study under what conditions the steady states of the system are contained in a parallel translate of a coordinate hyperplane. To this end, we focus mainly on ODEs consisting of generalized polynomials, and make use of algebraic and geometric tools to relate the local and global structure of the set of steady states. Specifically, we consider the local property termed zero sensitivity at a coordinate $x_i$, which means that the tangent space is contained in a hyperplane of the form $x_i=c$, and provide a criterion to identify it. We consider the global property termed absolute concentration robustness (ACR), meaning that all steady states are contained in a hyperplane of the form $x_i=c$. We clarify and formalise the relation between the two approaches. In particular, we …

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

Sec. 3, before turning to a discussion of the method's limitations and open challenges in Sec. 4.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.