FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better9 / 280 · beu-587d3034bbbe3477

Generalized Modeling: A survey and guide

Quantitative biology · 2112.00360v2

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Generalized Modeling: A survey and guide

8 Summary and Discussion

By contrast, generalized models, acknowledge that we have a limited view of reality and may hence be unable to perceive the exact laws that are at work in the system. Instead of postulating one definite reality, generalized models work with the whole infinite ensemble of possible realities that are consistent with the available structural knowledge. They explore dynamical implications within this ensemble, allowing the researcher to further narrow down the set of possible worlds.

As we have shown, some questions can be answered very efficiently by analyzing the whole ensemble of possible worlds captured by the generalized model. These questions include, the analysis of dynamical stability and bifurcations of steady states, prediction of responses to different types of perturbations, and identification of important parameters and parameter regions.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs are usable continuations, but they serve different argumentative functions. The draft has just listed questions that CAN be answered efficiently by generalized models (stability, bifurcations, perturbation responses, parameter identification). Output B provides a crucial balancing qualification about the LIMITS of what generalized models can deliver—namely that predictions are conditional and cannot single out 'the one true trajectory.' This creates an appropriate scholarly hedge and completes the argumentative arc: first establishing capabilities, then acknowledging boundaries. Output A, while factually accurate based on the sources (ecological food webs, laser systems, dynastic cycles are indeed examples from the generalized models paper), is redundant with material that should have appeared earlier in the manuscript if it were needed. The draft already explains the normalization procedure and Jacobian derivation in the first paragraph ('The approach is based on a normalization procedure...The Jacobian in a steady state is then derived'). Repeating these technical details at this position feels like misplaced exposition rather than forward movement. Output B's discussion of epistemic limits is more sophisticated and better fits the 'By contrast' framing that opened the section, which positioned generalized models against 'definite reality' approaches. The paragraph needs to land on what this contrast means practically, and B does that by explaining the conditional nature of predictions.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

The approach is particularly well suited to systems where the functional forms of the underlying processes are uncertain but the network structure is known, such as ecological food webs, coupled laser systems, and socioeconomic models of dynastic cycles. In each case, the normalization procedure identifies natural parameters of the system, and the Jacobian at a steady state is derived as a function of these parameters, so that local bifurcations can be computed analytically.

Passages supplied to the Evidence version

Card 1 · Bifurcations and Multistability in a Model of Cytokine-Mediated Autoimmunity

This paper investigates the dynamics of immune response and autoimmunity with particular emphasis on the role of regulatory T cells (Tregs), T cells with different activation thresholds, and cytokines in mediating T cell activity. Analysis of the steady states yields parameter regions corresponding…

Read full passage excerpt

This paper investigates the dynamics of immune response and autoimmunity with particular emphasis on the role of regulatory T cells (Tregs), T cells with different activation thresholds, and cytokines in mediating T cell activity. Analysis of the steady states yields parameter regions corresponding to regimes of normal clearance of viral infection, chronic infection, or autoimmune behavior, and the boundaries of stability and bifurcations of relevant steady states are found in terms of system parameters. Numerical simulations are performed to illustrate different dynamical scenarios, and to identify basins of attraction of different steady states and periodic solutions, highlighting the important role played by the initial conditions in determining the outcome of immune interactions.

Card 2 · Generalized models as a universal approach to the analysis of nonlinear dynamical systems

We present a universal approach to the investigation of the dynamics in generalized models. In these models the processes that are taken into account are not restricted to specific functional forms. Therefore a single generalized models can describe a class of systems which share a similar structure…

Read full passage excerpt

We present a universal approach to the investigation of the dynamics in generalized models. In these models the processes that are taken into account are not restricted to specific functional forms. Therefore a single generalized models can describe a class of systems which share a similar structure. Despite this generality, the proposed approach allows us to study the dynamical properties of generalized models efficiently in the framework of local bifurcation theory. The approach is based on a normalization procedure that is used to identify natural parameters of the system. The Jacobian in a steady state is then derived as a function of these parameters. The analytical computation of local bifurcations using computer algebra reveals conditions for the local asymptotic stability of steady states and provides certain insights on the global dynamics of the system. The proposed approach yields a close connection between modelling and nonlinear dynamics. We illustrate the investigation of …

Card 3 · Parameter regions that give rise to 2[n/2]+1 positive steady states in the n-site phosphorylation system

The distributive sequential n-site phosphorylation/dephosphorylation system is an important building block in networks of chemical reactions arising in molecular biology, which has been intensively studied. In the nice paper of Wang and Sontag (2008) it is shown that for certain choices of the react…

Read full passage excerpt

The distributive sequential n-site phosphorylation/dephosphorylation system is an important building block in networks of chemical reactions arising in molecular biology, which has been intensively studied. In the nice paper of Wang and Sontag (2008) it is shown that for certain choices of the reaction rate constants and total conservation constants, the system can have 2[n/2]+1 positive steady states (that is, n+1 positive steady states for n even and n positive steady states for n odd). In this paper we give open parameter regions in the space of reaction rate constants and total conservation constants that ensure these number of positive steady states, while assuming in the modeling that roughly only 1/4 of the intermediates occur in the reaction mechanism. This result is based on the general framework developed by Bihan, Dickenstein, and Giaroli (2018), which can be applied to other networks. We also describe how to implement these tools to search for multistationarity regions in a …

Card 4 · An algebraic method to calculate parameter regions for constrained steady-state distribution in stochastic reaction networks

Steady state is an essential concept in reaction networks. Its stability reflects fundamental characteristics of several biological phenomena such as cellular signal transduction and gene expression. Because biochemical reactions occur at the cellular level, they are affected by unavoidable fluctuat…

Read full passage excerpt

Steady state is an essential concept in reaction networks. Its stability reflects fundamental characteristics of several biological phenomena such as cellular signal transduction and gene expression. Because biochemical reactions occur at the cellular level, they are affected by unavoidable fluctuations. Although several methods have been proposed to detect and analyze the stability of steady states for deterministic models, these methods cannot be applied to stochastic reaction networks. In this paper, we propose an algorithm based on algebraic computations to calculate parameter regions for constrained steady-state distribution of stochastic reaction networks, in which the means and variances satisfy some given inequality constraints. To evaluate our proposed method, we perform computer simulations for three typical chemical reactions and demonstrate that the results obtained with our method are consistent with the simulation results.

Card 5 · Detecting parameter regions for bistability in reaction networks

Deciding whether and where a system of parametrized ordinary differential equations displays bistability, that is, has at least two asymptotically stable steady states for some choice of parameters, is a hard problem. For systems modeling biochemical reaction networks, we introduce a procedure to de…

Read full passage excerpt

Deciding whether and where a system of parametrized ordinary differential equations displays bistability, that is, has at least two asymptotically stable steady states for some choice of parameters, is a hard problem. For systems modeling biochemical reaction networks, we introduce a procedure to determine, exclusively via symbolic computations, the stability of the steady states for unspecified parameter values. In particular, our approach fully determines the stability type of all steady states of a broad class of networks. To this end, we combine the Hurwitz criterion, reduction of the steady state equations to one univariate equation, and structural reductions of the reaction network. Using our method, we prove that bistability occurs in open regions in parameter space for many relevant motifs in cell signaling.

Model-reported card use

Card 2 · fact

Affected output: The approach is particularly well suited to systems where the functional forms of the underlying processes are uncertain but the network structure is known, such as ecological food webs, coupled laser systems, and socioeconomic models of dynastic cycles.

Model explanation: This row supplied the concrete fact that generalized models are not restricted to specific functional forms and can describe a class of systems sharing a similar structure, which grounds the claim about the approach's suitability for systems with uncertain functional forms.

Card 2 · fact

Affected output: In each case, the normalization procedure identifies natural parameters of the system, and the Jacobian at a steady state is derived as a function of these parameters, so that local bifurcations can be computed analytically.

Model explanation: This row supplied the specific methodological steps—normalization to identify natural parameters, derivation of the Jacobian as a function of these parameters, and analytical computation of local bifurcations—which were used to describe the approach's implementation.

Card 2 · terminology

Affected output: ecological food webs, coupled laser systems, and socioeconomic models of dynastic cycles

Model explanation: This row provided the specific example systems from the source paper, which were used as illustrative applications of the generalized modeling approach.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

The ensemble perspective also clarifies what a generalized model cannot deliver: because it represents all dynamics consistent with the available structural knowledge, it cannot single out the one true trajectory of the system, and any prediction it yields is therefore conditional on the constraints that define the ensemble.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the no-Evidence version. Its original rationale appears below.

Read the original model judgment

Output A contains unsupported claims. It introduces specific applications ('ecological food webs, coupled laser systems, and socioeconomic models of dynastic cycles') and describes technical details ('normalization procedure identifies natural parameters,' 'Jacobian at a steady state is derived as a function of these parameters') that come from Source 2's description of generalized models. However, the draft itself does not mention these specific applications or the normalization/Jacobian procedure. The task is to continue the manuscript's paragraph, not to import content from sources that the draft hasn't already established. Output A presents these as if they follow from the draft's discussion, but the draft only mentions 'generalized models' in a conceptual sense without referencing these specific examples or methods. Output B, by contrast, stays entirely within the conceptual framework established by the draft: it discusses the 'ensemble perspective,' what generalized models 'cannot deliver,' and the conditional nature of predictions based on 'structural knowledge' and 'constraints that define the ensemble'—all concepts already present in the draft's language about 'ensemble of possible realities,' 'available structural knowledge,' and narrowing down 'possible worlds.' Output B is a coherent continuation that requires no external factual claims beyond what the draft establishes.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.