← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteAbout the same73 / 280 · beu-c37ab3fb8d383cb6

Theoretical foundations of studying criticality in the brain

Psychology and cognitive neuroscience · 2306.05635v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Theoretical foundations of studying criticality in the brain

6 Brain criticality: conclusions on current progresses and limitations

…non-negligible, a recent study presents an elegant theory to prove that two homeostatic adaptation mechanisms (i.e., the short-term depression of inhibition and the spike-dependent threshold increase) enable synaptic excitation/inhibition balance, AI characteristic, and SOqC to appear simultaneously in the same neural dynamics Girardi-Schappo et al. ((2021)) . Similarly, it is suggested that neural dynamics with criticality or with AI characteristic can be generated by the same neural populations if the synaptic excitation/inhibition balance is fine tuned appropriately J. Li & Shew ((2020)) .

…thout criticality Martinello et al. ((2017)) . Meanwhile, bistability phenomena, a kind of fine tuned or self-organized discontinuous phase transitions with limit cycles rather than critical points, can also create neural dynamics with power-law properties Buendía, Di Santo et al. ((2020)) ; di Santo et al. ((2016)) ; Cocchi et al. ((2017)) . Consequently, we emphasize that neural avalanches exponents alone are insufficient to prove or disprove any brain criticality hypothesis. These power-law exponents are meaningless for brain criticality hypothesis unless they satisfy the scaling relation.

Given what have been reviewed above, we arrive at a point to conclude on the current progresses and limitations in establishing theoretical foundations of different types of

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: About the same

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

This rationale combines reviews with different display orders.

Order-swapped judges disagreed. Pass 1: Both outputs are nearly identical and equally usable. They both complete the sentence by identifying the purpose of the conclusion: to summarize progresses/limitations and point to future work. The only difference is 'conceptual and empirical gaps' (A) versus 'open questions' (B). Both are appropriate academic phrasings that fit the context of reviewing theoretical foundations. 'Conceptual and empirical gaps' is slightly more specific about the nature of the limitations, while 'open questions' is more conventional. Neither introduces unsupported claims or contradictions. The draft clearly needs continuation—the sentence is grammatically incomplete after 'different types of'—so neither 'complete' action would be appropriate; however, both outputs provide text rather than abstaining. Both completions are concise, on-topic (brain criticality), and could be inserted without repair. Pass 2: Both outputs complete the sentence appropriately and fit the academic context of reviewing theoretical foundations of brain criticality. Output A uses 'open questions' while Output B uses 'conceptual and empirical gaps'—both are standard academic formulations. However, Output A is slightly more concise and direct. The phrase 'open questions' is a more natural and commonly used framing in review paper conclusions, while 'conceptual and empirical gaps' in Output B, though acceptable, is slightly more verbose and less idiomatic (the pairing 'conceptual and empirical' with 'gaps' works but 'open questions' is the more standard collocation after 'identify'). Both are usable, but Output A has marginally better fit and flow.

Evidence

Same model and position · retrieved passages supplied · first output

brain criticality, and to identify the open questions that future theoretical work must address.

Passages supplied to the Evidence version

Card 1 · Scaling properties of neuronal avalanches are consistent with critical dynamics

Complex systems, when poised near a critical point of a phase transition between order and disorder, exhibit a dynamics comprising a scale-free mixture of order and disorder which is universal, i.e. system-independent (1-5). It allows systems at criticality to adapt swiftly to environmental changes…

Read full passage excerpt

Complex systems, when poised near a critical point of a phase transition between order and disorder, exhibit a dynamics comprising a scale-free mixture of order and disorder which is universal, i.e. system-independent (1-5). It allows systems at criticality to adapt swiftly to environmental changes (i.e., high susceptibility) as well as to flexibly process and store information. These unique properties prompted the conjecture that the brain might operate at criticality (1), a view supported by the recent description of neuronal avalanches in cortex in vitro (6-8), in anesthetized rats (9) and awake primates (10), and in neuronal models (11-16). Despite the attractiveness of this idea, its validity is hampered by the fact that its theoretical underpinning relies solely on the replication of sizes and durations of avalanches, which reflect only a portion of the rich dynamics found at criticality. Here we show experimentally five fundamental properties of avalanches consistent with critic …

Card 2 · A toy model for brain criticality: self-organized excitation/inhibition ratio and the role of network clustering

The critical brain hypothesis receives increasing support from recent experimental results. It postulates that the brain is at a critical point between an ordered and a chaotic regime, sometimes referred to as the "edge of chaos." Another central observation of neuroscience is the principle of excit…

Read full passage excerpt

The critical brain hypothesis receives increasing support from recent experimental results. It postulates that the brain is at a critical point between an ordered and a chaotic regime, sometimes referred to as the "edge of chaos." Another central observation of neuroscience is the principle of excitation-inhibition balance: Certain brain networks exhibit a remarkably constant ratio between excitation and inhibition. When this balance is perturbed, the network shifts away from the critical point, as may for example happen during epileptic seizures. However, it is as of yet unclear what mechanisms balance the neural dynamics towards this excitation-inhibition ratio that ensures critical brain dynamics. Here we introduce a simple yet biologically plausible toy model of a self-organized critical neural network with a self-organizing excitation to inhibition ratio. The model only requires a neuron to have local information of its own recent activity and changes connections between neurons a …

Card 3 · Evidence for Quasicritical Brain Dynamics

Much evidence seems to suggest the cortex operates near a critical point, yet a single set of exponents defining its universality class has not been found. In fact, when critical exponents are estimated from data, they widely differ across species, individuals of the same species, and even over time…

Read full passage excerpt

Much evidence seems to suggest the cortex operates near a critical point, yet a single set of exponents defining its universality class has not been found. In fact, when critical exponents are estimated from data, they widely differ across species, individuals of the same species, and even over time, or depending on stimulus. Interestingly, these exponents still approximately hold to a dynamical scaling relation. Here we show that the theory of quasicriticality, an organizing principle for brain dynamics, can account for this paradoxical situation. As external stimuli drive the cortex, quasicriticality predicts a departure from criticality along a Widom line with exponents that decrease in absolute value, while still holding approximately to a dynamical scaling relation. We use simulations and experimental data to confirm these predictions and describe new ones that could be tested soon.

Card 4 · Identification of Criticality in Neuronal Avalanches: II. A Theoretical and Empirical Investigation of the Driven Case

The observation of apparent power laws in neuronal systems has led to the suggestion that the brain is at, or close to, a critical state and may be a self-organised critical system. Within the framework of self-organised criticality a separation of timescales is thought to be crucial for the observa…

Read full passage excerpt

The observation of apparent power laws in neuronal systems has led to the suggestion that the brain is at, or close to, a critical state and may be a self-organised critical system. Within the framework of self-organised criticality a separation of timescales is thought to be crucial for the observation of power-law dynamics and computational models are often constructed with this property. However, this is not necessarily a characteristic of physiological neural networks-external input does not only occur when the network is at rest/a steady state. In this paper we study a simple neuronal network model driven by a continuous external input (i.e. the model does not have an explicit separation of timescales from seeding the system only when in the quiescent state) and analytically tuned to operate in the region of a critical state (it reaches the critical regime exactly in the absence of input-the case studied in the companion paper to this article). The system displays avalanche dynami …

Card 5 · Self-organization without conservation: Are neuronal avalanches generically critical?

Such avalanches have been claimed to be generically scale-invariant – i.e. power-law distributed – with many exciting implications in Neuroscience. Recently, a self-organized model has been proposed by Levina, Herrmann and Geisel to justify such an empirical finding. Given that (i) neural dynamics i…

Read full passage excerpt

Such avalanches have been claimed to be generically scale-invariant – i.e. power-law distributed – with many exciting implications in Neuroscience. Recently, a self-organized model has been proposed by Levina, Herrmann and Geisel to justify such an empirical finding. Given that (i) neural dynamics is dissipative and (ii) there is a loading mechanism “charging” progressively the background synaptic strength, this model/dynamics is very similar in spirit to forest-fire and earthquake models, archetypical examples of non-conserving self-organization, which have been recently shown to lack true criticality. Here we show that cortical neural networks obeying (i) and (ii) are not generically critical; unless parameters are fine tuned, their dynamics is either sub- or super-critical, even if the pseudo-critical region is relatively broad.

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

brain criticality, and to identify the conceptual and empirical gaps that future theoretical work must address.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.