FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better145 / 280 · beu-ca8f737b287151a9

Predictive Coding: a Theoretical and Experimental Review

Quantitative biology · 2107.12979v4

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Predictive Coding: a Theoretical and Experimental Review

1 Introduction

…its own (predictable movement), such as during voluntary eye movements, but does not do so for external perturbations, such as when external pressure is applied to the eyeball, in which case we consciously experience visual movement arising from this (unpredicted) ocular motion. Helmholtz thus argued that the brain must maintain both a record of its own actions, in the form of a ‘corollary discharge’ as well as a model of the world sufficient to predict the visual effects of these actions (i.e. a forward model) in order to so perfectly cancel self-caused visual motion ( Huang & Rao (2011) ) .

…A.K. Seth (2014) ) . Cybernetics as a science is focused on understanding the dynamics of interacting feedback loops for perception and control, based especially around the concept of error minimization. Control and filtering theory have, in a related but distinct way, been based around methods to minimize residual errors in both perception or action according to some objective for decades. As we shall see, standard methods such as Kalman Filtering ( Kalman (1960) ) or PID control Johnson & Moradi (2005) can be shown as special cases of predictive coding under certain restrictive assumptions.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs provide coherent continuations that bridge from cybernetics/control theory to predictive coding. However, Output B is superior in fit and scope. The draft explicitly sets up 'As we shall see, standard methods such as Kalman Filtering or PID control can be shown as special cases of predictive coding under certain restrictive assumptions.' Output B directly honors this promise by describing how 'historical and engineering perspectives converge' and explicitly naming predictive coding's error-minimization framework, with appropriate citations (Rao & Ballard, 1999; Friston, 2005) that are standard in this literature. Output A, while conceptually accurate, shifts to a more general philosophical framing ('perception itself as the inference of the causes of sensory input') that slightly drifts from the draft's specific engineering-to-theory trajectory. Output B's 'common computational motif' language better echoes the draft's 'interacting feedback loops' and 'error minimization' terminology, creating smoother textual integration. Both are usable, but B more precisely continues the argumentative arc established.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

These historical and engineering perspectives converge on a common computational motif: the minimization of prediction error through interacting forward and feedback pathways. Predictive coding formalizes this motif as a theory of cortical function in which the brain's principal computation, underlying both perception and learning, is the minimization of prediction errors (Rao & Ballard, 1999; Friston, 2005).

Passages supplied to the Evidence version

Card 1 · Relaxing the Constraints on Predictive Coding Models

Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs, which underlies both perception and learning, is the minimization of prediction errors. While motivated by high-level notions of variational inference, detailed neurophysio…

Read full passage excerpt

Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs, which underlies both perception and learning, is the minimization of prediction errors. While motivated by high-level notions of variational inference, detailed neurophysiological models of cortical microcircuits which can implements its computations have been developed. Moreover, under certain conditions, predictive coding has been shown to approximate the backpropagation of error algorithm, and thus provides a relatively biologically plausible credit-assignment mechanism for training deep networks. However, standard implementations of the algorithm still involve potentially neurally implausible features such as identical forward and backward weights, backward nonlinear derivatives, and 1-1 error unit connectivity. In this paper, we show that these features are not integral to the algorithm and can be removed either directly or through learning additional set …

Card 2 · PID Control of Biochemical Reaction Networks

Principles of feedback control have been shown to naturally arise in biological systems and have been applied with success to build synthetic circuits. Here, we present an implementation of a proportional–integral–derivative (PID) controller as a chemical reaction network with mass-action kinetics.…

Read full passage excerpt

Principles of feedback control have been shown to naturally arise in biological systems and have been applied with success to build synthetic circuits. Here, we present an implementation of a proportional–integral–derivative (PID) controller as a chemical reaction network with mass-action kinetics. This makes the controller synthesizablein vitrousing DNA strand displacement technology, owing to its demonstrated capability of realizing arbitrary reaction-network designs as interacting DNA molecules. Previous related work has studied biological PID architectures using linearizations of nonlinear dynamics arising in both the controller components and in the plant. In this article, we present a proof of correctness of our nonlinear design in closed loop using arguments from singular perturbation theory. As an application to show the effectiveness of our controller, we provide numerical simulations on a genetic model to perform PID feedback control of protein expression.

Card 3 · PredNet and Predictive Coding: A Critical Review

PredNet, a deep predictive coding network developed by Lotter et al., combines a biologically inspired architecture based on the propagation of prediction error with self-supervised representation learning in video. While the architecture has drawn a lot of attention and various extensions of the mo…

Read full passage excerpt

PredNet, a deep predictive coding network developed by Lotter et al., combines a biologically inspired architecture based on the propagation of prediction error with self-supervised representation learning in video. While the architecture has drawn a lot of attention and various extensions of the model exist, there is a lack of a critical analysis. We fill in the gap by evaluating PredNet both as an implementation of the predictive coding theory and as a self-supervised video prediction model using a challenging video action classification dataset. We design an extended model to test if conditioning future frame predictions on the action class of the video improves the model performance. We show that PredNet does not yet completely follow the principles of predictive coding. The proposed top-down conditioning leads to a performance gain on synthetic data, but does not scale up to the more complex real-world action classification dataset. Our analysis is aimed at guiding future research …

Model-reported card use

Card 1 · fact

Affected output: Predictive coding formalizes this motif as a theory of cortical function in which the brain's principal computation, underlying both perception and learning, is the minimization of prediction errors

Model explanation: This row supplied the specific definitional proposition—predictive coding as a cortical theory whose principal computation is prediction-error minimization underlying perception and learning—that the generated sentence states.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

Predictive coding generalises this error-minimisation view by casting perception itself as the inference of the causes of sensory input, rather than as a passive feedforward transformation of that input. Under this account, the brain continuously generates predictions about its sensory consequences and updates its internal models only when those predictions are violated.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs provide coherent academic continuations, but B is more useful for different reasons. A introduces 'predictive coding generalises this error-minimisation view' but awkwardly positions itself as if continuing from Helmholtz/cybernetics discussion directly into predictive coding theory without smooth transition. The phrase 'this error-minimisation view' is vague—it's unclear whether 'this' refers to cybernetics, control theory, or the general concept just mentioned. A also lacks any citation markers, which while not disqualifying, makes it less immediately insertable in an academic context where the surrounding text has citations. B explicitly connects 'historical and engineering perspectives' (Helmholtz/cybernetics/control theory) to 'predictive coding,' providing a clear transitional bridge. It also includes parenthetical citations (Rao & Ballard, 1999; Friston, 2005) matching the draft's citation style, making it more immediately usable. However, B's citations are not from the supplied sources—Rao & Ballard 1999 and Friston 2005 are external references. Since citation markers are allowed per instructions and the substantive claim about predictive coding minimizing prediction errors is supported by Source 1 ('Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs...is the minimization of prediction errors'), this is acceptable. The core claim in B is directly grounded in Source 1. A's claims about 'predictions are violated' and 'passive feedforward transformation' are not directly supported by the sources—Source 1 mentions prediction error minimization but not the specific contrast with 'passive feedforward transformation' or the exact phrasing about 'only when predictions are violated.' B more precisely reflects Source 1's language. Both are usable, but B provides better transition, appropriate citation format, and more source-grounded content.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.