← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better70 / 280 · beu-c28d3eef7a240e3f

Survey of Consciousness Theory from Computational Perspective

Psychology and cognitive neuroscience · 2309.10063v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Survey of Consciousness Theory from Computational Perspective

1 Introduction > 1.4 Consciousness and Intelligence

" I do not wish to give the impression that I think there is no mystery about consciousness. There is, for instance, something of a paradox connected with any attempt to localise it. "

…n Fig. 1 (originally in Tegmark (2018) ), the intelligence-related problems are the easiest, which is also claimed by David Chalmers. This type of problem typically does not require to consider the subjective experience of the experimental subjects. A hard problem is to find the physically interpretable features for distinguishing conscious and unconscious processes. The next level question is how the consciousness happens and what the determined factors are. The final and hardest problems related to the explanation of the existence of consciousness, or why consciousness exists in any system?

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

The draft begins with a famous quote about consciousness and mystery, then abruptly jumps to a figure reference about intelligence-related problems being easiest, with a taxonomy of problems (intelligence → distinguishing conscious/unconscious processes → how consciousness happens → why consciousness exists). This is clearly an incomplete argumentative move: the quote sets up paradox/mystery, but the following text launches into a comparative taxonomy without explaining the connection. Output A attempts to bridge by explaining Chalmers' hard problem and linking it to the 'layered taxonomy' and 'hardest level.' However, this is largely redundant with what's already stated in the draft (which already mentions Chalmers, already describes the hardest problems). The draft already says the final problems concern 'why consciousness exists' — A merely restates this with more words about 'new physical laws,' adding little actual progression. The draft's quote about localisation paradox is left completely unaddressed, and A's text would insert somewhat repetitively. Output B takes a different, more productive angle: it identifies the implicit distinction between intelligence and consciousness that the draft's taxonomy presupposes but never articulates. The draft contrasts 'intelligence-related problems' (easiest, no subjective experience needed) with progressively harder consciousness problems. B explicitly draws out this distinction as 'not merely terminological,' noting that intelligent problem-solving can occur without resolving consciousness — this is a genuine argumentative move that clarifies the taxonomy's rationale. It connects to the draft's structure without being redundant, and provides a coherent philosophical framing that the next paragraph (about the taxonomy levels) would naturally follow from. B better serves author utility: it makes explicit what the draft assumes, advances the argument, and fits without requiring repair. A is usable but more repetitive and less additive.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

The hard problem of consciousness, as formulated by Chalmers, concerns why and how subjective experience arises from physical processes, and it has been argued that its resolution may ultimately require new physical laws or principles rather than being reducible to known physics. This framing motivates the layered taxonomy of problems adopted in this survey, in which the existence of consciousness constitutes the hardest and least tractable level.

Passages supplied to the Evidence version

Card 1 · A Compositional Model of Consciousness Based on Consciousness-Only

Scientific studies of consciousness rely on objects whose existence is assumed to be independent of any consciousness. On the contrary, we assume consciousness to be fundamental, and that one of the main features of consciousness is characterized as being other-dependent. We set up a framework which…

Read full passage excerpt

Scientific studies of consciousness rely on objects whose existence is assumed to be independent of any consciousness. On the contrary, we assume consciousness to be fundamental, and that one of the main features of consciousness is characterized as being other-dependent. We set up a framework which naturally subsumes this feature by defining a compact closed category where morphisms represent conscious processes. These morphisms are a composition of a set of generators, each being specified by their relations with other generators, and therefore co-dependent. The framework is general enough and fits well into a compositional model of consciousness. Interestingly, we also show how our proposal may become a step towards avoiding the hard problem of consciousness, and thereby address the combination problem of conscious experiences.

Card 2 · Integrated Information Theory and Isomorphic Feed-Forward Philosophical Zombies

Any theory amenable to scientific inquiry must have testable consequences. This minimal criterion is uniquely challenging for the study of consciousness, as we do not know if it is possible to confirm via observation from the outside whether or not a physical system knows what it feels like to have…

Read full passage excerpt

Any theory amenable to scientific inquiry must have testable consequences. This minimal criterion is uniquely challenging for the study of consciousness, as we do not know if it is possible to confirm via observation from the outside whether or not a physical system knows what it feels like to have an inside—a challenge referred to as the “hard problem” of consciousness. To arrive at a theory of consciousness, the hard problem has motivated development of phenomenological approaches that adopt assumptions of what properties consciousness has based on first-hand experience and, from these, derive the physical processes that give rise to these properties. A leading theory adopting this approach is Integrated Information Theory (IIT), which assumes our subjective experience is a “unified whole”, subsequently yielding a requirement for physical feedback as a necessary condition for consciousness. Here, we develop a mathematical framework to assess the validity of this assumption by testing …

Card 3 · Can quantum physics help solve the hard problem of consciousness? A hypothesis based on entangled spins and photons

The hard problem of consciousness is the question how subjective experience arises from brain matter. I suggest exploring the possibility that quantum physics could be part of the answer. The simultaneous unity and complexity of subjective experience is difficult to understand from a classical physi…

Read full passage excerpt

The hard problem of consciousness is the question how subjective experience arises from brain matter. I suggest exploring the possibility that quantum physics could be part of the answer. The simultaneous unity and complexity of subjective experience is difficult to understand from a classical physics perspective. In contrast, quantum entanglement is naturally both complex and unified. Moreover the concept of matter is much more subtle in quantum physics compared to classical physics, and quantum computing shows that quantum effects can be useful for information processing. Building on recent progress in quantum technology and neuroscience, I propose a concrete hypothesis as a basis for further investigation, namely that subjective experience is related to the dynamics of a complex entangled state of spins, which is continuously generated and updated through the exchange of photons. Spins in condensed matter systems at room or body temperature can have coherence times in the relevant r …

Card 4 · The "Hard Problem" of Life

Chalmer's famously identified pinpointing an explanation for our subjective experience as the "hard problem of consciousness". He argued that subjective experience constitutes a "hard problem" in the sense that its explanation will ultimately require new physical laws or principles. Here, we propose…

Read full passage excerpt

Chalmer's famously identified pinpointing an explanation for our subjective experience as the "hard problem of consciousness". He argued that subjective experience constitutes a "hard problem" in the sense that its explanation will ultimately require new physical laws or principles. Here, we propose a corresponding "hard problem of life" as the problem of how `information' can affect the world. In this essay we motivate both why the problem of information as a causal agent is central to explaining life, and why it is hard - that is, why we suspect that a full resolution of the hard problem of life will, similar to as has been proposed for the hard problem of consciousness, ultimately not be reducible to known physical principles.

Card 5 · The hard problem and the measurement problem: a no-go theorem and potential consequences

This is standardly done to explain “why are we more likely to find ourselves” in a branch with a higher multiplier. But this question already implies “us” and our subjective experience, and such a step in neither necessary nor easy to motivate in the philosophical zombie scenario. In such a scenario…

Read full passage excerpt

This is standardly done to explain “why are we more likely to find ourselves” in a branch with a higher multiplier. But this question already implies “us” and our subjective experience, and such a step in neither necessary nor easy to motivate in the philosophical zombie scenario. In such a scenario, there is actually no need to bother with the derivation of Born’s rule since all objective and impersonal experimental data can be explained without it. Therefore, we arrived at an important recognition that problems in MWI related to probability and the Born rule are again the same problems of subjective experience and of the unwarranted assumption that functioning of automata by itself produces consciousness (i.e. of the negation of the hard problem).

Model-reported card use

Card 4 · fact

Affected output: The hard problem of consciousness, as formulated by Chalmers, concerns why and how subjective experience arises from physical processes, and it has been argued that its resolution may ultimately require new physical laws or principles rather than being reducible to known physics.

Model explanation: This row supplied the specific content of Chalmers's formulation of the hard problem and the claim that its resolution may require new physical laws, which the generated sentence uses to ground the introduction's framing of the hardest level of problems.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

The distinction between intelligence and consciousness is thus not merely terminological: a system can exhibit intelligent problem-solving while the question of whether it is conscious remains entirely open. This asymmetry motivates treating the two as separate explanatory targets rather than assuming that progress on one automatically yields progress on the other.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs provide coherent academic continuations, but Output A is substantially more useful for this specific task. The draft establishes a paradox about localizing consciousness, then jumps to a figure reference about intelligence-related problems being 'easiest' with David Chalmers' endorsement, followed by a layered taxonomy of problems (distinguishing conscious/unconscious processes → how consciousness happens → why consciousness exists). Output A directly connects to this structure: it explains Chalmers' formulation of the hard problem, cites the claim about new physical laws (supported by source 4: 'The "Hard Problem" of Life'), and explicitly links this to 'the layered taxonomy of problems adopted in this survey'—mirroring the draft's progression from easier to hardest problems. Output B, while philosophically defensible, makes a more generic distinction between intelligence and consciousness that doesn't engage with the draft's specific layered structure or the Chalmers reference. The draft already implies this distinction by contrasting 'intelligence-related problems' with 'hard problem' levels; Output B states the obvious without advancing the manuscript's argument. Output A supplies substantive content about Chalmers' formulation and the need for new physical laws, which directly supports and extends the draft's taxonomy. Output B is usable but less helpful for a reader following the draft's specific argumentative thread.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.