← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better141 / 280 · beu-f1436e019da65e38

Generative AI for Controllable Protein Sequence Design: A Survey

Quantitative biology · 2402.10516v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Generative AI for Controllable Protein Sequence Design: A Survey

1 Introduction

…gorithms, have been propelling the protein design field towards an unprecedented revolution. In this survey, we systematically review recent advances in generative AI for controllable protein sequence design. To set the stage, we first outline the foundational tasks in protein sequence design in terms of the constraints involved and present key generative models and optimization algorithms. We then offer in-depth reviews of each design task and discuss the pertinent applications. Finally, we identify the unresolved challenges and highlight research opportunities that merit deeper exploration.

…ns, designing novel amino acid sequences that encode proteins with desired functions has been a central challenge in bioengineering. Nevertheless, due to the vast exploration space of possible proteins, conventional experimental methods, such as directed evolution Arnold 1998 , are prohibitively time-intensive and money-consuming. To overcome these challenges, recent years have witnessed the trend of leveraging generative AI technologies to explore the biochemical space intelligently. Fueled by advances in generative AI, the field of protein design is experiencing an unprecedented revolution.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs are coherent continuations, but they serve very different rhetorical purposes at this position. The draft already contains a complete introductory paragraph that establishes: (1) the central challenge of protein sequence design, (2) the limitations of conventional methods, (3) the rise of generative AI as a solution, and (4) a framing of 'unprecedented revolution.' Output A adds more detail about specific generative architectures (language models, diffusion processes) and a specific success rate statistic (~20%). However, this material is largely redundant with what the first paragraph already accomplishes—it piles on more evidence for a revolution already declared, without advancing the argument structurally. The draft's final sentence 'Fueled by advances in generative AI, the field of protein design is experiencing an unprecedented revolution' functions as a natural pivot to surveying what this review will cover. Output B capitalizes on this pivot by providing a clear roadmap sentence that mirrors the structure promised in the first paragraph ('To set the stage, we first outline... We then offer... Finally, we identify...'). This creates coherent bookends: the first paragraph establishes the problem and solution, Output B explicitly states the survey's organization, and then the body can proceed. Output B requires no repair and improves navigability; Output A, while accurate, would make the introduction feel bloated and delay the promised structural clarity.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

Generative architectures, including language models and diffusion processes, are increasingly capable of producing novel yet realistic proteins that display desirable properties and perform specified functions. State-of-the-art design protocols now achieve experimental success rates nearing 20%, widening access to de novo designed proteins.

Passages supplied to the Evidence version

Card 1 · Designing novel protein structures using sequence generator and AlphaFold2

Protein structures and functions are determined by a contiguous arrangement of amino acid sequences. Designing novel protein sequences and structures with desired geometry and functions is a complex task with large state spaces. Here we develop a novel protein design pipeline consisting of two deep…

Read full passage excerpt

Protein structures and functions are determined by a contiguous arrangement of amino acid sequences. Designing novel protein sequences and structures with desired geometry and functions is a complex task with large state spaces. Here we develop a novel protein design pipeline consisting of two deep learning algorithms, ProteinSolver and AlphaFold2. ProteinSolver is a deep graph neural network that generates amino acid sequences such that the forces between interacting amino acids are favorable and compatible with the fold while AlphaFold2 is a deep learning algorithm that predicts the protein structures from protein sequences. We present forty de novo designed binding sites of the PTP1B and P53 proteins with high precision, out of which thirty proteins are novel. Using ProteinSolver and AlphaFold2 in conjunction, we can trim the exploration of the large protein conformation space, thus expanding the ability to find novel and diverse de novo protein designs.

Card 2 · Efficient generative modeling of protein sequences using simple autoregressive models

Abstract Generative models emerge as promising candidates for novel sequence-data driven approaches to protein design, and for the extraction of structural and functional information about proteins deeply hidden in rapidly growing sequence databases. Here we propose simple autoregressive models as h…

Read full passage excerpt

Abstract Generative models emerge as promising candidates for novel sequence-data driven approaches to protein design, and for the extraction of structural and functional information about proteins deeply hidden in rapidly growing sequence databases. Here we propose simple autoregressive models as highly accurate but computationally efficient generative sequence models. We show that they perform similarly to existing approaches based on Boltzmann machines or deep generative models, but at a substantially lower computational cost (by a factor between 10 2 and 10 3 ). Furthermore, the simple structure of our models has distinctive mathematical advantages, which translate into an improved applicability in sequence generation and evaluation. Within these models, we can easily estimate both the probability of a given sequence, and, using the model’s entropy, the size of the functional sequence space related to a specific protein family. In the example of response regulators, we find a huge …

Card 3 · Generative artificial intelligence for de novo protein design

Engineering new molecules with desirable functions and properties has the potential to extend our ability to engineer proteins beyond what nature has so far evolved. Advances in the so-called "de novo" design problem have recently been brought forward by developments in artificial intelligence. Gene…

Read full passage excerpt

Engineering new molecules with desirable functions and properties has the potential to extend our ability to engineer proteins beyond what nature has so far evolved. Advances in the so-called "de novo" design problem have recently been brought forward by developments in artificial intelligence. Generative architectures, such as language models and diffusion processes, seem adept at generating novel, yet realistic proteins that display desirable properties and perform specified functions. State-of-the-art design protocols now achieve experimental success rates nearing 20%, thus widening the access to de novo designed proteins. Despite extensive progress, there are clear field-wide challenges, for example in determining the best in silico metrics to prioritise designs for experimental testing, and in designing proteins that can undergo large conformational changes or be regulated by post-translational modifications and other cellular processes. With an increase in the number of models be …

Card 4 · Design of metalloproteins and novel protein folds using variational autoencoders

The design of novel proteins has many applications but remains an attritional process with success in isolated cases. Meanwhile, deep learning technologies have exploded in popularity in recent years and are increasingly applicable to biology due to the rise in available data. We attempt to link pro…

Read full passage excerpt

The design of novel proteins has many applications but remains an attritional process with success in isolated cases. Meanwhile, deep learning technologies have exploded in popularity in recent years and are increasingly applicable to biology due to the rise in available data. We attempt to link protein design and deep learning by using variational autoencoders to generate protein sequences conditioned on desired properties. Potential copper and calcium binding sites are added to non-metal binding proteins without human intervention and compared to a hidden Markov model. In another use case, a grammar of protein structures is developed and used to produce sequences for a novel protein topology. One candidate structure is found to be stable by molecular dynamics simulation. The ability of our model to confine the vast search space of protein sequences and to scale easily has the potential to assist in a variety of protein design tasks.

Model-reported card use

Card 3 · fact

Affected output: Generative architectures, including language models and diffusion processes, are increasingly capable of producing novel yet realistic proteins that display desirable properties and perform specified functions.

Model explanation: This row supplied the concrete capability claim about generative architectures that the new paragraph opens with, rather than a generic statement about AI progress.

Card 3 · fact

Affected output: State-of-the-art design protocols now achieve experimental success rates nearing 20%, widening access to de novo designed proteins.

Model explanation: This row supplied the specific quantitative benchmark (nearing 20% experimental success) and its consequence for access to de novo designed proteins.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

This survey is organized around the constraints that define each protein sequence design task, the generative models and optimization algorithms that address them, and the applications that follow from successful designs.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the no-Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs provide coherent academic paragraphs, but they serve different purposes relative to the draft. The draft's blank paragraph appears immediately after the introduction's first paragraph and before a paragraph that begins mid-sentence with '...ns, designing novel amino acid sequences...' The draft's first paragraph already states: 'In this survey, we systematically review recent advances in generative AI for controllable protein sequence design. To set the stage, we first outline... We then offer... Finally, we identify...' This establishes the paper's organization. Output B directly continues this organizational framing, stating 'This survey is organized around the constraints that define each protein sequence design task, the generative models and optimization algorithms that address them, and the applications that follow from successful designs.' This mirrors and extends the structure already announced in the first paragraph, creating a smooth transition to the next section which discusses the foundational challenge of designing novel sequences. Output A, while grounded in source material (the 'Generative artificial intelligence for de novo protein design' source), introduces specific claims about experimental success rates nearing 20% and de novo designed proteins. These claims, though source-supported, feel more like content for a later section rather than a bridge paragraph. The draft's introduction already announced the survey structure; Output B fulfills that promise by elaborating the organization, making it more immediately insertable with less need for subsequent restructuring. Output A's content is valuable but would fit better in a later section about current capabilities or results, not as the immediate continuation after the introduction's structural preview.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.