FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better152 / 280 · beu-d7ee2ce48b83ab73

Large Language Models in Bioinformatics: A Survey

Quantitative biology · 2503.04490v3

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Large Language Models in Bioinformatics: A Survey

7 Conclusions and Future Directions > 7.1 Challenges and Limitations > 7.1.3 Multimodal Learning and Cross-Omics Integration

… large-scale, high-quality biological datasets for effective training, yet annotated genomic, transcriptomic, and proteomic data remain limited Lu et al. 2024 . Unlike natural language corpora, which are abundant and diverse, biological datasets are often noisy, incomplete, or biased toward well-studied species and diseases. Consequently, model generalizability suffers, leading to biased predictions which may not hold across diverse biological contexts. Additionally, batch effects and experimental noise complicate the development of robust foundation models for bioinformatics Yu et al. 2024 .

State-of-the-art LLMs, such as AlphaFold and DNABERT, require massive computational resources for both training and inference. This computational barrier limits accessibility, particularly for research groups with limited infrastructure (Table 2 and Table 3 ). Furthermore, relatively longer biological sequences significantly increase memory and processing requirements, making it challenging to apply standard Transformer architectures to genome-scale data Bernard et al. 2025 . Efficient methods of model compression and retrieval-augmented need to be further explored to enhance scalability.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs provide coherent continuations that shift from computational challenges to integration challenges across omics layers, which is a logical progression. However, Output B is stronger for several reasons: (1) It maintains tighter continuity with the draft's emphasis on data limitations (the opening sentence explicitly mentions 'annotated genomic, transcriptomic, and proteomic data remain limited'), making the scarcity of paired multi-omic measurements a natural extension of the data scarcity theme already established. (2) Output A's claim about 'dimensionality, noise structure, and biological interpretation' is more generic and somewhat overlaps with the draft's earlier discussion of biological data being 'noisy, incomplete, or biased'—it doesn't advance the argument as distinctly. (3) Output B introduces a specific, concrete constraint (paired multi-omic measurements) that explains *why* cross-omics integration fails, which is more actionable and academically precise than Output A's broader framing. (4) Output B's 'As a result' clause clearly traces the consequence, strengthening the causal argument flow. Both are usable, but B fits the draft's established concerns about data scarcity more precisely and provides more specific analytical value.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

Beyond computational cost, the integration of heterogeneous omics layers remains constrained by the scarcity of paired multi-omic measurements, since most public datasets profile only a single modality for a given sample. As a result, models trained to align genomic, transcriptomic, and proteomic representations are typically limited to the few cohorts in which such matched data exist.

Passages supplied to the Evidence version

Card 1 · DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome

Decoding the linguistic intricacies of the genome is a crucial problem in biology, and pre-trained foundational models such as DNABERT and Nucleotide Transformer have made significant strides in this area. Existing works have largely hinged on k-mer, fixed-length permutations of A, T, C, and G, as t…

Read full passage excerpt

Decoding the linguistic intricacies of the genome is a crucial problem in biology, and pre-trained foundational models such as DNABERT and Nucleotide Transformer have made significant strides in this area. Existing works have largely hinged on k-mer, fixed-length permutations of A, T, C, and G, as the token of the genome language due to its simplicity. However, we argue that the computation and sample inefficiencies introduced by k-mer tokenization are primary obstacles in developing large genome foundational models. We provide conceptual and empirical insights into genome tokenization, building on which we propose to replace k-mer tokenization with Byte Pair Encoding (BPE), a statistics-based data compression algorithm that constructs tokens by iteratively merging the most frequent co-occurring genome segment in the corpus. We demonstrate that BPE not only overcomes the limitations of k-mer tokenization but also benefits from the computational efficiency of non-overlapping tokenizatio …

Card 2 · Generative language models on nucleotide sequences of human genes

Language models, especially transformer-based ones, have achieved colossal success in natural language processing. To be precise, studies like BERT for natural language understanding and works like GPT-3 for natural language generation are very important. If we consider DNA sequences as a text writt…

Read full passage excerpt

Language models, especially transformer-based ones, have achieved colossal success in natural language processing. To be precise, studies like BERT for natural language understanding and works like GPT-3 for natural language generation are very important. If we consider DNA sequences as a text written with an alphabet of four letters representing the nucleotides, they are similar in structure to natural languages. This similarity has led to the development of discriminative language models such as DNABERT in the field of DNA-related bioinformatics. To our knowledge, however, the generative side of the coin is still largely unexplored. Therefore, we have focused on the development of an autoregressive generative language model such as GPT-3 for DNA sequences. Since working with whole DNA sequences is challenging without extensive computational resources, we decided to conduct our study on a smaller scale and focus on nucleotide sequences of human genes, i.e. unique parts of DNA with spe …

Card 3 · dnaGrinder: a lightweight and high-capacity genomic foundation model

The task of understanding and interpreting the complex information encoded within genomic sequences remains a grand challenge in biological research and clinical applications. In this context, recent advancements in large language model research have led to the development of both encoder-only and d…

Read full passage excerpt

The task of understanding and interpreting the complex information encoded within genomic sequences remains a grand challenge in biological research and clinical applications. In this context, recent advancements in large language model research have led to the development of both encoder-only and decoder-only foundation models designed to decode intricate information in DNA sequences. However, several issues persist, particularly regarding the efficient management of long-range dependencies inherent in genomic sequences, the effective representation of nucleotide variations, and the considerable computational costs associated with large model architectures and extensive pretraining datasets. Current genomic foundation models often face a critical tradeoff: smaller models with mediocre performance versus large models with improved performance. To address these challenges, we introduce dnaGrinder, a unique and efficient genomic foundation model. dnaGrinder excels at managing long-range …

Card 4 · DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings

LR) strategy. Empirical results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios. For example, it identifies twice more species from a mixture of unlabeled genomic sequences, doubles the Adjusted Rand Index (ARI) in species clustering, and outperf…

Read full passage excerpt

LR) strategy. Empirical results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios. For example, it identifies twice more species from a mixture of unlabeled genomic sequences, doubles the Adjusted Rand Index (ARI) in species clustering, and outperforms the top baseline's performance in 10-shot species classification with just a 2-shot training. Model, codes, and data is publicly available at https://github.com/MAGlCS-LAB/DNABERT_S.

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

Beyond computational cost, the integration of heterogeneous omics layers introduces challenges of alignment and representation. Genomic, transcriptomic, and proteomic modalities differ in dimensionality, noise structure, and biological interpretation, and effective cross-omics models must reconcile these differences without losing modality-specific signal.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.