← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better273 / 280 · beu-bd944843877752d6

The Missing Memory Hierarchy: Demand Paging for LLM Context Windows

Computer science · 2603.09023v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

The Missing Memory Hierarchy: Demand Paging for LLM Context Windows

body

Our work provides a specific mechanism (proxy-layer interposition with eviction policies) and the empirical production data that the field currently lacks. Prompt-level memory management.MemGPT [ 11] pi- oneered the OS virtual memory analogy: a FIFO message queue with recursive summarization at capacity, plus archival/recall databases accessible via tool calls. The model can save infor- mation but cannot explicitly release it; eviction is capacity- triggered (flush at 100%), not cost-driven. Contextual Mem- ory Virtualisation [13] models session history as a DAG with structurally lossless trimming that strips mechanical bloat (raw tool outputs, base64,

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs attempt to complete the sentence about CMV's trimming approach. The draft ends mid-phrase with 'raw tool outputs, base64,' clearly listing items that CMV strips. Output A directly continues this list with 'images, and metadata' which matches the source snippet exactly ('raw tool outputs, base64 images, and metadata'), then adds the quantitative claims about token reduction (20% mean, 86% max) which are also directly from the CMV source. This is a precise, source-grounded completion that fits the sentence structure and argument flow. Output B offers 'duplicated logs' which is not in the source and changes the list items, then pivots to a contrast about KV cache vs session history. While the KV cache distinction is arguably relevant to the broader paper's argument about proxy-layer interposition, it is not what the current sentence is doing—the sentence is describing CMV's specific trimming mechanism, not contrasting it with the authors' approach. Output B's completion requires a more abrupt thematic shift and introduces 'duplicated logs' without source support. Output A maintains coherence with the visible draft's purpose (enumerating what CMV strips) and uses the External Evidence row appropriately and honestly.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

images, and metadata) while preserving every user message and assistant response verbatim, reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead.

Passages supplied to the Evidence version

Card 1 · Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents

Borrowing from operat- ing system virtual memory, CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives that enable context reuse across independent parallel sessions. We introduce a three-pass structurally lossless trimming al- gori…

Read full passage excerpt

Borrowing from operat- ing system virtual memory, CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives that enable context reuse across independent parallel sessions. We introduce a three-pass structurally lossless trimming al- gorithm that preserves every user message and assistant response verbatim while reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead by strip- ping mechanical bloat such as raw tool outputs, base64 images, and metadata

Card 2 · A-MEM: Agentic Memory for LLM Agents

MemGPT (Packer et al., 2023) presents a novel virtual context management system drawing inspi- ration from traditional operating systems’ mem- ory hierarchies. The architecture implements a dual-tier structure: a main context (analogous to RAM) that provides immediate access during LLM inference, an…

Read full passage excerpt

MemGPT (Packer et al., 2023) presents a novel virtual context management system drawing inspi- ration from traditional operating systems’ mem- ory hierarchies. The architecture implements a dual-tier structure: a main context (analogous to RAM) that provides immediate access during LLM inference, and an external context (analogous to disk storage) that maintains information beyond the fixed context window. 4.4 Empricial Results In our empirical evaluation, we compared A-MEM with four competitive baselines including LoCoMo, ReadAgent, MemoryBank, and MemGPT on the LoCoMo dataset

Card 3 · Qwen2.5 Technical Report

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the h…

Read full passage excerpt

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well as multistage reinforcement learning. Post-training techniques enhance human preference, and notably improve long text generation, structural data analysis, and instruction following. To handle diverse and varied use cases effectively, we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base and instruction-tuned models, with quantized versions available. In addition, for ho …

Card 4 · Network-based Spatial Context Retrieval for Open-weight LLMs: A Faithfulness Benchmark for Grounded Geographic Reasoning

Large language models (LLMs) encode substantial latent geographic knowledge, yet they reason poorly over space and are unreliable when queried from coordinates alone. Useful behaviour emerges only when structured spatial context is supplied in the prompt. This raises a question geographic evaluation…

Read full passage excerpt

Large language models (LLMs) encode substantial latent geographic knowledge, yet they reason poorly over space and are unreliable when queried from coordinates alone. Useful behaviour emerges only when structured spatial context is supplied in the prompt. This raises a question geographic evaluation has left unexamined: once the right context is supplied, does the model reason from it, or override it with its own parametric recall? We take up this question with an open pipeline for network-based spatial context retriev-al. In it, the surroundings of a selected point are defined by the pedestrian street network, the area actually reachable on foot. Using only open data and open-weight models, the pipeline retrieves features from OpenStreetMap and the GHS-POP population grid, computes indicators over the network catchment in code, and injects them as a compact spatial brief. On this basis we build a faithfulness benchmark. It labels every claim a model makes by its source (grounded in th …

Card 5 · VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction

The rapid advancement of Large Multimodal Models (LMMs) for 2D images and videos has motivated extending these models to understand 3D scenes, aiming for human-like visual-spatial intelligence. Nevertheless, achieving deep spatial understanding comparable to human capabilities poses significant chal…

Read full passage excerpt

The rapid advancement of Large Multimodal Models (LMMs) for 2D images and videos has motivated extending these models to understand 3D scenes, aiming for human-like visual-spatial intelligence. Nevertheless, achieving deep spatial understanding comparable to human capabilities poses significant challenges in model encoding and data acquisition. Existing methods frequently depend on external depth sensors for geometry capture or utilize off-the-shelf algorithms for pre-constructing 3D maps, thereby limiting their scalability, especially with prevalent monocular video inputs and for time-sensitive applications. In this work, we introduce VLM-3R, a unified framework for Vision-Language Models (VLMs) that incorporates 3D Reconstructive instruction tuning. VLM-3R processes monocular video frames by employing a geometry encoder to derive implicit 3D tokens that represent spatial understanding. Leveraging our Spatial-Visual-View Fusion and over 200K curated 3D reconstructive instruction tunin …

Model-reported card use

Card 1 · fact

Affected output: images, and metadata) while preserving every user message and assistant response verbatim, reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead.

Model explanation: The row completes the author's unfinished enumeration of mechanical bloat and supplies the specific preservation and token-reduction figures for the cited prior work.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

duplicated logs) while preserving semantic content, but it operates on session history rather than the inference-time KV cache.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs complete the sentence about Contextual Memory Virtualisation [13], but B is more useful and precisely source-grounded. Output B directly quotes and paraphrases the CMV source: it correctly identifies what gets stripped ('images, and metadata' — completing the draft's interrupted list of 'raw tool outputs, base64,'), states the preservation guarantee ('every user message and assistant response verbatim'), and provides the exact quantitative results from the source ('reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead'). This is a rich, accurate completion that a researcher could use with minimal editing. Output A introduces 'duplicated logs' which is not in the source (the source says 'metadata', not 'duplicated logs'), and pivots to a comparison about 'inference-time KV cache' which is not mentioned in any source — while not strictly a false claim, it's an unsourced extrapolation that shifts focus away from what the draft was describing. The draft was in the middle of listing what CMV strips; B continues that list accurately and adds the results, while A changes the topic to a technical distinction not supported by the sources. B better preserves the draft's meaning and scope while being more substantially grounded in the provided source.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.