← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better38 / 280 · beu-4c6890f14f1f4645

An Overview and Recent Developments in the Analysis of Multistate Processes

Statistics · 2502.06492v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

An Overview and Recent Developments in the Analysis of Multistate Processes

7 Discussion

The R package simMSM ( Reulen, 2022 ) simulates event histories for multistate models. It enables the generation of event histories featuring potentially non-linear baseline hazard functions, as well as nonlinear time-dependent or time-independent covariates’ effect, while also accounting for dependencies on past history. The random generation of event histories is achieved through inversion sampling applied to cumulative all-cause hazard rate functions.

…ty-LTRC ) utilizes a pseudo-likelihood approach for marginalized Cox models outlined in Section 5.1.1, employing a gamma frailty. On the other hand, semicompAFT (available at https://github.com/leakats/semicompaft ) implements a semi-parametric AFT model under the multiplicative frailty setting discussed in Section 5.1.2, also employing a gamma frailty distribution. frailtypack ( Rondeau et al., 2019 ) is a package focusing particularly on handling frailty models and is versatile for time-to-event data in complex scenarios, including multistate models with recurrent events or competing risks.

In this paper, we have presented various approaches to modeling multistate processes, with a focus on both intensity-based and marginal models. Sections

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

The draft is an academic paper conclusion/transition sentence that needs to specify which sections covered which topics. Output A provides a clean, structurally appropriate continuation: it assigns sections 2–5 to intensity-based models and Section 6 to marginal approaches, which directly mirrors the 'both intensity-based and marginal models' dichotomy established in the draft. This is internally consistent and requires no repair. Output B claims 'Sections 2–6 outlined the key methodological frameworks, and Section 7 has surveyed the software implementations currently available for fitting these models.' This introduces a concrete claim about Section 7 surveying software implementations that is unsupported by the draft context and contradicted by the sources. The sources mention software packages in the context of joint models (first source) and pammtools (second source), but there is no indication in the draft that the paper itself contains a Section 7 surveying software implementations. This constitutes an invented structural claim about the manuscript's content, making it materially inaccurate and requiring substantial repair if inserted. The unsupported claim about Section 7's content is material to usability because it misrepresents the paper's organization.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

2–5 covered the intensity-based framework, while Section 6 addressed marginal approaches.

Passages supplied to the Evidence version

Card 1 · Joint models with multiple longitudinal outcomes and a time-to-event outcome: a corrected two-stage approach

Abstract Joint models for longitudinal and survival data have gained a lot of attention in recent years, with the development of myriad extensions to the basic model, including those which allow for multivariate longitudinal data, competing risks and recurrent events. Several software packages are n…

Read full passage excerpt

Abstract Joint models for longitudinal and survival data have gained a lot of attention in recent years, with the development of myriad extensions to the basic model, including those which allow for multivariate longitudinal data, competing risks and recurrent events. Several software packages are now also available for their implementation. Although mathematically straightforward, the inclusion of multiple longitudinal outcomes in the joint model remains computationally difficult due to the large number of random effects required, which hampers the practical application of this extension. We present a novel approach that enables the fitting of such models with more realistic computational times. The idea behind the approach is to split the estimation of the joint model in two steps: estimating a multivariate mixed model for the longitudinal outcomes and then using the output from this model to fit the survival submodel. So-called two-stage approaches have previously been proposed and …

Card 2 · pammtools: Piece-wise exponential Additive Mixed Modeling tools

Piecewise exponential additive mixed models (PAMMs) provide a flexible framework for analyzing censored and truncated time-to-event data, bridging classical hazard-based modeling with modern regression techniques. They enable the estimation of complex covariate effects, including non-linear and time…

Read full passage excerpt

Piecewise exponential additive mixed models (PAMMs) provide a flexible framework for analyzing censored and truncated time-to-event data, bridging classical hazard-based modeling with modern regression techniques. They enable the estimation of complex covariate effects, including non-linear and time-varying (cumulative) effects, and naturally incorporate time-varying covariates. Moreover, PAMMs are applicable across a wide range of survival settings, including non-proportional hazards, recurrent events, competing risks, and multi-state analyses. This article introduces the pammtools package, which facilitates data transformation, estimation, and interpretation for PAMMs within a unified workflow. The package provides a comprehensive, user-friendly, and extensible interface covering the full modeling pipeline, from data transformation to estimation and visualization. In addition, simulation-based inference allows the calculation of confidence intervals for arbitrary quantities of intere …

Card 3 · Penalized Estimation of Frailty-Based Illness–Death Models for Semi-Competing Risks

Semi-competing risks refer to the time-to-event analysis setting, where the occurrence of a non-terminal event is subject to whether a terminal event has occurred, but not vice versa. Semi-competing risks arise in a broad range of clinical contexts, including studies of preeclampsia, a condition tha…

Read full passage excerpt

Semi-competing risks refer to the time-to-event analysis setting, where the occurrence of a non-terminal event is subject to whether a terminal event has occurred, but not vice versa. Semi-competing risks arise in a broad range of clinical contexts, including studies of preeclampsia, a condition that may arise during pregnancy and for which delivery is a terminal event. Models that acknowledge semi-competing risks enable investigation of relationships between covariates and the joint timing of the outcomes, but methods for model selection and prediction of semi-competing risks in high dimensions are lacking. Moreover, in such settings researchers commonly analyze only a single or composite outcome, losing valuable information and limiting clinical utility-in the obstetric setting, this means ignoring valuable insight into timing of delivery after preeclampsia has onset. To address this gap, we propose a novel penalized estimation framework for frailty-based illness-death multi-state mo …

Card 4 · MENSA: A Multi-Event Network for Survival Analysis with Trajectory-based Likelihood Estimation

Most existing time-to-event methods focus on either single-event or competing-risks settings, leaving multi-event scenarios relatively underexplored. In many healthcare applications, for example, a patient may experience multiple clinical events, that can be non-exclusive and semi-competing. A commo…

Read full passage excerpt

Most existing time-to-event methods focus on either single-event or competing-risks settings, leaving multi-event scenarios relatively underexplored. In many healthcare applications, for example, a patient may experience multiple clinical events, that can be non-exclusive and semi-competing. A common workaround is to train independent single-event models for such multi-event problems, but this approach fails to exploit dependencies and shared structures across events. To overcome these limitations, we propose MENSA (Multi-Event Network for Survival Analysis), a deep learning model that jointly learns flexible time-to-event distributions for multiple events, whether competing or co-occurring. In addition, we introduce a novel trajectory-based likelihood term that captures the temporal ordering between events. Across four multi-event datasets, MENSA improves predictive performance over many state-of-the-art baselines. Source code is available at https://github.com/thecml/mensa.

Card 5 · Nonparametric efficient causal estimation of the intervention-specific expected number of recurrent events with continuous-time targeted maximum likelihood and highly adaptive lasso estimation

In our case, the entire data-generating distribution is parameterized by intensities, and we can carry out targeting according to the log-likelihood loss function and least favorable submodel presented by Rytgaard et al. (2022, Section 5.2) . A similar procedure was applied for competing risks setti…

Read full passage excerpt

In our case, the entire data-generating distribution is parameterized by intensities, and we can carry out targeting according to the log-likelihood loss function and least favorable submodel presented by Rytgaard et al. (2022, Section 5.2) . A similar procedure was applied for competing risks settings in Rytgaard et al. (2021) . We will assume for now that we have (initial) estimators for all clever weights and covariates from Theorem 1 . Estimating the clever covariates poses an additional challenge in recurrent events settings as compared to the competing risks setting, and we return to this specifically in Section 6 .

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

2–6 outlined the key methodological frameworks, and Section 7 has surveyed the software implementations currently available for fitting these models.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.