FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better35 / 280 · beu-1f090fd472eabddf

Shared Frailty Methods for Complex Survival Data: A Review of Recent Advances

Statistics · 2205.05322v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Shared Frailty Methods for Complex Survival Data: A Review of Recent Advances

1 INTRODUCTION

…oducing random variables to characterize unobserved dependence in the data is a popular practice in regression models. In the context of survival analysis (i.e., time-to-event outcome analysis), such models are known as shared frailty models: the cluster-specific random effect representing the unobserved common risk shared by cluster members is called the frailty. In these models, it is assumed that the random frailty factor captures all the within-cluster dependence, so that the failure times of cluster members are assumed independent given the observed covariates and the unobserved frailty.

…m observing the other event on this patient. Another classical example of competing risks is cause-specific mortality, such as death from heart disease, death from cancer, and death from other causes. Often, one event type is singled out as the event of interest, while the others may prevent the event of interest from occurring. For example, leukemia relapse or AIDS may be unobservable because the patient died before the diagnosis of these events. It is well known that caution is needed in estimating the distribution of the time to the event of interest in the presence of the competing risks.

Related to the competing risks setting the illness-death model setting, also known as the semi-competing risks setting. This setting

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs correctly define the semi-competing risks/illness-death model setting, but Output A is more accurate and complete. Output A correctly identifies that the intermediate (nonterminal) event can be censored by the terminal event, with the key constraint that the intermediate event is only observed if the terminal event hasn't occurred yet. This captures the asymmetry properly: the terminal event censors the intermediate event, but not vice versa. Output B states 'the reverse transition remains observable,' which is misleading phrasing—the 'reverse transition' from terminal to nonterminal event doesn't exist in this model; it's not that it's 'observable,' but that the terminal event can occur after the intermediate event. Output A's 'observed only if the terminal event has not yet occurred' precisely captures the censoring mechanism. Output B's 'while the reverse transition remains observable' could be read as suggesting a bidirectional model where both directions are possible transitions, which is incorrect for the standard illness-death model. The semi-competing risks setting specifically involves one direction of censoring (terminal censors nonterminal) but not the other, which Output A expresses more clearly.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

involves a terminal event that precludes the occurrence of a nonterminal event, while the reverse transition remains observable.

Passages supplied to the Evidence version

Card 1 · The choice to define competing risk events as censoring events and implications for causal inference

In failure-time settings, a competing risk event is any event that makes it impossible for the event of interest to occur. For example, death from cancer is a competing event for stroke because an individual cannot have a stroke once they have died of cancer. In follow-up studies, competing events c…

Read full passage excerpt

In failure-time settings, a competing risk event is any event that makes it impossible for the event of interest to occur. For example, death from cancer is a competing event for stroke because an individual cannot have a stroke once they have died of cancer. In follow-up studies, competing events cannot be prevented by design and can occur even in randomized experiments with full adherence to the assigned treatment strategies and no losses to follow-up.

Card 2 · A threshold‐free summary index of prediction accuracy for censored time to event data

Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores toscreenasymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecifiedfuturetim…

Read full passage excerpt

Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores toscreenasymptomatic population. The primary clinical interest is to predict the risk of having an event by a prespecifiedfuturetimet0. Accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cutoff value created barriers for practitioners and researchers. In this paper, we propose a threshold‐free summary index of positive predictive values that accommodates time‐dependent event status and competing risks. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between 2 risk scores for censored time to event data. We conduct a simul …

Card 3 · Siamese Survival Analysis with Competing Risks

Survival analysis in the presence of multiple possible adverse events, i.e., competing risks, is a pervasive problem in many industries (healthcare, finance, etc.). Since only one event is typically observed, the incidence of an event of interest is often obscured by other related competing events.…

Read full passage excerpt

Survival analysis in the presence of multiple possible adverse events, i.e., competing risks, is a pervasive problem in many industries (healthcare, finance, etc.). Since only one event is typically observed, the incidence of an event of interest is often obscured by other related competing events. This nonidentifiability, or inability to estimate true cause-specific survival curves from empirical data, further complicates competing risk survival analysis. We introduce Siamese Survival Prognosis Network (SSPN), a novel deep learning architecture for estimating personalized risk scores in the presence of competing risks. SSPN circumvents the nonidentifiability problem by avoiding the estimation of cause-specific survival curves and instead determines pairwise concordant time-dependent risks, where longer event times are assigned lower risks. Furthermore, SSPN is able to directly optimize an approximation to the C-discrimination index, rather than relying on well-known metrics which are …

Card 4 · Estimating causal effects in the presence of competing events using regression standardisation with the Stata command standsurv

When interested in a time-to-event outcome, competing events that prevent the occurrence of the event of interest may be present. In the presence of competing events, various statistical estimands have been suggested for defining the causal effect of treatment on the event of interest. Depending on…

Read full passage excerpt

When interested in a time-to-event outcome, competing events that prevent the occurrence of the event of interest may be present. In the presence of competing events, various statistical estimands have been suggested for defining the causal effect of treatment on the event of interest. Depending on the estimand, the competing events are either accommodated or eliminated, resulting in causal effects with different interpretation. The former approach captures the total effect of treatment on the event of interest while the latter approach captures the direct effect of treatment on the event of interest that is not mediated by the competing event. Separable effects have also been defined for settings where the treatment effect can be partitioned into its effect on the event of interest and its effect on the competing event through different causal pathways. We outline various causal effects that may be of interest in the presence of competing events, including total, direct and separable …

Card 5 · Competing risks regression for clustered survival data via the marginal additive subdistribution hazards model

A population‐averaged additive subdistribution hazards model is proposed to assess the marginal effects of covariates on the cumulative incidence function and to analyze correlated failure time data subject to competing risks. This approach extends the population‐averaged additive hazards model by a…

Read full passage excerpt

A population‐averaged additive subdistribution hazards model is proposed to assess the marginal effects of covariates on the cumulative incidence function and to analyze correlated failure time data subject to competing risks. This approach extends the population‐averaged additive hazards model by accommodating potentially dependent censoring due to competing events other than the event of interest. Assuming an independent working correlation structure, an estimating equations approach is outlined to estimate the regression coefficients and a new sandwich variance estimator is proposed. The proposed sandwich variance estimator accounts for both the correlations between failure times and between the censoring times, and is robust to misspecification of the unknown dependency structure within each cluster. We further develop goodness‐of‐fit tests to assess the adequacy of the additive structure of the subdistribution hazards for the overall model and each covariate. Simulation studies ar …

Model-reported card use

Card 1 · terminology

Affected output: a terminal event that precludes the occurrence of a nonterminal event

Model explanation: Row 1 supplies the field-native definition of an event that makes another event impossible, which licenses phrasing the terminal event as precluding the nonterminal event.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

involves an intermediate event that may be censored by a terminal event, so that the intermediate event is observed only if the terminal event has not yet occurred.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Both outputs attempt to define the semi-competing risks setting, but Output B is more accurate and precise. Output A incorrectly states that the intermediate event 'may be censored by a terminal event' and 'is observed only if the terminal event has not yet occurred.' This is backwards: in semi-competing risks, the nonterminal event (e.g., disease progression) can be observed even after it occurs, but the terminal event (e.g., death) prevents further observation of the nonterminal event after it occurs. More critically, Output A's 'observed only if the terminal event has not yet occurred' misrepresents the structure—the nonterminal event is observable until the terminal event occurs, not 'only if' it hasn't occurred at all. Output B correctly captures the asymmetry: the terminal event 'precludes the occurrence of a nonterminal event' (after death, disease progression cannot occur), while 'the reverse transition remains observable' (one can observe disease progression before death). This matches the standard definition of semi-competing risks where the terminal event censors the nonterminal event but not vice versa. Output B's phrasing is also more aligned with the sources' framing of competing events making the event of interest impossible.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.