← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better24 / 280 · beu-c0f5cda403b8fab0

An Introduction to Proximal Causal Learning

Statistics · 2009.10982v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

An Introduction to Proximal Causal Learning

7 Discussion

…sive least-squares yields results suggests a stronger protective effect β ^ a ¯ = \widehat{\beta}_{\overline{a}}= -0.37 (-0.67, -0.13), with strong evidence of confounding bias ( η ^ w , 0 , η ^ w , 1 ) = ( 0.785 , 0.524 ) \left(\widehat{\eta}_{w,0},\widehat{\eta}_{w,1}\right)=(0.785,0.524) with corresponding 95% confidence intervals ( 0.50 , 1.1 ) (0.50,1.1) and ( 0.33 , 0.71 ) (0.33,0.71) respectively. These results reinforce understanding of potential protective effects of MTX on disease progression. Results from this analysis are summarized in tables provided in the Supplemental Appendix.

…ying confounding factors. Our proximal causal learning framework provides a formal potential outcome framework under which one can articulate conditions to identify causal effects from proxies. We have described proximal g-formula and proximal g-computation algorithm for estimation in point treatment and time-varying treatment settings. The proximal approach is closely related to negative control methods recently proposed for detection and sometimes estimation of point treatment interventions 12,13,16,22 . We refer the reader to Shi et al 23 for a recent review of negative control literature.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs provide usable completions, but they serve different argumentative purposes at this position. The draft concludes a results section with 'Results from this analysis are summarized in tables provided in the Supplemental Appendix,' then moves to discussing confounding factors and the proximal causal learning framework, ending with a literature pointer to Shi et al. for negative control review. This is a natural transition point where the manuscript could either: (A) note limitations/future work, or (B) broaden to related methodological connections. Output A identifies 'open extensions' including unmeasured confounding of treatment-proxy relationships and efficiency theory. However, the draft has already presented results and situated the proximal approach relative to negative controls. Introducing new limitations about 'treatment-proxy relationship' confounding and 'efficiency theory' feels somewhat abrupt and speculative—the draft hasn't raised these as gaps, and this risks introducing concerns that weren't previously motivated. Output B connects the proximal framework to 'broader developments in causal inference with unmeasured confounding,' specifically mentioning instrumental variable methods for time-varying treatments and doubly robust estimation under misspecification. This fits better: the preceding text explicitly mentions 'point treatment and time-varying treatment settings' and cites negative control literature. The sources confirm instrumental variable methods for time-varying treatments (Robins 1997 extension) and doubly robust estimation under misspecification are active, relevant methodological areas. This broadening gesture is a standard academic move after presenting a specific framework and before concluding, and it naturally follows the literature review pointer just made. Output B maintains better continuity with the draft's existing trajectory (framework presentation → relation to negative controls → broader methodological context) without introducing unmotivated limitations. Both are accurate and well-scoped, but B's fit to the current argumentative position is superior.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

The proximal framework also connects to broader developments in causal inference with unmeasured confounding, including instrumental variable methods for time-varying treatments and doubly robust estimation strategies that remain valid under partial model misspecification.

Passages supplied to the Evidence version

Card 1 · Multiply Robust Causal Inference with Double Negative Control Adjustment for Categorical Unmeasured Confounding

Unmeasured confounding is a threat to causal inference in observational studies. In recent years, use of negative controls to mitigate unmeasured confounding has gained increasing recognition and popularity. Negative controls have a longstanding tradition in laboratory sciences and epidemiology to r…

Read full passage excerpt

Unmeasured confounding is a threat to causal inference in observational studies. In recent years, use of negative controls to mitigate unmeasured confounding has gained increasing recognition and popularity. Negative controls have a longstanding tradition in laboratory sciences and epidemiology to rule out non-causal explanations, although they have been used primarily for bias detection. Recently, Miao et al. (2018) have described sufficient conditions under which a pair of negative control exposure and outcome variables can be used to nonparametrically identify the average treatment effect (ATE) from observational data subject to uncontrolled confounding. In this paper, we establish nonparametric identification of the ATE under weaker conditions in the case of categorical unmeasured confounding and negative control variables. We also provide a general semiparametric framework for obtaining inferences about the ATE while leveraging information about a possibly large number of measured …

Card 2 · A Practical Introduction to Bayesian Estimation of Causal Effects: Parametric and Nonparametric Approaches

In this paper we reviewed causal effect estimation from a Bayesian perspective in point-treatment and time-varying treatment settings. For the latter, we outlined how to estimate causal effects of both static and dynamic treatment regimes. Both parameteric and nonparametric settings were discussed.…

Read full passage excerpt

In this paper we reviewed causal effect estimation from a Bayesian perspective in point-treatment and time-varying treatment settings. For the latter, we outlined how to estimate causal effects of both static and dynamic treatment regimes. Both parameteric and nonparametric settings were discussed. Along the way, we discussed the utility of priors both for providing interpretable shrinkage and also for conducting causal sensitivity analyses. Throughout, we emphasize that the ad-hoc procedures we often use correspond to strongly informative priors. Throughout, we have highlighted various BNP techniques used for causal estimation in the literature. We hope that these surveys will be useful literature overviews that can serve as a starting point for those who want to delve further into these methods.

Card 3 · Instrumental Variable Estimation of Marginal Structural Mean Models for Time-Varying Treatment

Robins 1997 introduced marginal structural models (MSMs), a general class of counterfactual models for the joint effects of time-varying treatment regimes in complex longitudinal studies subject to time-varying confounding. In his work, identification of MSM parameters is established under a sequent…

Read full passage excerpt

Robins 1997 introduced marginal structural models (MSMs), a general class of counterfactual models for the joint effects of time-varying treatment regimes in complex longitudinal studies subject to time-varying confounding. In his work, identification of MSM parameters is established under a sequential randomization assumption (SRA), which rules out unmeasured confounding of treatment assignment over time. We consider sufficient conditions for identification of the parameters of a subclass, Marginal Structural Mean Models (MSMMs), when sequential randomization fails to hold due to unmeasured confounding, using instead a time-varying instrumental variable. Our identification conditions require that no unobserved confounder predicts compliance type for the time-varying treatment. We describe a simple weighted estimator and examine its finite-sample properties in a simulation study. We apply the proposed estimator to examine the effect of delivery hospital on neonatal survival probability …

Card 4 · Honest data-adaptive inference for the average treatment effect under model misspecification using penalised bias-reduced double-robust estimation

The presence of confounding by high-dimensional variables complicates estimation of the average effect of a point treatment. On the one hand, it necessitates the use of variable selection strategies or more general data-adaptive high-dimensional statistical methods. On the other hand, the use of suc…

Read full passage excerpt

The presence of confounding by high-dimensional variables complicates estimation of the average effect of a point treatment. On the one hand, it necessitates the use of variable selection strategies or more general data-adaptive high-dimensional statistical methods. On the other hand, the use of such techniques tends to result in biased estimators with a non-standard asymptotic behaviour. Double-robust estimators are vital for offering a resolution because they possess a so-called small bias property (Newey et al., 2004). This means that their bias vanishes faster than the bias in the nuisance parameter estimators when the relevant smoothing parameter goes to zero, making their performance less sensitive to smoothing (Chernozhukov et al., 2016). This property has been exploited to achieve valid (uniform) inference of the average causal effect when data-adaptive estimators of the propensity score and conditional outcome mean both converge to their respective truths at sufficiently fast …

Card 5 · A Practical Introduction to Bayesian Estimation of Causal Effects: Parametric and Nonparametric Approaches

In this paper we reviewed causal effect estimation from a Bayesian perspective in point-treatment and time-varying treatment settings. For the latter, we outlined how to estimate causal effects of both static and dynamic treatment regimes. Both parameteric and nonparametric settings were discussed.…

Read full passage excerpt

In this paper we reviewed causal effect estimation from a Bayesian perspective in point-treatment and time-varying treatment settings. For the latter, we outlined how to estimate causal effects of both static and dynamic treatment regimes. Both parameteric and nonparametric settings were discussed. Along the way, we discussed the utility of priors both for providing interpretable shrinkage and also for conducting causal sensitivity analyses. Throughout, we emphasize that the ad-hoc procedures we often use correspond to strongly informative priors. Throughout, we have highlighted various BNP techniques used for causal estimation in the literature. We hope that these surveys will be useful literature overviews that can serve as a starting point for those who want to delve further into these methods.

Model-reported card use

Card 3 · direction

Affected output: instrumental variable methods for time-varying treatments

Model explanation: This row supplied the specific prior-work direction of instrumental variable methods for time-varying treatments as a related approach to unmeasured confounding.

Card 4 · direction

Affected output: doubly robust estimation strategies that remain valid under partial model misspecification

Model explanation: This row supplied the specific prior-work direction of doubly robust estimation retaining validity under partial misspecification as a related approach.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

Several extensions of the proximal framework remain open, including settings with unmeasured confounding of the treatment-proxy relationship and the development of efficiency theory for the proposed estimators.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the no-Evidence version. Its original rationale appears below.

Read the original model judgment

Output A introduces 'instrumental variable methods for time-varying treatments' and 'doubly robust estimation strategies that remain valid under partial model misspecification' as connections to the proximal framework. However, the draft and sources do not establish these connections. Source 3 discusses instrumental variables for time-varying treatments but in the context of marginal structural mean models, not proximal causal learning. Source 4 discusses doubly robust estimation but does not mention proximal methods. The draft mentions negative control methods as related to proximal approaches, but not IV methods or the specific doubly robust strategies in A. These are presented as factual connections ('also connects to') without source support. Output B discusses extensions of the proximal framework that are natural methodological open questions—unmeasured confounding of treatment-proxy relationships and efficiency theory—which are grounded in the draft's description of what the framework provides (identification conditions, estimation algorithms) without making unsupported factual claims about what has been done. B's claims about 'open' extensions are appropriately speculative and do not assert existence of unmentioned methods.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.