← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWith cards better166 / 280 · beu-b5be56b1263c5ae9

A Review of Statistical Methods for Handling Nonignorable Missing Data using Instrument Approach

Statistics · 2509.14520v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

A Review of Statistical Methods for Handling Nonignorable Missing Data using Instrument Approach

3 Conclusion and Discussion

…nsion reduction (SDR), including studies by Hu et al., (2010) , Ding and Wang, (2011) , Deng and Wang, (2017) , and Li et al., (2017) . Another line of research addresses the variable selection problem through feature screening, as seen in the work of Lai et al., (2017) , Wang and Li, (2018) , Tang et al., (2019) , Zambom and Matthews, (2019) and Ni et al., (2019) When high-dimensional data contains nonignorable missing values, some studies have also explored the problem from the perspective of dimension reduction or feature screening; see, for example, Wang, (2019) and Zhang et al., (2019) .

…eyed a range of methodological advances in handling nonignorable missing data through the use of instrumental variables, as illustrated in Figure 1 . We have focused on two main semiparametric pathways: one that assumes a parametric data model with a nonparametric propensity function, and another that assumes a parametric propensity model with a nonparametric data distribution. Each approach offers distinct advantages in terms of identifiability and estimation, and both have spurred significant research in areas such as pseudo-likelihood estimation, model selection, and doubly robust methods.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: With cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = without cards · B = with cards

Both outputs provide coherent continuations, but B better serves the manuscript's needs. The draft concludes a literature review by summarizing two semiparametric pathways and their research fruits (pseudo-likelihood, model selection, doubly robust methods). Output B naturally extends this by identifying a specific gap—the selection of instruments rather than assuming they're given—and connects directly to the source material about instrument/variable/model selection with nonignorable nonresponse. This mirrors the paper's own trajectory from 'existing work assumes X, we address how to find X.' Output A provides a generic limitations paragraph (identifiability assumptions, finite-sample sensitivity) that could apply to almost any instrumental variable method; it lacks the specific forward momentum and connection to the manuscript's apparent research contribution. B's scope is tighter and more purposeful: it identifies a concrete open problem that the paper likely intends to address, making it a better fit as a transitional paragraph.

Evidence

Same model and position · retrieved passages supplied · first output

Preferred by blind review

Several directions remain open for extending the instrument approach. One concerns the selection of instruments themselves: existing methods typically assume that a valid instrument is given, and recent work has begun to address how an instrument can be searched for from a set of candidate covariates while performing variable and model selection simultaneously.

Passages supplied to the Evidence version

Card 1 · Selective machine learning of doubly robust functionals

While model selection is a well-studied topic in parametric and nonparametric regression or density estimation, selection of possibly high-dimensional nuisance parameters in semiparametric problems is far less developed. In this paper, we propose a selective machine learning framework for making inf…

Read full passage excerpt

While model selection is a well-studied topic in parametric and nonparametric regression or density estimation, selection of possibly high-dimensional nuisance parameters in semiparametric problems is far less developed. In this paper, we propose a selective machine learning framework for making inferences about a finite-dimensional functional defined on a semiparametric model, when the latter admits a doubly robust estimating function and several candidate machine learning algorithms are available for estimating the nuisance parameters. We introduce a new selection criterion aimed at bias reduction in estimating the functional of interest based on a novel definition of pseudo-risk inspired by the double robustness property. Intuitively, the proposed criterion selects a pair of learners with the smallest pseudo-risk, so that the estimated functional is least sensitive to perturbations of a nuisance parameter. We establish an oracle property for a multi-fold cross-validation version of …

Card 2 · Robust Estimation and Model Selection for the Controlled Directed Effect with Unmeasured Mediator-Outcome Confounders

Controlled Direct Effect (CDE) is one of the causal estimands used to evaluate both exposure and mediation effects on an outcome. When there are unmeasured confounders existing between the mediator and the outcome, the ordinary identification assumption does not work. In this manuscript, we consider…

Read full passage excerpt

Controlled Direct Effect (CDE) is one of the causal estimands used to evaluate both exposure and mediation effects on an outcome. When there are unmeasured confounders existing between the mediator and the outcome, the ordinary identification assumption does not work. In this manuscript, we consider an identification condition to identify CDE in the presence of unmeasured confounders. The key assumptions are: 1) the random allocation of the exposure, and 2) the existence of instrumental variables directly related to the mediator. Under these conditions, we propose a novel doubly robust estimation method, which work well if either the propensity score model or the baseline outcome model is correctly specified. Additionally, we propose a Generalized Information Criterion (GIC)-based model selection criterion for CDE that ensures model selection consistency. Our proposed procedure and related methods are applied to both simulation and real datasets to confirm the performance of these meth …

Card 3 · Nonparametric regression on hidden $\Phi$-mixing variables: Identifiability and consistency of a pseudo-likelihood based estimation procedure

This paper outlines a new nonparametric estimation procedure for unobserved $\Phi$-mixing processes. It is assumed that the only information on the stationary hidden states $(X_{k})_{k\ge0}$ is given by the process $(Y_{k})_{k\ge0}$, where $Y_{k}$ is a noisy observation of $f_{\star}(X_{k})$. The pa…

Read full passage excerpt

This paper outlines a new nonparametric estimation procedure for unobserved $\Phi$-mixing processes. It is assumed that the only information on the stationary hidden states $(X_{k})_{k\ge0}$ is given by the process $(Y_{k})_{k\ge0}$, where $Y_{k}$ is a noisy observation of $f_{\star}(X_{k})$. The paper introduces a maximum pseudo-likelihood procedure to estimate the function $f_{\star}$ and the distribution $\nu_{b,\star}$ of $(X_{0},\ldots,X_{b-1})$ using blocks of observations of length $b$. The identifiability of the model is studied in the particular cases $b=1$ and $b=2$ and the consistency of the estimators of $f_{\star}$ and of $\nu_{b,\star}$ as the number of observations grows to infinity is established.

Card 4 · Doubly Robust Criterion for Causal Inference

The semiparametric estimation approach, which includes inverse-probability-weighted and doubly robust estimation using propensity scores, is a standard tool in causal inference, and it is rapidly being extended in various directions. On the other hand, although model selection is indispensable in st…

Read full passage excerpt

The semiparametric estimation approach, which includes inverse-probability-weighted and doubly robust estimation using propensity scores, is a standard tool in causal inference, and it is rapidly being extended in various directions. On the other hand, although model selection is indispensable in statistical analysis, an information criterion for selecting an appropriate regression structure has just started to be developed. In this paper, based on the original definition of Akaike information criterion (AIC; \citealt{Aka73}), we derive an AIC-type criterion for propensity score analysis. Here, we define a risk function based on the Kullback-Leibler divergence as the cornerstone of the information criterion and treat a general causal inference model that is not necessarily a linear one. The causal effects to be estimated are those in the general population, such as the average treatment effect on the treated or the average treatment effect on the untreated. In light of the fact that th …

Card 5 · Instrument, Variable and Model Selection with Nonignorable Nonresponse

With nonignorable nonresponse, an effective method to construct valid estimators of population parameters is to use a covariate vector called instrument that can be excluded from the nonresponse propensity but are still useful covariate even when other covariates are conditioned. The existing work i…

Read full passage excerpt

With nonignorable nonresponse, an effective method to construct valid estimators of population parameters is to use a covariate vector called instrument that can be excluded from the nonresponse propensity but are still useful covariate even when other covariates are conditioned. The existing work in this approach assumes such an instrument is given, which is frequently not the case in applications. In this paper we investigate how to search for an instrument from a given set of covariates. The method for estimation we apply is the pseudo likelihood proposed by Tang et al. (2003) and Zhao and Shao (2015), which assumed that an instrument is given and the distribution of response given covariates is parametric and the propensity is nonparametric. Thus, in addition to the challenge of searching an instrument, we also need to do variable and model selection simultaneously. We propose a method for instrument, variable, and model selection and show that our method produces consistent instru …

Model-reported card use

Card 5 · fact

Affected output: existing methods typically assume that a valid instrument is given, and recent work has begun to address how an instrument can be searched for from a set of candidate covariates while performing variable and model selection simultaneously

Model explanation: This row supplied the concrete prior-work limitation and the corresponding recent development, which the conclusion uses to name instrument selection as an open direction rather than a generic future-work statement.

No Evidence

Same model and position · no retrieved passages · first output

Not preferred in this comparison

Despite these advances, several challenges remain. The identifiability of instrument-based estimators typically relies on assumptions that are difficult to verify from observed data, and the finite-sample performance of these methods can be sensitive to the strength of the association between the instrument and the missingness mechanism.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Output A is grounded in the sources and draft. The source 'Instrument, Variable and Model Selection with Nonignorable Nonresponse' explicitly states that 'The existing work in this approach assumes such an instrument is given' and 'we investigate how to search for an instrument from a given set of covariates' while doing 'variable and model selection simultaneously.' This matches Output A's claims exactly. Output B introduces multiple unsupported claims: (1) 'identifiability... relies on assumptions that are difficult to verify from observed data' — no source discusses verifiability difficulty; (2) 'finite-sample performance... can be sensitive to the strength of the association between the instrument and the missingness mechanism' — no source discusses finite-sample sensitivity to instrument strength. These are plausible-sounding but unsupported field extrapolations. The draft discusses 'identifiability and estimation' as offering 'distinct advantages,' not difficulties, and no source mentions sensitivity to association strength or verification problems.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.