← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better90 / 280 · beu-02aa335b906ad03b

Causality for Earth Science -- A Review on Time-series and Spatiotemporal Causality Methods

Climate and environmental science · 2404.05746v2

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Causality for Earth Science -- A Review on Time-series and Spatiotemporal Causality Methods

1 Introduction

… extreme weather events, sea level rise, teleconnections, etc. Our survey paper will benefit the Earth science community interested in taking an AI-driven approach to study the causality of different dynamic and thermodynamic processes as we present the open challenges and opportunities in performing causality-based Earth science study. It will also serve as a primer for data science researchers interested in data-driven causal study as we share a holistic list of resources, such as Earth science datasets (synthetic, simulated and observational data) and open source tools for causal analysis.

… in data. Earth science can benefit greatly from causal methods as researchers have started utilizing causality to understand the complex interactions leading to climate change, long term weather patterns, extreme weather events, etc. This paper provides a primer to causal inference and causal discovery for different Earth science domains with some of the existing applications and open questions for future research. The paper also comprises a holistic list of available tools and datasets that can help both data science and Earth science communities in performing causal analysis on Earth data.

Earth science is a broad field that studies the Earth’s composition, processes, and physical characteristics ( Smith and Pun, 2013 ) . It encompasses several sub-disciplines

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

Both outputs provide a grammatically correct and factually accurate continuation listing standard sub-disciplines of Earth science. Output B is preferable because it adds meaningful value with the clause 'each of which investigates distinct components of the Earth system'—this reinforces the 'broad field' framing from the draft's opening, explains why these sub-disciplines matter, and creates a more complete definitional move. Output A stops abruptly after listing examples, leaving the sentence less purposeful. The additional text in B does not introduce redundancy or speculation; it tightens the logical arc of the sentence without requiring any repair to the existing draft.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

such as geology, meteorology, oceanography, and atmospheric science.

Passages supplied to the Evidence version

Card 1 · A Perspective on Gaussian Processes for Earth Observation

Some important challenges in causal inference for the Earth science are still to be solved: how to scale GP models to deal with millions of points, missing data and time aggregation as well as time sub-sampling, and complex spatial-temporal dependency structures. Testing scientific hypotheses, compa…

Read full passage excerpt

Some important challenges in causal inference for the Earth science are still to be solved: how to scale GP models to deal with millions of points, missing data and time aggregation as well as time sub-sampling, and complex spatial-temporal dependency structures. Testing scientific hypotheses, comparing model-vs-data causal graphs, and assessing the impacts of extreme events, are just some exciting further avenues of research.

Card 2 · Causal inference for process understanding in Earth sciences

There is growing interest in the study of causal methods in the Earth sciences. However, most applications have focused on causal discovery, i.e. inferring the causal relationships and causal structure from data. This paper instead examines causality through the lens of causal inference and how expe…

Read full passage excerpt

There is growing interest in the study of causal methods in the Earth sciences. However, most applications have focused on causal discovery, i.e. inferring the causal relationships and causal structure from data. This paper instead examines causality through the lens of causal inference and how expert-defined causal graphs, a fundamental from causal theory, can be used to clarify assumptions, identify tractable problems, and aid interpretation of results and their causality in Earth science research. We apply causal theory to generic graphs of the Earth system to identify where causal inference may be most tractable and useful to address problems in Earth Science, and avoid potentially incorrect conclusions. Specifically, causal inference may be useful when: (1) the effect of interest is only causally affected by the observed portion of the state space; or: (2) the cause of interest can be assumed to be independent of the evolution of the system's state; or: (3) the state space of the …

Card 3 · Causal Inference for Time series Analysis: Problems, Methods and Evaluation

Time series data is a collection of chronological observations which is generated by several domains such as medical and financial fields. Over the years, different tasks such as classification, forecasting, and clustering have been proposed to analyze this type of data. Time series data has been al…

Read full passage excerpt

Time series data is a collection of chronological observations which is generated by several domains such as medical and financial fields. Over the years, different tasks such as classification, forecasting, and clustering have been proposed to analyze this type of data. Time series data has been also used to study the effect of interventions over time. Moreover, in many fields of science, learning the causal structure of dynamic systems and time series data is considered an interesting task which plays an important role in scientific discoveries. Estimating the effect of an intervention and identifying the causal relations from the data can be performed via causal inference. Existing surveys on time series discuss traditional tasks such as classification and forecasting or explain the details of the approaches proposed to solve a specific task. In this paper, we focus on two causal inference tasks, i.e., treatment effect estimation and causal discovery for time series data, and provid …

Card 4 · Causal Inference in medicine and in health policy, a summary

A data science task can be deemed as making sense of the data or testing a hypothesis about it. The conclusions inferred from data can greatly guide us to make informative decisions. Big data has enabled us to carry out countless prediction tasks in conjunction with machine learning, such as identif…

Read full passage excerpt

A data science task can be deemed as making sense of the data or testing a hypothesis about it. The conclusions inferred from data can greatly guide us to make informative decisions. Big data has enabled us to carry out countless prediction tasks in conjunction with machine learning, such as identifying high risk patients suffering from a certain disease and taking preventable measures. However, healthcare practitioners are not content with mere predictions - they are also interested in the cause-effect relation between input features and clinical outcomes. Understanding such relations will help doctors treat patients and reduce the risk effectively. Causality is typically identified by randomized controlled trials. Often such trials are not feasible when scientists and researchers turn to observational studies and attempt to draw inferences. However, observational studies may also be affected by selection and/or confounding biases that can result in wrong causal conclusions. In this c …

Card 5 · A Perspective on Gaussian Processes for Earth Observation

Some important challenges in causal inference for the Earth science are still to be solved: how to scale GP models to deal with millions of points, missing data and time aggregation as well as time sub-sampling, and complex spatial-temporal dependency structures. Testing scientific hypotheses, compa…

Read full passage excerpt

Some important challenges in causal inference for the Earth science are still to be solved: how to scale GP models to deal with millions of points, missing data and time aggregation as well as time sub-sampling, and complex spatial-temporal dependency structures. Testing scientific hypotheses, comparing model-vs-data causal graphs, and assessing the impacts of extreme events, are just some exciting further avenues of research.

Model-reported card use

The model did not report that any card materially affected this completion.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

such as geology, meteorology, oceanography, and atmospheric science, each of which investigates distinct components of the Earth system.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

This case does not have both with-card continuation text and a model-reported use record. It remains among all 280 tests but does not enter the reported-use comparison.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.