FLOWING BENCHMARKS

两项评测,分别看写作质量和错误检测

覆盖 7 个学科、130 篇不同论文与 280 个固定写作点位:有 Evidence 的自动补全跨领域整体净胜 31 次;Evidence Card 对六类明确错误的检出率达 91.4%。

EVIDENCE CARD · 受控测试 · 独立复核

35 个已完成 Red 挑战中, 91.4% 被确认检出

Red 错误召回率

91.4%32/35

32/35 个获确认;若计入待复核项为 33/35。

36 个预设 Red 挑战kimi-k2.6 独立复核

Red 卡片精度

97.1%34/35

34/35 张获确认 · 1 张待复核 · 0 张未通过

发出的 Red 警报获确认34
46/48 个位置完成95.8%

Green 对照正确率

100.0%11/11

11/11 个 Green 对照判断正确;0/11 个被误报为 Red。

YELLOW 精度 · 自然样本

6/8 张经独立复核确认

75.0%

两项评测的任务和分母不同,结果不直接比较,也不合并计算。

评测口径

召回率、卡片精度、对照误报和完成率

四项指标分别计算。技术失败单独报告,不计作“未发现问题”。

46/48 个位置完成

Red 错误召回率

91.4%

32/35 个获确认;若计入待复核项为 33/35。

Red 卡片精度

97.1%

34/35 张获确认 · 1 张待复核 · 0 张未通过

Green 对照正确率

100.0%

11/11 个 Green 对照判断正确;0/11 个被误报为 Red。

测试完成率

95.8%

46/48 个位置完成;2 个技术失败单列,不当作“未发现问题”。

按错误类型拆分

六类错误的分项结果

数字、方向、因果关系、对象、适用范围和理论定义分别统计。

深色 = 已确认;浅色 = 待复核

数字错误

5/6

没有技术失败

方向与极性

6/6

没有技术失败

因果关系

4/5

1 个技术失败另列

对象错配

5/6

没有技术失败

适用范围

6/6

没有技术失败

理论定义

6/6

没有技术失败

YELLOW 精度 · 自然样本

75.0%+43.8 pp

6/8 张经独立复核确认

Yellow 卡优化前后对比

严格精度从 31.3% 提升到 75.0%。

优化前发出 16 张,其中 5 张获确认、1 张待复核、10 张未通过。完整复跑后发出 8 张,其中 6 张获确认、0 张待复核、2 张未通过;50/51 个位置完成,1 个技术失败单列。

完整复跑与独立逐卡复核均已完成

48 个公开案例

逐项查看输入、卡片与复核结果

每个案例包含待核查主张、冻结的原文依据、实际生成的卡片和独立复核结论。漏检与技术失败也保留在结果中。

显示 48 个案例
RED数字错误已确认检出

溶液法原位大面积制备钙钛矿光电薄膜成膜的同步辐射可视化结晶过程研究

该研究报道,单个器件面积约15 cm²的大面积全柔性三维钙钛矿太阳能电池转换效率提升至15.23%。

展开案例 ↓

待核查主张

该研究报道,单个器件面积约15 cm²的大面积全柔性三维钙钛矿太阳能电池转换效率提升至15.23%。

冻结的原文依据

转换效率提升至5.23%(单个器件面积约15 cm2)

实际生成的卡片 · RED

目标将该研究中约15 cm²全柔性三维钙钛矿太阳能电池的转换效率写为15.23%,而原文对同一器件报告的是5.23%,构成直接数值矛盾。

“基于各层均采用卷对卷 全溶液狭缝涂布方法制备的大面积全柔性三维钙钛矿薄膜太阳能电池转换效率提升至5.23% (单个器件面积 约15 cm2)”— 溶液法原位大面积制备钙钛矿光电薄膜成膜的同步辐射可视化结晶过程研究

独立复核 · valid

TARGET QUOTE声称'单个器件面积约15 cm²的大面积全柔性三维钙钛矿太阳能电池转换效率提升至15.23%',SOURCE QUOTE对同一器件(单个器件面积约15 cm²、大面积全柔性三维钙钛矿薄膜太阳能电池)报告转换效率为'5.23%'。两个数值15.23%与5.23%在相同实体(约15 cm²大面积全柔性三维钙钛矿太阳能电池)、相同关系(转换效率)上存在直接数值矛盾,构成直接反驳。

RED数字错误已确认检出

Genome-Scale CRISPR-Cas9 Knockout Screening in Human Cells

The GeCKO library targeted 18,080 human genes with 46,751 unique guide sequences.

展开案例 ↓

待核查主张

The GeCKO library targeted 18,080 human genes with 46,751 unique guide sequences.

冻结的原文依据

targeting 18,080 genes with 64,751 unique guide sequences

实际生成的卡片 · RED

The stated 46,751 unique guide sequences contradicts the source's 64,751 for the same GeCKO library. The source agrees with the stated 18,080 targeted genes.

“We show that lentiviral delivery of a genome-scale CRISPR-Cas9 knockout (GeCKO) library targeting 18,080 genes with 64,751 unique guide sequences enables both negative and positive selection screening in human cells.”— Genome-Scale CRISPR-Cas9 Knockout Screening in Human Cells

独立复核 · valid

The TARGET QUOTE states '46,751 unique guide sequences' for the GeCKO library, while the SOURCE QUOTE states '64,751 unique guide sequences' for the same GeCKO library targeting the same 18,080 genes. This is a direct numerical contradiction on the exact same entity (GeCKO library), same relation (number of unique guide sequences), and same scope. The source explicitly confirms the 18,080 genes figure but contradicts the 46,751 guide sequences figure. The proposed 'red' judgment is correct.

RED数字错误已确认检出

Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes

EMPA-REG OUTCOME treated a total of 8,020 patients and followed them for a median of 3.1 years.

展开案例 ↓

待核查主张

EMPA-REG OUTCOME treated a total of 8,020 patients and followed them for a median of 3.1 years.

冻结的原文依据

A total of 7020 patients were treated (median observation time, 3.1 years)

实际生成的卡片 · RED

The EMPA-REG OUTCOME report states that 7,020 patients were treated, not 8,020.

“A total of 7020 patients were treated (median observation time, 3.1 years).”— Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes

独立复核 · valid

The target quote states '8,020 patients' while the source quote explicitly states '7020 patients were treated.' This is a direct numerical contradiction on the identical claim of patient count in the EMPA-REG OUTCOME trial. The entity (EMPA-REG OUTCOME trial), relation (patient count), and scope (total treated patients) match exactly, with only the number differing. The proposed color red is correct because the source directly contradicts the target proposition on a material point.

RED数字错误已确认检出

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

The paper's small-task setting allows up to 10,000 training examples, 100 numerical features, and 10 classes.

展开案例 ↓

待核查主张

The paper's small-task setting allows up to 10,000 training examples, 100 numerical features, and 10 classes.

冻结的原文依据

≤ 1 000 training examples, ≤ 100 purely numerical features and ≤ 10 classes

实际生成的卡片 · RED

The paper defines its small-task setting as at most 1,000 training examples, not 10,000.

“has thus learned to solve novel small tabular classification tasks (≤ 1 000 training examples,≤ 100 purely numerical features without missing values and≤ 10 classes)”— TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

独立复核 · valid

The TARGET QUOTE states 'up to 10,000 training examples,' while the SOURCE QUOTE explicitly states '≤ 1 000 training examples.' This is a direct numerical contradiction on the same entity (training examples), same relation (upper bound on small-task setting), same scope (the paper's definition of small-task setting). The source directly contradicts the target proposition that the limit is 10,000 by specifying it is 1,000.

RED数字错误已确认检出

The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing

The survey investigates 24 state-of-the-art fuzzers closely related to directed greybox fuzzing.

展开案例 ↓

待核查主张

The survey investigates 24 state-of-the-art fuzzers closely related to directed greybox fuzzing.

冻结的原文依据

investigation of 42 state-of-the-art fuzzers that are closely related to DGF

实际生成的卡片 · RED

The survey explicitly reports investigating 42 state-of-the-art fuzzers closely related to DGF, not 24.

“Based on the investigation of 42 state-of-the-art fuzzers that are closely related to DGF, we conduct the first in-depth study to summarize the empirical evidence on the research progress of DGF.”— The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing

独立复核 · valid

The target quote claims '24 state-of-the-art fuzzers closely related to directed greybox fuzzing.' The source quote explicitly states '42 state-of-the-art fuzzers that are closely related to DGF' (where DGF = directed greybox fuzzing). The entity (fuzzers closely related to directed greybox fuzzing), relation (investigation of), and scope (state-of-the-art) match exactly; the only difference is the number (24 vs. 42), which is a direct numerical contradiction on a material point. The proposed color 'red' is correct.

RED数字错误漏检

Evaluating the Impact of Brazil's Bolsa Família: Cash Transfer Programs in Comparative Perspective

The review describes Bolsa Família as benefiting roughly 17 million families.

展开案例 ↓

待核查主张

The review describes Bolsa Família as benefiting roughly 17 million families.

冻结的原文依据

benefiting roughly 11 million families

没有生成卡片

RED方向与极性已确认检出

Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes

In EMPA-REG OUTCOME, cardiovascular death was significantly higher with empagliflozin than with placebo.

展开案例 ↓

待核查主张

In EMPA-REG OUTCOME, cardiovascular death was significantly higher with empagliflozin than with placebo.

冻结的原文依据

significantly lower rates of death from cardiovascular causes (3.7%, vs. 5.9% in the placebo group)

实际生成的卡片 · RED

The claim that cardiovascular death was higher with empagliflozin contradicts the account of the same trial’s cardiovascular results, which explicitly reports a lower rate of death from cardiovascular causes.

“Cardiovascular Outcomes The primary outcome that was reported previ ously showed a significantly lower risk of 3-point MACE in the pooled empagliflozin group than in the placebo group, a difference that was driven by a lower rate of death from cardiovas cular causes.23”— Empagliflozin and Progression of Kidney Disease in Type 2 Diabetes

独立复核 · valid

The target quote explicitly claims cardiovascular death was 'significantly higher with empagliflozin than with placebo' in EMPA-REG OUTCOME. The source quote directly contradicts this by stating there was 'a lower rate of death from cardiovascular causes' in the empagliflozin group, which 'drove' the composite benefit. Both quotes concern the same trial (EMPA-REG OUTCOME), the same outcome (cardiovascular death/mortality), the same comparison (empagliflozin vs. placebo), and the same directionality claim—but with opposite directions. The contradiction is material and unambiguous: 'higher' vs. 'lower' for the identical endpoint in the identical trial.

RED方向与极性已确认检出

Evaluating the Impact of Brazil's Bolsa Família: Cash Transfer Programs in Comparative Perspective

The review concludes that Bolsa Família increased inequality and extreme poverty.

展开案例 ↓

待核查主张

The review concludes that Bolsa Família increased inequality and extreme poverty.

冻结的原文依据

helped reduce inequality and extreme poverty

实际生成的卡片 · RED

The review attributes 21 percent of Brazil’s 1995–2004 Gini decline to Bolsa Família, contradicting the claimed increase in inequality.

“They found that the Gini index for Brazil fell by 4.7 percent from 1995 to 2004 and that Bolsa Familia was responsible for 21 percent of that fall (before 1995 there were no CCT programs in Brazil, so there is confidence that 1995 is a baseline year).”— Evaluating the Impact of Brazil's Bolsa Família: Cash Transfer Programs in Comparative Perspective

独立复核 · valid

The TARGET QUOTE states 'The review concludes that Bolsa Família increased inequality.' The SOURCE QUOTE from the same review states that Bolsa Família was responsible for '21 percent of that fall' in the Gini index, where 'fall' means a decrease in inequality. A decrease in the Gini index signifies reduced inequality. Therefore, the source directly contradicts the target proposition: the review concludes Bolsa Família decreased inequality (contributed to 21% of the Gini decline), not increased it. The scope matches—both concern the review's conclusion about Bolsa Família's effect on inequality.

1 / 6

适用范围与限制

  • 1受控改错测试衡量的是发现明确错误的能力,不代表真实论文中这些错误的自然发生率。
  • 2来源论文与早期开发试跑有重叠。这批数据适合用于回归测试和受控召回声明,不适合推断错误在论文中的普遍程度。
  • 3优化后只发出 8 张 Yellow 卡,因此必须同时展示 6/8 的分母,不能把 75% 当作总体精度估计。

← 自动补全评测