FlowingFlowing
← Back to all development cases
This case was judged by an anonymized AI reviewer and has not been independently reviewed by people; inspect the draft, cards, and both outputs to assess it.
AutocompleteWithout cards better232 / 280 · beu-1d2c1fb33c55fcda

Intelligent Energy Management Systems -- A Review

Climate and environmental science · 2206.03264v1

FLOWING EVIDENCE BENCHMARK

How do we compare writing with and without cards?

At the same writing position, the same model produces one first result with cards and one without. All 280 pairs enter the overall anonymized review, which assesses whether a decision not to continue fits the draft.

Overall: 138 with-card wins, 26 ties, 107 no-card wins, 5 pairs where neither was usable, and 4 inconclusive judgments. Both sides wrote text in 218 pairs; 48 had a deliberate stop and 14 included an invalid output.

SAME DRAFT · WITH OR WITHOUT CARDS

01 · FIXED WRITING POSITION

MANUSCRIPT

Same writing position ▌

Matched autocomplete at the same draft position

02 · TWO MATCHED VERSIONS

SHARED BY BOTH

Draft, writing task, and generation model

A · With cards

Relevant literature cards supplied

B · Without cards

No literature cards supplied

Generation model

Same model and version

A → first output

B → first output

Anonymized AI judge

First outputs are anonymized as X and Y

Continuation review: accuracy, fit to the writing task and usability
Returns: X preferred / tie / Y preferred / both unusable

Review again with the answer order swapped

HOW DOES BLIND JUDGING WORK?

① Anonymize both outputs
The AI judge sees the same draft and both first outputs without knowing which received cards.

② Compare and swap order
The judge applies task-specific criteria in X/Y and then Y/X order.

③ Review disagreements
A third pass resolves disagreements. Incomplete reviews remain in the denominator.

How are positions and cards selected?

We selected 40 writing positions in each of seven fields, for 280 comparisons. The position and relevant literature cards were set before each comparison, so both versions began from the same draft.

The main result retains every frozen position, including places where cards might not add a useful next move. This avoids selecting cases based on their outputs, but 49% describes the whole cohort, not a win rate where evidence clearly had an opportunity to help. This cohort did not record that opportunity label for each position before generation, so those two groups cannot yet be separated reliably.

CARD USE · 140/280

Reported card use in continuations with text

The model reports whether a card affected its continuation. In 140 of 280 tests it produced continuation text and reported card use. The previous count of 150 included 10 cases without continuation text and has been corrected.

COMPARISON · 70 / 7 / 30

Which written version was better?

Of the 107 reported-use cases where both versions produced text, the blinded AI judge preferred the with-card version 70 times, found 7 about the same, and preferred the without-card version 30 times.

How are the two table percentages calculated?

ADOPTION COVERAGE

140reported use with text
÷
280all positions
=
50%

WITH-CARD VERSION PREFERRED

70with cards better
÷
107comparable tests
=
65%

The percentages use different groups: 50% counts continuations with text reporting card use among all 280 tests; 65% covers the 107 such cases with text on both sides. The generating model reports card use, while an anonymized AI judge compares writing quality.

Card use is self-reported by the generator. The anonymized AI judge sees the draft and source snippets, so source-aware preference remains possible; there was no independent human review. The overall result describes these 280 frozen positions, not a win probability for any writing task.

Manuscript writing position

Text excerpt · not a PDF page

Research manuscript · excerpt

Intelligent Energy Management Systems -- A Review

3 Related Work

…at the environment. According to Cook et al. ((2009)) , there are various definitions about AmI depending on the features that are included. These environments offer environmental, comfort, safety and financial benefits. AmI is also an umbrella term which applies into technologies embedded into a physical space to create an invisible user interface augmented with AI ( Dunne et al. ((2021)) ). They presented an comprehensive survey on Ambient Intelligence (AmI) and Ambient Assistive Living (AAL) while referring to the state-of-the-art AI techniques and methodologies to implement these systems.

…ogramming, meta-heuristic search and heuristic scheduling techniques. Finally, a recent study by Ali et al. ((2021)) summarized research opportunities created by open issues in the field such as, blockchain-enabled IoT platforms for distributed energy management, deep learning models to handle, use and evaluate big energy data, peer-to-peer energy trading and demand-side energy management, context-aware pervasive future computing, resilience-oriented energy management, forecasting models, user comfort and real-time feedback systems as well as, Internet of Energy (IoE)-based energy management.

↓ Next paragraph was generated here

The highlighted text is near the writing position. This is a layout of the evaluation input, not a PDF screenshot.

A matched comparison at the same writing positionOverall AI review: Without cards better

Both arms use the same model, task and manuscript context. Only the retrieved passages differ. The first recorded output from each arm is shown below.

Read overall AI review rationale

A = with cards · B = without cards

The draft ends with a list of research opportunities from Ali et al. (2021) spanning blockchain IoT, deep learning, peer-to-peer trading, context-aware computing, resilience, forecasting, user comfort, and IoE-based management. The task is to continue with one coherent academic paragraph at this position. Output A provides a specific claim about blockchain-based architectures for transactive energy management in smart homes. While topically related to one item from Ali et al.'s list, it functions as a narrow factual insertion that doesn't advance the argument. It reads like a source summary rather than a continuation of the manuscript's flow. The draft has just summarized a survey's future research directions; simply elaborating on one direction without transitioning to the author's own framing feels like a non-sequitur that doesn't serve the paper's argumentative needs. Output B provides a critical synthesis paragraph that identifies a structural gap in the literature—fragmentation across individual techniques rather than integrated pipelines—and explains why this matters for comparing energy savings. This is a classic academic move: after surveying prior work, the author identifies limitations to motivate their own contribution. It fits the typical progression from literature review to problem statement. The paragraph is appropriately scoped, makes no unsupported concrete claims requiring source verification, and could be inserted without repair. It transitions naturally from the list of opportunities to a critique that would justify the author's proposed research direction. Output B is the stronger continuation because it performs necessary argumentative work at this position, whereas Output A merely adds descriptive detail without advancing the manuscript's purpose.

Evidence

Same model and position · retrieved passages supplied · first output

Not preferred in this comparison

Blockchain-based architectures have been proposed to secure transactive energy management in IoT-aided smart homes, where smart homes perform vertical transactions with the grid and horizontal peer-to-peer energy trading, and a privacy-preserving distributed algorithm executed via smart contracts enables users to manage their energy usages in parallel.

Passages supplied to the Evidence version

Card 1 · Authentication Protocols for Internet of Things: A Comprehensive Survey

In this paper, we present a comprehensive survey of authentication protocols for Internet of Things (IoT). Specifically, we select and in-detail examine more than forty authentication protocols developed for or applied in the context of the IoT under four environments, including: (1) Machine to mach…

Read full passage excerpt

In this paper, we present a comprehensive survey of authentication protocols for Internet of Things (IoT). Specifically, we select and in-detail examine more than forty authentication protocols developed for or applied in the context of the IoT under four environments, including: (1) Machine to machine communications (M2M), (2) Internet of Vehicles (IoV), (3) Internet of Energy (IoE), and (4) Internet of Sensors (IoS). We start by reviewing all survey articles published in the recent years that focusing on different aspects of the IoT idea. Then, we review threat models, countermeasures, and formal security verification techniques used in authentication protocols for the IoT. In addition, we provide a taxonomy and comparison of authentication protocols for the IoT in form of tables in five terms, namely, network model, goals, main processes, computation complexity, and communication overhead. Based on the current survey, we identify open issues and suggest hints for future research.

Card 2 · Combined Cooling, Heating, and Power System in Blockchain-Enabled Energy Management

The combined cooling, heating and power (CCHP) system is a typical distributed, electricity-gas integrated energy scheme in a community. First, it generates electricity by use of gas, and then exploits the waste heat to supply community with heat and cooling. In this paper, we consider a smart city…

Read full passage excerpt

The combined cooling, heating and power (CCHP) system is a typical distributed, electricity-gas integrated energy scheme in a community. First, it generates electricity by use of gas, and then exploits the waste heat to supply community with heat and cooling. In this paper, we consider a smart city consisting of a number of communities (CCHPs) and an agent of power grid (APG), where CCHPs can sell energy to the APG according to its bid. To study all utilities of entities in such a city from energy trading, a noncooperative Stackelberg game between APG and CCHPs is formulated. Here, the APG gives a bid for buying the energy from CCHPs, then CCHPs respond to the APG with their optimal energy supply that maximizing their utilities according to this bid. We show that the maximum profit to the APG and utilities to the CCHPs can be obtained at the Stackelberg equilibrium, which is guaranteed to exist and unique. Because the complete information about energy supply of each CCHP is unknown to …

Card 3 · On-line Building Energy Optimization using Deep Reinforcement Learning

Unprecedented high volumes of data are becoming available with the growth of the advanced metering infrastructure. These are expected to benefit planning and operation of the future power system, and to help the customers transition from a passive to an active role. In this paper, we explore for the…

Read full passage excerpt

Unprecedented high volumes of data are becoming available with the growth of the advanced metering infrastructure. These are expected to benefit planning and operation of the future power system, and to help the customers transition from a passive to an active role. In this paper, we explore for the first time in the smart grid context the benefits of using Deep Reinforcement Learning, a hybrid type of methods that combines Reinforcement Learning with Deep Learning, to perform on-line optimization of schedules for building energy management systems. The learning procedure was explored using two methods, Deep Q-learning and Deep Policy Gradient, both of them being extended to perform multiple actions simultaneously. The proposed approach was validated on the large-scale Pecan Street Inc. database. This highly-dimensional database includes information about photovoltaic power generation, electric vehicles as well as buildings appliances. Moreover, these on-line energy scheduling strategi …

Card 4 · Privacy-Preserving Transactive Energy Management for IoT-Aided Smart Homes via Blockchain

With the booming of smart grid, the ubiquitously deployed smart meters constitutes an energy Internet of Things (IoT). This article develops a novel blockchain-based transactive energy management (TEM) system for IoT-aided smart homes. We consider a holistic set of options for smart homes to partici…

Read full passage excerpt

With the booming of smart grid, the ubiquitously deployed smart meters constitutes an energy Internet of Things (IoT). This article develops a novel blockchain-based transactive energy management (TEM) system for IoT-aided smart homes. We consider a holistic set of options for smart homes to participate in transactive energy. Smart homes can interact with the grid to perform vertical transactions, e.g., feeding in extra solar energy to the grid and providing demand response service to alleviate the grid load. Smart homes can also interact with peer users to perform horizontal transactions, e.g., peer-to-peer energy trading. However, conventional TEM method suffers from the drawbacks of low efficiency, privacy leakage, and single-point failure. To address these challenges, we develop a privacy-preserving distributed algorithm that enables users to optimally manage their energy usages in parallel via the smart contract on the blockchain. Further, we design an efficient blockchain system …

Model-reported card use

Card 4 · fact

Affected output: Blockchain-based architectures have been proposed to secure transactive energy management in IoT-aided smart homes

Model explanation: This row supplied the concrete prior-work fact that a blockchain-based transactive energy management system was developed for IoT-aided smart homes, which the draft alone did not contain.

Card 4 · terminology

Affected output: vertical transactions with the grid and horizontal peer-to-peer energy trading

Model explanation: This row provided the field-specific terms vertical transactions and horizontal peer-to-peer energy trading used to describe the transaction types in the generated sentence.

Card 4 · fact

Affected output: a privacy-preserving distributed algorithm executed via smart contracts enables users to manage their energy usages in parallel

Model explanation: This row supplied the concrete prior-work fact that a privacy-preserving distributed algorithm enables parallel energy-usage management via blockchain smart contracts, which the draft alone did not contain.

No Evidence

Same model and position · no retrieved passages · first output

Preferred by blind review

Despite this breadth of prior work, the reviewed systems remain largely organized around individual techniques rather than an integrated pipeline from sensing to actuation. This fragmentation makes it difficult to compare reported energy savings across studies that differ in environment, sensing modality, and evaluation metrics.

Earlier card-use comparison

The earlier diagnostic compared only cases with reported card use and text on both sides; its scope differs from the overall AI review above.

The blinded model review preferred the Evidence version. Its original rationale appears below.

Read the original model judgment

Output A is grounded in Source 4 (Privacy-Preserving Transactive Energy Management for IoT-Aided Smart Homes via Blockchain), which describes blockchain-based transactive energy management with vertical/horizontal transactions and smart contracts for parallel energy usage management. This directly continues the paragraph's focus on energy management technologies mentioned by Ali et al. (2021), specifically blockchain-enabled IoT platforms and peer-to-peer energy trading. Output B makes an unsupported evaluative claim that prior work is 'organized around individual techniques rather than an integrated pipeline' and that 'fragmentation makes it difficult to compare reported energy savings.' These are material factual claims about the state of research that are not supported by any supplied source. The sources describe specific systems and methods but do not support this meta-analytical critique of the field's organization or comparability problems. The draft itself does not contain this evaluation either.

The passages are input material; their presence does not mean an output used them or that they support every claim. Draft and source passages are excerpted; no simulated scores or PDF appear here.