Evidence
Same model and position · retrieved passages supplied · first output
perceived training quality and effectiveness, including cognitive engagement, variety of activities, and instructor performance.
Passages supplied to the Evidence version
Card 1 · Corporate Training in Brazilian Software Engineering: A Quantitative Study of Professional Perceptions
However, there is a knowledge gap regarding the factors that drive quality and effectiveness of such training from the professionals’ perspective, and no validated instrument exists for assessing these factors in the software engineering (SE) domain.Objective:This study aims to quantitatively analyz…
Read full passage excerpt
However, there is a knowledge gap regarding the factors that drive quality and effectiveness of such training from the professionals’ perspective, and no validated instrument exists for assessing these factors in the software engineering (SE) domain.Objective:This study aims to quantitatively analyze which factors influence SE professionals’ per- ceptions of corporate training quality and effectiveness.Method: A quantitative survey was conducted with 282 Brazilian SE profes- sionals. A structured questionnaire was developed and polychoric correlation was adopted for data analysis.Results:Three tightly correlated factors (cognitive engagement, variety of activities, and instructor performance) emerged as the strongest predictors of per- ceived training quality and effectiveness.
Card 2 · Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selection
Recent advances in Large Language Models (LLMs) open new perspectives for automation in optimization, yet little is known about whether their internal representations capture problem structure or algorithmic behavior. We investigate whether representations learned by frozen, open-weight LLMs for com…
Read full passage excerpt
Recent advances in Large Language Models (LLMs) open new perspectives for automation in optimization, yet little is known about whether their internal representations capture problem structure or algorithmic behavior. We investigate whether representations learned by frozen, open-weight LLMs for combinatorial optimization instances can support downstream decision tasks. The goal is not to replace exact feature extractors or to propose a new algorithm, but to assess whether such representations are reusable for feature recovery and algorithm selection. Our methodology combines direct querying, which tests explicit feature extraction, with probing analyses of whether this information is implicitly encoded in the hidden layers. The probing framework is further extended to a per-instance algorithm selection task. Experiments span four benchmark problems, three instance representations, and five open-weight models from 3B to 120B parameters across the Llama instruct and GPT reasoning famili …
Card 3 · Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning
Recently, Multimodal Large Language Models (MLLMs) have achieved significant success across multiple disciplines due to their exceptional instruction-following capabilities and extensive world knowledge. However, whether these MLLMs possess human-like compositional reasoning abilities remains an ope…
Read full passage excerpt
Recently, Multimodal Large Language Models (MLLMs) have achieved significant success across multiple disciplines due to their exceptional instruction-following capabilities and extensive world knowledge. However, whether these MLLMs possess human-like compositional reasoning abilities remains an open problem. To unveil their reasoning behaviors, we first curate a \textbf{M}ultimodal \textbf{A}ssumptive \textbf{R}ea\textbf{s}oning Benchmark (MARS-Bench) in this paper. Interestingly, we find that most prevalent MLLMs can be easily fooled by the introduction of a presupposition into the question, whereas such presuppositions appear naive to human reasoning. Besides, we also propose a simple yet effective method, Active Deduction (AD), a novel reinforcement learning paradigm to encourage the model to actively perform composite deduction before reaching a final decision. Equipped with the proposed AD method, a MLLM demonstrates significant improvements in assumptive reasoning abilities with …
Card 4 · Emergent Abilities in Large Language Models: A Survey
Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence. The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked t…
Read full passage excerpt
Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence. The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked to various so-called emergent abilities that were previously unobserved. These emergent abilities, ranging from advanced reasoning and in-context learning to coding and problem-solving, have sparked an intense scientific debate: Are they truly emergent, or do they simply depend on external factors, such as training dynamics, the type of problems, or the chosen metric? What underlying mechanism causes them? Despite their transformative potential, emergent abilities remain poorly understood, leading to misconceptions about their definition, nature, predictability, and implications. In this work, we shed light on emergent abilities by conducting a comprehensive review of the phenomenon, addressi …
Card 5 · Composition of Experts: A Modular Compound AI System Leveraging Large Language Models
Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper introduces the Composition of Experts (CoE), a modular compound AI system leveraging multiple expert LLMs. CoE leverages a…
Read full passage excerpt
Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper introduces the Composition of Experts (CoE), a modular compound AI system leveraging multiple expert LLMs. CoE leverages a router to dynamically select the most appropriate expert for a given input, enabling efficient utilization of resources and improved performance. We formulate the general problem of training a CoE and discuss inherent complexities associated with it. We propose a two-step routing approach to address these complexities that first uses a router to classify the input into distinct categories followed by a category-to-expert mapping to obtain desired experts. CoE offers a flexible and cost-effective solution to build compound AI systems. Our empirical evaluation demonstrates the effectiveness of CoE in achieving superior performance with reduced computational overhead. Given that CoE comprises o …
Model-reported card use
Card 1 · fact
Affected output: perceived training quality and effectiveness, including cognitive engagement, variety of activities, and instructor performance.
Model explanation: Supplied the specific factors and the outcome variable that complete the draft's unfinished clause about the second phase's quantitative analysis.