Jiefeng Chen
Jiefeng Chen is a Research Scientist at Google Cloud AI Research, working on building more efficient and reliable Large Language Model (LLM)-based agents for real-world applications. For complete list of publications and latest updates, please check out his primary homepage or visit his Google Scholar page.
Research Areas
Authored Publications
Sort By
Preview abstract
Understanding long visual documents, where information is distributed across extensive pages of text and graphics, remains a critical challenge for modern Vision-Language Models (VLMs). This difficulty is rooted in two fundamental obstacles: poor evidence localization and a high tendency for model hallucination. To address these issues, we propose DocLens, a multi-agent framework that decomposes the task into two specialized stages.
First, a Lens Module leverages document parsing tools for fine-grained, hierarchical evidence localization at both the page and element level. Second, a Reasoning Module employs a sampling-adjudication mechanism to systematically analyze the localized evidence, mitigating hallucination and synthesizing reliable answers.
Paired with Gemini-2.5-Pro, DocLens achieves state-of-the-art performance on MMLongBench-Doc and FinRAGBench-V, even surpassing human experts. Furthermore, our framework offers a highly cost-effective variant that delivers comparable performance to strong baselines at a five-fold reduction in cost.
View details
Preview abstract
Automating AI research differs from general software engineering due to computationally expensive evaluation (e.g., model training) and opaque performance attribution. Current LLM-based agents struggle here, often generating monolithic scripts that ignore execution costs and causal factors. We introduce MARS (Modular Agent with Reflective Search), a framework optimized for autonomous AI research. MARS relies on three pillars: (1) Budget-Aware Planning via cost-constrained Monte Carlo Tree Search (MCTS) to explicitly balance performance with execution expense; (2) Modular Construction, employing a "Design-Decompose-Implement" pipeline to manage complex research repositories; and (3) Comparative Reflective Memory, which addresses credit assignment by analyzing solution differences to distill high-signal insights. MARS achieves state-of-the-art performance among open-source frameworks on MLE-Bench under comparable settings, maintaining competitiveness with the global leaderboard's top methods. Furthermore, the system exhibits qualitative "Aha!" moments, where 63% of all utilized lessons originate from cross-branch transfer, demonstrating that the agent effectively generalizes insights across search paths.
View details
FabScore: Fine-Grained Evaluation of Fabrications in Automated AI Research
James Xu Zhao
See-Kiong Ng
Dongfu Jiang
Bryan Hooi
Qianyun Guo
Hui Chen
Pang Wei Koh
Muhao Chen
Yiwei Wang
2026
Preview abstract
In automated AI research, scientific rigor is not merely a matter of producing coherent papers and executable code; rather, it fundamentally requires that the experimental results claimed in a paper are faithfully supported by the accompanying implementation and verifiable through actual execution logs. When this alignment breaks down, AI-generated research may contain fabrications: discrepancies between the methods described in the paper and what the code actually implements, or between the reported results and those obtained by running the code. This motivates a systematic evaluation of fabrications that systematically examines the consistency between a paper's claimed experimental outcomes and its accompanying code files. In automated AI research, scientific rigor is not merely a matter of producing coherent papers and executable code; rather, it fundamentally requires that the experimental results claimed in a paper are faithfully supported by the accompanying implementation and verifiable through actual execution logs. When this alignment breaks down, AI-generated research may contain fabrications: discrepancies between the methods described in the paper and what the code actually implements, or between the reported results and those obtained by running the code. This motivates a systematic evaluation of fabrications that systematically examines the consistency between a paper's claimed experimental outcomes and its accompanying code files. This is a placeholder. This is a placeholder. This is a placeholder.
View details
Preview abstract
Autonomous research agents can now produce competitive solutions and complete manuscripts, yet their papers routinely contain fabricated citations, method descriptions disconnected from the code, and scores on incorrect scales---failures invisible to evaluations that assess fluency rather than evidentiary grounding. The core problem is verifiability: no existing system maintains a traceable chain from each claim in the paper to its grounding evidence, and current evaluation protocols assess output fluency rather than evidentiary grounding. We address this with Chaine-of-Evidence (CoE), a verifiability standard requiring every claim to trace to its grounding evidence, and instantiate it in Scientist One, an end-to-end research system that maintains evidence chains natively, and CoE Audit, an evaluation protocol with four integrity checks targeting the most damaging chain failures. Auditing 60 papers from four systems, we find every baseline exhibits at least one failure: phantom citations at 4--25%, method-code alignment in at most 2/15 papers. Scientist One achieves zero phantom citations (0/830), the highest alignment rate (7/15), and competitive solver scores.
View details
Cost-effective Agent Test-Time Scaling
Samira Daruki
I-Hung Hsu
Jin Miao
Tengxiao Liu
2026
Preview abstract
Scaling test-time computation improves performance across different tasks on large language models (LLMs), yet mainstream scaling methods remain challenging for tool-augmented LLM agents. Sequential scaling tends to yield shallow tool use and under-exploration, whereas parallel scaling inflates cost through repeated tool calls. The dual costs of tokens and tool calls further complicate cost accounting and hinder fair comparison. In this work, we study the test-time scaling of widely used, tool-reliant search agents under resource constraints, analyzing performance with a unified cost metric that incorporates both tokens and tool calls. To this end, we propose Cost-effective Agent Test-Time Scaling (CATS), a budget-aware framework designed to support more cost-effective scaling by guiding resource allocation between sequential and parallel exploration. Experiments across search-intensive benchmarks show that CATS produces more favorable scaling curves, attaining higher accuracy with fewer tool calls and lower overall cost. Our work introduces a cost-conscious design for agent test-time scaling and contributes empirical insights that enable a more transparent and principled understanding of scaling in tool-augmented agents.
View details
Preview abstract
Automating data visualization from natural language is crucial for data science, yet current systems struggle with complex, multi-file datasets and iterative refinement. Existing approaches, including simple single- or multi-agent systems, often oversimplify the task, focusing on initial query parsing while failing to robustly manage data complexity, code errors, or final visualization quality. In this paper, we reframe this challenge as a collaborative multi-agent problem. We introduce CoDA, a multi-agent system that employs specialized LLM agents for metadata analysis, task planning, code generation, and iterative reflection. We formalize this pipeline, demonstrating how metadata-focused analysis bypasses token limits and quality-driven refinement ensures robustness. Extensive evaluations show CoDA achieves substantial accuracy gains, outperforming competitive baselines by up to 49.0%. This work advocates that future visualization automation should evolve from isolated code generation to integrated, collaborative agentic workflows.
View details
Preview abstract
Inference-time scaling has been successful in enhancing large language model (LLM) performance by increasing computation at test time, but it often relies on external verifiers or is not optimized for manageable computational budgets. To address these, we propose DynScaling, which addresses these limitations through two primary innovations: an integrated parallel-sequential sampling strategy and a bandit-based dynamic budget allocation framework. The integrated sampling strategy unifies parallel and sequential sampling by constructing synthetic sequential reasoning chains from initially independent parallel responses, promoting diverse and coherent reasoning trajectories. The dynamic budget allocation framework formulates the allocation of computational resources as a multi-armed bandit problem, adaptively distributing the inference budget across queries based on the uncertainty of previously sampled responses, thereby maximizing computational efficiency. By synergizing these components, DynScaling effectively improves LLM performance under practical resource constraints without the need for external verifiers. Experimental results demonstrate that DynScaling consistently surpasses existing verifier-free inference scaling baselines in both task performance and computational cost.
View details
Preview abstract
While Large Language Models (LLMs) have shown remarkable advancements in reasoning and tool use, they often fail to generate optimal, grounded solutions under complex constraints. Real-world travel planning exemplifies these challenges, evaluating agents' abilities to handle constraints that are explicit, implicit, and even evolving based on interactions with dynamic environments and user needs. In this paper, we present ATLAS, a general multi-agent framework designed to effectively handle such complex nature of constraints awareness in real-world travel planning tasks. Our framework introduces a principled approach to address the fundamental challenges of constraint-aware planning through dedicated mechanisms for dynamic constraint management, iterative plan critique, and adaptive interleaved search. ATLAS demonstrates state-of-the-art performance on the TravelPlanner benchmark, improving the final pass rate from 17.8% to 44.4% over its best alternative. More importantly, this is the first work to be evaluated in and demonstrate quantitative effectiveness on real-world travel planning with live information search and multi-turn feedback. In this realistic setting, ATLAS demonstrates its ability to adapt to multi-turn user feedback, achieving an 84% final pass rate which significantly outperforms baselines including ReAct (59%) and a monolithic agent (27%).
View details
Preview abstract
Data science, which transforms raw data into actionable insights, is critical for data-driven decision-making. However, these tasks are often complex, involving steps like exploring multiple data sources and synthesizing findings to deliver clear answers. While large language model (LLM) agents show significant promise in automating this process, they often struggle with heterogeneous data formats and generate sub-optimal analysis plans, as verifying plan correctness is inherently difficult without ground-truth labels for such open-ended tasks. To overcome these limitations, we introduce DS-STAR, a novel data science agent. Specifically, DS-STAR makes three key contributions: (1) a data file analysis module that automatically reads and extracts context from diverse data formats, including unstructured types; (2) a verification step where an LLM-based judge evaluates the sufficiency of the analysis plan at each stage; and (3) a sequential planning mechanism that starts with a simple, executable plan and iteratively refines it based the DS-STAR's feedback until its sufficiency is confirmed. This iterative refinement allows DS-STAR to reliably navigate complex analyses involving varied data sources. Our experiments show that DS-STAR achieves state-of-the-art performance, improving accuracy on the challenging DABStep benchmark from 41.0% to 45.2% and on Kramabench from 31.3% to 44.7%. These results demonstrate the effectiveness of our approach for practical, multi-step data science tasks.
View details
Preview abstract
Agents based on large language models (LLMs) for machine learning engineering (MLE) can automatically implement ML models via code generation. However, existing approaches to build such agents often rely heavily on inherent LLM knowledge and employ coarse exploration strategies that modify the entire code structure at once. This limits their ability to select effective task-specific models and perform deep exploration within specific components, such as experimenting extensively with feature engineering options. To overcome these, we propose MLE-STAR, a novel approach to build MLE agents. MLESTAR first leverages external knowledge by using a search engine to retrieve effective models from the web, forming an initial solution, then iteratively refines it by exploring various strategies targeting specific ML components. This exploration is guided by ablation studies analyzing the impact of individual code blocks. Furthermore, we introduce a novel ensembling method using an effective strategy suggested by MLE-STAR. Our experimental results show that MLE-STAR achieves medals in 64% of the Kaggle competitions on the MLE-bench Lite, significantly outperforming the best alternative.
View details
