#
As the AI development process itself becomes automated by AI, a core component—data engineering—is undergoing a paradigm shift. In the past, data preparation was a tedious, human-led task before model training. Now, an AI agent system named DataMaster is taking on the role of an “AI data engineer,” autonomously preparing and optimizing data for models.
This research, jointly published by Shanghai Jiao Tong University, Carnegie Mellon University, Zhejiang University, and Beihang University (Paper Title: DataMaster: Data-Centric Autonomous AI Research), aims to answer a core question: Can an AI improve its own capabilities through self-driven data engineering, given a fixed model architecture and training method?
The Data Bottleneck in AI R&D and the Trend Toward Autonomy
In practical machine learning development, data strategy adjustments are more frequent and critical than changes to the underlying models and training code, which are less often modified. The selection of data samples, filtering of noise, and methods of combining different sources together constitute the primary space for performance optimization. As models and training methods become more standardized, the concept of “Data-Centric AI” is gaining importance, but its execution has long relied on manual experience.
The emergence of DataMaster aims to automate this manual process. It transforms data engineering from a linear preparation task into a dynamic, iterative optimization process, enabling AI to proactively explore how to find and construct “better” data for itself.
DataMaster: A Learning Data Engineering Team
To tackle the non-linear and complex nature of data engineering, DataMaster is designed not as a one-off generation model but as a continuously optimizing search framework. This framework consists of three core components that work in synergy, simulating an experienced data engineering team.
Data Tree: As the core of the exploration, the Data Tree explores and evaluates different data processing paths. Each node on the tree represents a specific data engineering attempt. Red nodes act as “scouts,” responsible for searching the external world for potentially useful data sources, while black nodes act as “engineers,” responsible for cleaning, transforming, and formatting the data into training inputs ready for the model.
Data Pool: This is a globally shared repository of data assets. Any data source discovered and processed in an exploration branch is stored in the Data Pool for reuse by all subsequent exploration nodes. This mechanism avoids redundant work and improves the efficiency of data discovery and utilization.
Global Memory: This component records and accumulates experience from all attempts. It logs the data used, processing strategies adopted, final training outcomes, analysis of failures, and successful patterns worth generalizing. By learning from this history, the system can make more intelligent decisions in the future and avoid repeating mistakes.
The combination of these three components enables DataMaster to systematically solve data engineering problems like a team that accumulates experience and continuously evolves.
Experimental Validation: Performance Leaps Driven Solely by Data Optimization
The value of DataMaster lies in its proof that automated data iteration can yield real and significant performance gains. The research team validated its effectiveness in two benchmark scenarios.
In the MLE-Bench Lite task, which simulates traditional machine learning engineering, the agent could only optimize a given initial solution through data manipulation. The results showed that DataMaster increased the baseline solution’s medal rate from 35.91% to 68.18%, a 32.27 percentage point increase. The gold medal rate also rose sharply from 22.73% to 45.45%.
In the PostTrainBench scenario, which is closer to large model applications, the agent had to discover and construct post-training data from scratch for a fixed base model without any pre-existing training data. DataMaster lifted the model’s average score from 8.47% to 31.17%, an improvement of 22.70 percentage points, achieving the highest average score among all baseline comparisons.
Particularly noteworthy is its performance on the GPQA task. GPQA is a graduate-level scientific question-answering benchmark covering fields like physics, chemistry, and biology, demanding high levels of specialized knowledge and reasoning. Through iterative discovery and integration of relevant scientific literature, reasoning datasets, and medical Q&A data, DataMaster progressively raised the base model’s score on this task from 18.75% to 31.02%. This result not only surpassed several baseline models, including Codex and DataFlex, but also slightly exceeded the reference score (30.35%) of a model trained on instruction data meticulously curated by experts.
To ensure the rigor of the results, the researchers also conducted a strict data leakage check for the GPQA task, confirming that the performance improvement was not due to accidental exposure of the training set to the test set.
From Passive Input to Active Decision-Making: A New Paradigm in AI Development
The significance of DataMaster extends beyond a single tool’s innovation; it marks a major evolution in the AI development paradigm. It transforms data from a static, prerequisite “input” into a dynamic “variable” that can be actively decided upon and optimized by an AI agent.
This shift makes “data-centric” AI development tangible, moving it from an abstract concept to a practical reality where data itself, like code and algorithms, can be integrated into the continuous optimization loop of AI. When AI starts to autonomously decide what data it should learn, the boundaries of its capabilities are poised to expand further.
However, the automation of data engineering also brings new governance challenges. How can we ensure the compliance and traceability of external data? How can we audit the AI’s data selection decisions? How can we guarantee the transparency and controllability of the entire process? These are critical questions that must be addressed as autonomous data engineering moves toward real-world application. When AI begins to manage data, human responsibility shifts to managing how AI manages data.
