AgentSSL finds strong learning programs but struggles to choose them
AgentSSL lets agents construct learning programs using scarce labels alongside unlabeled data. In a new preprint, the strongest generated candidates beat the best human-built baseline in eight of twelve settings, but choosing with available feedback can sharply reduce performance. The researchers also introduce a collection of programs for testing how those choices are made.
Artificial Intelligence··Night
Available feedback can select a weaker program
AgentSSL, a system for searching over semi-supervised learning programs, exposes a selection problem in automated machine-learning work. The authors’ new preprint reports that search produces a top candidate exceeding the leading human-built comparison in eight settings out of twelve, with gains reaching 21.8 percent. Selecting a program using the feedback actually available to the agent can instead reduce performance by as much as 47 percent. The best candidate in the search and its eventual selection are measured separately.[1]
Agents combine labeled and unlabeled examples
The setting pairs a small supply of labeled examples with abundant unlabeled data. Agents write executable code that adapts and combines existing semi-supervised methods, addressing both the effort of labeling and the work of engineering a learning pipeline. Comparisons use matched computation budgets and weak feedback under limited supervision. Additional experiments examine ecology detection tasks, extending the investigation beyond the controlled benchmark settings.[1]
ProgramSelectBench tests candidate choice
To study the choice stage, the researchers assemble ProgramSelectBench, a collection containing 16,977 agent-generated and human-engineered programs. They evaluate unsupervised and active selection approaches, including spending some of the limited labels on choosing among programs rather than using all labels for training. This allocation is part of the experimental design: the available labels influence which result the agent can identify at the end of its search.[1]