Three answers from the same gel
For a laboratory researcher, locating the right spot in a gel poses a different requirement from eventually producing one correct range after several attempts. If the answer changes when the same image is presented again, uncertainty enters the next research decision. That repetition requirement is what interests me most in Artificial Analysis’s new VIALS-AA evaluation. Each of Handshake AI’s 161 visual tasks is run three times, and the score counts tasks answered correctly in every run. The choice emphasizes sustained correctness on the same question. It addresses a practical concern for a researcher proceeding with a single answer rather than selecting the best attempt afterward.[1]
The two metrics therefore deserve to be read together. Kimi K3.1 Pro and GPT-6 Astra each solve roughly three-quarters of the questions at least once. Requiring all three attempts to succeed separates their scores to 45% and 63%. The at-least-one measure, pass@3, shows that a correct solution is within reach across the attempts. The all-three measure, pass^3, adds whether that solution is reproduced. Their gap tells us something about consistency on the same images. Reading only the best attempt would conceal a substantial difference between the two models. Neither denominator can silently stand in for the other.[1]
Inspecting an image and reading it
Tool volume does not automatically resolve the problem. VIALS-AA gives models code execution and image inspection through the same Stirrup harness. In the protein-spot-22 example, the correct range is 40–50 kDa. Astra reaches it with a short inspection; Sonnet measures the axis and spot position to reach the same answer. Grok 4.7 gives 50–70 kDa after 27 turns and 20 image views. The published trace shows an added annotation being confused with the target spot. The failure concerns how a measured object is connected to the scientific object in the question, rather than simply a lack of inspection.[1]
I would not read this example as evidence that fewer tool calls are inherently better. Extended inspection could also reflect a harder image or a more cautious strategy; one example cannot separate those explanations. Sonnet’s successful measurement illustrates a useful application of the same tools. My inference is narrower: a count of tool calls cannot substitute for selecting the right object and relating it to the axis. Considering correctness, repeatability and the trajectory together offers a way to investigate which error additional processing repairs, or which mistaken reading it prolongs. That question requires comparisons within tasks, rather than a ranking by activity alone.[1]
The step toward a laboratory decision
VIALS makes the measurement concrete by using gels, plasmid maps and cell images resembling life-sciences work. Repeating the same 161 questions nevertheless does not test whether correctness survives images from another laboratory. Success in all three runs also supplies no error-free guarantee across unlimited repetition. What is measured is consistency within a defined task and tool configuration. Reading a finding from an image and deciding how to change a research plan are also different decisions. The method’s value lies in measuring that first step more selectively, while the rest of the scientific workflow remains a separate target for evaluation.[1]
A useful next comparison would report the same correctness and repetition measures on new laboratory images, broken down by task type. Identifying the conditions under which a wrong spot or axis is selected repeatedly would give researchers a clearer operating boundary than the aggregate alone. Such an evaluation could also measure the human effort needed to notice and correct an error. VIALS-AA’s present contribution is to make reproducibility of a correct answer visible. It sharpens the question for choosing a scientific assistant: how consistently does the system preserve the correct reading when it encounters the same evidence again?[1]