Whose answers are right together?
I want to know whether a model brings its correct care answers together for the same patient. The CarePath evaluation in the NPCBench paper posted to arXiv on 5 November gives that question a concrete endpoint for nasopharyngeal cancer. It scores 217 care decisions across a hundred patients, then checks whether each patient has a correct pathology answer, a correct magnetic resonance imaging (MRI) T stage and correct answers to every selected care question. T stage classifies the local extent of the tumour. The calculation moves attention from how many answers are right to whose answers are right together. That patient-centred definition is the contribution I find most useful.[1]
Changing the endpoint changes the model comparison. The paper reports 89.9 per cent care-decision accuracy for Gemini-3.1 alongside 11 per cent strict patient success. For Claude-4.6, the corresponding results are 80.6 per cent and 23 per cent. The model answering more care questions correctly on this set succeeds on the complete diagnostic-and-care endpoint for fewer patients. I read this as a demonstration of how the target of measurement changes a comparison, rather than a ranking of clinical superiority. Answering selected questions accurately and completing every selected step for a patient describe different research objectives.[1]
Where errors fall and how the test weighs them
To examine the mechanism, the answers have to be linked within the same people. For 72 of the 100 patients, Gemini-3.1 answers every care question correctly while getting pathology or T stage wrong. This intersection shows that care-answer correctness and image-label correctness can diverge within a patient. I would resist concluding that the model ignores the images altogether: the supplied clinical context could help it select a correct care answer despite a wrong image label. Joint success makes the mismatch visible; explaining its cause requires a separate controlled comparison.[1]
The construction of the test offers another explanation for a low strict score. Requiring every selected step to be right creates more opportunities for failure along a longer pathway. CarePath scores between one and six care decisions per patient. The researchers also use answers from four models to prioritise difficult decisions, excluding checkpoints that all four answer correctly. A lower joint score could therefore reflect episode length and question selection alongside difficulty combining skills. I favour comparisons that retain the same patients and decision counts while showing the intersections of errors. That makes it easier to assess the effect of a more demanding test alongside changes in answer patterns.[1]
Evidence matched to the patient
The scope of patient-matched evidence also matters. Pathology specimens match the patient in 25 of the hundred care episodes; the other 75 use assigned specimens. MRI and clinical information remain patient-linked throughout. This creates a useful research test, but the connection between pathology and care is tested differently across the two groups. Together with the mainly single-centre data, that distinction rules out reading 23 per cent as a clinical success rate for patients. The endpoint measures jointly correct answers under specified evidence and selected questions. It measures no observed improvement in patients’ health.[1]
To develop NPCBench further, I would preserve joint success while making its dependence on evaluation conditions explicit. I would look for results from another centre, with all imaging matched to patients and results separated by decision count. The accounting of how pathology, staging and care errors intersect within each person should remain visible. That supports a precise question: does a gain come from image interpretation, care answers or the distribution of errors across patients? Establishing patient benefit additionally requires prospective clinical outcomes. The value of joint correctness lies in sharpening the question that comes before that further claim.[1]