Following the same object
GIB-200K tests where a vision model gets its answer by putting the picture and the document in conflict. When a clear photo comes with a description that confirms it, the score gives the same credit whether the answer came from the picture or the text. Hold the photo steady and change one detail in the document, and the stability of that answer becomes measurable.[1]
The authors gave 4 open-weight models the same fixed set of examples. In matched questions, they swapped the answers suggested by the picture and the document. If a model names the same object both times, it may have a prior preference for that object. If it follows the document in both cases, its choice depends on what the document says. The reversal separates behaviors that look identical on the first question.[1]
That is why the image-only trial matters. When a model can identify the object unaided and changes its answer after reading a false description, the switch tells us something about how it uses available visual evidence. When the photograph is blurred and the model already struggles to identify the object, leaning on the document may be a reasonable response.[1]
The cost of a blurry picture
The preprint’s findings make a universal rule hard to defend. The tested LLaVA models leaned toward text relative to the evidence they could extract from images alone. InternVL3 leaned the other way. That difference emerges by accounting for the visual information each model can recover on its own. The measurement depends on asking the same question under each evidence condition.[1]
When a clear image comes with a false document, one option is to withhold an answer if the runs with and without text disagree. That leaves more questions unanswered. With a blurred image and an accurate document, falling back to the picture can spoil previously correct answers. Reporting accuracy and the share of questions answered separately for both conditions makes the cost of that safeguard visible.[1]
A useful next test would build its sample around the observed mix of misleading documents with clear pictures and accurate documents with degraded ones. The closed vocabulary here makes behavior on object and relation questions measurable. To judge the same safeguard in a service that needs free-form answers, the questions and acceptable responses should come from that service.[1]