A correct address, a different question
Finding an email address in a report is a small task. Making that address the destination of the right information adds another dependency. Gudmund Grov, chief scientist at Norway’s defence research establishment FFI, uses this example to explain the SEAL laboratory’s approach in an interview with Nettavisen. Researchers can separately examine information extraction, address identification and planning without sending an email. I find the useful methodological question at the handover: who supplied the information that the next step receives?[1]
An isolated test makes a capability easier to locate. If an evaluator supplies the correct extracted information before assessing a plan, the planning result concerns that supplied input. A chain receiving the model’s own earlier output asks how the capabilities work together. An error there could reflect poor planning, but it could also begin with a mistaken extraction. My inference is that intermediate inputs belong in the definition of the test. Combining the results without identifying those inputs would make the source of a failure harder to interpret.[1]
Let the earlier output enter the next step
Grov distinguishes a model that produces text from an agent connected to tools that can act. SEAL first seeks to understand the AI component; assessment of a complete application follows later. This leaves room for a comparison before any message is sent. I would use the same reports to compare planning from evaluator-supplied information with planning from information the model extracted itself. The intended destination and required content would stay fixed. This is a proposed extension of the described approach, rather than an experiment Grov reports as completed.[1]
The informative observation would be which plans change when the source of the intermediate information changes. If supplying the correct information repairs a plan, an earlier extraction error becomes a plausible explanation. A competing explanation is that the evaluator has also made the input clearer or easier to process. Keeping the presentation comparable would help examine that possibility. Conversely, an incorrect plan given correct information locates a difficulty beyond extraction. The comparison would investigate the dependency between steps, while leaving the tool’s execution of the plan for the application assessment.[1]
Local control and the scope of a result
FFI also tests open-weight models developed in China. Grov says these run in controlled systems and send no test data back to the originating laboratories. Open weights describe access to the model’s learned parameters; they are not a reported performance result. For this evaluation question, the important detail is the controlled setting in which inputs and outputs can be examined. Grov also says the capability and autonomy given to an agent can be controlled. A result obtained with one arrangement should identify that arrangement before being carried into another application.[1]
Grov’s account supplies a way to investigate capabilities, without a published score table establishing how reliably the parts compose. I would retain the separate tests and add the handover comparison alongside them. A team could then ask whether a weak chain needs better extraction, a more usable intermediate representation or better planning. Those alternatives call for different changes. The value I see in component evaluation is this ability to locate a research question precisely. Its next useful result would show what changes when the evaluator stops correcting the information between steps, while the final action still remains outside the test.[1]