Cohere has described CALIBER, a training method that gives reasoning models separate confidence targets before and after answering. Repeated attempts establish how often a question can be solved; the final estimate concerns the particular answer given. The team evaluated the method on mathematics and other question-answering tests, while its use for deciding when to verify remains untested.
AI company Cohere has described CALIBER, a method for teaching a reasoning model to express confidence at two points. One estimate comes before working through a question; the other concerns the answer that has actually been generated. The method puts these estimates around the reasoning and answer within a single response.[1]
Training the starting estimate uses several attempts at the same question. Their success rate becomes the shared target for that question. The final estimate receives a separate target according to whether the individual answer is right. A correct attempt and an incorrect attempt can therefore share the first target while receiving different final targets. Confidence training is combined with a reward for producing the correct answer.[1]
The researchers distinguish whether confidence percentages match observed accuracy from whether correct answers receive higher confidence than incorrect ones. Giving every question the same confidence can satisfy an aggregate accuracy comparison without separating reliable answers from mistakes. The team trained models on mathematics and evaluated additional question-answering tasks without further training for those tasks. Using confidence to decide when to check an answer, seek information or decline to answer remains outside the tested applications.[1]
