Label-only training leads financial adaptation in a LoRA comparison
A new preprint compares training on classification labels with adding a larger model’s written reasoning. In a financial-headline task, label-only adaptation improved the target metrics while keeping general capabilities near the unchanged model. Adding full rationales, splitting them across adapters or shortening them did not extend those gains under the study’s matched training setup.
Artificial Intelligence··Evening
Financial headlines become training examples
NVIDIA researchers compared several ways of adapting Qwen3-8B to classify financial headlines. Their preprint uses 200,000 examples from FinHeadlineMix, with labels and written rationales generated by a larger teacher model. Low-rank adaptation, or LoRA, trains a limited set of additional weights while keeping the backbone fixed. One regime learned only the final label. Another learned a rationale followed by the label. A third divided the available adapter capacity between the two training signals.[1]
Labels lead on the classification task
The label-only regime improved the financial task while leaving general capabilities close to the unchanged model. Its largest gains concerned calibration and which class received the highest probability. Calibration measures how predicted probabilities align with outcomes. Adding full teacher rationales left the task metrics behind their label-only counterparts under the matched configurations. A broad knowledge benchmark could improve slightly, while the other general-capability checks stayed within ordinary variation across runs. Each training configuration used three seeds.[1]
Compressed rationales and split adapters
Splitting the rank budget between a rationale adapter and a label adapter brought no further gain in the comparison. The researchers also asked the teacher to summarize its rationales to different target lengths before training. Shorter summaries moved calibration towards the label-only results but did not reach them. The experiment followed a staged hyperparameter search and used one student backbone, a financial classification task and separate tests of knowledge, scientific reasoning and instruction following.[1]