A new RoleQ preprint changes which weights share a scale when language models are compressed to very few bits. The choice follows whether a direction reads inputs or writes outputs. Its ternary configuration combines rotation with a short scale-training stage. Experiments measure storage and task accuracy; the single study does not measure inference latency.
Johannes Knittel worked with Alec Helbling and Hanspeter Pfister on RoleQ, a language-model compression framework that changes how weights share a scale. Their October 15 arXiv preprint assigns different scale axes to directions that read inputs and those that write outputs. This single study tests a structural choice based on computational function rather than the weight matrix’s storage layout.[1]
Its RoleQT configuration rotates the weight representation and expresses weights with ternary codes, which use three possible values. A short distillation stage then adjusts scales while keeping those codes fixed. The axis-selection rule needs no data-driven search, although unlabeled inputs are still needed to calibrate the quantization solver. For the base Qwen3-8B configuration, the authors give 1.59 as the mean number of bits stored per weight; extra ternary planes allocate more capacity to sensitive weights.[1]
Adding capacity improves measured task results but increases storage. Tests cover language modeling and a limited set of benchmarks across Qwen sizes and a Llama model. Some Llama test conditions favor a competing compression approach. Inference latency is not measured, and some smaller representations retain significant quality losses. The experiments therefore distinguish stored size and task accuracy from operational speed.[1]
