The Information Geometry of Softmax: Probing and Steering
We leverage information geometry to formalize the semantic structure of embedding spaces induced by the softmax architecture, common in LLMs and VLMs. Accepted to ICML 2026.
Vision Language Models