The Information Geometry of Softmax: Probing and Steering
We leverage information geometry to formalize the semantic structure of embedding spaces induced by the softmax architecture, common in LLMs and CLIP models. Under review.
Vision Language Models