The Geometry of Categorical and Hierarchical Concepts in LLMs
Building upon our ICML 2024 paper, we formalize the representations of categorical concepts in LLMs as convex polytopes and further show that hierarchically related concepts are represented as orthogonal vectors. Accepted to ICLR 2025 for oral presentation.