The Geometry of Categorical and Hierarchical Concepts in LLMs
Building upon our recent work, we formalize the representations of categorical concepts in LLMs as convex polytopes and further show that hierarchical relations between concepts are represented as orthogonality. Under Review. (Figure courtesy: Kiho Park)