Behavioral estimates of conceptual structure are robust across tasks in humans but not large language models
Published in Singapore, 2023
Recommended citation: Ranganathan, V., Suresh, S., Mathur, Y., Subramanyam, N., & Barbosa, D. (2020, March). Grcluster: a score function to model hierarchy in knowledge graph embeddings. In Proceedings of the 35th Annual ACM Symposium on Applied Computing (pp. 964-971). http://siddsuresh97.github.io/files/emnlp2023.pdf
‘This paper is about estimating conceptual structure in humans and language models using different behavorial tasks. We find that human conceptual structure is coherent across all tasks suggesting that humans have a conceptual core. However, we find the opposite in LLMs. In other words, LLMs do not have a conceptual core and are very sensitive to the context. Moreover, we also find that there are certain ways of prompting LLMs that can make them more human-like. Download paper here
Recommended citation: