Is Cosine-Similarity of Embeddings Really About Similarity?
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Cosine-similarity is the cosine of the angle between two vectors, or equivalently the dot product between their normalizations. A popular application is to quantify semantic similarity between high-dimensional objects by applying cosine-similarity to a learned low-dimensional feature embedding. This can work better but sometimes also worse than the unnormalized dot-product between embedded vectors in practice. To gain insight into this empirical observation, we study embeddings derived from regularized linear models, where closed-form solutions facilitate analytical insights. We derive analytically how cosine-similarity can yield arbitrary and therefore meaningless 'similarities.' For some linear models the similarities are not even unique, while for others they are implicitly controlled by the regularization. We discuss implications beyond linear models: a combination of different regularizations are employed when learning deep models; these have implicit and unintended effects when taking cosine-similarities of the resulting embeddings, rendering results opaque and possibly arbitrary. Based on these insights, we caution against blindly using cosine-similarity and outline alternatives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301
Cited by
Using natural language processing to analyse text data in behavioural science
Feuerriegel, S., Maarouf, A., Bär, D., Geissler, D., Schweisthal, J., Pröllochs, N., Robertson, C. E., Rathje, S., Hartmann, J., Mohammad, S. M., et al. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-024-00392-z · Cited by 140
“However, not all embeddings can be used together with cosine similarity, so caution is indicated 127 .”
Predicting cellular responses to perturbation across diverse contexts with State
Adduri, A. K., Gautam, D., Bevilacqua, B., Naghipourfar, M., Imran, A., Shah, R., Teyssier, N., Verma, R., Carpenter, C., Eraslan, B., et al. (2026) · Cell · doi:10.1016/j.cell.2026.07.052 · Cited by 100
Leveraging long context in retrieval augmented language models for medical question answering
Zhang, G., Xu, Z., Jin, Q., Chen, F., Fang, Y., Liu, Y., Rousseau, J. F., Xu, Z., Lu, Z., Weng, C., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01651-w · Cited by 81
Retrieval-Augmented Generation (RAG)
Klesel, M., & Wittmann, H. F. (2025) · Business & Information Systems Engineering · doi:10.1007/s12599-025-00945-3 · Cited by 92
“Additionally and importantly, the same embedding model that was used for the creation of the vector database must be used to translate the input query into a vector, as the similarity between query and chunks (or documents) in the database is measured using these vectors (Steck et al. 2024).”
Xu, S., Yan, Z., Dai, C., & Wu, F. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1635381 · Cited by 56
Metadata
- Authors
- Steck, H., Ekanadham, C., & Kallus, N.
- Venue
- Companion Proceedings of the ACM Web Conference 2024
- Publisher
- ACM
- Published
- 2024-05-13
- Pages
- 887-890
- OpenAlex
- W4392688329
- Cited by
- 137
- References
- 2
- FWCI
- 41.68
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.