Embeddings are dense numerical representations of complex data — words, images, users, products — that capture semantic meaning in a compact vector. They're the foundational technique behind modern recommendation systems, search engines, and large language models, converting everything into comparable geometry.

At their core, embeddings solve a fundamental problem in machine learning: how do you feed a neural network something like a word, a product, or a user identity when networks only understand numbers? The naive approach — one-hot encoding — creates massive sparse vectors that capture no relationships. Embeddings replace these with dense low-dimensional vectors, typically 128 to 1536 dimensions, where similar items cluster together in geometric space. Word2Vec, the seminal 2013 paper from Google, showed that embeddings could capture analogical relationships: king - man + woman ≈ queen in vector arithmetic. Today, embeddings are generated by transformer encoders (like BERT and OpenAI's text-embedding models) and trained on massive corpora. The result: a vector for 'dog' sits close to 'puppy' and 'canine', while staying distant from 'automobile'. Applications span semantic search (find documents by meaning, not keyword), collaborative filtering in recommendation engines, anomaly detection, and retrieval-augmented generation (RAG) pipelines. Understanding embeddings is foundational for any practitioner working with modern AI systems.

What Are Embeddings in Deep Learning?