🧠 AI with Python – 🔢 Vector Embeddings Explained
Posted on: October 20, 2026
Description:
Large Language Models can understand and generate human language, but how do AI systems identify documents that are similar in meaning, even when they use completely different words?
The answer lies in Vector Embeddings, one of the fundamental components of modern semantic search and Retrieval-Augmented Generation (RAG) systems.
In this project, we'll explore how text is converted into numerical vectors, how semantic similarity is calculated, and why embeddings are essential for building efficient RAG applications.
What Are Vector Embeddings?
Vector Embeddings are numerical representations of text generated by trained machine learning models. Instead of working directly with words, embedding models convert sentences or documents into vectors within a multidimensional space.
For example, consider these sentences:
- "How can I learn Python?"
- "What's the best way to study Python?"
- "The weather is sunny today."
Although the first two sentences use different words, they express similar meanings. An embedding model generally places their numerical representations closer together, while the weather-related sentence is likely to be farther away.
This allows AI applications to compare information based on semantic meaning rather than exact keyword matches.
Generating Embeddings with Python
We'll use Sentence Transformers and the pretrained all-MiniLM-L6-v2 model, which generates 384-dimensional embeddings.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentences = [
"How can I learn Python?",
"What's the best way to study Python?",
"The weather is sunny today."
]
embeddings = model.encode(sentences)
print(embeddings.shape)
Output:
(3, 384)
Each sentence is represented by 384 numerical values. These dimensions are learned features that collectively capture aspects of the text's meaning, rather than individually representing specific words.
Understanding Cosine Similarity
Once embeddings are generated, we need a way to measure how closely related they are.
Cosine Similarity is a commonly used technique that measures the angle between two vectors rather than their magnitude.
Its mathematical range is between -1 and 1, where higher values indicate greater directional similarity.
Using NumPy, we can calculate cosine similarity between our sentence embeddings:
import numpy as np
normalized = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
similarity_matrix = normalized @ normalized.T
print(np.round(similarity_matrix, 3))
The resulting matrix contains similarity scores for every pair of sentences. We generally expect the two Python-related questions to have a higher similarity score than either has with the weather-related sentence.
However, semantic similarity doesn't guarantee factual correctness. Two statements can discuss the same subject while expressing contradictory information.
How Vector Embeddings Power RAG
In a Retrieval-Augmented Generation system, embeddings help identify relevant information before an LLM generates its response.
A typical workflow follows these steps:
- Documents are divided into smaller chunks and converted into embeddings.
- The embeddings are stored in a vector store such as FAISS.
- A user's question is converted into an embedding using the same model.
- Similarity search retrieves the most relevant document chunks.
- The retrieved text is provided to an LLM as context for generating an answer.
This approach allows RAG systems to retrieve information based on meaning rather than relying entirely on keyword matching.
Embeddings themselves don't generate answers. Their primary role is to support efficient information retrieval.
Real-World Applications
Vector embeddings are useful beyond RAG systems. They power semantic search engines, document recommendation systems, duplicate-content detection, and document clustering.
For example, an enterprise knowledge assistant can use embeddings to identify relevant internal documentation, while a customer support system can retrieve troubleshooting instructions related to a user's question.
These applications demonstrate how numerical representations of language make information retrieval more flexible and context-aware.
Final Thoughts
Vector Embeddings form the foundation of semantic search and modern RAG applications. By converting text into numerical representations, AI systems can identify relationships between documents even when their wording differs.
In this project, we explored how to generate embeddings using Sentence Transformers, calculate cosine similarity using NumPy, and understand how vector representations support document retrieval.
Code Snippet:
import numpy as np
from sentence_transformers import SentenceTransformer
# =========================================================
# Step 1 - Load the Embedding Model
# =========================================================
print("=" * 60)
print("Loading Sentence Transformer Model...")
print("=" * 60)
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
print("Embedding model loaded successfully!")
# =========================================================
# Step 2 - Create Sample Sentences
# =========================================================
sentences = [
"How can I learn Python?",
"What's the best way to study Python?",
"The weather is sunny today."
]
print("\nSample Sentences:")
for i, sentence in enumerate(sentences, start=1):
print(f"{i}. {sentence}")
# =========================================================
# Step 3 - Generate Vector Embeddings
# =========================================================
embeddings = model.encode(sentences, convert_to_numpy=True)
print("\n" + "=" * 60)
print("Generated Vector Embeddings")
print("=" * 60)
print(f"Embedding Shape: {embeddings.shape}")
print(f"Embedding Dimensions: {embeddings.shape[1]}")
print(f"First 5 Values: {embeddings[0][:5]}")
# =========================================================
# Step 4 - Normalize Embeddings
# =========================================================
# Normalize vectors to unit length for cosine similarity
normalized_embeddings = embeddings / np.linalg.norm(
embeddings,
axis=1,
keepdims=True
)
print("\nEmbeddings normalized successfully!")
# =========================================================
# Step 5 - Calculate Cosine Similarity
# =========================================================
# Dot product of normalized vectors equals cosine similarity
similarity_matrix = normalized_embeddings @ normalized_embeddings.T
print("\n" + "=" * 60)
print("Cosine Similarity Matrix")
print("=" * 60)
print(np.round(similarity_matrix, 3))
# =========================================================
# Step 6 - Compare Sentence Similarities
# =========================================================
print("\n" + "=" * 60)
print("Sentence Similarity Comparison")
print("=" * 60)
for i in range(len(sentences)):
for j in range(i + 1, len(sentences)):
print(f"\nSentence 1: {sentences[i]}")
print(f"Sentence 2: {sentences[j]}")
print(f"Cosine Similarity: {similarity_matrix[i, j]:.4f}")
# =========================================================
# Step 7 - Find the Most Similar Sentence
# =========================================================
query_index = 0
scores = similarity_matrix[query_index].copy()
# Exclude the original sentence from the comparison
scores[query_index] = -np.inf
best_match_index = int(np.argmax(scores))
print("\n" + "=" * 60)
print("Most Similar Sentence")
print("=" * 60)
print(f"Original: {sentences[query_index]}")
print(f"Best Match: {sentences[best_match_index]}")
print(f"Similarity Score: {scores[best_match_index]:.4f}")
# =========================================================
# Step 8 - Perform Semantic Search with a New Query
# =========================================================
query = "What is a good way to start learning Python?"
# Generate an embedding using the same model
query_embedding = model.encode([query], convert_to_numpy=True)
# Normalize the query embedding
query_embedding = query_embedding / np.linalg.norm(
query_embedding,
axis=1,
keepdims=True
)
# Compare the query against stored sentence embeddings
query_scores = (query_embedding @ normalized_embeddings.T)[0]
# Retrieve the most similar sentence
best_result_index = int(np.argmax(query_scores))
print("\n" + "=" * 60)
print("Semantic Search Results")
print("=" * 60)
print(f"Query: {query}")
print(f"Best Match: {sentences[best_result_index]}")
print(f"Similarity Score: {query_scores[best_result_index]:.4f}")
# =========================================================
# Step 9 - Display Final Summary
# =========================================================
print("\n" + "=" * 60)
print("Vector Embeddings Summary")
print("=" * 60)
print("Embedding Model: all-MiniLM-L6-v2")
print(f"Total Sentences: {len(sentences)}")
print(f"Embedding Dimensions: {embeddings.shape[1]}")
print("Similarity Metric: Cosine Similarity")
print("Application: Semantic Search and RAG Systems")
# =========================================================
# Program Completed
# =========================================================
print("\n" + "=" * 60)
print("Vector Embeddings Demo Completed Successfully!")
print("=" * 60)
No comments yet. Be the first to comment!