Similarity Metrics

Interview answer (say this first). The three metrics you need are dot product, cosine similarity, and Euclidean (L2) distance. Cosine similarity compares direction only; dot product mixes direction and length; Euclidean distance measures straight-line separation. If every vector is normalised to length 1, all three produce the same ranking, so most text systems normalise once and then use whichever is fastest. pgvector exposes them as <-> (L2), <=> (cosine distance), and <#> (negative inner product).

Why this exists

Once text is a vector, retrieval becomes a ranking problem: given a query vector, order the stored vectors from most to least relevant. To order them you need a number that says “how close”.

Different numbers answer different questions:

• Are they pointing the same way? Cosine similarity.
• Are they aligned, and how strong is each? Dot product.
• How far apart are the points? Euclidean distance.

Pick the wrong one and ranking misbehaves in a specific, predictable way. A concrete failure: two chunks point in the same direction, but one is a short sentence and the other is a long repeated passage. With an unnormalised dot product, the long vector wins even when the short one is the better answer. The system retrieves the wrong chunk, and the query still looks like it worked.

The reverse mistake is just as common: a team normalises vectors when writing them but forgets to normalise the query, or switches the index operator without re-normalising, and every score is subtly off. Nobody gets an exception; recall just drops.

This page pins down the three metrics, the exact algebra that connects them, and the pgvector operators that implement them.

Start from zero

Three definitions do most of the work:

• Similarity means bigger is better; distance means smaller is better. Cosine similarity and dot product are similarities. L2 and cosine distance are distances. pgvector’s <#> is a negation, so it stays a distance-like operator.
• Normalisation removes magnitude. Once vectors are unit length, direction is all that remains, and the metrics collapse into each other.
• Ranking, not raw score, is what retrieval uses. You only need the top k, so any monotonic transform of a metric is as good as the metric itself.

The core idea

Imagine standing on a field. Two people are the same as each other if they face the same compass direction, even if one is one metre away and the other is a kilometre away. That is cosine similarity: it cares only about the angle.

Now imagine you also care about how far each person walked. The dot product rewards both facing the same way and walking far. Euclidean distance asks the completely different question of how far apart two people are on the ground.

The formulas make the relationship exact:

The bridge between them is the law of cosines:

‖a - b‖²  =  ‖a‖² + ‖b‖² - 2·(a·b)

If both vectors are unit length, ‖a‖² = ‖b‖² = 1, so it collapses to:

‖a - b‖²  =  2 - 2·cos(a, b)          for unit vectors

That single line explains why normalisation is so common: for unit vectors, L2 distance is a monotonic transform of cosine similarity, and ranking by one is identical to ranking by the other.

Choosing a metric is then a short decision:

flowchart TD
    A["Do you control the embedding model?"] -->|Yes, text| B["Normalise to unit length"]
    A -->|No, scores matter as given| C["Trust the model card"]
    B --> D["Dot product = cosine<br/>pick the fastest index"]
    C --> E{"Was the model trained<br/>for dot product?"}
    E -->|Yes, e.g. some retrieval models| F["Use inner product"]
    E -->|No| G["Use cosine distance"]
    H["Non-text: pictures, coordinates, counts"] --> I["L2 distance<br/>if magnitude is meaningful"]

The rest of the page is detail about that picture.

How it works

1. Get two vectors of the same dimension. Query q and stored chunk d. If dimensions differ, the comparison is invalid.
2. Compute the dot product. Multiply entry by entry and sum: q·d = Σ q[i]·d[i]. This is one pass over the vectors.
3. Compute each norm if needed. ‖q‖ = sqrt(Σ q[i]²). Cosine divides by both norms.
4. Average for large N. One dot product is D multiplications. Comparing against N vectors is N·D operations — the cost that vector indexes exist to reduce.
5. Normalise before storing (common practice). Divide each vector by its norm at write time and at query time. Now q·d already equals cosine, and the norm is never recomputed.
6. Rank. Sort by descending similarity, or ascending distance. Lower L2 is better; higher cosine is better; for pgvector <#>, lower (more negative) is better.
7. Take the top k. Only the order matters, which is why monotonic transforms are interchangeable.
8. Keep the choice consistent. The metric used to build the index must match the operator used to query it, or the index is bypassed or wrong.

Two subtleties follow from the algebra:

• Cosine is undefined for the zero vector. Dividing by a zero norm is invalid. pgvector simply does not index zero vectors for cosine distance.
• Floating-point ties break down. In high dimensions, many pairs have very close cosine values. Small precision loss from fp16 or int8 quantisation can flip near-ties, which is why re-ranking on full-precision vectors is common.

The syntax you will use

Dot product, norm, and cosine with numpy.

import numpy as np

a = np.array([1.0, 2.0, 3.0])
b = np.array([2.0, 4.0, 6.0])

dot = float(a @ b)                                    # 28.0
cos = float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))   # 1.0

@ is matrix multiplication; for 1-D arrays it is the dot product.

Euclidean distance.

l2 = float(np.linalg.norm(a - b))     # 3.7416573867739413

Subtract, then take the norm of the difference.

Normalise a batch of vectors.

def normalise(x: np.ndarray) -> np.ndarray:
    return x / np.linalg.norm(x, axis=-1, keepdims=True)

unit = normalise(np.array([3.0, 4.0]))    # [0.6, 0.8], norm 1.0

After this, unit_a @ unit_b is exactly the cosine similarity.

Rank stored vectors against a query.

sims = matrix @ q            # matrix shape (N, D), q shape (D,)
top_k = np.argsort(-sims)[:5]   # indices of the 5 best matches

argsort(-sims) sorts descending. This is brute-force search; an ANN index replaces it later.

pgvector’s three operators.

-- L2 distance: smaller is closer
SELECT id FROM chunks ORDER BY embedding <-> :q LIMIT 5;

-- cosine distance: smaller is closer
SELECT id FROM chunks ORDER BY embedding <=> :q LIMIT 5;

-- negative inner product: smaller (more negative) is higher dot product
SELECT id FROM chunks ORDER BY embedding <#> :q LIMIT 5;

<#> returns the negative inner product because Postgres index scans only support ascending order. Sorting ascending on the negation gives you the largest inner products first.

Recover a similarity from a distance.

SELECT 1 - (embedding <=> :q) AS cosine_similarity FROM chunks;
SELECT (embedding <#> :q) * -1 AS inner_product FROM chunks;

These conversions are useful for thresholds and for logging, but keep the bare operator in ORDER BY so the index can be used.

Match the index to the operator.

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);  -- for <=>
CREATE INDEX ON chunks USING hnsw (embedding vector_l2_ops);      -- for <->
CREATE INDEX ON chunks USING hnsw (embedding vector_ip_ops);      -- for <#>

One index per distance function. An index built for cosine does not serve an L2 query.

Normalise inside Postgres.

UPDATE chunks SET embedding = l2_normalize(embedding);
SELECT l2_normalize(embedding) <#> :q FROM chunks;   -- inner product of unit vectors

l2_normalize is a pgvector function; use it if you decide to normalise at the database layer.

Examples: simple to real

Example 1 — the three metrics on simple vectors.

a = np.array([1.0, 2.0, 3.0])
b = np.array([2.0, 4.0, 6.0])   # b is exactly 2x a
c = np.array([-1.0, -2.0, -3.0])  # c is exactly -1x a
d = np.array([1.0, 0.0, 0.0])
e = np.array([0.0, 1.0, 0.0])

a·b = 28.0      cos(a,b) =  1.0     L2(a,b) = 3.7417
a·c = -14.0     cos(a,c) = -1.0     L2(a,c) = 7.4833
d·e =  0.0      cos(d,e) =  0.0     L2(d,e) = 1.4142

a and b point the same way, so cosine is 1. a and c point opposite ways, so cosine is -1. d and e are perpendicular, so cosine is 0 and the dot product is 0.

Example 2 — same direction, different length.

p = np.array([1.0, 2.0, 3.0])
q2 = p * 10          # same direction, 10x longer
float(p @ q2)                                  # 140.0  — grows with length
float(p @ q2 / (np.linalg.norm(p) * np.linalg.norm(q2)))   # 1.0 — length removed

The dot product changed by 100×; the cosine did not move. This is exactly the difference between the two metrics.

Example 3 — the unit-vector identity. For any two unit vectors, ‖u - v‖² = 2 - 2·cos(u, v):

pair 0: ‖u-v‖² = 2.075396   2 - 2·cos = 2.075396   cos = -0.037698
pair 1: ‖u-v‖² = 3.063657   2 - 2·cos = 3.063657   cos = -0.531829
pair 2: ‖u-v‖² = 2.664080   2 - 2·cos = 2.664080   cos = -0.332040
pair 3: ‖u-v‖² = 1.447914   2 - 2·cos = 1.447914   cos =  0.276043
pair 4: ‖u-v‖² = 0.324101   2 - 2·cos = 0.324101   cos =  0.837950

The two columns match to six decimals. The algebra is not an approximation; it is an identity for unit vectors.

Example 4 — ranking is identical on normalised vectors. For six unit vectors scored against one unit query:

dot order        [2 4 1 3 5 0]
cosine order     [2 4 1 3 5 0]
neg L2 order     [2 4 1 3 5 0]

All three give exactly the same neighbour list. So on normalised vectors you can pick the metric by what your index computes fastest, not by what it means.

Example 5 — the magnitude trap, with a clear disagreement. A query is compared against aligned (short, exactly the same direction) and longoff (50× longer, but a little off-direction):

dot(q, aligned) =  1.8547     cos(q, aligned) = 1.0000     L2(q, aligned) =  1.0726
dot(q, longoff) = 43.9626     cos(q, longoff) = 0.9481     L2(q, longoff) = 49.1216

Dot product picks longoff. Cosine and L2 both pick aligned. The metrics disagree because longoff is long enough to dominate the unnormalised dot product despite pointing slightly away. This is the single most important reason text systems normalise: it makes the three metrics agree so this ranking bug cannot happen.

Example 6 — translating pgvector operators into the math. For one query and one stored unit vector:

<->  L2 distance       = 0.0
<=>  cosine distance   = 0.0     (1 - cosine similarity = 1 - 1.0)
<#>  negative inner product = -1.0    (so inner product = 1.0)

All three agree that the vectors are identical. <=> is always 1 - cosine. <#> is always -dot. And for unit vectors, <-> is sqrt(2 - 2·cos), so it ranks the same as <=>.

In production

• Normalise once, then use inner product for speed. Unit-length vectors make <#> and <=> rank identically, and inner-product scans avoid the per-row norm computation.
• Normalise at both write and query time. Normalising only one side reintroduces magnitude into every score. A common bug is a backfill that normalised old rows but a query path that does not.
• Keep the metric consistent with the index. vector_cosine_ops will not serve an <-> query. Mixing them silently drops you back to a sequential scan.
• Beware zero vectors. Cosine distance divides by the norm, so a zero vector is undefined. pgvector does not index zero vectors for cosine.
• Thresholds need the right scale. A cosine similarity of 0.8 means something different per model. Calibrate thresholds on labelled data instead of copying a number from a blog post.
• Do not compare scores across models. Dot products especially are not comparable between embedding models, and even cosine scales differ.
• Precision changes near-ties. High-dimensional vectors have many close scores. fp16 or int8 quantisation can change the top k slightly; re-rank the shortlist in full precision when it matters.
• Use squared L2 when you only rank. The square root is monotonic, so skipping it saves a sqrt per comparison. Only compute the true distance if you display or threshold it.
• L2 and cosine are not the same for unnormalised vectors. For unnormalised vectors, L2 cares about magnitude and cosine does not. Pick deliberately.
• <#> looks negative and that is correct. Order ascending on the negative inner product to get the largest inner products. Multiplying by -1 in ORDER BY is a common bug that disables the index.
• Similarity is not relevance. A high cosine score means “same direction in this space”, not “true answer”. Combine metric choice with good chunking and, often, a re-ranker.

Interview questions

1. What is the difference between dot product and cosine similarity?

Answer. The dot product is Σ a[i]·b[i] and reflects both direction and magnitude. Cosine similarity divides the dot product by both vector norms, removing length and leaving only the angle; it always lies in [-1, 1]. For text, cosine is the usual default because length often carries no meaning. For unit vectors the two are numerically identical.

Follow-up: “When is dot product better?” When the model was trained for it, or when vectors are pre-normalised and you want the cheaper inner-product index path.

Trap. Assuming a larger dot product always means more similar. A long vector can beat a shorter, better-aligned one.

2. How do Euclidean distance and cosine similarity relate?

Answer. The law of cosines gives ‖a - b‖² = ‖a‖² + ‖b‖² - 2·(a·b). For unit vectors this becomes ‖a - b‖² = 2 - 2·cos(a, b). So on normalised vectors, L2 distance is a monotonic transform of cosine similarity, and sorting by one gives exactly the same order as the other.

Follow-up: “Then why does pgvector offer both?” Because the index operator classes are different and one may be faster for your data; also, for unnormalised vectors the two metrics genuinely differ, and L2 is the natural choice when magnitude is meaningful.

Trap. Treating <-> and <=> as interchangeable on unnormalised vectors. They are only equivalent after normalisation.

3. Why does normalisation make ranking equivalent across metrics?

Answer. Normalisation removes magnitude, leaving only direction. On the unit sphere, cosine is just the dot product, and L2 distance is sqrt(2 - 2·dot). Both depend monotonically on the same dot product, so they order candidates identically. You can therefore choose the metric the index computes fastest.

Follow-up: “What is lost by normalising?” Magnitude information. For text embeddings that is usually noise; for image or behavioural vectors it may be signal.

Trap. Normalising the documents but not the query during a migration, which makes scores inconsistent between old and new rows.

4. What does each pgvector operator return?

Answer. <-> returns L2 (Euclidean) distance. <=> returns cosine distance, which is 1 - cosine similarity. <#> returns the negative inner product. There are also <+> for L1 (taxicab), and <~>/<%> for Hamming and Jaccard on binary vectors. All of them are used in ascending ORDER BY because Postgres only supports ascending index scans on operators.

Follow-up: “How do you get the actual similarity?” 1 - (embedding <=> q) for cosine, and (embedding <#> q) * -1 for inner product. Compute the conversion outside ORDER BY so the index still gets used.

Trap. Writing ORDER BY (embedding <#> q) * -1 DESC. That is an expression, not a distance operator, so the planner cannot use the index.

5. What happens with a zero vector?

Answer. Its norm is zero, so cosine similarity divides by zero and is undefined. Euclidean distance is still defined. pgvector avoids the problem by not indexing zero vectors for cosine distance, so they simply never appear in cosine results.

Follow-up: “How would a zero vector appear?” A failed embedding call that returned zeros, or an empty chunk embedded with a model that maps empty input to the zero vector. Validate embeddings before writing them.

Trap. Assuming a zero vector will match everything at distance 0. It will not match at all in a cosine index.

6. Why can two systems use “cosine” and still disagree on results?

Answer. Because cosine is only shape, not content: the vectors came from different embedding models, or different versions, or one side was normalised and the other not, or the stored vectors were quantised. The metric is identical; the vectors in the space are not.

Follow-up: “How do you debug that?” Check the model name on the rows, check norms on both query and document vectors, and compare exact-search results with index results to see whether the discrepancy is the index or the vectors.

Trap. Blaming the metric. The metric is arithmetic; almost all disagreements come from the vectors or from inconsistent preprocessing.

7. When would you choose L2 over cosine for embeddings?

Answer. When magnitude is meaningful — image features, sensor readings, or any vector where “how much” carries information. Also when the model is trained with an L2 objective. For normalised text embeddings the choice rarely matters, and cosine or inner product is the usual default.

Follow-up: “And what about the index?” L2 uses vector_l2_ops. On normalised data it ranks the same as cosine, so choose based on which index you already have or which is measurably faster on your hardware.

Trap. Saying L2 is “worse for text” without qualification. On unnormalised text vectors it is a different metric, not a broken one.

8. How do you handle near-ties and quantisation when ranking?

Answer. Retrieve a larger candidate set with the approximate index — for example ef_search well above k — and then re-rank those candidates using full-precision vectors. This recovers most of the accuracy lost to fp16 or int8 quantisation while keeping the index small and fast.

Follow-up: “How much does quantisation hurt cosine?” It depends on the data. Measure it: compare recall from the quantised index against exact search on a labelled sample rather than assuming a number.

Trap. Re-ranking with the same quantised vectors. Re-ranking only helps if the second pass uses more precision than the index did.

Remember this

• Dot product = direction and length; cosine = direction only; L2 = straight-line distance.
• On unit vectors, ‖a-b‖² = 2 - 2·cos, so all three metrics rank identically.
• Normalise at write and at query time, then inner product equals cosine and is cheaper.
• pgvector operators: <-> L2, <=> cosine distance, <#> negative inner product — all ascending, and each needs its matching operator class.
• Magnitude is the silent bug. Unnormalised dot product ranks long vectors higher regardless of topic.