Most explainers teach developers how to build with embeddings. This one is for people who publish content: how the numbers work, how AI engines use them to shortlist passages, and which parts of "embedding optimization" are real.
What Vector Embeddings Are
An embedding model turns a piece of content (text, an image, audio) into a long list of numbers, a vector, that captures what it means.
Passages with similar meaning get vectors that sit close together; unrelated passages sit far apart. Software can't compare two paragraphs the way you do, but it can compare two lists of numbers instantly, so "is this passage about the same thing as that query?" becomes a calculation.
A real embedding looks like [0.012, -0.738, 0.291, ...] and runs for hundreds or thousands of values. No single number means anything on its own, the same way no single pixel makes a photo.
It matters for publishers because ChatGPT, Perplexity, and Google AI Overviews all rely on retrieval to choose what to cite, and embeddings are core machinery in modern retrieval.
Similarity between the question's embedding and your content's embedding is part of how candidate passages get shortlisted. It is also the mechanism behind advice you've already heard: write clearly, cover one idea per section, use consistent terms. That advice works partly because of how this layer behaves.
How Embeddings are Made
They come from embedding models, usually neural networks trained on huge amounts of text until they learn which words and phrases appear in similar contexts.
The geometry that results supports odd arithmetic. In the classic word2vec demo, taking "king," subtracting "man," and adding "woman" lands nearest to "queen" (the demo excludes the input words from the answer, which is worth knowing before you repeat it on stage).
Early models like word2vec and GloVe were static: one vector per word, so a word like "set" with hundreds of senses got a single averaged vector.
Transformer models such as BERT read the whole input first, so "bank" near "river" and "bank" near "loan" get different vectors. Modern models extend this to whole sentences, passages, and documents.
For search, the form that matters is the passage: many AI retrieval systems split content into chunks and store an embedding for each.
| Model | Dimensions | Note |
| BERT base / large | 768 / 1,024 | 768 is an architecture choice (attention-head size times head count), not a magic number |
| OpenAI text-embedding-3-small | 1,536 | Can be shortened through a dimensions parameter |
| OpenAI text-embedding-3-large | 3,072 | Input capped at 8,192 tokens across these models |
More dimensions can capture finer distinctions at the cost of storage and speed. But dimension count isn't a content-quality lever: you can't choose what production engines use.
The embedding size GPT-4 uses internally isn't published, whatever a tool vendor claims.
How AI Engines Use Embeddings to Pick What They Cite
- The question is embedded, often after the engine splits one prompt into several sub-queries (query fan-out). It must use the same model that built the index, because vectors from different models aren't comparable.
- Candidates are retrieved: a similarity search pulls the passages closest to the query vector.
- Candidates are often reranked: a second model may rescore the shortlist, with signals like freshness and authority layered in.
- The model writes, citing a subset of the surviving passages.
One documented exception: Google states that AI Overviews pull candidates through its core Search ranking systems, with AI Mode adding query fan-out, so embeddings work inside classic ranking there rather than as a standalone vector index.
ChatGPT Search and Perplexity appear closer to the pipeline above based on observed behavior, but live web-search internals aren't fully published.
Three points get missed constantly:
- Retrieval is not citation. Similarity gets you onto the shortlist. A passage can be retrieved and never cited, outranked at the last step by a clearer or more authoritative one. Embeddings decide who is in the room, not who gets quoted.
- Production search is hybrid. Stacks pair keyword scoring (BM25) with vector similarity and merge results. Microsoft's Azure AI Search documentation says exact-match queries such as product codes, jargon, and names do better with keyword search. Anyone saying keywords are dead is ignoring how retrieval is built.
- "Does ChatGPT use embeddings?" has two answers. Internally, every LLM represents tokens as vectors, which has nothing to do with your visibility. Separately, retrieval decides what reaches the model; OpenAI documents that its file search uses vector stores with semantic and keyword search. The retrieval layer is where your content competes.
Training and retrieval are different doors: a page published yesterday can be retrieved and cited today, because retrieval reads the live index, not the model's training snapshot.
What Embeddings Mean for How You Write
Modern retrieval systems often chunk and embed passages, not whole pages, so your 3,000-word guide may compete as many separate units. Part of that is a hard limit: embedding models cap their input, and a vector forced to summarize five subtopics becomes an average of all of them, distinct for none. Systems chunk because focused chunks give sharper vectors. Three rules follow:
- One idea per passage. A section that opens with its point and stays on it produces a concentrated embedding that matches its question. One that wanders across three topics matches nothing decisively. This is the mechanical reason answer-first paragraphs and self-contained sections hold up, while "chunking hacks" sold around them don't.
- Make each passage survive alone. Retrieval pulls a passage out of its page. A chunk that says "this setting also affects it" with no referent arrives ambiguous. Name the subject.
- Keep terminology consistent. Embeddings handle synonyms far better than keyword matching, but calling one feature three names still spreads your semantic signal thinner.
A small public test points the same way. Chris Green embedded the same information in three formats and found Q&A format delivered the highest semantic relevance in every scenario, with dense prose worst. He labels it a small simulation, and vector similarity isn't ranking.
On long pages being "averaged out," the mechanism is plausible given chunk boundaries, but the guide found no controlled public evidence that length itself hurts retrieval once chunks are well formed.
Treat it as a claim. The defensible version: structure a long page so every section could be lifted out and still make sense.
Can You Optimize for Embeddings?
Mostly no, and the parts that work aren't what the hype suggests. You can't see or query the production models behind ChatGPT, Perplexity, or Google, so a tool selling a "vector similarity grade" measures your content against a model it chose, not the one deciding your visibility. Those numbers can disagree, and you can't check. There's also no stable target: engines swap models, which means re-embedding, so content tuned to one model's quirks inherits nothing.
What holds up is unglamorous:
- Disambiguation. "geotoolbox tracks citations across eight AI engines" embeds distinctly. "Our platform delivers results" embeds like ten thousand other vague pages. Specificity is separation in vector space.
- Structure and consistency, meaning the three rules above.
None of this is new advice, and that's the tell: when the practical recommendations match clear writing, "embedding optimization" is renaming fundamentals, not offering an edge.
One more caution. Embeddings inherit biases from their training text. A well-known 2016 paper (Bolukbasi et al.) found embeddings trained on Google News exhibited gender stereotypes "to a disturbing extent," so similarity scores carry the blind spots of the text a model learned from. The pragmatic discipline is to measure outcomes instead of intermediates: you can't audit an engine's vectors, but you can check whether engines cite you, for which prompts, against which competitors. That is noisy, so read trends, not single checks.
After Retrieval: What Wins the Citation?
Observable factors are reranked relevance (does the passage answer the exact question), self-sufficiency (can it be quoted without its page), and source-level signals such as authority and freshness that engines say they layer in. The weights aren't published, and anyone quoting exact percentages is improvising. Embeddings are the qualifying race; the writing quality of the individual passage is most of the final.
Embeddings, Entities and Semantic Search: Which Term Means What
| Term | What it is | Relation to embeddings |
| Keyword | A literal string you match | The complement, not the predecessor; hybrid retrieval scores both |
| Vector | Any list of numbers | In retrieval, "vector" and "embedding" are used interchangeably |
| Vector embedding | A vector encoding meaning, made by a model | The subject of this page |
| Entity | A specific named thing (person, brand, product) in a knowledge graph | A largely separate layer: explicit records with relationships, not statistical geometry |
| Semantic search | Search that matches meaning rather than exact words | The application embeddings make possible |
| Vector database | A database built to store and query embeddings at scale | The infrastructure (for example Pinecone or pgvector) |
| RAG | Retrieve passages, then feed them to a model before it answers | The pipeline that puts embeddings to work in AI search |
The mix-up that causes most arguments is entities versus embeddings. An entity is a database row: Google knows a brand is a software product with specific properties. An embedding is a position in space: this paragraph sits near other paragraphs about a topic. You can be strong in one layer and invisible in the other, so they deserve separate attention.
Frequently Asked Questions
What are Dense and Sparse Embeddings?
Dense embeddings are the compact vectors described here, where nearly every number carries information. Sparse embeddings are mostly zeros, with each position tied to an explicit term, like a classic keyword index. Hybrid systems use dense vectors for meaning and sparse signals for exact matches.
Does Perplexity Work like ChatGPT Search?
Directionally, yes: both retrieve passages before answering. They differ in what they retrieve from. The guide cites overlap studies finding Perplexity's citations track Google's results more closely than ChatGPT's do, so a page can be cited by one engine and invisible to the other.
Are Embeddings the Same as Keywords?
No. Keywords are literal strings; embeddings encode meaning, so "fix checkout errors" and "resolve payment failures" land close together despite sharing no words. Keywords still matter in hybrid systems for names, codes, and jargon, and as evidence of what people ask.
Does Schema Markup Change your Embeddings?
Not directly. Text embeddings come from your visible content, and structured data is a separate signal. Schema still helps with entity disambiguation and rich results, just not by moving your vectors.
Do Embeddings Go Stale?
The vectors don't decay, but they're snapshots: a rewritten page changes nothing until the engine recrawls and re-embeds it. Freshness in AI search comes from retrieval picking up current content, which is why crawl access matters more than any trick.
Related Searches: