AI search over documentation: RAG and semantic search

Semantic search across documentation: where it breaks

Search across a knowledge base doesn't break at implementation time; it breaks later: the corpus has grown from hundreds of documents to tens of thousands, while the retrieval logic has stayed the same.

An employee asks “how to process a return on a pre-order” — and gets the warranty policy: the words match, the meaning doesn't. After a few such misses, the person stops searching altogether.

The root cause is that exact-word search requires the person writing the query to guess the wording used by the document author. But terminology diverges: abbreviations, synonyms, cases and tenses, different names for the same entity across departments.

Lemmatization and a synonym dictionary cover some cases, but they don't cover paraphrases — and people ask precisely by paraphrasing.

A separate breakdown is fragmented document sources. The internal wiki, network folders, scanned contracts, email attachments, and exports from accounting systems live on their own: versions diverge, some files exist in two editions without indicating which one is current.

An index built from one source answers confidently and incorrectly — it simply doesn't see the rest.

The cost of irrelevant results is not measured in response seconds. An employee who has twice gotten the wrong thing goes to ask a knowledgeable colleague in chat — and the same question resurfaces again with the next newcomer.

The knowledge base is formally alive, but in practice people bypass it, and ready-made solutions get duplicated.

Semantic search across documentation addresses exactly this class of failures: the query is matched against the meaning of a fragment, not a set of matching words. We'll break down how it works and what it takes for search results to inspire trust again.

What changes after implementation: response time and accuracy

The effect is measured by four metrics: time to answer, accuracy of hitting the right fragment, share of answers with a source link, and volume of routine support requests.

Before implementation, these have to be collected manually — from search bar logs and reviews of closed tickets; after, they are calculated automatically on every query.

Time to answer is reduced not by “faster search” but by a change in mode. Keyword search requires the user to guess the wording: the wrong term means empty results, a second or third query iteration, then a call to a colleague.

Semantic search and the RAG approach search by meaning, so the answer comes on the first attempt — a task that took tens of minutes of waiting turns into a minute or two, including reading the source.

Accuracy grows because the model answers not “from memory” but from retrieved fragments, and every answer carries a link to the document, section, and version.

The share of answers with a source link stays close to full — this is the protection against the “answer off target” failure class: if no fragments are found, the system honestly says so instead of offering a plausible fabrication.

We separately maintain a relevance threshold and a list of topics that are routed to a human specialist by rule.

The support load drops unevenly, and that matters more than the overall number. First line stops being an information desk: routine questions about policies, contracts, and internal instructions are resolved through self-service, and only disputed cases reach engineers.

We record metrics before launch, repeat the measurement after a month of operation, and analyze queries with no hit — these show exactly which documents are missing from the index or which sections should be rewritten.

Indexing and retrieval modes: comparing approaches

The retrieval scheme determines what actually makes it into the answer: exact term matches, semantically close fragments, or a combined list.

This is not a choice of “better or worse” but different error profiles — and the mode should be selected for the query flow, not for a trendy term.

The key point for implementation: the modes are compatible with each other, not mutually exclusive. Both indexes are built from the same document corpus and run in parallel, while the retrieval method is switched by a service parameter — no reindexing is required.

This allows hybrid retrieval to be introduced gradually: first, reranking is enabled for part of the queries, the results are compared with the baseline on a reference set, and any change can be rolled back with a single switch.

Further down the chain, RAG takes the same selected fragments — only the number of candidates sent to generation changes. That's why answer quality rests on retrieval, not on the model: no reranking can save weak retrieval.

Implementation stages: from corpus audit to handover

The order of work is built from data to validation: first we understand what is in the corpus and in what form, then we design the search scheme, and only then do we configure processing.

This order removes the main risk — search that works confidently on showcase examples and falls apart on real policies and contracts.

1. Document corpus audit. We collect all sources, formats, and editions, flag duplicates, outdated versions, scans without a text layer, and documents with tables and embedded graphics. The output is a corpus map with volume, share of problematic files, and access rights by group.
2. Section markup. We define what counts as a whole search unit — a clause, subsection, or block of paragraphs — and preserve the heading hierarchy. Then the answer links to a specific document clause, not the entire file.
3. Scheme design. We define how vector representations of fragments are formed, how the link to the source is stored, and how access-right filters are applied.
4. Processing pipeline setup. Scan recognition, chunking, index updates when a document is edited — without full corpus reindexing.
5. Control questions. We prepare a set of several dozen questions with reference answers and the correct source, and measure whether the right fragment appears in the top lines of the results.
6. Result acceptance and handover. We review measurements, the change log, and the instructions for adding new documents, and train employees.

Each stage ends with an artifact that can be checked: a corpus map, a chunking scheme, a set of control questions with results. At acceptance, you see working semantic search on your documents, not a presentation.

What we hand over: schemes, policies, access

The solution remains in your environment, not in our infrastructure. That's why we hand over not only a working service but also a set of artifacts that the support team can use to develop it independently.

• Pipeline diagram — the document's path from intake to answer: format parsing, chunking into fragments, vectorization, search, and reranking. It notes at which step what breaks and how to notice it.
• Index update policy — schedule and triggers for rebuilds, rules for handling changed and deleted documents, and the point at which a rebuild is mandatory.
• Access policies — role and group separation: the answer shows only what the user has rights to in the source system.
• Integration map — sources, connection methods, service accounts, synchronization frequency, and failure points.
• Operations documentation — startup, monitoring, common failures, and procedures for when answer quality drops.
• A set of control questions with reference answers — to measure quality before and after changes to the model or data.
• Decision log — why this chunking, fragment size, and model were chosen: so that six months later you don't have to figure it out again.

We accompany the handover with a walkthrough with your team: we show what is where and how to verify a change. After that, sources, documents, and the model can be changed by your own team.

Case study: how answer accuracy and retrieval speed grew

A semantic search implementation case: logistics operator Severny Put stores 40,000 pages of policies, equipment instructions, and standard contracts.

Answers were searched for manually: an employee ran a keyword search, opened five to ten files, and still went to clarify with the document author.

The client waited 20–40 minutes for an answer, and employees gave some wording from memory — without confirmation from the primary source.

Metrics before and after

We split the documents into semantic fragments and linked each one to its source section and revision date. Search now works by the meaning of the query, not by word matching: the answer is assembled from the found fragments and must cite them.

If there is no confirmation in the database, the system says so directly instead of offering a plausible guess.

Control set and answer reliability

There were no eyeball estimates: we assembled a control set of 120 real employee questions, with the correct document known in advance for each. Answer reliability was calculated as the share of queries where the system found the right fragment and cited it.

We run the set after every change to the search index — we catch regressions before release, not while working with a client.

How does vector matching differ from word matching?

Word matching relies on an inverted index: a document is split into terms, for each term a list of fragments is stored, and weight is calculated by the term's frequency in the text and its rarity across the entire array.

A query finds exactly those paragraphs where the words appear literally — including SKUs, clause numbers, and abbreviations.

The flip side is synonyms and typos: “agreement for the purchase of machine tools” and “suplly contract” are already a different story for lexical search; it will see no matches.

Vector matching works with meaning: both the query and the fragment are turned into numerical representations whose proximity is compared directly.

The right section is found even if there are no shared words at all — it handles reformulations, typos, and colloquial variants confidently.

But where the letter matters — a product code, revision number, or policy clause — the vector leads to a thematically similar but incorrect fragment.

That's why in practice hybrid schemes most often win: both rankings run in parallel, the results are combined, and reranking resolves disputed positions.

This mode is natively supported, for example, by Elasticsearch — the lexical index, vectors, and metadata filters live in one store rather than in two disconnected systems.

Configuring a hybrid is calibration, not a switch. The weight of each ranking is selected on a set of real employee queries, and separate rules assign priority to exact matches and fresh document revisions.

We measure the result by the share of queries where the right fragment made it into the top five and by answer completeness — these two numbers show where synonyms and typos are still being lost.

Concerns about answer quality and data privacy

Fabricated answers are almost always a consequence not of search but of generation: if retrieval brings irrelevant fragments, the language model fills in plausible text.

That's why the answer is assembled only from retrieved chunks, with a mandatory link to the source document, and when relevance is low, the system honestly reports that there is no answer in the database.

The second source of doubt is access rights and markup quality. The access label is inherited from the document to each fragment and applied at the search stage, not after generation: an employee physically cannot get a paragraph they don't have rights to.

How accurately documents are split into fragments determines hit rate — we check this part on reference questions before launch.

How are access rights applied to fragments?

Rights are set at the fragment level, not the whole document — otherwise restricted sections leak into general retrieval. The access filter triggers before generation, so a forbidden chunk enters neither the answer nor the model context. When rights change, the document is reindexed.

Can search be deployed in a closed environment?

Yes: indexing, storage, and generation run on your infrastructure, without calls to external services. Documents do not leave the environment, login is through your corporate authorization, and query logs remain under your control.

Let's discuss the task: we'll prepare an estimate and work plan

For the estimate to be engineering-based rather than “plus or minus an order of magnitude,” we need not a retelling of the task but a picture of the data: how many documents, in what formats and languages, how often they are updated, who searches and under what roles.

With this set of inputs, it is already clear which fragment splitting and indexing strategy will fit, and where access coordination will be required.

In response to a task description, we send an estimate and plan: the search scheme, stage composition, and checkpoints. We immediately build quality metrics and the method for measuring them on your corpus into the plan.

• stage-by-stage scope of work: corpus preparation, markup, indexing, search layer, integration with your interfaces;
• data requirements: formats, languages, versioning, access rights, update frequency;
• architecture diagram: where vectors are stored, how only changed documents are reindexed;
• quality metrics and the methodology for measuring them on your corpus before work begins;
• launch order: a pilot on one subject area, then expansion to the rest;
• risks and rollback plan: return to the previous search without losing documents or settings.

Describe the task — what documents, who searches, and what result you consider a failure — and we will come back with an estimate, scheme, and implementation plan.