AI curation of the knowledge base: where answer search breaks
The knowledge base breaks not at publication, but after several months of operation. Duplicates appear in the knowledge base: the same instruction lives in three sections, each with its own edits, and it is unclear which is primary. No one retires outdated articles — they continue to be found by search and rank above current ones.
Document version desynchronization is a separate failure line. A regulation was updated in one repository, the copy in another remained unchanged, and the employee finds the one closest in query wording, not the one in effect.
Nearby are lost sections: material was moved between spaces, but links, access rights, and backlinks did not fully move.
For the user this looks like “search does not find the answer.” A person spends not a minute but dozens of minutes on one question, goes to ask in chat — and the knowledge again settles in the conversation, bypassing the base.
Such failures are measured by the share of queries without a result and time to first relevant answer, not by the number of articles in the catalog.
AI curation of the knowledge base works with content, not formatting.
A semantic index matches the question with a text fragment, not with a section heading; deduplication reduces copies to a single canonical source with an explicit owner; outdated articles get a review deadline and move down until they are confirmed.
Next, we break down by what signals this is detected during an audit and how to embed verification into a regular cycle so the base does not return to its original state.
What changes when the answer is found in one query
A typical client question is searched for minutes: some knowledge has settled in conversations, some in a wiki space, some remained only with the on-call engineer.
While the employee cross-checks three or four sources, the task is stalled, and support receives the same question again. Curation reduces the route to one step: query — single source of truth — answer.
First we measure how long the path to the answer takes and what share of articles are outdated. Usually a noticeable portion of materials has not been updated since the last product change, and it is precisely those that are referenced in answers.
After consolidating sources and marking owners, time to answer is reduced manyfold — from minutes to tens of seconds.
The second measurable effect is the load on support. When the answer is found in one query, the share of repeat requests decreases: a person does not return with the same question a week later. We count requests per topic and the share of those closed without an engineer's involvement.
Content freshness stops being a manual task: each article gets a review deadline and a responsible person. Materials without an owner go to the archive rather than surfacing in search.
In repositories like Confluence this is configured by rules, but it is not the setting that works — it is the regular revision cycle.
The metrics panel keeps the picture visible: time to answer, share of outdated articles, repeat requests, topic coverage by owners. It shows where the base is degrading, and curation stops being a one-off project.
Curation approaches: what to choose for the task
The choice of curation mode is determined not by the size of the base, but by the frequency of changes in the subject area and the cost of an error in the answer.
Where the rubric scheme is outdated and duplicates occupy a noticeable share of the reference, point edits only stretch out the process — a full reworking of the base with new markup and rebuilt links is needed.
Where individual cards diverge from practice, gradual updating is safer: the reference continues to answer while replacement is underway.
How the modes compare
| Mode | When it fits | Limitations |
|---|---|---|
| Full reworking of the base | Rubrics and links are outdated, answers lose trust, duplicates are more than a third | The base freezes during work, links are restored again, a rollback plan is needed |
| Gradual updating | The reference is working, but some cards contradict current practice | Outdated and new versions coexist for months, search ranks both equally |
| Manual curation | Regulated topics: standards, law, HR documents | Throughput is limited by people, the edit queue grows faster than the base |
| Automatic markup | Large flow of new documents, draft sorting is needed | Classification errors on edge cases, without verification there is no source of truth |
| Hybrid mode | Heterogeneous corpus: mass documents plus sensitive sections | Requires escalation rules and an explicit owner for a disputed card |
Pure modes are almost never found in practice: automatic markup covers volume, manual curation covers disputed and regulated topics, and hybrid mode fixes who confirms a draft label and on what basis.
Next we break down how to estimate the scope of work and not lose the reference's operability in the process.
How we run the project: stages from audit to handover
The project proceeds in waves: first we record what is actually in the base, then we build links, then we set rules, and only after that do we put automation into the working loop. Each stage closes with an artifact — without it we do not move to the next.
- Knowledge base audit. We export all materials and mark them by type, duplicates, and outdated versions. The output is a list of sections with an assessment of freshness and topic coverage.
- Content map. We link materials by meaning, mark gaps and places where two answers contradict each other. It is visible which topics are covered confidently and where a source is needed.
- Maintenance rules. We agree on who adds material, in what form, how versions and record lifespan are marked. This determines whether the base will be updated further or overgrown again.
- Curation setup. AI distributes new materials across sections, marks duplicates, and suggests merging close records — nothing is deleted without an editor's confirmation.
- Testing on live queries. We run real user questions: where the answer is complete, where it is outdated, where it diverges from a neighboring section. We fix the rules, not individual answers.
- Documentation handover. The link schema, update regulation, change log, and instructions for editors — after that the team maintains the base on its own.
What remains in your hands after handover
We hand over not a folder of reports, but a working curation loop that then lives without us. Each artifact has an owner, an entry point, and a procedure — editing the base does not require rebuilding the process from scratch.
- Knowledge base map: sections, sources, and links between documents. It immediately shows where duplicates are, which topics are not covered, and which sections contradict each other.
- Update regulation: how often the index is rebuilt, who initiates an edit, and within what period an outdated fragment leaves the search results.
- Roles and access schema: author, reviewer, publisher; separately — rights to export and delete sections. Access is granted by role, not by person, so an employee's departure does not break the process.
- Editor instructions: how to formulate a fragment, which metadata is mandatory, how to format headings. Quality stops depending on who exactly sat down to edit.
- Quality dashboard: share of answers without source support, volume of outdated fragments, topic coverage. Each metric has a threshold at which a review is triggered.
- Change log: who edited what and when, which source the change came from, and how to roll back to the previous version of a block.
- Set of regression checks: control questions are run after each edit — it shows whether quality has dropped on neighboring topics.
Such a kit allows you to take the base into operation on your own: update, verify, and roll back changes without contacting us.
Case: there were six sources of truth, now there is one
The reference lived in six places: a wiki space, regulation files, support conversations, a ticket base, pinned messages, and a spreadsheet with operating instructions.
The pre-curation audit showed: 41% of materials contradicted each other or described processes taken out of operation, about a third of cards duplicated one another with different wording, and finding an answer took up to 22 minutes and still ended with manually editing the text directly in the conversation.
We consolidated the sources into a single registry based on Confluence and marked every card: owner, confirmation deadline, link to the regulation, status.
A freshness rule moves to the archive any material the owner did not confirm on time, while the collector pulls data from the ticket system and highlights discrepancies between the card and the actual process — edits enter the review queue rather than getting lost in personal messages.
In the first quarter of work, the registry shrank from 4,800 cards to 1,700: duplicates were merged, outdated ones were moved to the archive rather than deleted — change history and rollback were preserved.
The share of outdated materials fell from 41% to 6%, time to answer — from 22 to 4 minutes, manual edits — from over three hundred to forty per month.
The metrics panel shows the share of overdue confirmations and the number of discrepancies per section, so degradation becomes visible earlier than support notices it.
What to do if the reference is already deployed and in use
A deployed and already working reference is not closed for rebuilding: AI curation is launched in a shadow loop that reads the same base as the working version but writes only to a separate revision space.
Readers get familiar answers, while sections simultaneously go through new rules for markup, links, and source verification. The divergence between loops is visible on the monitoring panel before it reaches the user.
Migration proceeds in stages, by section. Routing by namespace prefix: migrated branches are served by the new loop, while the rest continue to work under previous settings.
This way the transfer volume is split into portions, each checked against a control set of questions and rolled back without affecting neighboring ones.
We maintain link backward compatibility at the identifier level: the page number and address do not change, even if the heading or place in the tree has moved.
Old addresses and anchors to subsections live through a redirect layer — bookmarks, links from regulations, and internal transitions do not fall off. The share of broken transitions is zero; this is an acceptance criterion, not a wish.
Switching is also not simultaneous: first parallel indexing and verification of answers on reference questions, then traffic transfer in shares, with the old loop in reserve. Rollback is returning the pointer to the previous snapshot, minutes, not parsing an archive.
As a result, the base does not stop for even an hour, and the result is read by metrics: share of questions without a found answer, accuracy on the control set, number of sections that have passed a full curation cycle.
Common technical doubts before starting the project
Before starting knowledge base curation, the same set of questions comes up: who sees the documents, how long the work will take, who is responsible for the meaning of edits, and how to prove the effect.
We answer each point specifically, without "it depends on the task" where the mechanics can be named.
Where do our documents go and who sees them?
Processing takes place in your loop or on a dedicated stand: documents do not go to external training services. We grant access by role, requests are written to a log, and personal fields are masked during preparation. Upon completion — export of indexes and confirmation of deletion of working copies.
How long does curation take?
Auditing the structure, duplicates, and outdated versions takes from several days to two weeks, after which work proceeds in iterations according to an agreed schedule. A regular cycle keeps the base current: new documents take their place, obsolete ones are marked as outdated.
Who is responsible for the content of the base?
We are responsible for structure, markup, links, and the operation of update rules. Content decisions — what counts as a correct answer — remain with your experts: a semantic edit is confirmed by the owner of the area. This removes situations where wording was changed and the team learned about it from an incident.
How do you measure the result?
At the start we record a baseline: the share of questions for which an answer is found in the base, the share of conflicting and outdated fragments, time to a working link during search.
Then these indicators go to the panel, and the effect of curation is visible in dynamics, not by feel.
Let's discuss the task and return with an estimate and work plan
The estimate of the scope of work for AI curation of the knowledge base depends on the composition of the source data and how the reference lives now, not on the number of documents in it.
To return with a curation plan rather than general considerations, a primary slice is enough for us — send what you have at hand, there is no strict format:
- how the reference is organized: sections, tags, versions, links between articles, and whether there is a responsible person for each block;
- who uses it — support, engineers, managers, clients — and through which channels questions arrive;
- which topics are asked most often and where answers are searched longest before escalating;
- what has already been tried: search, manual markup, regular proofreading, automatic suggestions — and why it did not take hold;
- how content is updated: who writes, how often, what happens to outdated articles and duplicates;
- which systems the reference must remain compatible with and what access restrictions exist.
This is enough to discuss the task substantively and break the work into stages: audit of the current state, recommendations on structure and update rules, quality control dashboard.
Describe the task and send the slice — we will return with an estimate of the scope of work and a curation plan where each stage has a visible result and checkpoint.
