Why a social robot falls back into a template on the third reply
For the first two or three replies, the companion robot holds up confidently; after that, the conversation crumbles into memorized phrases.
The user notices this before the engineer does: they have already said twice what the conversation is about, and the robot greets them again and suggests the topic anew.
The cause is usually not the model itself, but the way the dialogue is assembled. If only the last reply is sent in the request, history is not preserved — and context resets after every phrase.
Service insertions and script prompts overflow the window, earlier replies are pushed out, and the companion robot rolls back to the beginning of the script branch.
A simple test: after three turns, ask what the interlocutor's name is and what they were talking about a minute ago.
In a noisy room, speech recognition makes mistakes twice: it misses a quiet reply and mistakes someone else's conversation for it. A TV, a second speaker at the table, clinking dishes produce false triggers, and the robot responds to the wrong person and the wrong thing.
This is treated by filtering by voice direction, a confidence threshold, and confirmation before action, rather than by raising microphone sensitivity.
Response delay breaks the rhythm more than inaccuracy does: a pause of several seconds reads as a malfunction, and the person leaves.
Short acknowledgment replies help while a substantive answer is being assembled, along with local branches such as a timer or weather and splitting a long reply into meaningful chunks.
Degrading to pre-recorded phrases is not protection but a cover-up: instead of a clarifying question, the robot repeats the script, and the conversation dies completely. The right behavior when unsure is to ask again, honestly change the subject, or admit that it did not hear.
We examine how this is addressed at the level of dialogue history storage, audio processing, and companion robot behavior logic.
What managed dialogue delivers: the effect in measurable metrics
Managed dialogue is evaluated not by the beauty of individual answers, but by behavior over a long series of similar sessions. The main metric is the share of autonomous responses: what portion of replies the social robot handles on its own, without transferring to an operator.
In launches with guest scenarios, it reaches a plateau of 82–91% and then grows not through the number of scripted phrases, but through refining the rules for parsing unclear wording.
The second metric is dialogue stability under load. We maintain a separate clarification loop: if the system is unsure of the guest's intent, the robot does not answer at random but narrows the question and returns to the topic after one or two replies.
This way, the session does not break into irritation and early termination — this is visible in the share of dialogues brought to the end of the scenario.
We measure recovery time after a failure from the first incorrect response to return to normal mode.
Previously, this was a manual reboot taking several minutes with loss of context; now it is an automatic rollback to the previous stable configuration with session preservation: it fits within tens of seconds and is noticeable to the guest only as a short pause.
Failure classes decrease as telemetry accumulates: stuck states, repeated answers, memory loss between sessions, responses outside the character role.
Operator load drops to observation and review of disputed cases — one operator on duty supports a fleet of several dozen robots, not just one.
The engineering foundation here is ROS and external speech recognition and synthesis services connected by a unified integration layer. This allows changing an individual model without rewriting the robot's behavior logic.
Where a social robot works autonomously and where an operator is needed
Autonomous mode is built on local processing: speech recognition, dialogue logic, and response synthesis run onboard the robot.
Response does not depend on network quality, and if the network is lost, the dialogue does not break off — for a children's room, reception area, or exhibition hall, this is a basic requirement.
The mode's limitation is a predefined subject area: beyond its boundary, the system recognizes misunderstanding and does not make up an answer.
Hybrid mode leaves typical replies to the machine and routes complex ones to an operator. Transfer to an operator triggers on clear signs: repeated clarifications, a direct request for help, an emotional outburst, a topic outside the scenario.
The human joins the same dialogue and sees the context of recent replies, so the conversation does not start over and the thread is not lost.
Comparison of modes
| Mode | When it fits | Mode limitations |
|---|---|---|
| Autonomous | Typical scenarios, sites without stable connectivity, tasks with high privacy requirements | Works only within a predefined subject area, does not go beyond the scenario |
| Hybrid | Mixed flow of requests, there is an on-duty operator shift | Requires a transfer protocol and an available operator at the moment of the request |
| Fully operator-based | Unclassified situations, interviews, communication with the press | The number of simultaneous dialogues is limited by the number of operators |
The choice of mode is determined not by technology but by the class of requests: the higher the share of unpredictable replies, the sooner a human is needed.
We design the transfer protocol separately — by what sign the robot calls the operator, what happens if the operator does not respond within the allotted time, and how these cases accumulate for further training of the autonomous loop.
How we implement an AI system: stages of work
For a social robot, dialogue does not live separately from behavior: a reply triggers a gesture, light, and head turn, and speech recognition quality depends on room noise.
Therefore, work proceeds sequentially — each next stage relies on the verified result of the previous one, not on assumption.
- Audit of scenarios and bottlenecks. We analyze real conversation recordings and logs: where the robot loses context, where the pause before a response becomes uncomfortable, at which phrase the guest leaves the dialogue. The output is a map of problem areas with repetition frequency.
- Dialogue logic design. We describe the scenario tree and transition rules: attention-retention branches, reaction to interruption, correct exit from a dead end, escalation to an operator when recognition is uncertain.
- Integration setup. We connect speech modules, the knowledge base, external services, and the device itself — motors, light, sound. We verify that a synthesis command correctly turns into a physical action without desynchronization.
- Domain data tuning. We assemble a dictionary of names, terms, and typical site wording so the robot confidently understands the speech of a specific audience, not just reference phrases.
- Acceptance testing. We run the system through real scenarios: we measure the share of successfully completed dialogues, response time, and frequency of repeated questions. We analyze every failed run and fix it until the result is stable.
- Handover to operation. We provide scenario update regulations, a dialogue quality control panel, and clear instructions for site staff. Content edits are made without stopping the robot.
What remains with you after project handover
We hand over not an archive of source code, but a working loop that your team can change without us. Below is a list of what is recorded at acceptance: each item is checked on a live bench, not just on paper.
- System configurations — reply scenarios, trigger thresholds, reaction timings, and rules for switching between modes are placed in separate files. Robot behavior changes by editing the configuration, without rebuilding the logic.
- Integration schemes — a map of data flows between the voice loop, dialogue core, and external services, with failure points. Behavior during network loss is described separately: the robot falls back to the basic scenario instead of freezing.
- Data access policies — role separation: who can see dialogue recordings, how long they are stored, how they are deleted on request. For scenarios with children and the elderly, this is a mandatory part, not an option.
- Operator work regulations — when a dialogue is transferred to a human, how to review disputed answers, what to do with recurring complaints. Without this, a well-tuned system degrades within weeks.
- Documentation — architecture, deployment diagram, a log of known limitations, and procedures for typical failures. We write it for an engineer who will open it six months after us.
- Team training — practical sessions on a test bench: add a scenario, view response quality metrics, roll back an unsuccessful change.
- Set of reference replies — fixed dialogue examples for regression. After any scenario change, you see within minutes what did not break.
Acceptance follows a checklist: your engineers make a change themselves, run regression, and roll back the version. If they can do this without our prompting, the project is handed over.
Case study: museum guide — from frequent failures to stable dialogue
In the exhibition space of Museum of Urban History, a social guide robot operated: it led visitors along a route and answered questions about exhibits. The logic was built on a rigid scenario — the robot confidently answered only pre-written formulations.
As soon as a visitor interrupted it or asked “and why is this item displayed separately?”, the dialogue fell apart: the robot repeated its last reply or went into a restart, and a caretaker picked up the conversation.
What we changed
We rewrote the dialogue logic: the robot began to hold the context of the entire session rather than a single reply, ask a clarifying question instead of guessing, and explicitly acknowledge when there is no answer.
We added fact-checking against the museum's database so the guide would not invent dates and names, and transfer of the dialogue to the caretaker with a brief summary — the human continues from where the robot stopped.
We also set up nightly runs of recorded conversations: edits are tested on real dialogues before going live in the hall.
| Metric | Before implementation | After launch |
|---|---|---|
| Dialogues completed without failure | 62% | 94% |
| Off-script questions with an answer | 25% | 87% |
| Caretaker interventions per 100 dialogues | 34 | 6 |
| Average conversation length, replies | 3 | 9 |
Stability rests not on a single model but on the loop: if a new logic build performs worse than the previous one in the nightly run, it is rolled back, and the working version remains in the hall.
Dialogue logs are collected in one place, so the cause of a failure is found by a specific reply, not by a visitor complaint.
How a scenario framework differs from a generative response
A scenario framework is a state tree: the robot knows which replies are permissible in the current context, how each branch ends, and how to react to silence.
Behavior is reproducible: the same situation produces the same answer, and the same scenario passes tests deterministically. The price is a narrow funnel: deviating from the prepared script runs into “I didn't understand, please repeat.”
A generative response produces a reply for a specific context, so it sustains free conversation, answers an unexpected question, and picks up a change of topic.
Predictability of behavior decreases: the answer is not reproduced from run to run, and it is harder to keep topic boundaries and tone.
Hybrid logic separates areas of responsibility. The framework holds critical branches — greeting, introduction, intent recognition, escalation to an operator, safe refusal — and decides when to pass the turn onward.
The generative layer handles the free part within boundaries: a given tone, reply length limit, forbidden topics, and conditions for returning to the scenario.
A pure framework hits the ceiling of phrase recognition — this was clearly demonstrated by the story of Jibo and other companion robots.
The opposite extreme is no better: without a framework, the system loses predictability and, in maintenance, turns into endless tuning for every new dialogue.
Therefore, system maintenance is built around the framework: scenario branches are updated by editing the tree and re-running tests, while the generative layer is updated with a set of test dialogues and analysis of deviations.
Technical doubts worth resolving before the start
Before the start, the client checks three things: where dialogue data physically resides, what happens if connectivity is lost, and how expensive it is to change robot behavior after launch. Below are answers on these points and on integration with an already selected robot platform.
Do audio and dialogue text remain with us?
We deploy speech recognition, synthesis, and the response model on your infrastructure or in a private cloud under your control. Recordings do not leave the perimeter; logs are stored locally with a configurable retention period and role-based access. An external service, if needed at all, is disabled with a single configuration parameter.
What happens if connectivity is lost?
Basic scenarios — greeting, navigation, typical replies — are embedded in the module and work autonomously. After connectivity is restored, accumulated sessions are uploaded to the common analytics. This is graceful degradation, not failure: the robot continues serving people.
Is the model trained on our materials?
We load scripts, regulations, operator recordings, and the service catalog into the knowledge base, on which the model builds its answers. The robot relies on your wording and does not invent what is not in the materials.
How are scenarios updated and equipment connected?
We keep replies and behavior branches in separate configurations: we roll out an edit to part of the fleet, watch the metrics, and roll back without reflashing. Microphones, actuators, and sensors remain native — we work through the manufacturer's standard interfaces.
Let's discuss the task and come back with an estimate and plan
To make the task review take one conversation instead of three, it is enough to prepare several inputs — they come from current work and do not require separate documents.
The more specific the source data, the more accurate the stage composition and timeline estimate: without them, the plan turns out either inflated or with missing work.
- Interaction scenario. Where the social robot meets the person — reception area, hall, waiting area — and what it must manage to do in the first minutes of communication.
- Current platform capabilities. Which sensors and output channels are already available, whether there is flexibility: this determines what is configured and what is developed from scratch.
- External data. Systems the robot must query for answers: knowledge base, schedule, internal service, accounting system.
- Site limitations. Noise level, lighting, passage width, network stability — they directly affect the choice of speech recognition models and behavior logic.
- Who accepts the result. A person on the client side who confirms “it works”; otherwise, the pilot implementation will stretch into approvals between departments.
- Success criterion. The metric by which we judge the result: share of dialogues brought to completion, number of failures per hundred requests, time to a meaningful answer.
After the review, we come back with a work plan: stage composition with order and dependencies, timeline estimate for each, and points where feedback from your side is needed.
Usually the first step is a pilot implementation on one robot and one site, where scenarios are tested and failure statistics are collected, and only then scaling to a fleet of devices. Describe the task — we will come back with an estimate and plan.
