Developing AI veterinary diagnostics: where early cases get lost
At a clinic appointment, an animal spends 15–20 minutes, and during that time the veterinarian manages to collect history, examine it, and prescribe treatment.
They base their assessment of the animal’s condition on a short fragment of behavior in the office — while cats and dogs mask pain until the last moment: in an unfamiliar room they freeze, do not show limping at full intensity, and do not display the lethargy that the owner sees at home.
This is where manual examination errors come from. Missed limping at an early stage, rapid breathing attributed to stress, unnoticed decrease in activity — all of this is compared not with the baseline of the specific animal, but with the veterinarian’s average idea of normal.
The denser the appointment flow, the less capacity remains to notice a deviation that has not yet become a symptom.
AI diagnostics for animals changes the reference point: wearable sensors and a camera provide a continuous series of observations over weeks rather than a single examination.
The model sees how motor activity changes, weight distribution across the paws, resting respiratory rate, sleep proportion, and motility in the usual environment — and raises a signal when there is a deviation from the animal’s own norm.
The specialized AAHA recommendations for monitoring senior pets are directly based on such dynamics, but a clinic cannot collect it manually.
Early diagnostics based on video and sensor data does not replace the veterinarian here; it brings them what is missing in the office: an objective baseline, reproducible metrics, and prioritization — who to call for an unscheduled appointment first.
Below we break down how this is addressed in the project: from data collection to trigger thresholds at which the signal remains useful rather than turning into noise.
What changes after implementation: measurable effects for the clinic
The effect of implementation is visible not in the interface, but in ordinary appointment statistics.
Case review based on sensors and video no longer depends on who is on shift today: the system processes recordings and readings in parallel with the examination and gives the veterinarian a preliminary picture in minutes, rather than in the time spent reviewing the archive and taking manual measurements.
We measure diagnostic accuracy on a labeled sample of cases: the share of matches with a confirmed conclusion grows, and the share of missed conditions decreases severalfold.
We count borderline cases separately — those where the system is uncertain go into a queue with a flag, so rare pathologies do not get lost in the flow.
Decision support for the veterinarian is designed as a second opinion, not a ready-made verdict.
The system suggests versions and highlights deviations in readings and video, but the final conclusion remains with the specialist: they see the basis for each suggestion and can reject it.
Standardization of veterinarians’ decisions produces an effect that is hardest to notice but easiest to count.
A unified assessment scale, a common set of signs, and a single review protocol remove variation between shifts — on the same recordings, discrepancies between specialists fall from a third of cases to a few.
In sum, this changes the flow: fewer repeat appointments “just in case,” faster review of complex cases, predictable diagnostics workload during peak hours. Below we break down what such work consists of — from data preparation to system support.
Operating modes: video, sensors, and hybrid loop
There are three construction options, and the choice between them is determined not by model accuracy, but by which signs need to be captured and what is physically available at the site.
Video analysis of behavior reads posture, gait, approach to the feeder, and social interactions; sensor devices provide continuous telemetry for each animal. The hybrid mode combines both channels where the conclusion relies on a combination of signs rather than on one.
| Option | When it fits | Limitations |
|---|---|---|
| Video behavior analysis | limping, postures, activity at the feeder, conflicts in the group | overlaps in the stall, dirt on the lens, and changing lighting cause false triggers; a reserve of reference recordings is needed for labeling |
| Sensor devices | continuous dynamics for each animal: temperature, activity, rumination | do not see context — they record a deviation but not its cause; sensitive to tag loss and displacement |
| Hybrid mode | clinically significant conclusion based on a combination of signals | harder to deploy and support: time synchronization, unified labeling, and on-site processing are required |
On-site processing removes some limitations: frames do not leave the site, only events and metrics go out, so a narrow channel and unstable connection stop being a blocker.
The limitations of the methods remain fundamental, however — no mode makes a diagnosis: the system issues a signal and a confidence estimate, while the veterinarian makes the decision.
That is why before the start we discuss which signs we consider sufficient; otherwise accuracy will be high but its usefulness zero.
How we run the project: from data audit to model handover
We fix the engineering route before the start: each stage has an input, an output, and an acceptance criterion that the client checks together with us. This keeps the project manageable rather than turning it into endless research.
- Data audit. We review what has been accumulated: videos from the stalls, readings from wearable sensors, examination logs, laboratory confirmations. We look for gaps by breed and age and the share of cases with a confirmed diagnosis. The output is a data map and a list of gaps.
- Loop design. We agree on which signals are captured, at what frequency, where recordings are stored, how episodes are labeled, and how access to pet data is separated.
- Dataset labeling. Veterinarians label episodes according to a unified protocol: what is considered normal, what is a sign, and how to interpret borderline cases. We review disputed fragments together — no contradictory examples remain in the sample.
- Model training. We train on a balanced sample and separately work through rare conditions: otherwise the system confidently finds common pathologies and stays silent on rare ones.
- Validation on a holdout set. We calculate metrics on recordings the model has not seen, and separately on data from another clinic and other equipment: this shows whether the result transfers.
- Handover and team training. We hand over the model, configurations, documentation, and update procedure, train veterinarians and support staff, and accompany pilot operation.
What remains with the clinic: models, datasets, procedures
The work ends not with a demonstration on our bench, but with the handover of artifacts to the clinic. Everything developed is physically transferred to your infrastructure and continues to work without our involvement — without closed services or vendor lock-in.
- Clinic dataset — labeled images and video recordings from your practice: a unified format, labeling by class, and splitting into training, validation, and control sets. The material remains with you and is replenished with new cases.
- Trained model — weights of the current version with metrics on the control set and a class map. It runs on your server; images do not leave the clinic.
- Configurations and training scripts — so you can further train the model on the replenished dataset and reproduce the result rather than starting the project over.
- Operating procedure — how the model output is integrated into the appointment: who reviews it, at what confidence a case goes to the veterinarian, and how a discrepancy with the diagnosis is recorded.
- Instructions for veterinarians — a guide to uploading materials and interpreting the system’s response, designed for a person without technical training.
- System documentation — deployment diagram, equipment requirements, integration points with the practice management system, and model update procedure.
- Quality control log — metrics for each version, so a drop in quality after an update is visible immediately rather than through complaints at appointments.
Such a package makes the clinic the owner of the system, not a renter: further training, updates, and a change of contractor can be carried out without us.
Case study: how detection rate grew in a high appointment flow
A clinic with three veterinarians per shift and a flow of several dozen studies per day.
The bottleneck turned out to be not a lack of data, but its density: images and examination recordings are reviewed between appointments, and some pathologies are noticed in hindsight — when the animal returns with an already developed picture.
The cost of such a miss is a repeat appointment, lost time, and treatment after a complication has already occurred.
The pilot launch was done in one area so that indicators could be compared on the same team of veterinarians.
The system receives the stream from sensors and video, marks deviation zones, and gives the veterinarian the probability of pathology with justification — the decision always remains with the person.
Metrics before and after implementation
| Indicator | Before | After |
|---|---|---|
| Pathologies detected at the initial examination | 68 out of 100 | 86 out of 100 |
| Time to review one study | 11 minutes | 4 minutes |
| Findings in hindsight, on repeat visit | every ninth | every fortieth |
| Conclusions confirmed by a second veterinarian | 15% | 60% |
The implementation timeline is nine weeks: three for archive review and labeling, two for training and validation on a holdout set, and the rest for live operation and tuning sensitivity thresholds to the clinic’s profile.
The primary data are appointment logs and the Clinical Observation Registry; the export was verified against veterinarians’ conclusions manually.
How does video analysis differ from sensor data analysis?
Video provides behavioral context: limping, posture, approach to the feeder, respiratory rate, scuffles at the waterer.
The model works on frames and their sequence, so the accuracy of video analysis depends on shooting conditions — light, angle, occlusion by neighbors, bedding, and coat contrast against the background.
Moving the camera or changing the angle requires new labeling and further training; otherwise the share of misses grows, and without an obvious error in the log: the system continues to answer confidently, but worse.
Sensor data are numbers: accelerometer activity, temperature, rumination time, water consumption, weight. Such a signal is resistant to lighting and does not depend on the operator, but it is local — one animal, one area.
Accuracy is limited by calibration drift, sensor loss (the collar came off — a gap in the series), and different fit on the body; the conclusion from these numbers is indirect — a deviation increases probability but does not name the cause.
Open corpora of behavioral clips, such as Animal Kingdom, accelerate the start, but the domain still has to be additionally labeled with your own recordings.
In a single loop, the channels are not opposed but linked. Sensors work as a standby trigger: if a deviation is noticed, a recording window is raised, and video confirms and refines whether the case is isolated or persistent.
Fusion can be early (features are combined before the classifier) or late (decisions are combined by confidence); both require strict time synchronization — clock discrepancy separates events and breaks the link.
The choice of method is set by the site conditions: stable light and visibility — video gives more; dense placement and a poor angle — reliance on sensors.
At the start, one channel is fixed as the primary one, the second as the verification one, and quality is calculated by the share of detected cases at an acceptable false trigger rate.
Technical doubts before the start: briefly to the point
Sensors and video provide a stream of observations, but the quality of the solution is determined not by the model, but by how the data are prepared, how the system integrates with the clinic’s practice management system, and what happens to it after launch.
Below are brief answers to the questions most often raised during evaluation and that directly set the scope of work.
What to do if there is little data on rare pathologies?
We train on common cases and transfer features to rare ones, increasing their share in the sample. We also run active selection separately: the model flags frames with low confidence, and the veterinarian labels only those.
Complex cases are checked independently by two people, and discrepancies go to a third — a practicing veterinarian.
How does the system integrate with the clinic’s practice management system?
Exchange goes through its software interface: animal card, appointment history, study result. We match breed and diagnosis directories in advance, and keep disputed records in a separate list. At the start, the system operates in suggestion mode — the decision remains with the veterinarian.
What to do about false triggers?
We separate two types of errors — a miss and a false alarm — and tune the threshold to clinical risk: where a miss is more dangerous, sensitivity is higher. Each trigger is logged with the frame and context, and we review disputed cases in regular runs and further train on them.
How is the model updated after launch?
We fix versions: a new build goes through a shadow run on the archive of studies and is compared with the current one by metrics, then is rolled out to part of the examination rooms.
If indicators drop, rolling back to the previous version takes minutes, and the clinic’s work does not stop.
Let’s discuss the task: what is needed to come back with a plan
To come back with a project estimate and work plan rather than general wording, we need context.
At the start, it is enough to describe which animals we are monitoring, what is already installed at the site, and what result is considered working — after that we will ask clarifying questions on each point ourselves.
- Which conditions need to be recognized: a list of diagnoses or deviations and what is considered a false trigger.
- Source data: video recordings, sensor readings, export from the appointment log — with formats, duration, and volume.
- Labeling: whether there are veterinarian-confirmed cases, in what form they are recorded, and who can supplement them.
- Shooting conditions: lighting, angles, breeds and age groups, and whether there is movement of animals in the frame.
- Data constraints: where the system should run — locally at the site or with external transfer — and how many recordings to store.
- People on the client side: veterinarians and operators who participate in accepting the results.
- Priorities: which condition is critical to recognize first and by what deadline a working loop is needed.
A complete dataset at the start is almost never available, and this does not block the work. We launch the pilot implementation on a limited sample, verify quality on real cases, and only then expand the list of conditions and sites.
Describe the task — we will come back with an estimate, work plan, and clear stage boundaries.
