Building Real-Time AI Vision UIs in Vue.js

TL;DR: In a real-time AI vision interface, lag is an accuracy problem: if overlays draw late, reviewers correct the wrong object. On Natif AI, overlays built to track the live recognition stream, confidence heatmaps and a deliberately small Vuex state raised interaction accuracy 35%. Keep the raw model stream out of deep reactivity, store only what users act on, and render at the display rate, not the model rate.

Most AI vision demos look great in a screen recording. Then a real reviewer sits down with them, the model streams detections faster than the interface can draw them, and every click lands a few frames behind.

On Natif AI I worked on exactly this kind of interface: a Vue.js frontend with real-time object recognition, heatmaps and annotation overlays. Interaction accuracy rose 35% after the overlay and heatmap workflow shipped. The lesson that stuck with me is that in AI interfaces, frontend performance and data quality are the same thing.

Why lag becomes an accuracy problem

A reviewer sees a bounding box around a detected object and clicks to correct it. If the screen is showing a detection from a moment ago, the box has already moved, or the model has already replaced it with a new one. The correction gets attached to the wrong object.

The model can be accurate and the dataset you build from human corrections still ends up wrong. Nobody sees an error message. You just get worse training data and nobody knows why.

So the first requirement for a vision UI isn’t pretty overlays. It’s overlays that show what the model is reporting right now.

Keep the raw stream out of deep reactivity

Vue’s reactivity is great for forms and dashboards. It’s expensive for a stream of hundreds of detections arriving many times per second, because by default Vue makes every nested property reactive and tracks every change.

What works better:

• Hold the live stream in a non-deep container. In Vue 3, shallowRef (replace the whole array on each frame) or markRaw for objects that never need tracking. In Vue 2, freezing the incoming arrays with Object.freeze stops Vue from walking them.
• Keep the store small. On Natif, Vuex state was kept tight, holding only what the overlays and reviewers needed. The store should hold what a person acts on, like the selected detection, edits and confirmed labels. It shouldn’t hold every frame the model emits.
• Derive instead of storing. Things like “detections above threshold” or “count per class” should be computed from the current frame, not stored as separate state that can fall out of sync.

Render at display rate, not model rate

A model might emit results faster or slower than the screen refreshes. Neither should drive rendering directly.

Buffer the latest result and draw on requestAnimationFrame. If three frames arrive between two paints, draw only the newest. Reviewers never needed the two in between, and skipping them keeps the main thread free for their clicks.

For dense scenes, drawing boxes on a <canvas> layer usually beats rendering hundreds of DOM elements. Keep DOM elements for the one or two interactive things, such as the selected box and its edit controls, where you need accessibility and real event handling.

Show confidence, not only boxes

A bounding box says something is here. It doesn’t say how sure the model is.

On Natif, heatmaps added that layer. Reviewers could see where the model was confident and where it was guessing, and spend their attention on the uncertain regions instead of checking every detection equally. Together with overlays that stayed in sync, that’s where the accuracy gain came from.

I wrote more about this side of the problem in designing human-in-the-loop review UIs.

A checklist for AI vision frontends

1. Measure display lag: the time from a model result arriving to it appearing on screen. Treat it as a quality metric.
2. Keep the incoming stream out of deep reactivity (shallowRef, markRaw or frozen arrays).
3. Store only user actions and selections in Vuex or Pinia.
4. Draw on requestAnimationFrame and drop stale frames.
5. Use canvas for dense overlays and DOM for interactive elements.
6. Show model confidence visually.

FAQ

Is Vue.js a good choice for real-time AI interfaces?

Yes, if you keep high-frequency data out of deep reactivity. Vue handles the interactive parts well. The raw model stream should sit in shallow or non-reactive containers.

Should overlays use canvas or DOM elements?

Canvas for many boxes that update constantly, DOM for the few elements users interact with. Mixing both is common and works well.

Why does UI lag affect AI accuracy?

Human corrections get attached to whatever is on screen. If the screen is behind the model, corrections land on the wrong detections, and that bad data flows back into training.

---

Full project context in the Natif AI case study. Building an AI interface? See software architecture.