Production computer vision is mostly plumbing

Two of my computer-vision deployments started from the same place: a custom-trained YOLOv11 detector. In both, most of the engineering effort went everywhere except the model.

At Pertamina Patra Niaga (through my internship at Telkom Indonesia), the job was to flag missing helmets and vests across a facility's existing CCTV network. With Neopark, it was per-slot parking availability from low-cost ESP32-CAM boards instead of a sensor in every bay. Different problems, same lesson: production computer vision is roughly 80% data and infrastructure engineering.

Here is what that 80% looked like.

1. Assume every camera will let you down

Real camera networks are not benchmark datasets. The facility's NVR mixed heterogeneous RTSP streams, and some cameras simply dropped out. A naive loop — read a frame from each camera, run the detector, repeat — has a nasty property: one slow or stalled stream holds up detection for every other camera.

So ingestion and inference became separate concerns:

• every stream is read independently and normalised on ingest, so the detector only ever sees one frame format;
• flaky cameras get reconnection logic instead of taking the pipeline down with them;
• the detector consumes whatever frames are ready, so no single camera can stall monitoring.

It is more moving parts to operate than one loop. It is also the difference between a demo and a system that runs unattended.

2. Stabilise the output, not just the model

Neopark's early dashboard flickered: a parked car would read occupied, free, occupied across adjacent frames. Each frame could be "accurate" and the display still impossible to trust.

The tempting fix is a heavier model. The one that worked was a temporal smoothing layer on top of the detector: a slot only changes state once the new evidence holds for a short window. A minimal version of the idea (not the production code):

from collections import deque


class SlotSmoother:
    """Flip a slot's state only after the opposite state holds for `window` frames."""

    def __init__(self, window: int = 5) -> None:
        self.window = window
        self.state: dict[str, bool] = {}
        self.recent: dict[str, deque[bool]] = {}

    def update(self, slot_id: str, occupied: bool) -> bool:
        recent = self.recent.setdefault(slot_id, deque(maxlen=self.window))
        recent.append(occupied)
        current = self.state.get(slot_id, occupied)
        if len(recent) == self.window and all(seen != current for seen in recent):
            current = not current
        self.state[slot_id] = current
        return current

The trade-off: a slot flips slightly after the raw detection does. That is a small price for a display people actually believe.

3. Calibrate per camera, not globally

Lighting and viewing angles differ from camera to camera, so one global confidence threshold misjudged some bays while being fine for others. Two things fixed it:

1. training on a curated dataset that spans varied lighting, and
2. tuning thresholds per camera angle.

The cost is operational — every new camera needs a calibration pass before it goes live — so make that pass a documented step, not tribal knowledge.

4. Put the compute where it can breathe

ESP32-CAM boards are great frame sources and far too constrained to run a modern detector. Rather than squeezing the model onto the edge, Neopark keeps the edge thin and batches frames on a GPU-backed inference node, which held inference latency around 150 ms. The honest trade-off: the system now depends on the camera-to-server link, so that link deserves the same monitoring as the model.

5. Own the data, own the lifecycle

A detector has to perform on this facility's camera views, not on generic footage. For the PPE system that meant curating and annotating the dataset in-house, then training, deploying, and monitoring the model ourselves. The accuracy you can rely on comes from that loop far more than from architecture tweaks.

6. Ship something operations can run

The PPE system had to be handed over to the facility's operations team, so every service shipped as a Docker container. Neopark's stack runs in Docker for the same reason: deployments that are repeatable rather than hand-assembled.

A checklist before you call it done

• What happens when one camera stalls? When it disappears for an hour?
• Is the displayed state stable, or does it flicker from frame to frame?
• Can you add a camera without retraining — and is calibration written down?
• Where does inference run, and what happens when the link to it drops?
• Can someone else deploy and restart it without you?

When those answers are solid, the model usually is the easy part.