How to evaluate real time object detection models on video footage?

Greetings everyone,

I’m working on a real-time object detection project, where I’m trying to detect and track multiple animals moving around in videos. I’m struggling to find an efficient and smart way to evaluate how well my models perform.

Specifically, I’m using and training RF-DETR models to perform object detection on video segments. These videos vary in length (some are just a few minutes, others are over an hour long).

My main challenge is evaluating model consistency over time. I want to know how reliably a model keeps detecting and tracking the same animals throughout a video. This is crucial because I’ll later be adding trackers and using those results for further forecasting and analysis.

Right now, my approach is pretty manual. I just run the model on a few videos and visually inspect whether it loses track of objects which is not ideal to draw conclusions.

So my question is:

Is there a platform, framework, or workflow you use to evaluate this kind of problem?

How do you measure consistency of detections across time, not just frame-level accuracy or label correctness?

Any suggestions appreciated.

Thanks a lot!