VLM-Based Method Extracts Expert Actions and Decision-Making Scenes from Maintenance Videos

Jul 15, 2026

A new method uses vision-language models (VLMs) to detect anomalous frames between maintenance task videos, enabling automatic extraction of expert-specific actions and contextual decision-making scenes. In simulated maintenance experiments, the approach achieved extraction rates of 65% for actions and 61% for decision-making scenes, outperforming conventional methods. The technique leverages frame-wise visual descriptions and intra-video self-similarity to identify key moments of expert know-how.

Why it matters: This method could facilitate the transfer of expert knowledge to less experienced workers by automatically identifying and extracting critical scenes from maintenance videos.

Full story at: arXiv Computer Vision ↗