ย Two papers have been accepted to TMLR
Title: Revisiting Learning-based Video Motion Magnification for Real-time Processing
Authors: Hyunwoo Ha (POSTECH)*, Oh Hyun-Bin (POSTECH)*, Kim Jun-Seong (POSTECH), Kwon Byung-Ki (POSTECH), Kim Sung-Bin (POSTECH), Linh-Tam Tran (KYUNG HEE UNIVERSITY), Ji-Yun Kim (POSTECH), Sung-Ho Bae (KYUNG HEE UNIVERSITY), Tae-Hyun Oh (KAIST)
Video motion magnification is a technique to capture and amplify subtle motion in a video that is invisible to the naked eye. The deep learning-based prior work successfully models outstanding quality better than conventional signal processing-based ones. However, it still lags behind real-time performance, which prevents it from being extended to various online systems. In this paper, we revisit the first learning-based model and present experimental analyses, in particular on the identification of redundant components, the insertion of spatial bottlenecks, and the trade-off relationship between channel reduction and layer addition. By integrating the findings of each experiment, we present a real-time, deep learning-based motion magnification model that achieves a computational speed ranging from a minimum of 2.7 times to a maximum of 34.9 times faster than existing learning-based methods, while maintaining perceptually sufficient generation quality. To the best of our knowledge, this is the first learning-based motion magnification model that runs in real-time on Full-HD resolution videos even without ad hoc quantization.
Title: Hypothesize and Verify: Natural-Language Explanations of Vision Model Errors
Authors: Nam Hyeon-Woo (POSTECH), Yoonsu Kim (KAIST), Kihoon Son (KAIST), Juho Kim (KAIST), Tae-Hyun Oh (KAIST)
LLM- and agent-based assistants now bring non-experts into direct ML work, where they probe model failures by asking the assistant in plain language. When such a classifier misclassifies an image, the non-expert needs a reliable account of why. Two obstacles stand in the way. No benchmark scores free-form natural-language explanations of such errors, and existing retrieval-based methods can only return sentences from a fixed error corpus. We close both. NEMO is a task and benchmark of 1,200 misclassified images across ImageNet-R, ObjectNet, and ImageNet-D, each varying along a controlled factor (artistic style, viewpoint, low-level attributes), scored by an LLM-as-a-Judge (LLM Match) protocol that asks whether the explanation describes that factor. SciTX is a generation-based method that emulates the scientific method: retrieve observations, hypothesize candidate causes, verify each with a counterfactual intervention, and retain the hypothesis whose intervention shifts the modelโs prediction farthest toward the ground-truth class. The shift is captured by our Counterfactual Explanation Impact (CEI), which serves as both SciTXโs selection signal and a complementary evaluation metric. On NEMO, SciTX outperforms retrieval-based and MLLM-augmented baselines on both LLM Match and CEI, and 30 AI practitioners rank it first across all five helpfulness dimensions, including factuality, specificity, and actionability.