The first author of this paper is Minghang Zheng, a Ph.D. student at the Wangxuan Institute of Computer Technology, Peking University, and the corresponding author is Assistant Professor Yang Liu. The team has published multiple representative works at top conferences such as TPAMI, CVPR, ICCV, and ICML in recent years, and extensively collaborates with renowned domestic and international universities and research institutions.
This article primarily introduces the latest research outcomes from this team and the Huawei Central Media Technology Institute in the fields of multimodal video understanding and temporal grounding.
To address the problem where existing reinforcement learning-based video large models tend to generate superficial reasoning during the inference process—failing to provide effective guidance for precise temporal grounding—this work proposes a novel Temporal-Aware Reasoning Optimization (TaRO) training framework. This method explicitly enhances the model's ability to think with time, achieving state-of-the-art zero-shot performance on multiple public benchmarks. The relevant code is currently open-source.
Paper Title: Temporal-Aware Reasoning Optimization for Video Temporal Grounding
Paper Link: https://arxiv.org/abs/2606.09248v1
Open Source Code: https://github.com/oceanflowlab/TaRO
Project Homepage: https://minghangz.github.io/publication/taro
Video Introduction: https://www.youtube.com/watch?v=GJaodMUG9Vc
Background and Motivation
Video Temporal Grounding (VTG) aims to precisely locate the start and end times of corresponding events in untrimmed videos based on natural language queries. Recently, Multimodal Large Language Models (MLLMs) combined with Reinforcement Learning (RL) have shown great potential in generating reasoning paths that guide temporal grounding. However, the reasoning generated by existing RL methods is often superficial description, failing to identify the specific video evidence required for the answer.
As shown in Figure 1(a), this paper trains and infers existing models under two settings: with reasoning paths and with direct answer output (no reasoning). It finds that the performance of both is nearly identical. This phenomenon demonstrates that although existing models are trained to reason, the superficial reasoning they generate contributes almost nothing to the final localization prediction. This paper analyzes two major reasons behind this:
Inefficient Random Exploration Mechanism. Existing RL paradigms lack effective guidance when exploring the vast video reasoning space. Blind random rollouts cause the model to primarily explore low-quality trajectories, resulting in suboptimal and superficial reasoning.
Reasoning Quality-Ignorant Reward Design. Current reward functions primarily focus on the correctness of the final answer (e.g., calculating IoU) while completely ignoring the quality of the reasoning process itself. This allows reasoning paths that do not truly rely on visual temporal evidence to also be reinforced, leading the model to depend on spurious correlations.
Figure 1: Background and Motivation
Technical Approach
To overcome the aforementioned challenges, this paper proposes the Temporal-Aware Reasoning Optimization (TaRO) framework, designed to train multimodal large models to explicitly think with time. As shown in Figure 2, the TaRO framework consists of three components:
Constructive Reasoning Exploration: To provide high-quality initial guidance and break away from inefficient random exploration, this paper utilizes pre-generated dense video captions with explicit timestamps to construct reasoning trajectories. By sequentially concatenating sampled captions, the model learns which visual cues are critical for localization and which are distractors, thereby avoiding blind trial-and-error.
Temporal-Sensitivity Reward: To evaluate reasoning quality and ensure it is strictly anchored to correct visual segments, this paper designs an instance-level reasoning path reward mechanism. The core idea is: high-quality reasoning should be anchored to specific events and timestamps. If the frames near the ground truth event boundaries are perturbed, this reasoning should fail, causing the probability (logit) of the reasoning path to drop. TaRO leverages this probability drop as a reward signal, forcing the model to generate reasoning tightly coupled with critical timestamps.
Progressive Curriculum: The TaRO framework follows a progressive learning strategy. In the warm-up phase, the model learns using constructive exploration data, mastering how to focus on visual cues and establishing a paradigm for thinking with time. Subsequently, the model transitions to a free exploration phase, autonomously generating and refining its reasoning strategies guided by the temporal-sensitivity reward.
Figure 2: Temporal-Aware Reasoning Optimization (TaRO) Framework
Experimental Results
Zero-Shot Video Temporal Grounding Performance: As shown in Table 1, the video large model trained with the TaRO framework comprehensively surpasses existing state-of-the-art methods on four public benchmarks: Charades-STA, ActivityNet Captions, QVHighlights, and TVGBench. For instance, when using Qwen2.5-VL-7B-Instruct as the base model, TaRO leads the baseline model by 8.4% on the R1@0.5 metric on TVGBench.
Furthermore, TaRO demonstrates consistent performance improvements on the smaller Qwen2.5-VL-3B model and the newer Qwen3-VL-8B architecture, proving the method's versatility.
Table 1: Comparison of Zero-Shot Video Temporal Grounding Performance
Scalability in Long Video Scenarios: To further verify TaRO's performance on long videos, this paper conducted zero-shot evaluations on two major long video datasets, including TACOS (average length 367 seconds) and Ego4D NLQ (average length 499 seconds) datasets. As shown in Table 2, when using the same base model, the video large model trained with the TaRO framework still maintains excellent performance, significantly outperforming existing baseline methods. Particularly on the Qwen3-VL-8B architecture, TaRO brings even more pronounced improvements; for example, R1@0.3 on TACOS increased by 13.7%, and R1@0.3 on Ego4D NLQ increased by 8.7%. This demonstrates the effectiveness and robustness of temporal-aware reinforcement learning optimization when dealing with long videos.
Table 2: Comparison of Long Video Temporal Grounding Performance
Ablation Study: Table 3 verifies the effectiveness of each core design of TaRO. First, on a baseline model with purely random exploration, adding only the temporal-sensitivity reward (TR) increases R1@0.5 from 61.1% to 63.1% (rows 1, 2), proving the effectiveness of the temporal-sensitive reward. However, if the model is only made to entirely imitate externally constructed reasoning paths (CRE) during training without the subsequent free exploration phase (PC), the localization performance severely drops (rows 3, 4). This is because, during the testing phase, without relying on external caption inputs, the model must internalize its own reasoning strategies. Introducing the Progressive Curriculum (PC) bridges this gap and achieves optimal performance (rows 5, 6).
Table 3: Ablation Study
Visualization Results: The visualization in Figure 3 demonstrates TaRO's performance in handling complex multimodal scenarios. A strong distractor appears at the beginning of the video (a woman wiping her face with her hand), whose visual dynamics are highly similar to the text query (wiping face with a brush). By generating fine-grained intermediate temporal reasoning, TaRO precisely anchors the key action from 19.0s to 37.0s and eliminates subsequent irrelevant segments, ultimately providing the correct temporal prediction.
Figure 3: Visualization Comparison
Conclusion
To address the issue of superficial reasoning and the lack of genuine temporal perception in multimodal large models for video temporal grounding, this paper introduces the TaRO framework. By incorporating a constructive reasoning exploration mechanism to efficiently guide the model to think with time, and utilizing a temporal-sensitivity reward to quantify reasoning quality, TaRO successfully enhances the temporal reasoning capabilities of multimodal large models. Extensive experiments prove that this framework not only significantly improves the robustness and interpretability of model reasoning but also achieves the best video temporal grounding performance across multiple public benchmarks.