V↗Viscot-steer

Visual Chain-of-Thought · Activation Steering

Viscot-steerDisentangling and Transferring
Visual Chain-of-Thought
via Activation Steering

Transfer “thinking with images” to a new model.
Separate what to transfer, where to steer, and when to intervene.

Explore the method ↗View resultsBibTeX
TL;DR

Transfer visual-CoT without retraining. Extract separate grounding and reasoning vectors, align them across models, and steer each one only at the layer and generation phase where it is needed.

Viscot-steer contrasts separate grounding and reasoning vectors with unified steering, showing layer and stage specialization and Qwen3-VL-4B benchmark results.
The central idea. Visual-CoT combines two capabilities with different roles and preferred model depths. Disentangling them makes transfer more targeted.
0Model-parameter updates during transfer
5Target LVLMs in the DeepEyes experiments
5Main benchmarks across three task families
01

The research question

Can visual-CoT travel
across models?

“Thinking with images” is powerful. Relearning it for every backbone is expensive.

Viscot-steer transfers these capabilities through inference-time interventions in hidden activations.

WhatDisentangled grounding & reasoning
WhereCapability-specific model layers
WhenCorresponding generation phases
02

The framework

Disentangle. Align. Steer.

Extract two capability vectors

Keep the image and question fixed. Compare visual-CoT prompting with a no-grounding or no-reasoning counterpart, and average activation differences at capability-specific source layers.

Align across architectures

Use LMN frequency-domain resampling to match the target hidden dimension, then restore the source-vector norm. Neither the source nor the target model is updated.

Route by generation phase

Inject the grounding vector during bounding-box prediction and the reasoning vector during reasoning-trace generation. Leave final-answer generation unsteered.

Pipeline of contrastive vector extraction, LMN dimension alignment, and phase-aware grounding and reasoning activation steering.
Method overview. A source visual-CoT model provides two capability directions. The target model receives each direction at the appropriate depth and phase.

Explore the routing policy

One generation, different interventions.

Interactive explanation of the method; no model inference is run in this page.
03

Experimental evidence

Transfer across families
and model scales.

DeepEyes-7B is the source model. Target models span Qwen2.5-VL, Qwen3-VL, and InternVL2.5, from 3B to 32B parameters. All scores below are overall benchmark scores in percent; higher is better.

Main-text results: Qwen2.5-VL-7B, Qwen3-VL-4B, and InternVL2.5-8B. The 3B and 32B results are from the appendix. HR-Bench 4K and 8K are settings of one benchmark. Bold marks the highest score among the displayed methods.

+17.80percentage points · V*
Same-architecture transfer

A base model can surpass its source.

Qwen2.5-VL-7B improves from 75.92% to 93.72% on V*, exceeding the DeepEyes-7B source model’s 88.48% in this setting.

+14.13percentage points · HR-Bench 4K
Cross-family transfer

Grounding transfers beyond the source family.

InternVL2.5-8B rises from 36.25% to 50.38% on HR-Bench 4K. Its MathVista score also improves from 58.0% to 61.5%.

Transfer from a different source: GRIT-3B

With GRIT-3B as the source, Qwen3-VL-4B improves from 55.96% to 64.26% on VSR and from 51.53% to 54.58% on TallyQA. InternVL3-2B also improves on both benchmarks, although L2V-CoT scores higher on VSR (48.01% vs. 46.57%).

04

Why it works

What, where, and when
all matter.

05

Qualitative examples

Look at the right evidence.

The manuscript compares base-model and steered responses on fine-grained attributes, counting, sign recognition, and chart reading. The examples show how changing the attended region can correct the final answer.

Four manuscript cases: cap color corrected from black to white, dog count from three to four, sign shape from triangular to rectangular, and a chart value identified as 991.
Representative visual-CoT trajectories. Red and blue boxes in the original figure distinguish the base and Viscot-steer outputs. Click the figure to inspect the original examples at full resolution.

Scope & reproducibility

Targeted transfer,
with clear boundaries.

Training-free means the transfer process updates no model parameters. It still requires source-model activations, contrastive extraction examples, and inference-time steering.

Grounding and reasoning layers are chosen separately. The manuscript provides steering-layer and strength analyses; improvement is not uniform across every model and benchmark. For example, Qwen3-VL-32B stays at 78.1% on MathVista, below L2V-CoT’s 79.4%.

The main extraction setup uses 1,000 examples from DeepEyes-Datasets-47k, covering real-world, mathematical, and chart reasoning. Detailed prompt and hyperparameter settings are described in the manuscript appendix.

Reference

Citation

Manuscript citation. Publication details and a persistent URL can be added when available.

Download BibTeX ↓