Training-Free Memory Policies for Revisit Consistency in Video World Models

Contributions

Core Contributor: Nicola Dall'Asen
Project Lead: Shiqi Yang
Creative Vision Team, Multimodal AI Department

Introduction

Point a camera at a tree, pan away to explore the rest of a generated world, then pan back. Nothing guarantees the same tree is there with the same shape and color; the model may just have made up a new one.

This is a central problem for interactive video world models. Autoregressive video world models extend a scene one chunk at a time, and for real-time inference the active context has to stay small. Old views get evicted, and after that the model simply cannot look at them.

None of the standard fixes is free. A bigger window costs more attention than we can afford. Compressing history throws away detail. Retrieval needs a rule for deciding which old frames are worth keeping, and camera-aware warping of what you retrieve only pays off when the geometric match is good, which for a retrieved view it usually is not.

This post is a status report on long-term revisit consistency in LingBot-World Fast. We compare full attention, bounded windows, persistent attention sinks, explicit memory retrieval, camera-aware attention, and a training-free compressed cache inspired by RELIC. Every method is applied at inference time to the same base model, so the comparison stays focused on what can be gained without additional training.

Short version: there is no single winner. Which policy comes out ahead depends on how the camera comes back.

Top left: base model, after a 360° rotation the scene is not the same.
Top right: camera-aware retrieval, the barns on the left and the right remain consistent.
Bottom left: camera-aware retrieval with sink, during the diagonal return movement, the red house returns.
Bottom right: base model, after moving towards the red container, the original position and color of the objects are lost.

Overview of the study

Our base model is LingBot-World Fast [1], a few-step causal video world model with camera control. It generates three latent frames per autoregressive chunk and uses a rolling KV cache to avoid recomputing the full past.

We evaluate a simple question: when two frames are rendered from the same or a strongly overlapping viewpoint, how similar are their pixels, perceptual features, and semantic content?

The workflow has four parts:

Construct camera paths with controlled revisits;
Generate the same scene with different memory policies and three seeds;
Pair repeated or geometrically overlapping views;
Evaluate PSNR, SSIM, LPIPS, and DINO similarity, then inspect the videos and attention behavior.

Key challenges

A local window is efficient but forgets distant views; full attention retains everything but grows with the rollout.
Retrieval can find a useful frame, but the model must still know how to use a feature created at another camera pose.
Compression preserves distant evidence but can damage a recent view that would have fit in the normal cache anyway.
A useful memory policy must preserve representations that the unchanged base model can still interpret.
A method that wins on an exact 360° revisit may lose when the camera oscillates, returns early, or only partly revisits the field of view.

How we make memory observable

We began with two different camera trajectories.

	Scene 1: long-horizon exact revisit	Scene 2: short-horizon oscillating revisit
Content	lakeside scene	stylized courtyard
Camera path	two complete 360° in-place pans	repeated ±88° in-place pans
Pairing	frame i versus i + 180	best earlier view with sufficient frustum overlap
Main question	can distant content be recalled?	does memory processing damage evidence that is still recent?

Both trajectories are pure rotations, so the first study isolates orientation
and memory without translation or parallax. The LingBot runs use
832×480 generations, three seeds, and a canonical attention budget
of 18 latent frames. For explicit retrieval, nine frames remain local and up to
nine are retrieved.

The four metrics expose different failure modes:

PSNR rewards direct pixel alignment;
SSIM measures local structural similarity;
LPIPS measures perceptual appearance difference (lower is better);
DINO similarity measures higher-level semantic and scene identity.

Because the pairing rules differ, values should not be compared directly between scenes. Rankings within a scene are the meaningful comparison.

Training-free memory approaches

Local window, full attention, and persistent sinks

The simplest baseline retains only the most recent 18 latent frames. A sink variant permanently keeps the first three frames as a stable anchor. Full attention keeps the entire history and serves as an upper-cost reference; as we show below, having everything in context does not mean the model attends to the right frames.

Explicit retrieval

All fixed-budget retrieval runs attend to at most 18 latent frames. Nine are the most recent local frames; the other nine are selected from history. This keeps attention cost comparable with the local-window baseline and lets us test frame selection separately from how the model uses the retrieved frames.

We test two selectors:

FOV retrieval ranks old frames by estimated camera-frustum overlap with the current view. It directly asks which camera saw the largest portion of what should be visible now.
Pose retrieval uses a WorldKV-style [2] distance combining translation and rotation. It is cheaper conceptually and can retrieve nearby viewpoints even when the coarse frustum estimate is ambiguous, but geometric proximity does not guarantee shared visible content.

Each selector is tested with sink 0 and sink 3. Sink 3 permanently reserves the first three latent frames as an initial-scene anchor; sink 0 spends the entire budget on recent and retrieved frames. Unless stated otherwise, retrieved keys retain their original temporal RoPE positions.

Camera-aware retrieval

Selection alone returns features encoded from an earlier camera. We therefore adapt the relative-camera projection from Matrix-Game 3.5 [3] as a training-free attention overlay. The variants isolate three independent choices:

Full camera-aware transform (MG) transforms queries and keys for camera-relative matching, transports values into the current camera frame, and maps the attention output back. It provides the strongest geometric intervention, but an approximate view match can make value transport blend incompatible content.
Score-only (MG score-only) transforms queries and keys but leaves values and outputs untouched. Camera geometry influences where attention looks without rewriting what is retrieved.
Offset RoPE (MG offset) places retrieved keys in a compact temporal block immediately before the local window instead of at their true past times. This tests whether temporal distance itself suppresses useful memory.
Sink retrieval (MG sinkretr-offset) lets the three pinned sink frames also enter the retrieval pool as explicit copies, using the compact offset layout. Ordinary sink attention can always see the anchor, but this variant asks whether selecting it into the memory slots makes it easier to use.
True-past sink retrieval (MG sinkretr-past) performs the same sink retrieval while keeping every copied key at its original RoPE time. The comparison with sinkretr-offset isolates temporal placement from retrieval content.

On the first exact-revisit scene, the full transform raises sink-0 PSNR from 11.28 for plain FOV retrieval to 12.02. Score-only reaches similar PSNR while improving SSIM, LPIPS, and DINO, suggesting that zero-shot camera geometry is more reliable for attention scores than for transporting values.

Compressed full history

For the compressed-history baseline, we borrow RELIC's [4] central idea and turn it into a training-free cache policy. A rolling window keeps either six or nine recent latent frames at full spatial resolution. Every older key/value frame remains accessible after spatial average pooling with a cyclic 1×/2×/4× schedule. Unlike fixed top-k retrieval, this preserves coverage of the entire past, but older evidence is progressively represented with fewer tokens.

No weights are trained or distilled for this cache format: the experiment asks
whether broad compressed coverage is already useful to the unchanged LingBot
model. It trails the best retrieval rows on the two original scenes but wins a few metrics on some multi-revisit paths, so we keep it as a coverage baseline.

Results

Scene 1: Exact long-horizon revisit

Selected three-seed results are shown below. Parentheses denote the standard deviation across seeds.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	DINO ↑
Camera-aware sink retrieval, true-past RoPE	12.44 (0.57)	0.518 (0.026)	0.541 (0.037)	0.760 (0.023)
Camera-aware score-only, sink 0	12.04 (0.15)	0.546 (0.022)	0.548 (0.015)	0.747 (0.008)
Camera-aware retrieval, sink 0	12.02 (0.47)	0.518 (0.015)	0.553 (0.006)	0.714 (0.013)
Full attention	11.54 (0.31)	0.524 (0.003)	0.550 (0.012)	0.700 (0.040)
Plain FOV retrieval, sink 3	11.21 (0.42)	0.507 (0.024)	0.583 (0.003)	0.770 (0.016)
RELIC-inspired compressed memory, w=9	10.91 (0.58)	0.491 (0.038)	0.586 (0.024)	0.691 (0.047)
Local window, sink 0	9.70 (0.20)	0.453 (0.023)	0.632 (0.009)	0.608 (0.039)

Among the matched LingBot rows, retrieving the sink at its true past RoPE time has the best PSNR/LPIPS, score-only camera conditioning has the best SSIM, and plain FOV retrieval with sink 3 has the best DINO similarity. So the geometry helps when it steers attention scores; warping the values themselves gives no consistent gain.

Top left: base model with local window only.
Top right: base model with full attention.
Bottom left: RELIC-like memory.
Bottom right: Camera-aware sink retrieval.

Scene 2: Oscillating short-horizon revisit

Method	PSNR ↑	SSIM ↑	LPIPS ↓	DINO ↑
Pose retrieval, sink 3	15.65 (0.11)	0.409 (0.008)	0.303 (0.002)	0.893 (0.015)
Plain FOV retrieval, sink 3	15.32 (0.40)	0.402 (0.008)	0.320 (0.015)	0.885 (0.026)
Full attention	15.15 (0.25)	0.397 (0.011)	0.323 (0.007)	0.894 (0.007)
Local window, sink 3	13.35 (0.72)	0.347 (0.005)	0.424 (0.039)	0.822 (0.021)
RELIC-inspired compressed memory, w=9	12.47 (0.52)	0.348 (0.019)	0.456 (0.023)	0.757 (0.008)
Camera-aware score-only, sink 0	11.64 (0.69)	0.322 (0.009)	0.479 (0.026)	0.778 (0.050)

Here the ranking changes again. Pose retrieval with sink 3 wins the three pixel/perceptual metrics and nearly matches full attention on DINO; ordinary FOV retrieval with the same sink is close behind. The useful evidence is recent and the camera keeps coming back to the initial anchor, so a small, unmodified retrieval cache is enough to compete.

Camera-aware value transport also becomes harmful when retrieved views are only approximate geometric matches. The score-only variant does less damage, which fits the reading that the geometry in the attention scores is useful but the transported values end up blending content from different angles.

Top left: base model with local window and no sink.
Top right: base model with local window and sink.
Bottom left: RELIC-like memory.
Bottom right: Pose retrieval and sink.

What we learned

1. The right policy depends on the revisit regime

Long, exact revisits want retrieval plus a pinned anchor, and there camera geometry in the attention scores helps while warping the values does not. Short oscillating paths are the opposite case. The useful evidence is still recent, plain local context plus a sink already does well, and the extra memory machinery mostly gets in the way. If we had to ship one policy, it would need to check first whether the current view overlaps the resident window or only evicted history, then spend the budget accordingly.

2. Selection and transformation are separate bottlenecks

FOV and pose distance usually pick similar frames, so the selector is rarely what decides the metric. What decides it is what happens next. The retrieved key was encoded under a different camera, and the base model was never trained to reconcile that. Score-only conditioning gets around most of this; full value transport blends content across viewing angles and can end up worse than not retrieving at all.

3. The first three latent frames do a lot of work

Pinning them helps almost every configuration we tried, at essentially no cost. On the oscillating scene the sink is also the recurring target itself, since the camera keeps coming back near the initial pose, so the anchor is generally the answer.

4. RoPE placement and value transport are separate choices

sinkretr-past and sinkretr-offset select exactly the same content and still rank differently, so where a retrieved key sits in time matters independently of what was retrieved. Same story for score-only against the full transform. Bundling the retrieval rule, the temporal position and the feature warping into one "camera-aware" method hides which of the three actually matters.

Expanded experiments: 11 comparable revisit patterns

Revisit-pattern suite

We expanded the benchmark using the trajectory families in WorldTrace [5]. Unlike the original two rotation-only scenes, the topology, rollout-length, and multi-revisit rows translate the camera through A/B/C/D waypoints; the orientation row rotates in place. To add visual diversity without confounding controlled comparisons, one Context-as-Memory [6] scene is shared within each trajectory family, with a different source scene across families. The four sources cover a coastal lighthouse, two visually distinct container yards, and a European square with a fountain.

The 832×480 sweep evaluates the same memory configurations with three seeds. The table gives the winner for each metric within every reported case (MG is the camera-aware overlay; sinkretr makes the initial sink retrievable).

Case	Best PSNR ↑	Best SSIM ↑	Best LPIPS ↓	Best DINO ↑
translated ABA, N=8	MG sinkretr-past 11.338	MG sinkretr-past 0.319	MG sinkretr-past 0.620	MG sinkretr-offset 0.837
translated ABA, N=16	MG sinkretr-offset 11.258	MG sinkretr-offset 0.290	MG sinkretr-offset 0.665	full attention 0.820
translated ABA, N=32	MG sinkretr-past 10.658	MG sinkretr-past 0.282	MG sinkretr-past 0.684	MG, sink 3 0.833
topology ABA, N=16	MG sinkretr-offset 15.322	MG sinkretr-offset 0.645	MG sinkretr-offset 0.356	MG sinkretr-offset 0.577
topology ABCA, N=17	pose retrieval, sink 3 14.819	MG sinkretr-offset 0.659	MG sinkretr-offset 0.363	MG sinkretr-offset 0.503
topology ABCDA, N=16	pose retrieval, sink 0 15.049	MG, sink 0 0.663	MG sinkretr-offset 0.352	MG offset 0.518
180° look-back, N=8	pose retrieval, sink 0 12.386	MG score-only 0.402	pose retrieval, sink 0 0.453	pose retrieval, sink 3 0.879
360° full pan, N=8	pose retrieval, sink 3 10.151	full attention 0.226	MG sinkretr-offset 0.683	MG offset 0.488
ABABA, N=32	RELIC w=9 13.911	pose retrieval, sink 3 0.373	MG sinkretr-past 0.409	MG sinkretr-past 0.966
ABCBA, N=20	RELIC w=6 16.166	RELIC w=6 0.375	RELIC w=6 0.339	MG offset 0.976
ABCDBA, N=20	pose retrieval, sink 0 15.745	MG score-only 0.382	pose retrieval, sink 0 0.331	FOV retrieval, sink 3 0.976

Sink retrieval wins LPIPS at all three ABA horizons, while topology changes the winner: straight ABA strongly favors sink retrieval, while pose retrieval takes PSNR on ABCA
and ABCDA. Training-free compressed history is competitive only on some paths: w=9 wins ABABA PSNR and w=6 wins three ABCBA metrics. On the diagonal-shortcut ABCDBA path, pose retrieval wins PSNR/LPIPS and plain FOV retrieval wins DINO. The nominal score-only SSIM winner is highly seed-variable (standard deviation 0.130). Overall, compressed coverage is worth having alongside retrieval, but it does not replace it.

Top left: base model with full attention.
Top right: base model with local window.
Bottom left: RELIC-like memory.
Bottom right: Sink retrieval at true past position.

Conclusion

More tokens alone do not give a video world model long-term memory. The right evidence has to survive eviction, be found from the current camera, and reach the model in a form it was trained to read.

Our training-free study shows that exact distant revisits reward retrieval and
persistent anchors, while camera-aware score geometry is safer than untrained
value transport. The expanded translation suite adds to the evidence for sink retrieval, though pose retrieval and compressed history each win on particular trajectories.

Reference

[1] LingBot-World: arXiv:2601.20540
[2] WorldKV: Efficient World Memory with World Retrieval and Compression: arXiv:2605.22718
[3] Matrix-Game 3.5: arXiv:2608.29910
[4] RELIC: Interactive Video World Model with Long-Horizon Memory: arXiv:2512.04040
[5] Addressable Memory for Video World Models (WorldTrace): arXiv:2608.07408
[6] Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval: arXiv:2506.03141