One model outputs poses and new views
LVSPM addresses a split that has shaped this area: view synthesis methods usually assume known camera poses, typically obtained from structure-from-motion pipelines that are computationally expensive and fragile, while geometry models such as DUSt3R and VGGT estimate poses but do not synthesize views and rely on dense 3D supervision. LVSPM frames the task as sequence-to-sequence prediction over unposed images and produces both outputs from the same forward pass.
Concretely, given N uncalibrated images, the model regresses a translation and a quaternion rotation per view plus horizontal and vertical fields of view in radians, and it renders a target image from a requested target camera configuration. The first view is treated as the canonical reference frame with zero translation and identity rotation, and translation scale is normalized by setting the distance between the first and furthest points to unit length, so scenes share a consistent coordinate system.
The design deliberately avoids explicit 3D inductive biases. Instead of feeding input-view Plücker ray encodings, each input image is split into patches and given a learnable camera token, with a special token for the reference frame; target views are still conditioned on Plücker rays computed from their requested camera parameters. That asymmetry is what lets the network infer camera information from images rather than receive it.
Test-time training layers carry long sequences
The scaling claim rests on the backbone. LVSPM stacks 24 Large-Chunk Test-Time Training (LaCT) blocks, each combining per-view windowed self-attention for local spatial dependencies, a test-time training layer with a SwiGLU-MLP, and a feed-forward MLP for channel mixing. The architecture is described as processing variable-length sequences up to 1M tokens.
Inside a TTT layer, the MLP's weights are updated at inference from a gradient computed on key and value projections of the tokens, then the updated MLP processes the query projections. Only input-view tokens pass through the key and value path and generate that gradient; target-view tokens are read-only. Because target tokens never interact with one another, each novel view can be rendered independently, which the paper reports as 58.8 FPS regardless of the number of targets.
The authors describe the TTT hidden state as a compressed scene representation queried by both camera tokens, for pose, and target ray tokens, for synthesis. Their ablation attributes a virtuous cycle to this joint setup: supervision from view synthesis supplies geometric signal that improves pose accuracy, removing the need for dense 3D supervision. This is the paper's own explanation of the mechanism, not an independently established causal claim.
Pose gains are largest at strict thresholds
Pose accuracy is reported as AUC3 and AUC5, area-under-curve measures at strict rotation or accuracy thresholds, so a higher value means more views land within a tight tolerance. These are not the same as looser-threshold AUC30, and the paper's headline advantage is concentrated in the strict ones.
On RealEstate10k at 16 views, LVSPM reaches 40.0 AUC3 and 53.8 AUC5 against VGGT's 14.0 and 25.3, roughly three times and twice the precision. At 128 views the gap widens to 61.3 versus 20.6 AUC3 and 72.5 versus 34.1 AUC5. Against DA3, which uses dense geometric supervision, LVSPM wins 10 of 12 metrics, leading on AUC30 and AUC5 at all view counts while remaining competitive on AUC3.
On Co3Dv2 the model beats VGGT at every metric, from 39.2 versus 33.9 AUC3 at 16 views to 68.8 versus 62.1 at 128 views. DA3 remains a strong sparse-view reference, but LVSPM surpasses it on AUC3 at 64 and 128 views and on AUC5 at 128 views without dense geometry labels. On DL3DV at 16 views it records 88.4 AUC3 against VGGT's 74.7 and DA3's 84.0, and at 256 views it holds 90.1 AUC3 while Fast3R falls to 8.0 and Cut3R to 19.2.
More views mean larger scenes, not denser sampling
The synthesis comparison depends on an evaluation choice that readers should not skip. Under the paper's equi-temporal protocol, N input views are sampled from the first N/Nmax portion of a video, so adding views expands scene coverage instead of sampling the same area more densely. Within each window, views are randomly sampled for pose estimation and evenly sampled for synthesis.
The authors argue that standard uniform sampling over a full sequence conflates two variables, scene coverage and sampling density, leaving it unclear whether gains come from seeing more of the scene or from denser observation. Holding density roughly constant isolates scene scale, and it makes the task progressively harder as views increase. For RealEstate10k and Co3Dv2, Nmax is 128 with N in {16, 32, 64, 128}; for DL3DV, Nmax is 256 with N up to 256.
This means the reported numbers describe performance as coverage grows, not performance under a fixed scene with more redundant frames. The paper also reports standard sparse-view protocols in supplementary material and states that results there confirm competitiveness, but those tables are not in the readable evidence here.
Synthesis quality degrades more gracefully than baselines
On DL3DV-10K, LVSPM reaches 25.77 PSNR at 16 views, above the pose-dependent DepthSplat at 24.80 and the pose-free AnySplat at 23.15, so it leads without receiving poses. PSNR measures pixel-level reconstruction fidelity; SSIM and LPIPS, reported alongside, capture structural similarity and perceptual distance and can rank methods differently.
The scaling pattern is the more decision-relevant result. Across a 16-fold increase in scene scale, LVSPM's PSNR falls from 25.77 to 22.09, a drop of 3.68 dB, while AnySplat falls from 23.15 to 18.55, a drop of 4.60 dB, and DepthSplat collapses from 24.80 to 18.39 at just 64 views before failing on model constraints. The gap over AnySplat widens from 2.62 dB at 16 views to 3.54 dB at 256 views, and LPIPS moves from 0.152 to 0.250 for LVSPM versus 0.158 to 0.339 for AnySplat.
The paper also reports zero-shot generalization to Tanks and Temples and MipNeRF360 under dense and sparse inputs, and sparse-view results on RealEstate10k, MipNeRF360 and LLFF, including a fine-tuned variant that outperforms baselines trained only on RealEstate10k. Those quantitative tables sit in supplementary material and are summarized rather than shown in the readable text.
Where the evidence stops
Several boundaries qualify the results. Training uses a multi-stage pipeline that the authors call more complex, and they note that performance may degrade under extreme lighting changes or in weakly textured scenes where reliable geometric reasoning is difficult. They also concede that specialized geometry-focused models may still do better in certain edge cases, so the pose advantage is not a blanket claim across all conditions.
The comparison set matters too. DepthSplat requires ground-truth poses and cannot run beyond 64 views, so its collapse partly reflects a model constraint rather than pure quality loss, and the pose-free comparison against AnySplat is the cleaner test of the paper's own setting. The strict-threshold AUC3 and AUC5 gains should not be read as equivalent improvements on looser metrics, where the paper itself reports only competitiveness against DA3 on AUC3.
The priority is long-sequence joint estimation
The paper's distinctive claim is not that it wins every metric but where it wins. Its advantage grows with sequence length and scene coverage: on RealEstate10k the AUC3 gap over VGGT expands from 26 points at 16 views to about 41 at 128, and on DL3DV the PSNR gap over AnySplat widens from 2.62 dB to 3.54 dB as coverage grows 16-fold. A reader deciding whether this matters should look at the longest sequence and largest scene in their own setting, because that is the regime the design targets.
The second priority is the joint formulation itself. LVSPM matches or beats models trained with dense geometric supervision while using only RGB images and camera poses, and it delivers poses and rendered views from one model. The trade-off is a more complex multi-stage training pipeline and reported fragility in extreme lighting and weakly textured scenes, which is where a geometry-specialized alternative may still be the safer choice.
Sources used in this answer
LVSPM: Long Sequence View Synthesis and Pose Estimation Model.
LVSPM jointly estimates camera poses and synthesizes novel views from uncalibrated long image sequences, surpassing VGGT on pose estimation across 16-256 views and achieving state-of-the-art pose-free synthesis on DL3DV-10K.