[CVPR 2025] 4DEquine: Disentangling Motion and Appearance for High-Fidelity 4D Horse Reconstruction

TL;DR

4DEquine is a revolutionary framework that reconstructs high-fidelity 4D models (3D + Time) of horses from simple monocular videos. By splitting the problem into motion estimation and appearance generation, it eliminates the need for time-consuming per-video optimization, achieving State-of-the-Art (SOTA) results while remaining efficient enough for real-world applications.

Background & Positioning

Reconstructing animals in 4D has historically been a choice between two evils: Template-based methods (like SMAL) which provide stable geometry but lack surface detail, and Optimization-based methods (like BANMo or GART) which produce high detail but take hours to process a single clip and fail under sparse viewpoints. 4DEquine occupies a unique spot in the academic coordinate system—it is a feed-forward, disentangled framework that uses the advanced VAREN parametric model to bridge the gap between efficiency and realism.

Problem & Motivation: Why is Horse Reconstruction Hard?

Horses possess complex non-rigid muscle deformations and are rarely filmed in perfect 360-degree "turntable" settings.

1. Temporal Jitter: Most single-frame estimators produce "jittery" motions when applied to video.
2. Observation Incompleteness: If you only see one side of a horse, traditional models can't "guess" the appearance of the other side accurately.
3. Efficiency Bottleneck: Waiting 15+ minutes for an optimization-based model (like GART) to converge per video is impractical for massive datasets.

Methodology: The "Disentangled" Secret Sauce

The authors split the pipeline into two specialized modules, connected by the VAREN Model (a high-resolution parametric model with muscle-aware deformation).

1. AniMoFormer (Motion)

Instead of looking at frames in isolation, AniMoFormer uses a Spatio-Temporal Transformer. It looks at a window of 16 frames to understand the "physics" of the movement, ensuring legs don't snap unnaturally. A subsequent Post-Optimization (PO) step nudges the 3D mesh to perfectly align with 2D silhouettes and keypoints.

2. EquineGS (Appearance)

This module creates a 3D Gaussian Splatting (3DGS) avatar from a single frame. The core innovation is the Dual-Stream Transformer Gaussian (DSTG) Decoder, which fuses 2D image features from a DINOv3 backbone with 3D point features from a VAREN template to predict the color, opacity, and shape of over 55,000 Gaussian primitives.

Figure 1: The 4DEquine framework. AniMoFormer handles the 'how it moves', while EquineGS handles 'what it looks like'.

3. Synthetic Data Dominance

Since real-world 4D horse data with ground truth is non-existent, the authors created:

• VarenPoser: 1,171 video clips with precise motion annotations.
• VarenTex: 150,000 multi-view images generated via diffusion models to teach the network photorealistic appearance.

Experiments & Results: SOTA Performance

4DEquine was tested on real-world datasets like APT36K and AiM.

• Geometry Precision: It achieved a PCK@0.05 of 84.2% on the AiM horse subset, crushing previous methods like AniMer (55.5%) and 4D-Fauna (46.5%).
• Visual Fidelity: While fully optimized models like GART achieve slightly higher PSNR (pixel-perfect matching of the input view), 4DEquine wins on SSIM and LPIPS, meaning its reconstructions look more "correct" and structurally sound to the human eye, especially in novel views.

Figure 2: Qualitative comparison. 4DEquine (Ours) demonstrates far superior leg alignment and texture consistency compared to 4D-Fauna and GVFDiffusion.

Zero-Shot Generalization: From Horses to Zebras

Perhaps the most impressive feat is the model's ability to generalize. Even though it was trained only on horses, it successfully reconstructs donkeys and zebras (Zero-shot) with high accuracy, proving it has learned robust features rather than just memorizing textures.

Figure 3: Reconstructing a donkey from a single internet image. The model generalizes the parametric "horse" prior to similar species.

Conclusion & Critical Analysis

Takeaway: 4DEquine represents a major leap in animal digitization. By moving away from per-video optimization and leveraging high-quality synthetic data, it makes 4D reconstruction practical.

Limitations:

• Extreme Occlusion: As shown in the paper's failure cases, if the input image is heavily truncated (e.g., the front half of the horse is missing), the appearance model struggles to "dream up" the missing biological parts.
• Dynamic Elements: The current model doesn't handle the fluid physics of manes and tails perfectly, as these aren't fully captured by the VAREN skeleton.

Future Outlook: The next logical step is Multi-Keyframe Fusion—allowing the model to peek at a few different frames in a video to build an even more perfect avatar.