Why 2D priors limit 3D drags
Existing 3D drag methods follow two routes: optimizing a 3D representation such as a mesh or 3D Gaussians through Score Distillation Sampling, or explicitly warping geometry and then refining it with a 2D diffusion model. Both depend on 2D diffusion priors, which the paper argues lack true 3D awareness. The consequences it names are expensive iterative optimization, multi-view inconsistency such as the Janus problem, and geometric violations including unnatural stretching and topological artifacts.
The paper's premise is that a consistent deformation should be computed where the object already lives: in the native 3D latent space of the generative model. That reframing is what makes the rest of the design necessary, because a latent-space edit has to solve two problems at once — deciding where each piece of geometry moves, and making the generative model accept that move without losing the object's identity.
A displacement field that respects occupancy
3DRAG extends an elasticity-based deformation model into 3D to produce a displacement field: for a source point near a handle, the displacement approaches that handle's drag vector and decays smoothly toward the deformation boundary. With multiple handles, each source point is assigned to the handle it is closest to by a distance-based weight.
Projecting points onto a discrete voxel grid creates collisions, since several source points can land on the same destination voxel. A plain winner-takes-all rule would let an empty voxel override an occupied one, producing cavities or topological tearing in the final mesh. The Occupancy-Aware Winner-Takes-All strategy adds a binary occupancy indicator to the weight, so occupied source points score above 1 and unoccupied ones at most 1. Occupied points therefore strictly dominate, which the paper reports yields meshes with better topological consistency and fewer artifacts.
The same displacement field partitions space into four regions that later control generation: a destination region receiving displaced content, an inpainting region with no source mapping, a preserving region left untouched, and a transition region that blends the boundary.
Warping the latent, then regenerating
The input asset is projected into a sparse Structured Latent space and inverted along the exact ODE trajectory, caching the final inverted latent together with the attention Query, Key and Value tokens at every layer and timestep. The initial latent is then assembled region by region: destination coordinates take the warped source latent, preserving and transition regions keep the original latent, and the inpainting region is filled with Gaussian noise so the model generates rather than interpolates. In the SLAT stage a destination point is kept only if it exists in the generated voxels and maps back to an occupied source voxel; otherwise it is reassigned to inpainting.
Two interventions run during denoising. For every spatial coordinate in the preserving region, the Query, Key and Value tokens are replaced with their cached inversion counterparts, which blocks the global self-attention from drifting into unedited areas. For the destination region, Logits-Level Token Fusion intervenes in the pre-softmax attention logits. The paper explains why this is needed: frameworks such as TRELLIS inject absolute 3D positional encodings into latent tokens before attention, so destination tokens would assign negligible attention to cached source keys under ordinary attention and lose the source features.
How the comparison was run
The method is built on TRELLIS and Voxhammer and runs on a single NVIDIA H20 GPU, operating natively in the sparse SLAT space at 64-cubed resolution. Inversion uses RF-Solver along the exact trajectory, with 25 sampling steps for both inversion and denoising and a text classifier-free guidance scale of 5.0 in both stages.
Baselines cover Gaussian-based and mesh-based drag editing. 3DGS-Drag and Drag Your Gaussian were run with official implementations and default hyperparameters, with Gaussians inside the user-defined mask treated as editable. ARAP and APAP operate on meshes, so the authors mapped user points to nearest mesh vertices as handles and chose vertices farthest from the handles in non-editable regions as anchors. The evaluation uses a newly constructed dataset of diverse 3D assets and complex multi-point drag instructions.
Where 3DRAG leads the table
In the reported table, 3DRAG records a chamfer distance of 0.008, PSNR of 31.50, SSIM of 0.974 and LPIPS of 0.025. The next-best chamfer distance is 0.023 from 3DGS-Drag, and the next-best PSNR is 29.37 from Drag Your Gaussian. Lower chamfer distance and LPIPS indicate closer geometry and perceptual similarity, while higher PSNR and SSIM indicate better reconstruction; these are distinct measures and should not be read as one score.
The margins are larger on the distribution-level metrics. 3DRAG reports FID of 21.99 and FVD of 42.32, against 63.31 and 181.29 for the strongest baseline on those columns, APAP. Its GPTEval3D score of 1316.46 is the highest in the table. Runtime is 227.74 seconds per edit, far below the 752.70 seconds of 3DGS-Drag and 1820.93 seconds of Drag Your Gaussian, though ARAP is much faster at 3.32 seconds while scoring far worse on every quality column.
The paper also reports DAI of 0.187 and MD of 15.89 for 3DRAG, both the lowest in the table, which it associates with drag accuracy and unedited-region preservation respectively. The qualitative comparison reaches the same conclusion, showing results against four baselines on inputs marked with editable masks, handle points, target locations and drag directions.
What the 64-cubed grid cannot resolve
Two limitations bound how far these results transfer. First, slight color shifts can occur during editing, which the authors attribute to the VAE of TRELLIS failing to guarantee perfect color consistency on reconstruction; they measure a mean Delta E76 of 8.14 and a mean CIEDE2000 of 5.59 relative to the original meshes. Second, drag precision is bounded by the resolution of the sparse active voxel grid, currently 64-cubed, so fine-grained instructions on extremely thin structures or highly intricate details may deform suboptimally.
Both limits are properties of the underlying generative model rather than of the drag formulation, and the authors expect the method to benefit as native 3D foundation models improve in spatial resolution and reconstruction fidelity. Until then, the reported numbers describe performance at that resolution and on the paper's own dataset, not at arbitrary geometric detail.
The priority shift from 2D priors to latent space
The paper's central claim is a priority rather than a single score: for point-based 3D drag editing, the deformation should be computed in the model's native 3D latent space instead of being distilled from 2D priors. The evidence for that priority is the combination of the lowest chamfer distance and LPIPS in the table with the lowest FID and FVD, achieved without SDS optimization or iterative point tracking and at 227.74 seconds per edit.
The practical reading is that 3DRAG is the stronger choice when an edit must stay view-consistent and preserve the untouched region, and when per-edit optimization time matters. The trade-off to watch is scale: at 64-cubed voxel resolution, thin or highly detailed structures are where the method's precision is most likely to fall short, and color fidelity is capped by the underlying VAE's reconstruction error.
Sources used in this answer
3DRAG: 3D-Native Object Drag Editing with Generative Models
3DRAG performs drag editing natively in a generative model's explicit 3D latent space and, in the paper's table, beats 3DGS-Drag, Drag Your Gaussian, ARAP and APAP on drag accuracy, geometric fidelity and unedited-region preservation while running in 227.74 seconds per edit.