PointGT: Simultaneous Geometry and Texture Editing for Point-Based Representations

Zhang, Yanshu; Shramko, George; Srinivasan, Pratul P.; Li, Ke

PointGT: Simultaneous Geometry and Texture Editing for Point-Based Representations

Yanshu Zhang¹, George Shramko¹, Pratul P. Srinivasan², Ke Li¹

¹Simon Fraser University ²Google DeepMind

Paper (21 MB)

Overview. Given multi-view captures (left), PointGT reconstructs a point-based 3D representation and learns a global UV mapping onto 2D charts (middle left). By maintaining a deformation-aware correspondence between canonical and deformed space, PointGT ensures persistent texture edits under non-rigid geometry changes. Users can simultaneously deform and retexture a reconstructed dress (top) or a real-world table (bottom), with geometry and appearance fully decoupled and editable.

Abstract

We present PointGT, a point-based 3D representation that enables simultaneous editing of object geometry and appearance. Existing reconstruction and view synthesis techniques produce volumetric 3D representations that are high-quality and photorealistic, but are difficult to edit. In particular, recent efforts to enable texture editing for 3D Gaussian Splatting representations are not compatible with geometry edits and deformations. Our method combines an attention-based point representation that is well-suited for geometry deformations with a learned UV mapping technique that enables high-resolution texture editing. We demonstrate that PointGT enables fine-grained editing of both geometry and appearance in point-based neural representations while maintaining high rendering quality.

Editing Results

Simultaneous geometry and texture editing results across multiple scenes.

Gaussian Splatting vs. Attention-Based Point Rendering

Gaussian Splatting vs. PointGT under deformation

Left: Since each Gaussian is a linear primitive, these methods struggle to preserve surface smoothness under large non-rigid deformations. Right: PointGT maintains surface continuity after deformation by predicting ray–surface points via attention-based interpolation on a point cloud.

How to maintain UV mapping under deformation

Deformation-aware canonical correspondence

In deformed space (left), a ray intersects the deformed point cloud to predict a surface point (red). Each deformed neighbor (dark blue) has a stored canonical counterpart (light blue); the per-neighbor displacements (dashed gray arrow) are fused with the deformed-space attention weights to transfer the intersection back to canonical space (middle), where UV lookup retrieves the texture (right).

Interactive Editing Demo

Screen recording of the interactive geometry and texture editing workflow.

BibTeX

@inproceedings{zhang2026pointgt,
  title     = {PointGT: Simultaneous Geometry and Texture Editing for Point-Based Representations},
  author    = {Zhang, Yanshu and Shramko, George and Srinivasan, Pratul P. and Li, Ke},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}