2D Triangle Splatting for Direct Differentiable Mesh Training

Kaifeng Sheng*, Zheng Zhou*, Yingliang Peng, Qianwei Wang

*Equal Contribution

Amap, Alibaba

Code Paper

Interactive Viewer

Explore the mesh reconstructed by 2D Triangle Splatting (2DTS) method on the Nerf-Synthetic dataset and compare it with other reconstruction methods like 2D Gaussian Splatting (2DGS) [3] and Nvdiffrec [4].
Use the controls below to switch among different methods and rendering options. You can also use the mouse to rotate, zoom, and pan the view.
If the model doesn't change when you select a different method, give it a moment to load the new model!

Ship

Ficus

Abstract

Differentiable rendering with 3D Gaussian primitives has emerged as a powerful method for reconstructing high-fidelity 3D scenes from multi-view images. While it offers improvements over NeRF-based methods, this representation still encounters challenges with rendering speed and advanced rendering effects, such as relighting and shadow rendering, compared to mesh-based models. In this paper, we propose 2D Triangle Splatting (2DTS), a novel method that replaces 3D Gaussian primitives with 2D triangle facelets. This representation naturally forms a discrete mesh-like structure while retaining the benefits of continuous volumetric modeling. By incorporating a compactness parameter into the triangle primitives, we enable direct training of photorealistic meshes. Our experimental results demonstrate that our triangle-based method, in its vanilla version (without compactness tuning), achieves higher fidelity compared to state-of-the-art Gaussian-based methods. Furthermore, our approach produces reconstructed meshes with superior visual quality compared to existing mesh reconstruction methods.

Features

🔄 Fast Training

Train colored meshes end-to-end with speeds matching state-of-the-art Gaussian-based methods

⚙️ Efficient Representation

Scales to large scenes with thousands of images where traditional mesh reconstruction methods fail

🎮 Interactive 3D Viewer

Explore reconstructed triangle splats and meshes with intuitive controls and fluid navigation

📁 Seamless Integration

Export results in GLB and PLY formats for direct integration with modern game engines

Methods

2D Triangle Splatting (2DTS) replaces the Gaussian primitives from 3DGS [1] with triangle primitives and combines the compactness parameter from GES [2] to approximate a solid mesh representation. The triangle primitives are rendered using the splatting and alpha-blending methods introduced in 3DGS. To enable the gradient back-propagation through the rendering process, we use a decaying opacity function based on the barycentric coordinates of each point on the triangle plane. The depth and normal of each primitive can be calculated naturally from the normal of the triangle plane and the depth of the triangle vertices. A normal consistency loss similar to the one used in 2DGS [3] is applied to constrain the rendered normals and depth image.

Triangle Splatting Method Overview
Figure 1: Overview of the 2D Triangle Splatting approach. Our method represents the 3D scene with 2D triangle facelets and uses a compactness parameter γ to control the sharpness of the triangle edges. This method creates a discrete mesh-like structure while maintaining the benefits of continuous volumetric modeling.

References

  1. Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuhler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023.
  2. Abdullah Hamdi, Luke Melas-Kyriazi, Jinjie Mai, Guocheng Qian, Ruoshi Liu, Carl Vondrick, Bernard Ghanem, and Andrea Vedaldi. GES: Generalized exponential splatting for efficient radiance field rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19812-19822, 2024.
  3. Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2D gaussian splatting for geometrically accurate radiance fields. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024.
  4. Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Muller, and Sanja Fidler. Extracting Triangular 3D Models, Materials, and Lighting From Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8280-8290, 2022.