The majority of existing large 3D shape datasets contain meshes that lend themselves extremely well to visual applications such as rendering, yet tend to be topologically invalid (i.e, contain non-manifold edges and vertices, disconnected components, self-intersections). Therefore, it is of no surprise that state of the art studies in shape understanding do not explicitly use this 3D information, but rather focus on rendering based approaches. In conjunction with this, triangular meshes remain the dominant shape representation for many tasks, and their connectivity remain a relatively untapped source of potential for more profound shape reasoning. In this paper, we introduce ROAR, an iterative geometry/topology evolution approach to reconstruct 2-manifold triangular meshes from arbitrary 3D shape representations, that is highly suitable for large existing in-the-wild datasets. ROAR leverages the visual prior large datasets exhibit by evolving the geometry of the mesh via a 2D render loss, but still observes sub-pixel resolution features using a novel 3D projection loss, the Planar Projection. After each geometry iteration, our system performs topological corrections. Self-intersections are reduced following a geometrically motivated attenuation term, and triangular resolution is added to required regions using a novel face scoring function. These steps alternate until convergence is achieved, yielding a high-quality manifold mesh adhering to the requested triangle count budget. We evaluate ROAR on the notoriously messy yet popular dataset ShapeNet, and present ShapeROAR --- a topologically valid yet still geometrically accurate version of ShapeNet. We compare our results to various state-of-the-art reconstruction methods and demonstrate superiority in shape faithfulness, topological correctness, and triangulation quality. In addition, we demonstrate reconstructing a mesh from a neural Signed Distance Functions (SDF), and achieve comparable Chamfer distance with much fewer SDF sampling operations than the commonly used Marching Cubes approach.
The pipeline for reconstruction is shown below. After preprocessing the input, we iteratively refine the geometry and topology of the mesh, alternating between geometry and topology steps. Our main contributions include the usage of a planar projection loss, and a novel face score criterion for detection of regions that require additional triangulation.
When using Chamer distance as a loss term, the reconstruction suffers from artifacts and overlaps near sharp features. We instead propose using a planar projection operation formulated as a loss, which is more robust to such artifacts. The image below shows iso-contours of the two loss terms landscape in 2D. Notice how the corner is part of the zero set for the planar projection loss.
To detect regions that require additional triangulation, we propose a novel face score function:
Where nN is the mean of the face's vertex normals, and nf is the face normal. This function naturally captures local surface variations in the target shape, and is used to guide the addition of new triangles. Below a face on the evolving mesh considered for a split is super-sampled and projected onto the target, where the face score is evaluated (heatmap)
@misc{2307.00690,
Author = {Amir Barda and Yotam Erel and Yoni Kasten and Amit H. Bermano},
Title = {ROAR: Robust Adaptive Reconstruction of Shapes Using Planar Projections},
Year = {2023},
Eprint = {arXiv:2307.00690},
}