CeRF
Convolutional Neural Radiance Fields for New View Synthesis with Derivatives of Ray Modeling
2023 arxiv - First author arxiv
In recent years, novel view synthesis has gained popularity in generating high-fidelity images. While demonstrating superior performance in the task of synthesizing novel views, the majority of these methods are still based on the conventional multi-layer perceptron for scene embedding. Furthermore, light field models suffer from geometric blurring during pixel rendering, while radiance field-based volume rendering methods have multiple solutions for a certain target of density distribution integration. To address these issues, we introduce the Convolutional Neural Radiance Fields to model the derivatives of radiance along rays. Based on 1D convolutional operations, our proposed method effectively extracts potential ray representations through a structured neural network architecture. Besides, with the proposed ray modeling, a proposed recurrent module is employed to solve geometric ambiguity in the fully neural rendering process. Extensive experiments demonstrate the promising results of our proposed model compared with existing state-of-the-art methods.
Model
© minisal
Network
© minisal
Results
© minisal
Coffee
© minisal
Drums
© minisal
Toaster
© minisal
AnyDoor-NeRF
An Efficient and Hierarchical Framework for Large Extended Indoor Scene
2023 arxiv - First author
Neural implicit representations have made significant advancements in rendering new views. However, the dynamic loading of NeRF has remained less explored. Despite the tremendous practical benefits, loading large scenes poses a grand challenge to efficient scene representation and memory requirements. In this paper, we propose a novel framework named AnyDoor-NeRF for the dynamic loading of large extended indoor scenes. Specifically, we introduce a hierarchical sparse voxel representation to model the scenes. With such an adaptive design, our method is able to partition large scenes into different levels for fast modeling with less memory cost. Furthermore, the partitioned scene can be dynamically loaded and joined for a better immersive experience. Quantitative and qualitative results on the ScanNet datasets show the effectiveness of AnyDoor-NeRF, which requires only about 5% training steps and about 70% GPU memory cost compared with NeRFusion.
Teasor
© minisal
Quality Results
© minisal
Quantity Results
© minisal
Dynamic Loading
© minisal
Demo
© minisal
AdapMVSNet
Efficient Multi-View Stereo with Adaptive Convolution and Attention Fusion
Pengfei Jiang, Xiaoyan Yang, Yuanjie Chen, Wenjie Song, Yang Li
CAD&CG 2023 oralMulti-View Stereo is a crucial technique for reconstructing the geometric structure of a scene, given the known camera parameters. Previous deep learning-based MVS methods have mainly focused on improving the reconstruction quality but overlooked the running efficiency during the actual algorithm deployment. For example, deformable convolutions have been introduced to improve the accuracy of the reconstruction results further, however, its inability for parallel optimization caused low inference speed. In this paper, we propose AdaptMVSNet which is device-friendly and reconstruction-efficient, while preserving the original results. To this end, adaptive convolution is introduced to significantly improve the efficiency in speed and metrics compared to current methods. In addition, an attention fusion module is proposed to blend features from adaptive convolution and the feature pyramid network. Our experiments demonstrate that our proposed approach achieves state-of-the-art performance and is almost 2x faster than the recent fastest MVS method.
Module
© minisal
Overview
© minisal
Results
© minisal
Realtime Reconstruction Demo
Recon Demo
© minisal