Adaptive sedimentation and patch optimization for multi-viewed stereo reconstruction.

Abstract

This dissertation presents two main contributions towards the Patch-based Multi-View Stereo (PMVS) algorithm. Firstly, we present an adaptive segmentation method for preprocessing input data to the PMVS algorithm. This method applies a specially developed grayscale transformation to the input to redefine the intensity histogram. The Nelder- Mead (NM) simplex method is used to adaptively locate an optimized segmentation threshold point in the modified histogram. The transformed input image is then segmented using the acquired threshold value into foreground and background data. This segmentation information is thus applied to the patch-based method to exclude the background artefacts. The results acquired indicated a reduction in cumulative error whilst achieving relatively similar results with a beneficial factor of reduced time and space complexity.

Secondly, two improvements are made to the patch optimisation stage. Both the optimisation method and the photometric discrepancy function are changed. A classical quasi-newton BFGS method with stochastic objectives is used to incorporate curvature information into stochastic optimisation method. The BFGS method is modified to introduce stochastic gradient differences, whilst regularising the Hessian approximation matrix to ensure a well-conditioned matrix. The proposed method is employed to solve the optimisation of newly generated patches, to refine the 3D geometric orientation and depth information with respect to its visible set of images. We redefine the photometric discrepancy function to incorporate a specially developed feature space in order to address the problem of specular highlights in image datasets. Due to this modification, we are able to incorporate curvature information of those patches which were deemed to be depleted in the refinement process due to their low correlation scores. With those patches contributing towards the refinement algorithm, we are able to accurately represent the surface of the reconstructed object or scene. This new feature space is also used in the image feature detection to realise more features. From the results, we noticed reduction in the cumulative error and obtained results that are denser and more complete than the baseline reconstruction.