Accurate high-resolution 3D surface reconstruction and localisation using a wide-angle flat port underwater stereo camera: towards autonomous ship hull inspection
Thesis DisciplineComputer Science
Degree GrantorUniversity of Canterbury
Degree NameDoctor of Philosophy
The goal of this thesis is to scan a ship hull with high 3D accuracy and resolution using an underwater stereo camera so as to enable the future autonomous detection of invasive biofouling organisms with autonomous underwater vehicles (AUVs). However, turbidity in most harbours necessitates being within a metre of the hull and thus requires ultra wide-angle camera lenses. But such ultra wide-angle lenses embedded in an underwater housing with a flat port lead to significant distance dependent image distortions. Prior research in this area has only considered narrower fields of view and so has not solved for the significant image distortions arising from wide-angle high resolution flat port underwater cameras. This thesis proposes a solution to modelling and calibrating the underwater camera for accurate 2D imaging and 3D reconstruction, and additionally demonstrates an accurate underwater real-time pose estimation system required for future ship hull relative AUV navigation.
In this thesis an ultra wide-angle, short-baseline stereo camera is used, which is embedded in a flat port underwater housing. Flat port underwater housings represent a cost efficient way to use arbitrary in-air cameras underwater. However, the flat port of the underwater housing is subject to light refraction and causes distance dependent distortion, which is particularly visible at the large angles of the ultra wide-angle stereo camera used. To incorporate the effects of refraction, the thesis uses the well-known and accurate physics-based refractive underwater camera model. In contrast to the perspective camera-based underwater camera model, the refractive underwater camera model accurately describes the distance dependency of distortion.
In the beginning of this thesis, the effects of refraction caused by a thick flat port underwater housing are summarised and extended. In this context, the fundamental magnification function is proposed, which enables the description of numerous known and also newly discovered effects. An additional quantitative analysis is carried out in which the importance to model the thickness of the port and the wavelength of light is revealed.
In refractive geometry with a thick flat port, refractive forward projection represents a fundamental operation and describes where a 3D object point is observed in a 2D camera image. Refractive forward projection is required in numerous applications, such as refractive calibration, bundle adjustment, simultaneous localisation and mapping (SLAM) or image restoration. Unlike perspective projection in air, this operation is non-linear and computationally more expensive. This thesis compares existing and proposes new refractive forward projection methods and shows in contrast to previous research that refractive forward projection is efficient enough for real-time applications.
The thesis also investigates the impact of the port and the impact of the indices of refraction on the camera's projection and reconstruction accuracy. A novel investigation shows that the water pressure, water salinity, water temperature, air pressure and the wavelength of light significantly affect the projection and reconstruction accuracy of wide-angle flat port underwater stereo cameras and should not be neglected by standard refractive indices.
Moreover, this thesis proposes an accurate and efficient calibration method for thick flat port underwater stereo cameras. The proposed calibration method mainly achieves its high accuracy by the use of a significantly higher number of calibration images. In contrast to prior research, the computation of the reprojection error does not represent a bottleneck if the proposed refractive forward projection method is used. In this way, the calibration is similar to standard in-air camera calibration techniques and minimises the reprojection error.
In combination with the proposed more accurate indices of refraction and refractive calibration, the underwater reconstruction accuracy of the novel configuration of a wide-angle flat port underwater short baseline stereo camera is evaluated under real-world conditions. In this context, a method is proposed, which enables the evaluation of the accuracy of the reconstructed 3D object space.
Both chromatic aberration and pincushion distortion are effects of refraction and are particularly visible at the large angles of wide-angle underwater cameras. In order to obtained distortion-free images with minimised chromatic aberration to texturize reconstructed 3D ship hull surfaces, this thesis proposes accurate real-time methods to minimise chromatic aberration and to correct the distortion in the underwater camera images.
The refractive camera model is based on image coordinates of images, which are distortion-free in air. But these in-air undistorted images are strongly distorted in-water by refraction, particularly at the large angles of wide-angle flat port underwater cameras. Image correspondence in these images is difficult. For that reason, this thesis proposes pseudo rectified images in which these distortions are minimised. Moreover, an accurate and efficient representation of epipolar curves is presented, which enables, for example, real-time constrained correspondence search or dense stereo.
This thesis concludes with the demonstration of a pose estimation system for future ship hull relative navigation. The proposed pose estimation system is the first underwater SLAM and visual odometry system, which is based on the more accurate refractive underwater camera model. This thesis shows that the proposed pose estimation system is very accurate in a water tank experiment and efficiently works in real-time, and thus is superior to prior underwater SLAM research, which is based on the less accurate perspective camera-based underwater camera model.