GIMPhI: A Novel Vision-Based Navigation Approach for Low Cost MMS

GIMPhI: A Novel Vision-Based Navigation Approach for Low Cost MMS
of 8
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  See discussions, stats, and author profiles for this publication at: GIMPhI: A novel vision-based navigationapproach for low cost MMS  Article  · May 2010 DOI: 10.1109/PLANS.2010.5507286 CITATIONS 6 READS 44 4 authors:Some of the authors of this publication are also working on these related projects: Visual odometry View projectGeomatics operations for SKA (Square Kilometers Array)   View projectMattia De AgostinoGeographic Information Systems 31   PUBLICATIONS   92   CITATIONS   SEE PROFILE Andrea LINGUAPolitecnico di Torino 109   PUBLICATIONS   418   CITATIONS   SEE PROFILE Francesco NexUniversity of Twente 72   PUBLICATIONS   745   CITATIONS   SEE PROFILE Marco PirasPolitecnico di Torino 66   PUBLICATIONS   219   CITATIONS   SEE PROFILE All content following this page was uploaded by Andrea LINGUA on 17 February 2014. The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the srcinal documentand are linked to publications on ResearchGate, letting you access and read them immediately.  GIMPhI: A Novel Vision-Based Navigation Approach for Low Cost MMS Mattia De Agostino, Andrea Lingua, Francesco Nex, Marco Piras Land, Environment and Geo Engineering Department (DITAG) Politecnico di Torino Turin, Italy (mattia.deagostino, andrea.lingua, francesco.nex, marco.piras)  Abstract — Over the last two years, the Geomatics research group at the Politecnico di Torino has developed a Low Cost System, in which only low cost sensors are involved. The system is equipped with webcams, an MEMS IMU and up to four GNSS receivers. During this development, several (non negligible) problems have been solved in order to obtain good quality after the data processing. One of the main problems of the low cost systems concerns the occurrence of GNSS outages. In this case, the IMU can only estimate the trajectory and the attitude of the vehicle for short periods. For this reason, considering the high number of frames available (about 3-5 frame per second, fps), a vision-based navigation (VBN) approach, called GIMPhI (GNSS IMU and PHotogrammetry Integration), has been adopted and tested. The navigation solutions have been refined by means of integration with a photogrammetric approach (bundle block adjustment) and a rigorous weight matrix has been adopted in order to consider the different accuracies of the various sensor observations (GNSS, IMU and images). A detailed description of this integrated approach is presented in this paper. The first tests and the achieved results are then shown in order to evaluate the reliability of the proposed approach.  Keywords: Multisensor Integration, GNSS/INS, Bundle Block, Vision-based navigation, Mobile Mapping System I.   I NTRODUCTION  Nowadays, Mobile Mapping Systems (MMSs) are used to acquire spatial information, e.g. road paths, images and point clouds that can be directly georeferenced using an integrated GNSS/INS system ([1]). Thanks to the high performance and to the speed in acquiring the final products (geo-information), the use of MMSs will be extended to new application fields. During early impact assessment, the availability of an MMS could be of great importance, in particular if a quick investigation of all the damage is requested. On the other hand, the available MMSs are not flexible enough to be installed on all vehicle models and they are too expensive (more than 200 000 €) to be easily available to all countries. The remarkable technology improvements, that have been made, have increased the role of low cost sensors on the market, especially for navigation applications. Single frequency receivers are currently able to work with an accuracy of a few centimeters or decimeters under particular conditions. The new generation of low cost inertial sensors which use MEMS technology has been employed in several fields (mobile phones, games) and also for Geomatics applications. Digital cameras have also undergone incredible improvements. Commercial webcam can in fact obtain high definition videos and clear images (from 2 to 7 Megapixels), with very low costs and allow implementation of the device control. The Geomatics research group at the Politecnico di Torino has realized a Low Cost Mobile Mapping System (LCMMS) that merges low cost sensors and the mobile mapping systems. This system has been realized with the purpose of having an MMS that can be installed on different types of vehicles, which is cheaper than other systems and, at the same time, efficient and offering good performances. The main problem of this low cost system concerns the occurrence of GNSS outages. In this case, the IMU can estimate the trajectory and the attitude of the vehicle and adopt particular integration algorithms, such as loosely or tightly coupled algorithms considering both a Kalman filter ([2]) and Least-squares based integration algorithms, especially calibrated for low cost GNSS and INS data. These methods allow improving the IMU performance to be improved because the gyroscope and accelerometer drifts are reduced. These results, however, get gradually deteriorate for long GNSS outages (more than 25 s) e.g. when driving in long tunnels or in dense urban areas. In these conditions, the information obtained from the images (about 5-10 fps) acquired by the system can be used to improve the positioning and increase the redundancy of the system. Position and attitude data can, in particular, be corrected by means of a photogrammetric approach, exploiting the overlap between adjacent images: each point captured by the webcams is in fact visible in at least 3 or 4 frames. The GIMPhI approach (GNSS/IMU and PHotogrammetry Integration) tries to improve the navigation solution, by integrating the “a priori” solution (GNSS-IMU) with the photogrammetric information extracted from the images. In order to realize an automated approach it has been necessary to automatically extract points from the images. In this context, valuable help is obtained from the use of Computer Vision and its feature extractor and matchers, such as the SIFT operator. A modified version of the SIFT operator, the A 2 SIFT ([1]), has been adopted: in this way, the feature extraction from the images and the preliminary matching  between homologous points is made possible thanks to feature matching. This process is then refined by the robust Least Median Square (LMS) relative orientation estimation ([4]): in this way, all the feature matching blunders are deleted and only correct homologous points are considered. Finally, the orientation of all the images is computed using some of the homologous points and the preliminary GNSS-IMU information by means of a Bundle Block Adjustment (BBA). The achieved information (GNSS/IMU/image sequences) is integrated in ad hoc software using a rigorous weight matrix in order to consider the different degrees of precision of the various observations (GNSS, IMU, images). In general, each sensor contributes in a different way to the final solution, according to the operative conditions and this balance between different weights sensors is one of the most critical aspects that must be evaluated. A description of this approach and the first tests performed are presented in this paper. The goal of the work was to evaluate the reliability and the effectiveness of the proposed method. Different operative conditions have been considered in order to consider all the operative conditions usually encountered by MMSs. Dedicated tests have been carried out for this purpose, and the results have been compared with a reference dataset in order to evaluate the accuracy of our solutions. II.   D ESCRIPTION OF THE L OW C OST M OBILE M APPING S YSTEM  The Geomatics research group at the Politecnico di Torino has designed and built a universal system for Mobile Mapping, called LCMMS (Low Cost Mobile Mapping System, [5]-[6]). This system is made up of a metallic bar that can be used with any vehicle and that can hold up to four GNSS antennas, three low cost inertial sensors (Crossbow IMU 700CA, Crossbow IMU 400CC and XSens MTi) and three web-cameras (Logitech Quickcam Pro 9000, with a resolution of up to 2 megapixels). Figure 1. LCMMS layout The MMS realized prototype is composed of low cost sensors which have a medium level of accuracy in order to reduce the costs but not decrease the quality. It is necessary to pay special attention to the system calibration procedure, in order to avoid external errors. Calibration of the optical lens, definition of the reference frame transformation, synchronization of the time devices and integration between the sensors has been considered and estimated ([5]). After the calibration, LCMMS allows the images to be georeferenced to the position and attitude of the vehicle on each occasion, without any additional information, as described in the following section. As an alternative, all the sensors can be used to define the final solution (GIMPhI approach), as will be described in the next sections. III.   T RADITIONAL M OBILE M APPING I NTEGRATIONS  A brief description of the traditional integration between GNSS and IMU devices ([7], [8]) is presented. This solution represents the basis of our system since it gives a first estimation of the position and attitude of the vehicle in each epoch. Academic or commercial software devoted to realizing the GNSS/IMU integration are already available, but only if INS calibration models (bias, drift of gyros and accelerometers and their variations) are available. Unfortunately, it is very difficult to find a correct individual MEMS sensors calibration model and dedicated software, which allows the bias and drifts of the low cost IMU to be estimated and applied, is necessary. The Authors have in particular developed a software in which loosely coupled and tightly coupled algorithms are implemented. These algorithms and the software have been tested for several cases, and interesting results have been obtained ([7]).   A.    Loosely coupled approach This algorithm uses the information from the INS sensor and three GNSS receivers. When three GNSS antennas are used, it is also possible to obtain, in addition to the positions and the velocity, the attitude of the vehicle. The attitude of the vehicle can be determined in the GNSS process, using double-differenced carrier phase measurements between two of the three receivers and one, which is assumed as the master receiver (Lu, 1995) or, in analogy with the loosely-coupled architecture, directly from the computed positions of the three antennas. The ultimate goal of this loosely-coupled integration is to estimate the position and velocity corrections, and also the attitude corrections that they are applied to the inertial navigation solution. This estimate is performed when INS epochs are synchronized with GNSS epochs. This is generally carried out using a Kalman filter, and it is especially useful when low-cost inertial sensors, which have considerable gyroscope drifts, are used. The loosely-coupled algorithm is realized through a 15 state Kalman filter ([11]) composed of the variation error of each navigation state (position, velocity and attitude), and the variation of the accelerometer and gyroscope biases.  B.   Tightly coupled approach The tightly coupled approach for GNSS/INS integration is not a new innovation in itself but it has only recently been used  in the autonomous vehicle community ([12]). In this algorithm, GNSS pseudorange observables are directly fused to the INS states (usually the positions and velocities). In a low GNSS availability situation (e.g. urban canyons, tree-lined roads) a tightly coupled configuration, unlike the loosely-coupled solution, allows the navigation equations with two visible satellites to be solved. The integration model may also include variables such as GNSS signal propagation delays, accelerometer scale factor errors and system time delays, and the estimated values of these variables may be used to improve the inertial solution performance during GNSS signal outages, and for faster ambiguity reacquisition after GNSS outages ([13]). In addition, a tightly coupled algorithm directly processes the GNSS signals directly. In a well designed system this increases the chance of optimal solution performance. In our case, the tightly coupled approach had to be modified in order to consider a different computational schema. Double differences are available in order to define the position of each antenna and three constraint equations can be included in the mathematical system in order to obtain more robustness. This tightly-coupled approach is able to solve navigation problems even when only three satellites are tracked. In fact, it is possible to define four double difference equations from the GNSS data (two for each couple of antennas), another four double difference equations from the INS solution and three constraint distance equations. The value of redundancy is greater than one, therefore, a Least-Square approach or Kalman filter can be applied. IV.   GIMP H I:  THE NEW APPROACH  The information collected by the images can be used, in order to increase the redundancy of the MMT and correct the IMU drift. In particular their orientation parameters have to be estimated, in order to define the position and the attitude of each image during the acquisition. This process is achieved following several steps. The first step is the extraction and the matching of the feature of interest extracted from the images. Then, the matching process has to be refined by a robust relative orientation and, finally, these data can be used in the orientation itself by means of a Bundle Block Adjustment. A scheme of the proposed workflow is presented, in Figure 2. Figure 2. Workflow of the GIMPhI integration approach  A.   Feature extraction and matching Homologous points have to be extracted from adjacent images, in order to perform the orientation of the images acquired by the LCMMS. The SIFT operator ([14]) is one of the most frequently used operators in the photogrammetry and computer vision application field. SIFT extracts image features that are invariant to image scaling and rotation and partially invariant to changes in illumination and 3D camera viewpoints (affine transformation). The features (keypoints) are detected in a Difference of Gaussians (DoG) scale space, which represents the difference of Gaussian convolutions of the srcinal image (Figure 3). A predominant orientation of the radiometric gradients, which assures the invariance to rotations, is assigned to each local maximum of the DoG function. Finally, a “descriptor” is associated to each keypoint. The “descriptor” is a 128 dimension vector which summarizes the radiometric content of the keypoint neighborhood. The correspondence between two candidate points is found through the evaluation of the minimum distance between the “descriptors”. A detailed description of the SIFT algorithm can be found in [14]. The SIFT operator can give different results according to the dynamic range of the images or the texture distribution: some papers ([15]) have underlined the importance of SIFT contrast thresholds in relation to the number of extracted points. This aspect influences the performances of the SIFT detector, especially over areas around roads, such as grasslands, pavements or wooded zones. In these cases, the local dynamic range of the image is quite low and the image can be defined as “badly textured”. Therefore, some threshold parameters proposed in [14] for the removal of low-contrast regions has to be corrected. a) b) c) Figure 3. (a) DoG scale space; (b) predominant orientation of the radiometric gradient; (c) SIFT descriptor  A modified version of the SIFT detector has been developed and implemented for this purpose. The Auto-Adaptive SIFT (A 2 SIFT) allows the contrast threshold parameters of the SIFT detector to be defined, in relation to the local radiometric content around each feature. Some experimental tests on the MMS have already shown that A 2 SIFT ([1]) allows the feature extraction and matching to be increased, especially in areas with a high rate of repetitive-patterns or bad textures. In this implementation, the srcinal SIFT algorithm was modified to fit the contrast threshold according to the texture: in other words, each keypoint has a different threshold according to the texture of the image in its neighbourhood. In order to do this, a coefficient ( Tx_coef  ) which is able to define the local texture of the image was implemented. This texture coefficient allows the local radiometric content of the DoG scale to be evaluated around a keypoint: if the texture is good, the Tx_coef   will be high and vice versa. In this way, it is possible to predict the image areas where the keypoint extraction is more difficult. As a consequence, a lower contrast value  xˆ  D  can be used in these areas to extract a higher number of keypoints; vice versa, a higher contrast value ( )  xˆ  D  must be adopted in well-textured areas. For a more detailed description of the algorithm, reference can be made to ([1]). The distribution of the features extracted on the image is a fundamental aspect for the relative orientation and the bundle block adjustment: points that are too close or blank areas can compromise the stability of the relative estimation between images. On the other hand, the number of points alone does not assure the quality of the image orientation. It has been shown that the SIFT operator can match a high number of points on well-textured areas, while it does not allow any points to be matched on poor textures. A 2 SIFT allowed this problem to be partially solved.  B.    Matching refinement and robust relative orientation   The feature extraction and matching techniques provide a set of homologous points which are usually affected by outliers and gross errors. Therefore, adjustment techniques must be used to eliminate the inconsistency in the measurements. For this purpose, a robust estimation of the symmetric relative orientation has been carried out between the two images. The Least Median Square (LMS) ([16]) estimator has been considered. This approach is extensively used in photogrammetry ([4]), especially for the computation of the relative orientation between image pairs (in aerial applications) and the estimation of the fundamental matrix in close-range and Computer Vision applications. The algorithm removes outliers by means a two-step standard residual analysis according to the rejection threshold  L . LMS provides good results in the data set, which can have a number of outliers of the 80%. Nevertheless, it is not an efficient estimator, and therefore it does not supply accurate solutions. The unknown parameters must be re-estimated using the Least Square estimator in order to achieve the final results. The IMU attitude data are used as approximate values and initialize the LMS algorithm, in order to estimate the orientation parameters. The realized LMS algorithm leads to good results when a good point distribution is assured all over the image. A homogenous distribution strengthens the solution and ensures that the orientation parameter is determined in a reliable way. On the other hand, bad distribution points (clustered points) determine ambiguous solutions and unreliable parameters. For this reason the image texture and their semantic information are of particular significance in the proposed integration approach. The LMS solution can be influenced by mobile objects (cars, pedestrians, etc.) and can define erroneous parameter solutions; a manual deletion of tie points on these objects must be performed for this reason. The authors are at present developing an automatic car recognition algorithm in order to speed up this process. C.    Bundle Block Adjustment The image orientation has been conducted by means of a bundle block adjustment: the IMU, the image and the GNSS information (if available) is considered at this step in order to determine the position and the attitude of the vehicle at each epoch. This algorithm is implemented considering a rigorous weight matrix which considers the precision of the different sensors at each epoch. The weights are achieved considering, on one hand the root mean squares of the GNSS-IMU solution, and on the other, the residuals of each stereo-pair relative orientation. In this way, it is possible to consider the performance of the integrated sensors according to different operative conditions, and to exploit their information in the best way. In particular, if the GNSS solution is not available, the weight of the GNSS-IMU solution decreases progressively, and the solution is more influenced by the photogrammetric information. Instead, when an unreliable relative orientation is reached, the GNSS-INS solution might not be influenced by this information. Figure 4. Image region for the point extraction; the lower part of the image has not been considered Due to the high number of processed images, the Bundle Block Adjustment has high computational costs. The high number of extracted points increases the dimension of the normal matrix in the Bundle Block Adjustment. Nevertheless, it has been noticed that a lower number of well distributed points can be sufficient to perform the same process with the
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks