Religious & Philosophical

A Fast Evolutionary Algorithm for Real-Time Vehicle Detection

A Fast Evolutionary Algorithm for Real-Time Vehicle Detection
of 16
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 62, NO. 6, JULY 2013 2453 A Fast Evolutionary Algorithm forReal-Time Vehicle Detection Vinh Dinh Nguyen, Thuy Tuong Nguyen, Dung Duc Nguyen, Sang Jun Lee, and Jae Wook Jeon,  Member, IEEE   Abstract —The evolutionary algorithm (EA) is an effectivemethodforsolvingvariousproblemsbecauseitcansearchthroughvery large search spaces and can quickly come to nearly opti-mal solutions. However, existing EA-based methods for vehicledetection cannot achieve high performance because their fitnessfunctionsdependonsensitiveinformation,suchasedgeorcolorin-formation on the preceding vehicle. This paper focuses on improv-ing the performance of existing evolutionary-based methods forvehicle detection by introducing an effective fitness function thatcan more accurately capture a vehicle’s information by combininga disparity map, edge information, and the position and motion of the preceding vehicle. The proposed method can detect multiplevehicles by using a turn-back genetic algorithm (GA) and can pre-vent false detection by using motion detection. Our fitness functionis designed in a typical manner along with the fitness parameters.These parameters are usually selected using heuristic methods,making the choice of optimal parameters difficult. Therefore,this paper proposes a new approach to estimating optimal fitnessparameters using EA and the least squares method. Robustnesstesting showed that the proposed method provides detection rate(DR) results close to those obtained using a state-of-the-art systemand outperforms other dominant vehicle-detection-based EAs.  Index Terms —Distance estimation, evolutionary algorithm(EA), stereo vision, vehicle detection. I. I NTRODUCTION I NTELLIGENT driving assistance systems offer importantadvantages, particularly in their ability to save lives bypreventing accidents. The key purpose of driving assistancesystems is to obtain traffic and road information, such as lanewidth [1], free space [2], vehicle distance [3], road surface [4], driving states of oncoming vehicles [5], and security margin[6]–[8]. This paper focuses on reducing accidents by detectingand estimating the distance to the preceding vehicles. Twomethods are currently used to estimate vehicle distance: usingonly vision sensors (e.g., cameras) or fusing vision sensorswith other sensors (e.g., radars) [9]. Both vehicle detection Manuscript received July 8, 2012; revised October 14, 2012 andDecember13,2012;acceptedJanuary14,2013.DateofpublicationJanuary25,2013; date of current version July 10, 2013. This work was supported in part bythe Korean Ministry of Knowledge Economy under Grant ITRC NIPA-2012-(H0301-12-3001) and in part by the National Research Foundation of Koreaunder Grant MEST(2012-0005861) through the Pilot Research CooperationProgram. The review of this paper was coordinated by Dr. A. Chatterjee.V. D. Nguyen, D. D. Nguyen, S. J. Lee, and J. W. Jeon are with the School of Information and Communication Engineering, Sungkyunkwan University, Su-won 440-746, South Korea (e-mail:;;; T. Nguyen is with Image Mining Group, Institut Pasteur Korea, Seongnam463-400, Korea (e-mail: versions of one or more of the figures in this paper are available onlineat Object Identifier 10.1109/TVT.2013.2242910 and distance estimation (VDDE) using vision sensors are verychallenging due to the great deal of variability in vehicleappearance. Camera-only-based vehicle detection can be cate-gorized into two main techniques: monocular vision and stereovision. Monocular-vision-based techniques [10], [11] use priorknowledge of the vehicles in the image, such as symmetry,color, shadow, texture, vehicle light, vehicle motion, and therelationship between vehicle size and distance. Among stereo-vision-based method, disparity map and inverse perspectivemapping are two types used for vehicle detection in [12].Every intelligent driving assistance has shortcomings. Avehicle’s feature based on colors is usually not effective underthe varied illumination conditions of outdoor environments. Asystem based on symmetry, corners, or texture yields a highnumber of false positives in complex environments where thereare many clusters. Using shadow information on a precedingvehicle is unsuccessful under imperfect weather conditions. Adetection approach using horizontal and vertical edges dependson a number of thresholds that might impact performance.Both vehicle-motion-based and stereo-vision-based methodsare time-consuming. Moreover, stereo methods work well onlywhen the two cameras are strongly fixed on the vehicle andaccurately calibrated. It is difficult to ensure optimal conditionsin any on-road scenario. Recently, a few approaches to genetic-based methods for vehicle detection have been proposed. Themost common genetic algorithm (GA) for vehicle detectioninvolves randomly generating all possible vehicle candidates,each as a chromosome. Subsequently, all chromosomes con-tinue to evolve via natural selection and genetics until a finalsolution is obtained. Two dominant applications of evolutionaryalgorithms (EAs) for vehicle detection are proposed in [6]and [7]. Obstacles are detected based on a set of 3-D pointsgenerated on the surface of an obstacle in [6] or according toedge information in [7]. Genetic-based methods can detect ob-stacles on the road, but they are unable to address noise comingfrom the pavement, buildings, or surrounding environments. Inaddition, most genetic-based methods are time-consuming.Many vehicle detection methods have been proposed basedon features, learning, and motion, but few approaches to vehicledetection are based on EAs. The processing time and accuracyof the existing EAs [6], [7] are not appropriate for real-time sys- tems. This motivated us to develop a new system based on EA.In a previous study [8], we proposed a genetic-algorithm-basedmethod to sustain various types of driving noise. That methodcan detect and estimate the distance of any preceding vehiclewith high accuracy and a rapid processing time. However, themaindisadvantageofthatmethodisthatonlythevehicleclosestto the detection system can be detected. Thus, in this paper, we 0018-9545/$31.00 © 2013 IEEE  2454 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 62, NO. 6, JULY 2013 focus on multiple vehicle detection using stereo vision and EA(SEA). This paper has two objectives. First, we present a brief review of stereo-based vehicle detection tools and their use indistance estimation applications. In the review, we concentrateon the common models, methods, system configurations, andfeatures of stereo-based VDDE systems. The second objectiveis to propose a system to overcome the shortcomings of currentVDDE systems. Additional abilities of the proposed systeminclude the following: 1) Its functionality works in differentillumination, complex environmental, and imperfect weatherconditions; 2) it improves computational speed by fusing thefield-programmable-gate-array-based (FPGA) stereo matchingimplementation with a fast central-processing-unit-based GA;3) it reduces the search space using V-disparity information;and 4) it automatically estimates the fitness parameters. Thedetails of these contributions are presented later.The remainder of this paper is organized as follows. Sec-tion II briefly reviews existing stereo-based VDDE systemsbased on several characteristics: stereo configuration, possiblerange, feature extraction, and evaluation of results. Section IIIdescribes the proposed genetic-algorithm-based VDDE system.Section IV presents the experimental results. This paper con-cludes in Section V.II. B RIEF  R EVIEW OF  S TEREO -V ISION -B ASED V EHICLE  D ISTANCE  E STIMATION  S YSTEMS The relatively few surveys that have been conducted inthe field of intelligent transportation have included excellentreports about vehicle detection [12], lane estimation [13], and vehicle speed measurement [14]. The work of one existingstudy [12] reviewed vision-based on-road vehicle detection sys-tems; however, it did not provide a detailed analysis of stereo-based systems or the stereo configuration. Surveys in the fieldof VDDE began in the year 2000, and much of the informationtherein is still active and relevant (including the first proposedsystem by the authors of the current study) due to the growth of efficient computer visiontechniques. Some ofthesystemssharecommon models, methods, configurations, and features. Othersystems have srcinated from diverse approaches to VDDE.The existing stereo-based VDDE systems have been presentedin terms of their characteristics, including their stereo config-urations, maximum detected distance (MDD), stereo match-ing, feature extraction, and evaluation of results. Most stereosystems have been evaluated by comparing their output to theactual distance [15]–[20] or using ground-truth data [21]. Most VDDE systems have followed four main stages: preprocess-ing, vehicle detection, vehicle tracking, and vehicle distanceestimation. Generally, the tasks required to estimate vehicledistance in the surveyed systems are as follows: 1) Setting upand calibrating the camera system; 2) obtaining successive leftand right images; 3) detecting and tracking preceding vehi-cles; and 4) estimating vehicle distance based on the obtainedstereo information. The main characteristics of stereo-basedVDDE systems are presented in the following, including theircommon assumptions and requirements, camera calibration andstereo matching techniques, and vehicle detection and trackingtechniques. Fig. 1. Possible detection ranges of our system with respect to four differentconfigurations (focal length f   and baseline b ).  A. Common Assumptions and Requirements 1) The camera system is securely fixed to the vehicle be-cause a very small change in camera position mightimpact the stereo results.2) The detection range is limited (e.g., 15–80 m) accordingto the various ranges of camera focal lengths.3) Vehicles travel either away from or toward the camerasystem.4) Various types of vehicles (cars, trucks, buses, and vans)are supported.5) Systems can deal with varying conditions, such as trafficand varying speeds in different lanes.6) Systems can deal with varying lighting conditions.7) Systems operate in real time.  B. Camera Calibration and Stereo Matching Camera calibration plays a crucial role in stereo-vision-basedsystems, particularly transportation systems. The accuracy of cameracalibrationdirectlyimpactstheestimateddistancetotheobject, and when an accurate camera calibration technique isapplied, the problem of the determining depth information fromtwo stereo images is substantially simplified. Most systems usecameras with focal lengths of 6 to 12 mm, and the baselinesvary from 15 to 150 cm. Fig. 1 shows the possible detectionrange of our system with respect to four different baselines andfocal lengths.The sum of absolute differences (SAD) and dynamic pro-gramming (DP) are the most common techniques for stereomatching in existing VDDE systems. DP has been efficientlyused in several prominent systems [16], [17] and, therefore, has the SAD-based method [21], [22]. The remaining VDDE systems use various matching methods, such as high-accuracystereo vision system for driving environments [3], mean-freecross-correlation function (KKFMF) [15], corner-feature-basedmatching [19], edge-feature-based matching [18], [23], the sum of squared differences [24], and line-segment-based matching[25]. For up-to-date systems based on graphic processing unit(GPU) and FPGA, other methods, such as belief propagation[26] or semiglobal matching (SGM) [27], can be implemented. Many modern systems are based on SGM to obtain high-qualitydisparitymapsupportingvehicle detection [5],[28]ortobuilda 3-Dmodelofobstaclesonrealroads[29].Incontrasttoexistingsystems, our system used census-based correlation due to itsrobustness for accommodating random noise within a matchingwindow. Moreover, the census transform uses bit-oriented costcomputation that is significant to FPGA implementation [30].  NGUYEN  et al. : FAST EA FOR REAL-TIME VEHICLE DETECTION 2455 C. Vehicle Detection and Tracking Many methods have been used to extract vehicle features fordetection and tracking. VDDE systems have srcinated fromdiverse feature detection approaches: 3-D points and edges areextracted and clustered in [15]–[17]; symmetry characteristicsare used in [17] and [20]; and edge features are another option for feature detection in [21] and [25]. In addition, lane recog- nition can be used [18]; therefore, vehicles can be effectivelydetected in their own lanes. After obtaining the characteristicsthat indicate vehicles, the next step is to estimate the move-ment of each individual vehicle. Image correlation and Kalmanfiltering are used in most systems due to their simplicity andeffectiveness. The image correlation technique measures thesimilarities between vehicles detected in the previous frame andthose in the current frame. A Kalman filter has been used topredict and smooth the distance estimates and to reject outliers[21], [23].To detect obstacle at short distances, [31] proposed a stereobox providing easy, fast, and reliable stereo obstacle detection.The fusion of radar and stereo vision was introduced to detecta vehicle and a railroad in [9]. Radar was used to limit thesearch space, and vehicles were detected based on the verticalsymmetry information in [9]. Recently, real-time dense stereomatching algorithms have become more and more popularin on-road applications because they can be implemented onFPGA or GPU. To handle a huge amount of stereo data, [29]introduced a middle level called Stixel World, which separatesthe pixel-based level and the object-based level. Stixel Worldis a flexible solution for modeling traffic situations in 3-D. Todetect a moving obstacle from a moving system, a segmen-tation of the dynamic Stixel World is proposed in [28]. Ina moving situation, the background subtraction method doesnot work well. Therefore, another algorithm also used is thesegmentation of dense scene flow [32], which is introduced in[33] to detect a moving object. In this approach, each pixel inthe image plane is modeled with motion likelihood, and themin-cut max-flow is used to solve the global optimal spatiallycoherent labeling. To detect and track multiple objects, [34] and[35] used an elevation map [4]. More recently, [36] has defined the occupancy grid to model and track a moving object in thedriving environment.In addition, many proposed methods (such as in [37] and[38])haveusedthemotionfromopticalflowtodetectprecedingobstacles. To detect moving obstacles in the case of partialocclusion of foreground or background, [39] introduced 6-Dvision by fusing 3-D position and 3-D motion.In sum, here, we examined the various techniques of existingstereo-based VDDE systems. It is important to look at the sur-veyed systems as a whole and to compare them with respect toperformancebasedonstereoconfigurationandmatching,possi-ble view ranges, features, result evaluations, and requirements.Table I summarizes and compares the existing stereo-basedVDDE systems, including our proposed system. The advan-tages and disadvantages of the existing systems are highlightedwithin our comparison and analysis. Although not all existingsystems are covered herein, the systems included for reviewsufficiently highlight the major characteristics in this field.III. V EHICLE  D ETECTION AND  D ISTANCE E STIMATION  B ASED ON  S TEREO  V ISIONAND  E VOLUTIONARY  A LGORITHMS Here, we describe the proposed SEA system using stereocameras, along with the proposed methods for vehicle detec-tion. To obtain more stable detection results, a simple trackingstrategy is also introduced. Two basic steps, i.e., hypothesisgeneration (HG) and hypothesis verification (HV), for on-roadvehicle detection were summarized in a previous report [12].The HG step is used to determine the location of the potentialobstacles. The HV step is used to remove the false detections.Our paper proposes a vehicle detection system that incorporatesthesetwosteps.Inadditiontoensuremorestableresults,weusepostprocessing and a simple tracking model. A block diagramof the proposed method is shown in Fig. 2. Our system usestwo main processes. The offline process consists of calibratingthe stereo camera and estimating the weighting constants of thefitness function. The online process is comprised of four mainstages: the disparity calculation, preprocessing [40], vehicledetection, and the extension process.  A. Stereo Camera Calibration and Stereo Matching First, we conducted calibration offline to determine intrinsiccamera parameters, such as the principal point and focal length.We applied the calibration method from [41], which was ef-fective with a chessboard at different view angles. Our camerasystem was calibrated with four different configurations basedon two focal lengths ( f   =  16 and 25 mm) and two baselines( b  =  35 and 60 cm). The proposed system uses FPGA-basedcensus correlation for stereo matching [30] because of advan-tages such as its robustness to accommodate random noisewithin a window and its efficient bit-oriented cost computation.Fig. 3 shows our model with a chessboard, three cameras  ( f   = 25 mm ) , and the FPGA and PC systems.  B. Vehicle Candidate Detection Based on SEA Here, we introduce a new EA-based algorithm for detectingvehicle candidates. First, a chromosome structure was intro-duced to represent a vehicle candidate. The possible chromo-somes were then generated inside the region of interest (ROI)in the left image. Second, the fitness function was designed toestimate the fitness value of each chromosome. Tournamentselection and tournament crossover were then performed to se-lect and generate strong chromosomes for the next generation.Finally, after reaching the terminal condition, the strongestchromosomes surviving were considered as real vehiclecandidates. 1) Chromosome Structure:  Selecting an appropriate geno-type representation is critical work for reducing the processingtime of an EA. Three genotype representations commonly usedin GAs and genetic programming are string, matrix, and treestructures [42]. This paper proposes a chromosome structurebased on the string representation because it is flexible andallowsustoeasilyextendthechromosomesize.Thephenotypictrait of a chromosome has a rectangular shape, as shown in  2456 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 62, NO. 6, JULY 2013 TABLE IC OMPARISON OF  S TEREO -V ISION -B ASED  VDDE S YSTEMS Fig. 2. Flowchart of our proposed system based on the general vehicledetection strategy [12]. Fig. 4(a). Each chromosome is organized by four genes, asshown in Fig. 4(b).  G ( i ) ux  and  G ( i ) uy  represent the  x -coordinateand the  y -coordinate of the  i th chromosome’s upper left corner,respectively, whereas  G ( i ) lx  and  G ( i ) ly  represent the  x -coordinate Fig. 3. Proposed system with two charge-coupled device cameras, a chess-board, an FPGA board, and a PC for stereo camera calibration.Fig. 4. (a) Initial population based on a Gaussian process. (b) Chromosomestructure.  NGUYEN  et al. : FAST EA FOR REAL-TIME VEHICLE DETECTION 2457 and the  y -coordinate of the  i th chromosome’s lower rightcorner, respectively. 2) Initial Population:  The population of chromosomes rep-resenting vehicle candidates is randomly initialized within theregion of the left image as x ci  = x i  + w ci,x y ci  = y i  + w ci,y  (1)where  ( x ci ,y ci )  is the position of the upper left corner of thechromosome at time index  i ,  ( x i ,y i )  is the position of theimage’s center, and  w ci,x  and  w ci,y  are zero-mean Gaussianprocesses with standard deviations of   σ cx  and  σ cy , respectively.Experimental results show that suitable width and height valuesof the initial rectangle are 20 and 15 pixels, respectively. Theseparameters were determined by measuring the width and heightin pixels of a sample vehicle at 140 m. After generating thepopulation, the EA needs a fitness function to evaluate whichchromosome is stronger in natural competition. 3) Fitness Evaluation:  Reference [6] proposed obstacle de-tection based on an evolutionary method. The fitness functionwas designed based on color information in the left and rightimages. However, it is very difficult to ensure that the same con-ditions exist for two cameras due to brightness differences orimagedevice changes inrealscenes.Meanwhile, [7]proposed amethod to detect a preceding vehicle based on a GA by solvingthe correspondence problem based on edge information in theleft and right images. However, that method can easily lead tofalse detections when multiple obstacles appear on the road be-cause the fitness function is sensitive to edge information frombuildings, trees, traffic, and other sights. Moreover, the fitnessfunction’s coefficients were heuristically designed and cannotguarantee that the method can obtain the optimal solution undervarious road conditions. Motivated by these two approaches,our proposed fitness function incorporates robust features, suchas edge information, disparity information, vehicle positionrelated to the image center, and vehicle motion. These featuresguarantee that our proposed method can address various roadconditions and obtain better results than existing EA methods.The fitness of an individual in a GA is given by the valueof an objective function for its phenotype. To apply the ideaof EA in the vehicle detection problem, the fitness functionhas to be designed to capture the vehicle’s features, assumingthat the disparity inside the vehicle region is homogenous. Thefirst feature for the chromosome was the disparity error locatedinside the chromosome’s region. The chromosome’s regionwas determined by mapping chromosomes into the disparitymap. This feature helped us to decrease the fitness values of chromosomes located in nonhomogenous regions, such as theroadsurfaceor trees.Anonhomogenous disparityregion occursdue to the quality of stereo matching results in real scenes. Thedisparity error is then described as follows: G ( i,t )de  =  1 w ( i,t ) h ( i,t )  w ( i,t ) h ( i,t ) − h ( i,t )  j =0 w ( i,t )  k =0 d (  j,k,t )  (2) Fig. 5. (Left) Edge view of a sample chromosome. (Right) Chromosomefeatures based on edge information. where  w ( i,t ) and  h ( i,t ) are the width and height of   i  chro-mosome at time index  t , respectively. The auxiliary function d (  j,k,t )  is defined as follows: d ( i,j,t ) =  1 ,  if   d v ( i,j,t ) =  D ( i,t ) v 0 ,  if   d v ( i,j,t )   =  D ( i,t ) v (3)where  d v ( i,j,t )  is the disparity value at position  ( i,j )  andtime index  t , and  D ( i,t ) v  is the dominant disparity value in-side the chromosome’s region. The dominant disparity wasthe disparity value with the highest occurrence located insidethe chromosome’s region. After analyzing various shapes of vehicles appearing along the road, we realized that the fourcorners of a vehicle could be used to distinguish vehicles fromother objects. Therefore, the second feature of the chromosomeis the density of the edge pixels along the left, right, top, andbottom sides of the  i th chromosome after applying Canny edgedetection. The density of edge pixels along these sides can becalculated as follows: G ( i,t ) l  =  1 ( 2 η + 1 ) h ( i,t ) h ( i,t )  j =0 η  k = − η I  eg  G ( i,t ) uy  +  j,G ( i,t ) ux  + k   (4) G ( i,t ) r  =  1 ( 2 η + 1 ) h ( i,t ) h ( i,t )  j =0 η  k = − η I  eg  G ( i,t ) uy  +  j,G ( i,t ) lx  + k   (5) G ( i,t )to  =  1 ( 2 γ  + 1 ) w ( i,t ) γ   j = − γ w ( i,t )  k =0 I  eg  G ( i,t ) uy  +  j,G ( i,t ) ux  + k   (6) G ( i,t ) b  =  1 ( 2 γ  + 1 ) w ( i,t ) γ   j = − γ w ( i,t )  k =0 I  eg  G ( i,t ) ly  +  j,G ( i,t ) ux  + k   (7)where  G ( i,t ) l  ,  G ( i,t ) r  ,  G ( i,t )to  , and  G ( i,t ) b  are the density of edgepixels along the left, right, top, and bottom sides, respectively,of the  i th chromosome at time index  t ;  I  eg ( i,j )  is the intensityvalue at pixel  ( i,j )  in the edge image; η  is the expected width of the left and right sides (in pixels); and  γ   is the expected heightofthetopandbottom sides(inpixels).Weuse η  =  γ   =  1inthispaper. A visual view of the second feature of the chromosomeis shown in Fig. 5. Thus, if the chromosome were locatedalong the corner of the vehicle region, the fitness value wouldbe higher than that for other regions. In addition, a precedingvehicle can be dangerous when it is located near our system.Therefore, the third feature of the chromosome is the distancefrom the camera system to the  i th chromosome, based on thedisparity information. This feature increased the fitness value
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks