Entertainment & Humor

A natural image quality evaluation metric

Description
A natural image quality evaluation metric
Published
of 8
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  A natural image quality evaluation metric Xuelong Li a,  , Dacheng Tao b , Xinbo Gao c , Wen Lu c a School of Computer Science and Information Systems, Birkbeck College, University of London, London WC1E 7HX, UK  b School of Computer Engineering, Nanyang Technological University, Singapore c School of Electronic Engineering, Xidian University, Xi’an 710071, China a r t i c l e i n f o  Article history: Received 31 July 2008Received in revised form4 October 2008Accepted 7 October 2008Available online 22 October 2008 Keywords: Image quality assessmentReduced referenceHybrid wavelets and directional filter banks a b s t r a c t Reduced-reference  (RR)  image quality assessment   (IQA) metrics evaluate the quality of adistorted (or degraded) image by using some, not all, information of the original(reference) image. In this paper, we propose a novel RR IQA metric based on  hybridwavelets and directional filter banks  (HWD). With HWD as a pre-processing stage, thenewly proposed metric mainly focuses on subbands coefficients of the distorted andsrcinal images. It performs well under low data rate, because only a  threshold  andseveral  proportion  values are recorded from the original images and transmitted.Experiments are carried out upon well recognized data sets and the results demonstrateadvantages of the metric compared with existing ones. Moreover, a separate set of experiments shows that this proposed metric has good consistency with humansubjective perception. &  2008 Elsevier B.V. All rights reserved. 1. Introduction The past decades have witnessed the tremendousgrowth of digital image processing techniques for visualinformation representation and communication. Theperformances of these techniques depend greatly on thequality of input and output images, especially, in basicapplications such as compression, enhancement, anddigital watermarking. However, the assessment of imagequality stills an open problem and an area of activeresearch.Subjectively, to  have a look  is probably the best way toevaluate image quality, because human beings, e.g., endusers, should make ultimate assessment on performancesof algorithms on digital images. However, subjectivemethods normally take/cost much time/resource as endusers have to be highly involved — i.e., these methodscannot be routinely performed as there could even bedifference assessment between different (groups of) endusers.So, effective and efficient subjective  image qualityassessment   (IQA) metrics are desirable but too hard todevelop in real time systems. Therefore, objective IQA ismore demanding.Briefly, objective IQA metrics are classified into threemajor categories, namely,  no-reference  (NR),  full-reference (DR), and  reduced-reference  (RR). The criterion for thisclassification is: how much information from the srcinal(reference) image is required during the quality assess-ment procedure at the receiver end. Each of the threecategories of metrics has its own advantages anddisadvantages.Basically, NR metrics do not need any information of the original image, and the assessment fully dependson human perception upon an image itself. As we know,it is highly related to semantic image understanding,which is still beyond current visual information proces-sing models and techniques. So, the popular and doableIQA metrics are still the ones with reference, including FR and RR ones. Contents lists available at ScienceDirectjournal homepage: www.elsevier.com/locate/sigpro Signal Processing ARTICLE IN PRESS 0165-1684/$-see front matter  &  2008 Elsevier B.V. All rights reserved.doi:10.1016/j.sigpro.2008.10.007  Corresponding author. E-mail addresses:  xuelong@dcs.bbk.ac.uk (X. Li),dacheng.tao@gmail.com (D. Tao), xbgao@mail.xidian.edu.cn (X. Gao), terry.luwen@gmail.com (W. Lu).Signal Processing 89 (2009) 548–555  FR metrics need full information of the srcinal image.Their basic ideas are to compare two images based ontheir corresponding pixels, regions, and/or frequencyfeatures. FR metrics normally work well as full informa-tion is utilized — however, this rarely happens in realapplications.So, RR metrics are the most popular ones for IQA tasks.Digital image processing algorithms and application canbe regarded as communication systems, in which theinput (srcinal/reference) images are at the sender endand the output (distorted/degraded) images are obtainedat the receiver end. With RR metrics, only the mostimportant information for IQA is sent from the sender endto the receiver end. As the transmitted information is notof a large amount, the requirement of bandwidth does notchange much.Over the past years, a number of RR IQA metrics havebeendeveloped to extractharmonicamplitudeinformation[1,2], statistical model in Wavelet domain [3,4], and other ways. The key factor of RR metrics is the statistic analysison the changed/modified image feature(s) between thesrcinal and corresponding distorted/degraded images. Inother words, how to extract such effective and efficientfeatures to identify the difference between images is themajor concern of all RR IQA researches and this paper.Image geometric transforms, a group of capable toolsfor image description and feature extraction have beendeveloped and well studied over the past some 20 years.As their representatives, Wavelet transform has beenproposed and is able to detect dot singularity [5], andContourlet has been developed to detent the contourinformation [6]. They both achieve good performancesunder many situations for signal processing, especially forimages and videos. However, they also encounter someintrinsic problems when dealing with direction informa-tion. As we know, in IQA tasks, image distortions normallylead to the change of image texture and directioninformation. Wavelet extracts only the local feature andContourlet produces much distortion for the singledirection. Therefore, it is very demanding to furtherimprove the conventional image geometric transforms toeffectively represent not only texture but detailed direc-tion information of images.Aiming at that, this paper presents a novel RR IQAmetric with  hybrid wavelets and directional filter   (HWD).We first carry out the HWD operations upon both thesrcinal and the distorted/degraded images. Thereafter,with the results, the change between two images ismeasured by taking characteristics of human visualperceptual system into account.The organization of this paper is as follows: HWD isintroduced in Section 2. In Section 3, we present the newHWD based RR IQA model in detail. Section 4 reportsexperimental results and necessary analysis. Finally,Section 5 concludes. 2. Hybrid wavelets and directional filter  HWD[7] is an image transform, whichincorporatestheadvantages of waveletand  directional filter banks  (DFB) [8].HWD processes have two steps: (1) conventional waveletdecomposition is employed; and (2) an improved DFB isutilized to further extract directional features. For non-linear approximation of natural images, HWD has manyadvantages, e.g., (1) it is a sparse and non-redundancytransform, and can obtain scale and direction informationeffectively; (2) it provides better visual effect because itkeeps much texture and other details; and (3) it caneliminate some traditional distortion in flat image fields,which are produced by Contourlet etc.As for DFB, there are three main types, namely  verticalDFB  (VDFB),  horizontal DFB  (HDFB), and  normal DFB .After Wavelet decomposition, each wavelet subband isfurther decomposed by DFB with  m d  levels, where  m d o L and  L  is the number of wavelet levels. We choose differentfilters according to different details in each subband.The HWD family consists of two basic functions, andtheir major difference is only the order of applying VDFBand HDFB on the data:HWD1 Step  1: apply normal DFB to  m d  finest diagonal waveletsubbands  ð HH  i ;  ð 1 p i p m d ÞÞ ; Step  2: apply VDFB to  m d  finest vertical waveletsubbands  ð HL i ;  ð 1 p i p m d ÞÞ , and Step  3: apply HDFB to the  m d  finest horizontal waveletsubbands  ð LH  i ;  ð 1 p i p m d ÞÞ .HWD2 Step  1: apply normal DFB to  m d  finest diagonal waveletsubbands  ð HH  i ;  ð 1 p i p m d ÞÞ ; Step  2: apply the HDFB to  m d  finest vertical waveletsubbands  ð HL i ;  ð 1 p i p m d ÞÞ ; and Step  3: apply the VDFB to  m d  finest horizontal waveletsubbands  ð LH  i ;  ð 1 p i p m d ÞÞ .It can be seen that: in HWD1, vertical and horizontalwavelet coefficients are decomposed by vertical andhorizontal filters, respectively, and the normal DFB isemployed to reduce the complexity and minimize thedistortion to some extent.Different from HWD1, in HWD2 horizontal subbandsare decomposed vertically while vertical subbands aredecomposed horizontally. This is because: the decom-position is not full-construction, i.e. there are somevertical/horizontal coefficients in horizontal vertical sub-band, and HDFB (VDFB) is employed to enhance thedirectionality of these subbands. As a result, the distortionof non-linear approximation is minimized.Note that: in both HWD1 and HWD2, the normal DFBwith  D  directions is used to decompose the level  i  ¼  1,then decreases the directions of DFB along with the levels.Herein, VDFB and HDFB have  d  ¼  D /2 directions. 3. RR IQA metric based on HWD Fig.1 illustrates the framework of the proposed RR IQAmetric. In this Section, wefirst summarize the main stagesof the algorithm, thereafter detailed information isprovided in Sections 3.1–3.5.In general, images are decomposed into subbands byusing HWD. Variation of visual sensitive coefficients isanalyzed as a measurement of IQA. ARTICLE IN PRESS  X. Li et al. / Signal Processing 89 (2009) 548–555  549  At the sender end, the srcinal image is decomposed byHWD to different scale- and direction-subbands. Herein,in order to achieve a uniformvisual sensitivity to differentscale- and direction-subbands for human perception, eachsubband is masked with  contrast sensitivity masking   (CSF).After that, on the basis of characteristics of   human visionsystem  (HVS), we define a rational  sensitivity threshold , andcompute the proportion of sensitivity coefficients in eachsubband. This  sensitivity threshold  value and a smallnumber of   proportion  values are sent from the senderend to the receiver end for use as the reduced-referenceinformation.At the receiver end, the distorted/degraded images areprocessed with the same procedure as aforementioned.Say, HWD operations, threshold and proportion valuecomputation.With the above steps, an objective RR IQA metric canbe built by comparing values from the sender and receiverends as illustrated in Fig. 1.  3.1. HWD decomposition In the HWD decomposition stage of the proposed RR IQA metric, an image is first decomposed into threewavelet levels. The number of DFB decomposition levelsat the two finest scales is 3. Herein, to decompose theimage, the followings are applied: (1) wavelet withDaubechies ‘‘db1’’ filters; and (2) DFB with McClellantransform and ‘‘9–7’’ filters. Both HWD1 and HWD2 have16 directional subbands at the two finest scales. To furtherreduce the bandwidth in communication, only the finestdirectional subband is used. As shown in Fig. 2, half of thedirectional subbands are selected in the finest and finerscales, except the for the low frequency parts.  3.2. Contrast sensitivity masking  After HWD, an image is decomposed and useful (forIQA) subbands are selected for following steps.It has been reported that human eyes have differentsensitivity to signals with different frequencies. So, tomake full use of the above obtained coefficients indifferent scales and directions, there should be properprocesses and much researches have been done regardingto it [9,10]. Among those previous efforts,  contrast sensitivity function  (CSF) [11] has been demonstrated asan effective tool making sensitivity of human eyes thesame to different frequencies.In the basic CSF, it is defined: H  ð  f  Þ ¼  2 : 6 ð 0 : 192 þ 0 : 114  f  Þ e ½ð 0 : 114  f  Þ 1 : 1   f   ¼  ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  f  2  x  þ  f  2  y q   ¼  f  n n  f  s  (1)where  f   stands for the spatial frequency, which has theunit of   cycles/degree , and  f   x  (  f   y ) is spatial frequency in thehorizontal (vertical) direction. It is obvious that they sharethe same unit with  f. f  n  stands for the normalized spatialfrequency with units of   cycles/pixel  and  f  s  is the samplingfrequency with units of   pixels/degree , which can becomputed by (2) as introduced in [12].  f  s  ¼ ð 2 v   tan ð 0 : 5  Þ   r  Þ = 0 : 0254 (2)where  v  is the  eye-to-monitor   distance and it has the unitas  meter  . Practically,  v  is set to 0.8 — as between 2 and 2.5times of a monitor’s height.  r   is the monitor’s resolutionand has the unitof   pixels / inch . In our tests,the monitor is a21-inch one and of a 1024 by 768 resolution. Therefore,the value of   r   can be easily computed as  r   ¼  ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1024 2 þ 768 2 p   = 21  ¼  61  pixels/inch . ARTICLE IN PRESS srcinalImageSenderSideDistortedChannelReceiverSideHWDDecompositionCSF MaskingSensitivityThresholdAncillaryChannelNormalization of SensitivitySensitivityThresholdCSF MaskingHWDDecompositionPoolingSensitivityCompre-hensionQuality of Image... .........DistortedImage Fig. 1.  A framework of the proposed RR IQA metrics.  X. Li et al. / Signal Processing 89 (2009) 548–555 550   3.3. Sensitive threshold With previous analysis, CSF masking can now beapplied to each of the selected coefficient subbands tomake them having  same sense  to human’s naked eyes.However, further research on this masking pointed outthat: actually, HVS is mainly sensitive to the largercoefficients rather than all of them [13]. Therefore, it isdesirable to define a  sensitive threshold T  as given below in(3), and only the coefficients, which are larger than thisthreshold, will be utilized in our RR IQA metric. Theselarger coefficients are named as  visual sensitive coefficient  .Because of this threshold, the transmitted information isfurther reduced (only several proportion values, ratherthan a large number of coefficients, details see nextsubsection), i.e. in this digital communication system thebandwidth requirement is even lower. T   ¼  a M  X M i ¼ 1  ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 N    1 X N  j ¼ 1 ð  x i ;  j    ¯  x i Þ 2 v uut  (3)where  x i,j  is the  j -th coefficient of the  i -th subband infinest scale,  ¯  x i  stands for the mean value of the  i -thsubband coefficients,  M   is the amount of the selectedsubbands in finest scales, and  N   stands for the number of coefficients of each selectedsubband. Finally, the selectionof  a will be given in the following experiments. ARTICLE IN PRESS 1718192025272628 2  1  2  2  2   3  2  4  2   9   3   0   3  1   3  2   123491110125 7 8 13 14 15 16HWD1HWD221222324 1   7  1   8  1   9  2   0   25 2627 28 2   9   3   0   3  1   3  2   1 2 3 49 1011 1213 14 15 165678Low FrequencySelected high FrequencySubbands Fig. 2.  HWD1/HWD2 decomposition and the selection of subbands.  Table 1 Subjective vs. Objective: Consistency comparison between the proposed RR IQA metric and existing metrics.MODEL JPEG JPEG2000CC ROCC OR MAE RMS CC ROCC OR MAE RMSPSNR 0.9229 0.8905 0.1886 7.1180 9.1540 0.9330 0.9041 0.0947 6.4070 8.3130MSSIM 0.9674 0.9485 0.0400 4.7710 5.8320 0.9490 0.9368 0.0651 5.4220 6.7090WNISM 0.9291 0.9069 0.1486 6.0236 8.2446 0.9261 0.9135 0.1183 6.1350 7.9127Q (HWD1)  a  1 0.9662 0.9460 0.0571 4.5191 5.9398 0.9264 0.9118 0.1006 6.2399 7.95752 0.9704 0.9526 0.0514 4.3305 5.5646 0.9447 0.9277 0.0651 5.5687 7.15503 0.9702 0.9473 0.0400 4.3328 5.5910 0.9494 0.9300 0.0651 5.3850 6.96704 0.9695 0.9482 0.0343 4.3879 5.7127 0.9511 0.9306 0.0473 5.4155 6.96075 0.9658 0.9402 0.0343 4.6709 6.0960 0.9540 0.9333 0.0493 5.2857 6.79726 0.9554 0.9257 0.0629 5.3885 6.9388 0.9488 0.9287 0.0533 5.5366 7.1261Q (HWD2)  a  1 0.9666 0.9482 0.0629 4.4626 5.8784 0.9270 0.9131 0.1065 6.2216 7.92012 0.9715 0.9527 0.0457 4.2981 5.4605 0.9473 0.9307 0.0710 5.3992 6.95493 0.9728 0.9543 0.0400 4.1135 5.3206 0.9518 0.9344 0.0651 5.1876 6.73624 0.9729 0.9535 0.0343 4.2532 5.3827 0.9539 0.9363 0.0592 5.1559 6.63265 0.9674 0.9436 0.0343 4.4260 5.9697 0.9540 0.9362 0.0473 5.1357 6.63126 0.9652 0.9387 0.0343 4.6894 6.2104 0.9537 0.9354 0.0355 5.1316 6.6555  X. Li et al. / Signal Processing 89 (2009) 548–555  551  ARTICLE IN PRESS 15202530354045102030405060708090       M      O      S PSNRPSNR (JPEG,   = 4)PSNR (JPEG2000,   = 4)MSSIM (JPEG,   = 4)MSSIM (JPEG2000,   = 4)RR-WISM (JPEG,   = 4)RR-WISM (JPEG2000,   = 4)HWD1 (JPEG,   = 2)HWD1 (JPEG2000,   = 5)HWD2 (JPEG,   = 3)HWD2 (JPEG2000,   = 5) JPEG imagesFitting with Logistic Function 1520253035404550PSNR JPEG2000 imagesFitting with Logistic Function 0.40.50.60.70.80.91102030405060708090       M      O      S MSSIM0.40.50.60.70.80.91MSSIM90919293949596979899100102030405060708090       M      O      S D86889092949698100D0.10.20.30.40.50.60.70.8102030405060708090       M      O      S Q0.20.30.40.50.60.70.80.91Q0.10.20.30.40.50.60.70.8102030405060708090       M      O      S 102030405060708090       M      O      S 102030405060708090       M      O      S 102030405060708090       M      O      S 102030405060708090       M      O      S 102030405060708090       M      O      S Q 0.20.30.40.50.60.70.80.91Q JPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic FunctionJPEG2000 imagesFitting with Logistic Function Fig. 3.  Non-linear scatter plots of subjective MOS vs. four objective IQA metrics.  X. Li et al. / Signal Processing 89 (2009) 548–555 552
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks