How To, Education & Training

A Robust Resolution-Enhancement Scheme for Video Transmission Over Mobile Ad-Hoc Networks

Historically, Error-Resilient (ER) video transmission and Super-Resolution (SR) image reconstruction techniques have evolved separately. In this paper, we propose a coordinated application of ER and SR to enhance the resolution of image transmitted
of 10
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  312 IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 2, JUNE 2008 A Robust Resolution-Enhancement Scheme for Video Transmission OverMobile Ad-Hoc Networks Liang Zhou  , Student Member, IEEE  , Baoyu Zheng  , Member, IEEE  , Anne Wei, Benoît Geller, and Jingwu Cui  Abstract— Historically, Error-Resilient (ER) video transmissionand Super-Resolution (SR) image reconstruction techniques haveevolved separately. In this paper, we propose a coordinated appli-cationofERandSRtoenhancetheresolutionofimagetransmittedover mobile ad-hoc networks. In order to combat error propaga-tion,aflexiblemultipledescriptioncodingmethodbasedonshifted3-D SPIHT (3-D Set Partitioning In Hierarchical Trees) algorithmis presented to generate variable independent descriptions (sub-streams) according to the network condition. And then, a novelunequal error protection strategy based on the priority level isprovided to assign a higher level of error protection to more im-portant parts of bitstream. Moreover, a robust SR algorithm isproposed in the presence of different kinds of packet loss rate toenhance the image resolution. Experimental results indicate thatthe proposed robust resolution-enhancement scheme outperformsthe competing methods from the aspects of PSNR (Peak-Signal-to-Noise Ratio) and visual quality under different packet loss rates.  Index Terms— Error-resilient, mobile ad-hoc networks, resolu-tion-enhancement, super-resolution. I. I NTRODUCTION I N RECENT years, mobile ad-hoc networks (MANETs) hasemerged as one of the high growth applications of wirelesscommunication technology, and the interest in the transmissionof video over MANETs has increased dramatically. However,this error-prone network is packet based where many potentialreasons may result in packet loss which has a devastating effecton the visual quality of images at the receiver. Furthermore, inmost electronic imaging applications, image with high resolu-tion (HR) is desired and often required because HR image canoffer more details that may be critical in various applications.Unfortunately,itischallengingtoprovideHRimagetransmittedoverMANETswherenoQualityofService(QoS)isguaranteedat the network level. In order for HR, it would be essential tosolve the following primary technical challenges: Manuscript received December 29, 2006; revised January 7, 2008. Thiswork is supported by the International Project PRA-SI (financed by France andChina government) under Grant SI04-03, the Key Project of Nature ScienceFoundation of Jiangsu (China) under Grant BK2007729 and also in part by theClimbing Plan of Nanjing University of Posts and Telecommunications underGrant NY207061.L. Zhou, B. Zheng, and J. Cui are with the Electrical Engineering Depart-ment, Shanghai Jiao Tong University, Shanghai 200240, China, and also withtheInstituteofSignalProcessingandTransmission,NanjingUniversityofPostsand Telecommunications, Nanjing 210003, China (e-mail:;; Wei is with the Laboratory of CEDRIC, Conservation National des Artset Métiers, Paris 75003, France (e-mail: Geller is with the Laboratory of LEI, Ecole Nationale Supérieure desTechniques Avancées, Paris, France, and also with the Laboratory of SATIE,Ecole Normale Supérieure (E.N.S.) Cachan, Paris 75015, France ( versions of one or more of the figures in this paper are available onlineat Object Identifier 10.1109/TBC.2008.917746 • Trade-off between coding efficiency and error resilience.Most current standardized video codecs, includingMPEG-2/4 and H.263/4 are designed to achieve highcompression efficiency at the expense of error resilience.The coding efficiency in these codecs is achieved by usingmotion-compensated prediction to reduce the temporaland statistical redundancy between the video frames. Thisbrings a severe problem, namely error propagation, whereerrors due to packet loss in a reference frame propagate toall of the dependent frames leading to visual artifacts thatcan be long lasting and annoying [1].Therefore, Error resilience is needed to achieve robustvideo transmission. One strategy to achieve resilience isto insert redundant information systematically into com-pressed video signals so that the decoder can compensatetransmission errors. The redundant information can beerror correction codes [2], [3] or multiple descriptions[4], [5]. The former one combined with layered codingcan provide good performance in prioritized networkswhile the latter is suitable for delivery over multiplechannels to enhance reliability. However, error resilienceis achieved at the expense of coding efficiency in bothmethods. Another way can be achieved with feedback mechanism to request retransmission or adjust encodingmodes according to conditions. The methods proposedin [6], [7] rely on feedback from the decoder and are,therefore, application-limited.• Enhance resolution under the scenario of packet loss. Inorder for HR image, one promising approach, which iscalled super resolution (SR), uses signal processing tech-niques to obtain a HR image from observed multiple lowresolution (LR) images [8]. In the past few decades, a va-riety of SR methods have been proposed for estimating theHR image from a set of LR images without taking into ac-count the packet loss during the transmission. The criticalrequirement for traditional SR approach is that the obser-vations contain different but related views of the scene [8],however, it can not be guaranteed under the framework of error-prone networks where the packet loss destroys thecorrelation between the related views of scene. Therefore,how to apply the SR approach to the packet loss scenariois still an open problem. Here, it is necessary to differen-tiateerrorconcealment(EC)withSR.EChidesorrecoversthe errors by using correctly received image informationwithout modifying source or channel coding schemes [9],which can only produce a visually acceptable (rather thanexact) image from the available data, and can not enhancethe physical resolution of the image. While SR extracts theexact detail information hidden among the different but re-lated video frames to enhance the image resolution.To meet these challenges, in this paper, we propose an en-tire scheme to get HR video transmitted over MANETs by in- 0018-9316/$25.00 © 2008 IEEE  IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 2, JUNE 2008 313 Fig. 1. The total architecture of video transmission and processing. tegrating efficient ER strategy with robust SR algorithm, whichnot only provides relatively efficient compression and transportperformance but also provides robust resolution-enhancementperformanceinthepresenceofvariouspacketlossrates.Anout-line of the remainder of this paper is as follows. In Section II,we describe the whole system and provide the related techniquepreliminaries used in this work; in Section III, we present anadaptive error-resilient strategy to reduce the distortion due tothe packet loss; and then, a robust SR algorithm to enhance theresolution of received image is proposed in Section IV; Sec-tion V provides the simulation results and compares the perfor-mance of the proposed scheme with other comparison systems.Finally, we give some concluding remarks.II. P RELIMINARIES In this section, we first overview system framework of thevideo transmission and processing, and then present some re-lated technical preliminaries used in this work, such as shifted3-D SPIHT algorithm and multiple description coding.  A. System Overview Thetotalarchitectureofvideotransmissionandprocessingil-lustratedinFig.1iscomposedofthreeprocesses,suchasimagedegradation, image transmission over error-prone networks andthe image SR reconstruction process.Generally, all of the video sequences we observe are LR im-ages comparing to the real-world scenes which are viewed asthesrcinal HR images.That is because thedegradationprocessaffects the quality of images acquired by digital video camerawhich results from the lens’ physical limits, such as motionwarping, optical blur and additive noise. In addition, these LRimages are usually down-sampled convenient for transmissionorstorage.Next,theobservedLRimagesareencodedandpack-etized preparing for transmission over error-prone networks. Inthis paper, the method of encoding and packetizing is based onthe shifted 3-D SPIHT algorithm to generate variable descrip-tions (substreams) at the sender, and different descriptions em-ploydifferenterrorprotectionstrategiesaccordingtoitspriority.As to the MANETs system, suppose that there are mobilenodes in the system and senders (sources)stream complementary substreams to a single receiver (desti-nation) over different paths. In this system, sender- streamssubstreams- to the receiver over path- .At the receiver, the received images can be reconstructed afterdepacketizing and decoding the received data.  B. Shifted 3-D SPIHT Algorithm Wavelet zero-tree image-coding technique was first ad-dressed by Shapiro [10], and further developed by Said andPearlman [11], and have provided unprecedented high per-formance in image compression with low complexity. Later,extended 2-D zero-tree wavelet coding (EZW) by [11] hadbeen introduced to three dimensions (3-D EZW) by Chen andPearlman [12], and had shown promise of an effective andcomputationally simple video-coding system without motioncompensation, obtaining excellent visual results. And then,Kim and Pearlman developed the 3-D SPIHT [13] codingalgorithm based on the 3-D EZW mentioned in [12].3-D SPIHT algorithm provides excellent rate-distortion per-formancealongwithalowencodingcomplexity,andspatiotem-poral trees are defined as groups of wavelet transform coeffi-cients organized into trees rooted in the lowest frequency sub-band and the spatially and temporally related coefficients inthe higher frequency subbands, which is helpful to reduce the  314 IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 2, JUNE 2008 Fig. 2. Structure of the spatiotemporal relation of 3-D SPIHT. (a) Traditional 3-D SPIHT. (b) Shifted 3-D SPIHT. error propagation [13]. Fig. 2(a) shows how coefficients in a3-Dtransformarerelatedaccordingtotheirspatialandtemporaldomains. The parent-children linkage except at the highest andlowest pyramid levels (which do not have offspring) is:(1)where represents a 3-D of coordinates of all the off-spring at node .Although the SPIHT-coded bitstream has many advantages,it is also sensitive to data losses because of the dependenceamong wavelet coefficients in constructing a significance map.Motivated by the packetized 2-D SPIHT coding algorithm [14]partitions the wavelet coefficients into independent packets tocombat error propagation by shifting 2-D wavelet tree struc-ture, here we apply it to 3-D case by shifting the 3-D wavelettree structure. The essential aim is the wavelet coefficients fromdifferent sub-bands are interleaved to form independent packetsthat can be decoded independently. The formation of the shiftedwavelet trees is shown in Fig. 2(b). It should be noted that al-though a lot of multi-interlaced partitioning and packetizingmethods have been presented in recent years [23], they couldenhance the encoding burden greatly. Therefore, we don’t em-ploy them here. C. Multiple Description Coding As to the way to protect data from packet losses induced bythe error-prone channels, one of the most common ways is toadd the redundant information at the bitstream so that the src-inal video can be recovered in presence of the packet loss, andthe code should be devised in such a way that the decoder isable to recover the lost information or conceal the irrecoverableerrors.One such popular approach is multiple description coding(MDC), which is similar in spirit to the multiple-substream ap-proach. The fundamental principle of MDC is to generate mul-tiplecorrelateddescriptionsofthesourcesuchthateachdescrip-tion approximatesthesource witha certainlevelof fidelity[15].MDC does not impose any dependency among its descriptionssothateachextrasuccessfullyreceiveddescriptionimprovesthequality further regardless of what has been received so far.The benefits of using MDC in video streaming can be furtheramplifiedwhenMDCiscombinedwithpathdiversity(PD)[16].In this approach, each substream (or description) is explicitlytransmittedoveranindependentpathtoreceiver.PDexploitsthefact that the probability of having all the paths simultaneouslycongested is relatively low. As a result, the use of PD in videostreaming can achieve higher throughput and increase toleranceto packet loss.III. A DAPTIVE  E RROR -R ESILIENT  S TRATEGY In this section, a novel error-resilient strategy is proposedbased on partitioning the GOF (group of frames) into variablesubstreams with different priority levels adapting to the currentnetwork condition.  A. Unequal Error Protection Progressive bitstreams provide a natural basis for unequalerror protection (UEP), by which perceptually more importantparts of the bitstream are assigned a greater level of error pro-tection [17]. In this work, we propose a novel UEP based onthe expected lifetime of the path to guarantee the important part(high priority) of substreams are received at the receiver. Thatis to say, important substream corresponds to the “good” path.Here, we set the priority of the substreams according to theimportance of the subband derived from the wavelet decompo-sition. In general, the provides more visual informationthan , and (where denotes the decompositionlevel), so we set priority level of as , while the otherthree subbands are . To the priority of path, we use theratio of remaining energy to transmit power to denotethe expected lifetime of the node. So the higher the ratio is themore lifetimes of the node. In order to maximize the lifetime of the whole network system, to an arbitrary node , the criterionof its next hop should satisfy that(2)where denotes the maximum expected lifetime for nodecorresponding to the optimal energy transmission to next hop  IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 2, JUNE 2008 315 which locateson thedesired path. Assuming that thediscoveredpath consists of nodes such as ,, the energy consumer function of path , here we define(3)This means that the minimum expected lifetime of each nodedetermines the whole lifetime of the path. Because once one of the nodes in the path dies, thewhole path dies as well. The morethehigherpriorityofthepath,andcorrespondtohigherlevelofencodedsubstream.Notethat,inordertorealizetheproposedUEP,wemodifythetraditionalDSR(DynamicSourceRouting)Protocol as follows: besides adding the node’s ID to the requestpacket, each node also adds the information of transmit powerand remaining energy to the request packet, if the node receivespacket, the information of the received power is also added tothepacket.Sowhenthesendernodereceivestherequestpacket,the packet should consist of the route nodes’ ID, the remainingenergy of each route node and each node’s transmit power andreceived power.  B. Flexible MDC  We focus on how to design the MDC according to the net-working condition: one is how many descriptions are needed toguarantee the reconstructed video quality as well as keep thetotal bitstream as little as possible, while the other one is how todistribute these encoded data to the determinate substreams.Obviously, the more substream the more data received at thedestination,butitisinfeasibleforthepracticalwirelessnetwork.Here, we give an oversimplified method to compute the min-imum needed substream number according to the packet lossrate of the obtained channels.With regard to the channel model, we use a two-state Markovmodel (i.e. Gilbert model) to simulate the bursty packet lossbehavior [18]. The two states of this model are denoted as G(good) and B (bad). In state G, packets are received correctlyand timely, whereas, in state B, packets are assumed to be lost.This model can be described by the transition probabilitiesfrom state G to B and from state B to G. Then the averageis given by(4)And the average length of burst errors is given by(5)So the average channel of path is . Suppose thatall the transmit probability of potential paths are independent,the minimum number of substreams is(6)where is the tolerance threshold which depends onthe practical application requirements. Usually, its value variesfrom [22], in this paper, we set .In the case of data distribution, it also contains two aspects:one is the decision of the wavelet decomposition level, and theotheroneisthedatadistributionamongthesedeterminatepaths.As to this point, we employ three basic principles:• Highprioritylevelofthesourcedatacorrespondingtohighpriority of the path, which is called equity principle.• The whole transmission system only guarantees the mostimportant source data, which has the highest priority level.• As to other parts of the data, we use the best-effort strategyto transmit.If a -level dyadic wavelet decomposition is used, the numberof wavelet coefficients in the level spatial-frequency subband is given by(7)where and represent the frame width and height respec-tively. The level of the wavelet decomposition is decided by theexpected lifetime of the highest priority path . The essentialrequirement is that the GOF of the most important data shouldbe guaranteed to transmit from source to destination over thehighest priority path, so(8)where denotes the ratio of the most important partin the total data streaming; is the frame number of oneGOF; is the total source coding rate in bytes/s; is the framerate in frames/s; is the packet size in bytes.The fundamental distribution rule of data distribution is thatthe highest priority level data should be distributed at the eachpathwhiletheothersonlydistributedonceatthepath.Assuming, and expected life of path is , soeach path should satisfy that(9)If there is the which not transmitted bythispath,itistransmittedbytheotherpotentialpathswithlowerpriority level.IV. R OBUST  S UPER -R ESOLUTION  A LGORITHM Although adaptive error-resilient video transmission canreduce the transmission distortion, it can not exterminate thepacket loss which has a devastating effect on visual quality.In this section, we propose a robust SR algorithm taking intoconsideration the various packet loss scenarios to enhance theresolution of received image. At first, we propose a simplifiedestimator to estimate the lost wavelet coefficients. And then, aseries of convex sets which extract the exact detail informationhidden among the adjacent images are constructed by takingadvantage of the correlation of the wavelet coefficients.  A. Simplified Estimator  Motivated by the model of [19], here we propose a simplifiedestimator to estimate the lost coefficients. Since LL subbandprovide basic information (low frequency) for srcinal image,missing samples in approximation subband LL are estimated
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks