Press Releases

Mobility change-of-state detection using a smartphone-based approach

Understanding the mobility of people with physical disabilities is important for rehabilitation decision making. This paper presents a smartphone-based approach to mobility monitoring. The BlackBerry-based system is clipped to the person's belt.
of 9
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011 3153 Wearable Mobility Monitoring Using aMultimedia Smartphone Platform Gaëtanne Haché,  Member, IEEE  , Edward D. Lemaire,  Member, IEEE  , and Natalie Baddour  Abstract —Understanding mobility is important for effectiveclinical decision making in the area of physical rehabilitation.Ideally, a person’s mobility profile in a nonclinical setting, suchas the home or community, could be obtained. This profile wouldinclude the environment and context in which the mobility takesplace. This paper introduces a novel wearable mobility monitor-ing system (WMMS) for an objective ubiquitous measurementof mobility. This prototype WMMS was created using a smart-phone-based approach that allowed for an all-in-one WMMS.The wearable system is freely worn on a person’s belt, such asa normal phone. The WMMS was designed to monitor a user’smobilitystateandtotakeaphotographwhenachangeofstatewasdetected. These photographs were used to identify the context of mobilityevents(i.e.,usinganelevator,walkingup/downstairs,andtype of walking surface). Validation of the proposed WMMS wasperformed with five able-bodied subjects performing a structuredsequence of mobility tasks. System performance was evaluated byits ability to detect changes of state and the ability to identifycontext from the photographs. The WMMS demonstrated goodpotential for community mobility monitoring.  Index Terms —Acceleration, cameras, mobility, monitoring,multimedia systems, wearable. I. I NTRODUCTION M OBILITY can be defined as the ability to move in-dependently from one point to another [1] and is es-sential for maintaining independence. Mobility is required toperform many activities of daily life,such as,cooking, dressing,shopping, and visiting friends. According to Statistics Canada,mobility problems are one of the issues that affect the greatestnumber of adults [2]. Mobility disabilities can affect an individ-ual’s quality of life, health, productivity, and independence andalso affect the lives of their family and the people around them. Manuscript received September 5, 2010; revised December 15, 2010;accepted December 16, 2010. Date of publication March 28, 2011; date of current version August 10, 2011. This work was supported in part by theResearch In Motion, by The Ontario Graduate Scholarships in Science andTechnology Program, and by the Ontario Centers of Excellence. The AssociateEditor coordinating the review process for this paper was Dr. Salvatore Baglio.G. Haché is with the Ottawa-Carleton Institute for Biomedical Engineer-ing, Department of Mechanical Engineering, University of Ottawa, Ottawa,ON K1N 6N5, Canada (e-mail: D. Lemaire is with the Ottawa-Carleton Institute for Biomedical Engi-neering, Department of Mechanical Engineering, University of Ottawa, Ottawa,ON K1N 6N5, Canada, and also with the Institute for Rehabilitation Re-search and Development, The Ottawa Hospital Rehabilitation Centre, Ottawa,ON K1H 8M2, Canada.N. Baddour is with the Department of Mechanical Engineering, Universityof Ottawa, Ottawa, ON K1N 6N5, Canada.Color versions of one or more of the figures in this paper are available onlineat Object Identifier 10.1109/TIM.2011.2122490 Preserving mobility is paramount for staying independent andactive at home and in the community.Accurate and objective mobility assessment is required fordecision making in rehabilitation medicine. Such assessmentscan be used to determine mobility issues outside a hospi-tal environment, evaluate the progress made during and afterrehabilitation, and enhance clinical decision making about arehabilitation program (i.e., assistive devices, exercises, treat-ment, etc.). Currently, many types of mobility assessments areperformed in a clinical setting and are supervised by the re-habilitation physician. These assessments include clinical tests,quantitative measures, and subjective feedback from client topatient. Although clinical mobility tests have a value, suchassessment tools may not be appropriate for determining thecontributing factors for independent community walking andthe impact of the environment on the individual’s mobility [3],[4]. Monitoring the mobility outside a clinical setting is im-portant because mobility in the real world is typically differentfrom the mobility measured in the clinic [5].Wearable mobility monitoring systems (WMMSs) are de-signed to be worn on the body and allow mobility monitoringin the person’s home and the community [6]. Many wear-able mobility monitoring studies measure biomechanical and/orlocation parameters [5], [7]–[10], but most lack environmentalor contextual information. In community mobility monitoring,contextual information is important since it provides insight onwhere, how, and on what a person is moving. A camera couldprovide contextual information from a person’s surroundingenvironment.Somewearablesystemsthatusecontextualinformation,suchas context-aware systems [11] and life logs [12], are not meantfor community mobility monitoring for people with physicaldisabilities. Other context-aware wearable systems use contextinformation to better recognize activities [13]–[15], but theenvironmental characteristics in which activities take place arenot analyzed for their impact on mobility.We propose a novel WMMS that provides unsupervisedobjective mobility measurements in a cost-effective way, usingsmartphone technology that has already achieved consumeracceptance. In addition to monitoring biomechanical parame-ters, our WMMS also aims to identify mobility tasks andtheir context. This paper uses the smartphone as the centralprocessing hub for data capture, data processing, multimediacapture, outcome storage, and the option for wireless outcomedata transmission. The novelty of this approach is the combina-tion of biomechanical task identification methods and contextidentification via mobile multimedia tools. 0018-9456/$26.00 © 2011 IEEE  3154 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011 This WMMS is the first to combine change-of-state detec-tion, sensor-based activity classification, and environmental-image capture into an integrated system designed to improvethe contextual information available for mobility assessment.Photographs provide contextual information that cannot besupplied by inertial sensor systems, such as using an elevator,walking on carpet or grass, or walking in a crowded room. TheWMMS provides information on the context and environmentin which mobility events take place, which will help identifymobility challenges in a person’s own environment. For thispaper, changes of state include starting or stopping an activ-ity, postural changes, walking on stairs, indoor and outdoortransitions, and using transportation. This paper describes theWMMS design from a system perspective, including hardwareintegration, data processing methods, mobility assessment, andcontext identification outcomes.II. M ETHODOLOGY  A. WMMS Prototype Development  The BlackBerry smartphone platform was chosen for theWMMS due to capability, acceptance in the health-care sector,and device/platform security. In our preliminary study, theBlackBerry platform demonstrated the multitasking, commu-nication, and processing capabilities required for a WMMShub [16].BlackBerry smartphones possessthe required securityfeatures, built-in global positioning system (GPS), integratedcamera, video recording, Wi-Fi, Bluetooth, data encryp-tion, adequate processing speed, and large storage capacity.Furthermore, newer models provide access to accelerometerraw data that could enable the design of an all-in-one WMMS.A mature Java environment and many secure developmentinterfaces [i.e., application programming interference (API)]are also available with the BlackBerry devices. Based on thelatest available BlackBerry Java development environment andAPI at the time of project inception, BlackBerry Bold 9000was used in this study.While recently released phones have integrated accelerom-eters and the potential to test ambient light, the BlackberryBold 9000 did not possess these features; therefore, an externalboard with mobility analysis sensors was designed. The ex-ternal sensors were required because BlackBerry smartphoneswith all the required capabilities were not available during theproject development phase (i.e., accelerometer, GPS, Wi-Fi,Bluetooth, and camera). The external board design, integratedinto the phone’s holster (i.e., Smart Holster), provided a flex-ible approach to add other measurement sensors or tools inthe future. The board was connected to the BlackBerry viaBluetooth.As shown in Fig. 1, a microcontroller CY8C27443(Cypress Semiconductor Corporation, San Jose, CA, USA),Bluetooth Module F2M03GLA (Free2Move AB, Halmstad,Sweden), triaxial accelerometer LIS344alh (STMicroelec-tronics, Geneva, Switzerland), and light sensor APDS-9005(Avago Technologies Limited, San Jose, CA, USA) weremounted on the board. A rechargeable lithium battery pow-ered the board. This external board could continuously run Fig. 1. Sensors and board for attachment to the holster. for approximately 14 h on one charge. However, the lengthof time that the WMMS could run without recharging wasapproximately 3 h with the BlackBerry’s Li-Ion 1500 mAhbattery. The sensors were not put to sleep if no activity wasdetected.To start and stop sensor data sampling, sampling delaycommands were sent from the BlackBerry to the board micro-controller. Bluetooth serial port profile (SPP) protocol was usedfor communication with the external board. The data fromthe accelerometer and the light sensor were stored in a bufferon the microcontroller. At every sampling delay, the last datapacket stored in the buffer was sent to the host (BlackBerry)via SPP. A data packet (21 B) was sent to the host (Black-Berry) every 20 ms (50 Hz). In the data packet, 6 B wererequired for raw acceleration data and 2 B for light sensor data.The data packet-sampling rate equaled the data-sampling rate;therefore, the sampling rate for the accelerometer and lightsensor was 50 Hz.All signal processing and the state detection algorithms wereimplemented and performed on the BlackBerry. These algo-rithms are subsequently discussed. Following signal process-ing, the outcome data for each record were stored on theBlackBerry’s 8-Gb secure-digital card. The WMMS softwarewas developed using the BlackBerry Java Development Envi-ronment version 4.6.1.  B. Signal Processing The accelerometer and light sensor sampling rate was 50 Hzsince body-fixed accelerometers placed at the waist must beable to measure acceleration with frequencies up to 20 Hz[17]. The accelerometer was calibrated to remove the direct-current offset [17]. The calibration method involved rotatingthe sensor to known angles, as suggested in the manufacturerdatasheet [18]. For example, under static conditions, the outputfrom an axis pointed toward the center of the earth should beequal to 1 g. If the axis is then rotated by 180 ◦ , its outputshould be equal to − 1 g. The sensitivity  s  and the offset  o  of a  HACHÉ  et al. : WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM 3155 particular axis of the sensor can be calculated using the follow-ing equations: s  = ( u max − u min )2  (1) o  = ( u max  +  u min )2  (2)where  u max  and  u min  are the maximum and minimum acceler-ation measured during the rotation between ± 1 g. The output  a of one accelerometer can then be expressed as a  = ( u − o ) s  (3)where  u  is the uncalibrated acceleration.On the Blackberry, the calibrated acceleration data werepassed through a median filter  ( n  = 3)  to remove spikes [7].The signal was then passed through a low-pass digital filter(0.25 Hz) to separate the static component from the dynamiccomponent [19]. A nonoverlapping sliding window of 1.02 s(51 samples) [7], [20] was used to extract signal features fromthestaticanddynamicaccelerationcomponentsandtheaveragelight sensor output. These extracted features were used as inputto calculate the parameters that determined the user’s mobilitystate. C. Mobility Features Various mobility and activity classification variables wereselected from the literature to help determine mobility changesof state. One of the largest challenges in the development of the WMMS is the integration of individual components into acoherent and functional system that meets the overall systemobjectives. Therefore, the decision was made to focus on thesystem-level development and to incorporate previously provenmobility classification variables. Preliminary evaluations, overall activities, with two able-bodied subjects were used to ver-ify these measures and to determine the threshold values foractivity classification. 1) Inclination Angle:  Inclination angle was used to helpclassify posture [9], [21]–[23] and identify postural transition[24]. The inclination angle was calculated for every windowperiod using the two-axis method presented by Freescale Semi-conductor [25]. The static components of the acceleration,obtained from low-pass filtering (0.25 Hz), were used to calcu-late the inclination angle  Φ  using the two-argument arctangentfunction, i.e., Φ = arctan  GAzGAy  ( ◦ )  (4)where GAz and GAy are the averaged static accelerations of the z -axis (forward) and  y -axis (vertical), respectively. An offset of 180 ◦ was then added to the inclination angle to give a range of 0to360 ◦ .Ideally,aninclinationvalueof90 ◦ shouldbeobtainedwhen the person lays on their belly, a value of 180 ◦ whenthe person stands, and a value of 270 ◦ when the person layson their back. The averaged inclination angle was comparedwith high and low thresholds (200 ◦ and 160 ◦ ) to determineif the person was in a standing position. If the person wasnot standing, the angle was compared with different high andlow thresholds (320 ◦ and 250 ◦ ) to determine if the person waslying on their back. If both conditions were false, the positionwas determined to be somewhere in between. The thresholdvalues were based on the study by Culhane  et al.  [23] and ourpreliminary observations. 2) Standard Deviation of Vertical Acceleration:  Standarddeviation is a well-supported measure for activity classifica-tion [17], [21]–[23], [26]. Since most daily activities can beclassified by changes in vertical axis acceleration, vertical ac-celeration ( y -axis) was used to differentiate between static anddynamic states by comparing the standard deviation of the  y -axis acceleration with two thresholds (static and dynamic). Thisalgorithm was defined as the double threshold (DT) algorithm.With a DT algorithm, if the initial state is static, the activityclassification remains static until the signal crosses the dynamicthreshold (0.120 g). Then, the state is set to dynamic andstays dynamic until the signal passes below the static threshold(0.075 g). When the person stands still, the standard deviationshould be close to 0 g. 3) Skewness of Vertical Acceleration:  The skewness valueof the vertical acceleration is a time-domain feature that wasused by Baek  et al.  [26] to differentiate walking/running fromgoing up/down stairs. Skewness is a measure of asymmetry of the  y -axis (vertical) acceleration about the average acceleration(e.g., a skewness value of 0 indicates a symmetric distributionof accelerations). Skewness of the  y -axis acceleration wascalculated asskewness  =  n ( n − 1)( n − 2) n  i =1  x i − xσ  3 (5)where  n  is the number of points,  x i  the  y -axis accelerationat point  i , and  σ  and  x  are the standard deviation and themean of the  y -axis acceleration signal, respectively. The DTalgorithm was also applied to the skewness value but onlywhen in a dynamic classification state. Based on preliminaryworks, skewness values greater than 1 were observed for stairdescent [16], [26]. Skewness increased when ascending stairs,but values were less than stair descent. Similar skewness valueswere sometimes observed for both stair ascent and normalwalking, which could result in a false positive change-of-statedetection. High (0.6) and low (0.2) thresholds were chosen todetect stair descent and detect stairs ascent with minimal falsepositive results. 4) SMA of Three-Axis Acceleration:  Signal magnitude area(SMA) is another viable activity and mobility measure [7], [9].The SMA normalized to the length of the signal  T   can becalculated fromSMA  = 1 T   T    t =0 | a x | dt  + T    t =0 | a y | dt  + T    t =0 | a z | dt   (6)where  t  is the time in seconds and  a x ,  a y , and  a z  are theacceleration of   x -,  y -, and  z -axes, respectively. During pre-liminary testing, peaks occurred in the SMA signal for sit-ting, rising from a chair, and lying down. The SMA also  3156 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011 Fig. 2. User’s state detection algorithm. helps identify activity intensity changes that could indicate achange of state. Therefore, three thresholds were used, andthree states were determined: no peak with normal intensity,no peak with increased intensity, or a peak. A DT algorithmwas used to determine increases in intensity and peak detection.The “increased intensity” low and high thresholds were 0.100and 0.190 g, respectively. Since an increase in intensity shouldhappen when the person is moving, the algorithm verified thatthe person was in a dynamic state when detecting the “no peakwith increase in intensity” state. The peak detection low andhigh thresholds were 0.100 and 0.320 g, respectively. 5) Light Intensity:  The light sensor measured light intensityof the ambient environment. To detect indoor and outdoor statesduring the day, the DT algorithm was applied to the lightintensity feature. Based on preliminary light sensor calibration,a high threshold of 1000 mV and a low threshold of 300 mVwould differentiate outdoor from indoor states during the day.However, during preliminary testing while driving, many falsechangesofstatewererecordedduetothelightintensitychangesin the car. To remove those false changes of state, the DTalgorithm was only applied to the light intensity feature whennot moving in a vehicle. 6) GPS Speed:  GPS data have been used in mobility mon-itoring to complement motion data, improve activity recogni-tion, and provide contextual data [5], [8], [27]–[29]. Therefore,GPS location coordinates and speed were collected and addedto the WMMS output file when available (outdoors). The GPSdata were extracted from the BlackBerry Bold every 9 s. Forthis WMMS prototype, only the speed was considered forthe change-of-state detection algorithm, to detect whether theperson was in a vehicle. This feature was passed through a DTalgorithm, with the low and high thresholds of 1 and 7 m/s,respectively. When the speed was above the 7 m/s threshold,the state was identified as “in a vehicle.” The state stayed thesame until the GPS speed measured below the low threshold.  D. Determination of State and Change of State The user’s state was assessed for every data window (1.02 s)(see Fig. 2). A change of state was determined by subtracting  HACHÉ  et al. : WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM 3157 the three previous states from the current state. If the answerwas different from zero for one of the subtractions, a change of state occurred. As a result of a change of state, the algorithmdetermined if a picture should be taken. From our cameraperformance test, approximately 0.7 s was required to take apicture, and another 0.9 s was required before another picturecould be taken. Therefore, it was decided to wait at least twowindows before taking another picture (i.e., 3 s later). Pictureencoding was set to Joint Photographers Expert Group, i.e.,640  ×  480 pixels, and the quality was set to normal. Thememory size of a picture with this encoding was 10 to 70 KB.  E. Mobility Evaluation A convenience sample of five able-bodied subjects (threemales and two females; age: 36.6  ±  6.4 years old; height:173.82  ±  13.17 cm; weight: 69.32  ±  16.09 kg) was recruitedfrom the staff at The Ottawa Hospital Rehabilitation Center(TOHRC, Ottawa, Canada) and the community. Informed con-sent was obtained from all the participants. People with injuriesor gait deficits were excluded.Data collection took place within the TOHRC (hallways,elevator, stairs, and Rehabilitation Technology Laboratory),outside the TOHRC on a paved pathway, and in a car drivingaround the Ottawa Hospital campus.The subjects were asked to wear the WMMS on their waist,with the holster attached on a belt, on their right hip, with thedevice pointing forward. No additional instructions were givenfor positioning the instrumented holster. The subjects wereasked to follow a predetermined path with a series of mobilitytasks. For every trial, the subjects were filmed with a digitalvideo camera. Three trials per subject were performed. Thedigital camera was synchronized with the WMMS by havingthe subject block the light sensor with their hand for 5 s whenstarting data collection. A digital video was necessary to vali-date the change-of-state detection, to determine the change-of-state timing, and to provide context information for validation.The predefined mobility tasks were sequentially performed.Moving from one task to another should trigger a change of state, providing a total of 38 changes of state per trial. Thegiven sequence of tasks was as follows: 1) stand; 2) walk onlevel ground (25 m); 3) stand-to-sit transition; 4) sit; 5) sit-to-stand transition; 6) walk on level ground (60 m); 7) stand andwait for an elevator; 8) walk to get in the elevator; 9) take theelevator to the second floor; 10) walk out of the elevator andkeep walking on level ground (30 m); 11) stand and wait for theelevator; 12) walk to get in the elevator; 13) take the elevator tothe first floor; 14) walk out of the elevator and keep walking onlevel ground (50 m); 15) ascend stairs (13 steps); 16) walk onstair intermediate landing (level ground for 1.5 m); 17) ascendstairs (13 steps); 18) walk on level ground (30 m); 19) descendstairs (13 steps); 20) walk on stair intermediate landing (levelground for 1.5 m); 21) descend stairs (13 steps); 22) walk onlevel ground (20 m); 23) stand-to-lie transition; 24) lie on back;25) lie-to-stand transition; 26) walk on level ground (45 m);27) ascend and descend a 7 ◦ angled ramp (5 m); 28) walk onlevel ground (15 m); 29) transition indoor/outdoor and keepwalking outdoors on level ground (60 m); 30) transition out-door/indoor and keep walking indoors on level ground (40 m);31) transition indoor/outdoor and keep walking outdoors onlevel ground (30 m); 32) stand-to-sit transition to get in a car;33) sit in the car; 34) start car and ride around campus; 35) stopcar ride; 36) sit-to-stand transition; 37) walk on level ground(30 m); 38) transition outdoor/indoor and keep walking indoorson level ground; 39) and finally stand.Changes-of-state timing from the digital video were com-pared with the WMMS change-of-state timestamps. Each datawindow was analyzed to determine if the change of state was atrue or false positive, or true or false negative. The number of true and false positives and true and false negatives were usedto calculate WMMS sensitivity and specificity, i.e., Sensitivity = #TruePositives#TruePositives + #FalseNegatives  × 100 (7) Specificity = #TrueNegatives#TrueNegatives + #FalsePositives  × 100 . (8)Two research assistants independently evaluated the Black-Berry Bold images. The evaluators were asked to identify thecontext (i.e., stairs, elevator, ramp, floor, outdoor, etc.) fromthe digital images. Only the images taken due to a real changeof state (true positives) were evaluated. The evaluators weregiven a list of context options to choose from. The evaluatorswere not informed of the mobility tasks represented by theimages prior to evaluation. The results from the two evaluatorswere then analyzed to determine if the context was successfullyidentified from the pictures. Table II shows the various contextsthat were identified for each mobility task. For the context “typeof ground,” the evaluators also had to choose between floor,grass, and pavement.III. R ESULTS An overall sensitivity of 77.7% ± 2.5% and a specificity of 96.4% ± 2.2% were obtained across all the activities. Averagedsensitivity of the different change-of-state categories are givenin Table I. Context identification from the photographs had anoverallsuccessrateof72.5% ± 33%.TableIIshowsasummaryof the results divided by type of context.IV. D ISCUSSION Understanding mobility in a nonclinical setting is importantwhen making rehabilitation/health-care-related decisions. Ourresults suggest that smartphones, particularly the newer ver-sions to come, have great potential for community mobilitymonitoring. By using the integrated camera, information on thecontext/environment in which mobility events take place canbe identified. Additionally, the BlackBerry has the necessaryprocessing power to log and process data, run algorithms,collect GPS data, and take pictures, all without data loss.Our approach of taking a photograph when a change of state occurred demonstrated that mobility tasks such as taking
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks