Music & Video

A vision-based three-dimensional capture system for maxillofacial assessment and surgical planning

A vision-based three-dimensional capture system for maxillofacial assessment and surgical planning
of 5
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  A vision-based three-dimensional capture system for maxillofacial assessment and surgical planning A. F. Ayoub*, P. Siebert’, K. F. Moos:, D. WrayQ, C. Urquhart+, T. B. Niblett’ ‘Muxillojbcial Unit nt Cunnieshurn Hospitul und The Univemitl~ oj’ Glus,go:oll~; The Turing Institute, iThe West qj’ Scotlund Regional Plustic. and Muxillqfilciul Unit: ~Glu.~gow Dental Hospital und School, The Univecsity c~f Glmgon; Glus,p~; SrotlundB UK SUMMARY. We describe a vision-based three-dimensional facial data capture system designed for the planning of maxillofacial operations. We describe the system requirements and outline the methods used to develop a complete three-dimensional facial capture system. Our approach is based upon imaging the face using two stereo-pair sets of cameras. Scale-space-based stereo-matching is then used to recover correspondences between each of the captured stereo-pairs. Photogrammetric routines based on adjustment of bundles are used off-line to calibrate the system by imaging a single object that references all cameras to the same co-ordinate frame. This calibration scheme allows us to convert stereo correspondences to world points for each pair of cameras without the need for any subsequent fusion of data. Initial results show that we are able to capture key facial landmarks to within 0.5 mm. INTRODUCTION For many years there has been a need for a tool that enables the maxillofacial surgeon to plan ortho- gnathic operations in detail. The human face is three- dimensional, so the clinicians must have accurate three-dimensional information about the maxillo- facial region to plan their operations effectively. Without this, the surgeons cannot precisely estimate the outcome of a particular procedure. Patients may therefore have to undergo further operations before the desired result is achieved. The most commonly used method is to cut up pro- file photographs magnified to the same size as the standardized lateral skull radiograph (natural head size). These are superimposed over the cephalo- graphs.‘J The various portions of soft tissues and underlying bone are moved around to produce the most acceptable result, guided by the known ratios of soft tissue movements to the surgical changes of the underlying bones (Figs 1 2). Unfortunately, radio- graphic and photographic registration and super- imposition are approximate because of the distortion inherent in the photograph ~ the imaging geometry of the camera that took the photograph and the X-ray machine that took the radiograph are different. The radiographic photographic superimposition is done manually using the soft tissue profile and is subject to human error. Recently, various computer packages have become available that have partially replaced the manual method of simulating orthognathic and maxillofacial operations.3.4 A video camera is used to capture the facial profile. Skeletal and dental landmarks are digi- tized from the lateral cephalograph and superimposed on the facial image. Having achieved a bone-face registration, the surgeon can then analyse the face and plan the operation. Sarver et ul.? described a video-imaging system that captured facial profiles and then manipulated them for planning and counselling before orthognathic surgery. They described the movement of operator-defined rectangular ‘windows’ and artistic reconstruction using conventional ‘cut and paste’ software techniques.? They reported that image distortion as a result of magnification was a major problem. On the radiographic side, the first step towards computerization of cephalometric data was their digitization.’ Attempts were made to generate images of patients’ profiles and cephalometric radio- graphs using various camera and computer configu- rations.h x The main disadvantages were the poor resolution, and the fact that the landmarks identified on the soft tissue profile did not correspond with those identified from radiographic examination. In addition, analysis and prediction of postoperative appearance were limited to two dimensions. The main shortcomings of both the traditional and computer-based two-dimensional techniques cur- rently used are the distortion of the perspective in the facial image and the lack of precision in the super- imposition of the photograph and cephalograph. Distortion of the perspective facial image results from misregistration of the facial photograph on the lateral cephalograph. This is exacerbated by the change in alignment of the head in the interval between taking the lateral cephalograph and facial photograph. In addition, the assessment of soft tissues is inaccurate because it is impossible to show in the soft tissue changes that occur after movement of the hard tissue (bone and teeth) in three-dimensions using only a profile photograph of the full face. The key to archiving a full three-dimensional simu- lation of soft tissue displacement hinges on being able to capture the three-dimensional geometry of the soft tissue air boundary ~~ hat is, the three-dimensional shape of the face itself.  354 British Journal of Oral and Maxillofacial Surgery Fig. 1 ~ Two-dimensional facial photograph superimposed on the lateral cephalograph. Rabey used morphoanalysis to try and reconstruct the facial form three-dimensionally, by superimposing radiographic on photographic material, but the method did not gain wide acceptance.9 Moire fringe techniques have been used to assess the changes on smoothly curving surfaces in three dimensions. The surface of the face and a reference plane are illumi- nated at an angle with a straight line grid. Moire fringes are formed between the distorted facial lines and the straight lines on the reference plane. The technique is, however, unsuitable for a complicated structure such as the face.‘O Previous approaches to obtaining three-dimen- sional facial data have included computerized tomog- raphy (CT) and magnetic resonance imaging (MRI).” However, the resolution of facial structures is limited by the separation of the CT scans. In addition, soft tissue resolution is poor. Both MRI and CT can also produce distorted facial reconstruction because arte- facts are generated by metallic objects such as fillings in teeth. Finally, neither CT nor MRI provide the nat- ural photographic appearance of the texture of the facial surface. There was therefore a need for a new method of three-dimensional collection of facial data to support the planning of maxillofacial surgery. Laser scanning is a non-invasive method of captur- ing the face in three dimensions.12,13 The system consists of two vertically fanned-out, low-power helium-neon laser beams, which are projected on to the face and viewed from an oblique angle by a television camera. To scan the whole face, the patient is seated on a chair that is rotated by a stepper motor under computer con- trol. A computer system has been developed for the simulation of facial surgery with interactive three- dimensional graphic techniques. Data were derived Fig. 2 - Planning orthognathic surgery and prediction of soft tissue changes as a result of orthognathic surgery. from CT scans and a purpose-built laser scanning sys- tem. The technique was useful in recording soft tissue changes after orthognathic surgery.‘ ” Facial changes were analysed using colour-coded surface mapping. There is no doubt that the laser technique is a simple, non-invasive method for measuring shapes in three dimensions. However, its disadvantage is the slow cap- ture of the patient’s face. It scans the face in 13-15 s and any overall changes in the patient’s head during scanning or any alteration in facial configurations during scanning will distort the captured image. The patient’s eyes should be closed during scanning for protection, and this may bring into question the iden- tity of the captured image. In addition, the surface tex- ture of the soft tissue is not captured by this technique. A method of three-dimensional recording of the soft tissues of the face was first recorded by Zeller as an example of short-base stereophotogrammetry.18 A stereo-pair of facial photographs were recorded in a stereometric camera and placed in a plotting machine to make contour maps of facial morphology. This method produced an accurate three-dimensional record of the face, but the mapping equipment was large and expensive. In view of its unsuitability for clinical use, an effort was made to simplify the technique without loss of accuracy.19m2’ Nevertheless, the technique has not been widely used to assess acial morphology due to the complexity of facial mapping and the inaccuracy inherent in the technique, as a result of the interval between contours. Moreover, analysis of facial measurements is labour intensive and they do not lend themselves easily to developing automatic methods for producing a surface-textured three-dimensional facial model for assessment and planning.  Vision-based 3D capture system 355 Fig. 3 Two video cameras and a central light source are placed at each side of the patient and the face is located within the cephalostat of an X-ray machine. NEW TECHNIQUE We have based our approach on the use of an existing non-contact videometric system, developed as a result of on-going collaboration between Glasgow Dental Hospital, and The Turing Institute (the centre for computing sciences), that does not have the limita- tions of previous approaches to collection of three- dimensional facial data. Based on recent advances in stereophotogrammetry, a three-dimensional (C3D) software was developed which processes stereo-pairs of images to produce metrically accurate 3D com- puter graphics models. Stereo-pair images are captured from two camera stations (each comprising a stereo pair of cameras), one on each side of the face, com- bined with integral illumination (Fig. 3). A computer- controlled slide projector illuminates the subject with either a texture pattern to facilitate stereo matching, or with plain light that is used when capturing the nat- ural appearance of the face. C3D software matches the images captured by the station to recover triangu- lated distances to each surface point imaged by the pair of cameras. Off-line calibration recovers both the internal and external camera measurements and enables stereo disparities to be converted into X, Y, Z world points (defined by the co-ordinate system of a calibration object) to be recovered by automatic stereo matching. Camera configuration planning is critical to achieve the specified working volume accuracy and resolution performance requirements. Because of the overall curved shape of the human face, it was quite clear that a single stereo pair of cameras would not provide adequate stereo coverage to achieve complete three-dimensional reconstruction of the area of anatomical interest. Two camera stations have been used to image each half of the face while maintaining a central region of overlap. The working volume of each half-face view corresponds to 276 mm x 2 13 mm x 150 mm in height, width, and depth, respectively. Tools for camera configuration analysis developed by Urquhart12 were used to search the space of camera baselines, mean working distances and commercially available focal lengths, to achieve an acceptable design solution. The solution chosen comprises: standard video cameras of 756 x 576 usable pixel resolution, lenses of 50 mm focal length and a mean working distance of 1.75 m, a camera base-lint separation of 300 mm and an inter-camera station separation of 1.75 m. C3D stereo matching provides both X and Y disparities, sub-pixel matching accuracy, as described, and will match a pair of images in under 5 min on a personal computer. This algorithm operates satisfac- torily clinically in a conventional X-ray room, using projected texture illumination under conditions of  356 British Journal of Oral and Maxillofacial Surgery Fig. 4A & 4B - Photorealistic rendering of the three-dimensional facial model generated by the twin video camera system. reduced ambient illumination, in accordance with common radiological practice (Fig. 3). After the surface of the patient’s face has been captured, modern three-dimensional computer graphics are used to manipulate the facial three- dimensional geometry. High-quality visualization of the imaged face is provided from any desired view- point in a process called photorealistic rendering (Figs 4A B). In addition, the three-dimensional polygonized face model (Fig. 5) can be exported in Fig. 5 - Three-dimensional polygonized facial model for facial measurements. various formats to be used in volumetric facial measurements. A user interface designed to support interactive measurements of facial landmarks has also been developed. The system has now been tried to capture various objects, volunteers, and a dummy head. Initial results from these trials indicate that we are able to capture the relative locations of key facial landmarks on the face to within an accuracy of 0.5 mm. A further test to establish the accuracy of the system has been made by imaging a flat aluminium plate that has been painted matte grey to suppress illumination high- lights. The recovered three-dimensional surface was then fitted to a flat plane and the root mean square (RMS) error was computed. Results for this trial yielded errors lower than 0.2 mm RMS for a range of plate angles from roughly -15” to +50” from the cam- era station. Clearly, imaging a planar surface is the ideal, and additional experiments are required to eval- uate the system’s performance when imaging curved surfaces that are more typical of human faces. DISCUSSION Any proposed method for three-dimensional capture for planning must satisfy the following basic require- ments: the full face must be imaged from the hairline to the hyoid bone, vertically, and from the back of the left ear lobe to the back of the right ear lobe horizontally. This comprises a maximum working volume of 280 mm x 180 mm x 165 mm (height x width x depth, respectively) to accommodate a wide variety of patients. A relative accuracy of 0.5 mm RMS is required within the specified working volume. In addi- tion to capturing the three-dimensional topography of the face, a two-dimensional image of the corresponding  Vision-based 3D capture system 357 natural surface texture of the face must also be cap- tured within this region. Obtaining the natural photo- graphic appearance of the face is crucial, as it shows the landmarks from which measurements can be recorded by the clinician. Another requirement is that the system be able to acquire both three- and two- dimensional facial data within 2 s to allow the system to be used with children and to avoid any possible changes in facial configurations during capture. Finally, any proposed technique should be non- invasive and cost-effective. Our presented method of stereophotogrammetry fulfils these requirements. Initial trials of the system indicate that we have achieved the target spatial accuracy of 0.5 mm and further trials to verify these results are ongoing. Our current work centres on further developing the three-dimensional visualization and manipulation capabilities of the system in anticipation of developing surgical planning programs based on our abilities to collect three-dimensional facial data. The method has also been used to capture the dental casts and store them in digital format.‘? It has been also used in foren- sic medicine and facial identification, details of which are beyond the scope of this paper. It will also be used to assess surgical changes and stability after repair of facial clefts. Acknowledgements The project was supported by the University of Glasgow and Scottish Higher Education Funding Council. We would like to thank Mr C. Richardson, Chief Photographer, Canniesburn Hospital for his help in our preliminary trials. and the School 01 Manufacturing and Mechanical Engineering at Birmingham University, who measured our calibration object. References 1. 2. 1 4. 5. 6. 7. 8. 9. IO. I I. Henderson D. The assessment and management of bony deformities of the middle and lower face. Br J Plast Surg 1974: 27: 287-296. Fanibunda K. Photoradiography of facial structures. Br J Oral Surg 1983; 21: 246-258. Sarver DM, Johnston MW, Matukas VJ. Video imaging for planning and counselling in orthognathic surgery. J Oral Maxillofac Surg 1988: 46: 939 945. Lowey MN. The development of a new method 01 cephalometric and study cast mensuration with a computer controlled video image capture system. I. Br J Orthod 1993: 20: 203-2 14. Houston WJB. Automated measurements of photographs and radiographs. Trans Br Sot Study Orthod 1970; 5: 1996-1970. DuiT MJB. Review of the CLIP image processing system. Processing of National Computer Conference. Montvale. NJ: AFIPS Press. 1978: 1055 1060. Cohen AM, Ip HHS. Linney AD. A preliminary study 01 computer recognition and identification of skeletal landmarks as a new method of cephalometric analysis. Br J Orthod 1984: II: I43 154. Chaconas SJ. Engel GA. Gianelly AA et cl/. The digraph work station. I. Basic concepts. J Clin Orthod 1990; 24: 360-367. Rabey Cl? Current principles of morphoanalysis and their applications in oral surgical practice. Br J Oral Surg 1978: 15: 97~mlo9. Takasaki H. Moire topography. Appl Opt 1978: 9: 1467-1472. Hoenhen KH. Hanson WA. Interactive 3D segmentation of MRI and CT volumes using morphological operations. J Comput Assist Tomogr 1992: 16: 285-294. 12. 13. 14. 15. 16. 17. IX. 19. 20. 21. 22. 23. Moos J, Linney A, Grindord S, Arridge S, Clifton J. Three dimensional visualisation of the face and skull using computerised tomography and laser scanning techniques. Eur J Orthod 1987; 9: 247-253. McCance AM, Moss JP. Wright WR, Linney AD, James DR. A three dimensional soft tissue analysis of I6 skeletal class III patients following bimaxillary surgery. Br J Oral Maxillofac Surg 1992; 30: 221-232. McCance AM. Moss JP, Fright WR. Linneyt AD, James DR. Three dimensional analysis technique: three dimensional soft- tissue analysis of 24 adult cleft patients following Le Fort I maxillary advancement: a preliminary report. Cleft Palate Craniofac J 1997; 34: 27-45. McCance AM. Moss JP. Fright WR, Linneyt AD, James DR. Three dimensional analysis techniques. 2. Laser scanning: a quantitative three-dimensional soft tissue analysis using a colour coding system. Cleft Palate Craniofac J 1997; 34: 46-S I McCance AM. Moss JI? Fright WR. Linneyt AD, James DR. Three dimensional analysis techniques. 3. Color-coded system for three dimensional measurement of bone and ratio of soft tissue to bone: the analysis. Cleft Palate Craniofac J 1997: 34: 52-57. McCance AM, Moss JP. Fright WR. Linneyt AD, James DR. Three dimensional analysis techniques. 4. Three dimensional analysis of bone and soft tissue to bone of movements in 24 cleft patients following Le Fort I osteotomy: a preliminary reuort. Cleft Palate Craniofac J 1997: 34: 58 62. Zeller M. Textbook of stereophotogrdmmetry. Zurich: L Miskin, 1952. Burke PH. Beard L. Stereophotogrammetry of the face. Am J Orthod 1967; 53: 769-782. Burke PH. Stcreophotogrammetric measurement of normal facial asymmetry in children. Human Biol 1971; 43: 536-548. Burke PH. Serial stereophotogrammetric measurements of the soft tissues of the face. Br Dent J 1983: 155: 373- 379. Urquhart CW The active stereo probe: the desibm and implementation of an active videometrics system: 1997. University of Glasgow: Department of Computing Science, PhD Thesis. Ayoub AF, Wray D, Moos KF et al. A three dimensional imaging system for archiving dental study model: a preliminary report. Int J Adult Orthod Orthognath Surg 1997; 12: 79 -84. The Authors A. F. Ayoub PhD, MDS, FDS RCS Lecturer Maxillofacial Unit at Canniesburn Hospital and The University of Glasgow Glasgow. P. Siebert, PhD, BSc, MIEE Consultant in Computing Science The Turing Institute K. F. Moos MBBS, BDS, FDS RCS, FRCS Consultant in Oral and Maxillofacial Surgery The West of Scotland Regional Plastic and Maxillofacial Unit, Canniesburn Hospital D. Wray MD, FDS RCS Associate Dean for Research Glasgow Dental Hospital and School University of Glasgow C. Urquhart PhD, MEng, AMIEE. T. B. Niblett PhD Consultants in Computing Science The Turing Institute Glasgow L1 K Correspondence and requests for offprints to: Dr Ashraf F. .4youb Department of Oral and Maxillofacial Surgery, Glasgow Dental Hospital and School. 378 Sauchiehall Street, Glasgow G2 352. LJK Paper received 8 December 1997 Accepted 8 April I998
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks