SpringerOpen Newsletter

Receive periodic news and updates relating to SpringerOpen.

This article is part of the series Facial Image Processing.

Open Access Research Article

Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA

Jian-Gang Wang1*, Hui Kong2, Eric Sung2, Wei-Yun Yau1 and EamKhwang Teoh2

Author Affiliations

1 Institute for Infocomm Research, 21 Heng Mui Keng Terrace, 119613, Singapore

2 School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore

For all author emails, please log on.

EURASIP Journal on Image and Video Processing 2007, 2007:038205  doi:10.1155/2007/38205


The electronic version of this article is the complete one and can be found online at: http://jivp.eurasipjournals.com/content/2007/1/038205


Received:27 April 2006
Revisions received:22 October 2006
Accepted:18 June 2007
Published:28 August 2007

© 2007 Wang et al.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper presents a novel approach for face recognition based on the fusion of the appearance and depth information at the match score level. We apply passive stereoscopy instead of active range scanning as popularly used by others. We show that present-day passive stereoscopy, though less robust and accurate, does make positive contribution to face recognition. By combining the appearance and disparity in a linear fashion, we verified experimentally that the combined results are noticeably better than those for each individual modality. We also propose an original learning method, the bilateral two-dimensional linear discriminant analysis (B2DLDA), to extract facial features of the appearance and disparity images. We compare B2DLDA with some existing 2DLDA methods on both XM2VTS database and our database. The results show that the B2DLDA can achieve better results than others.

References

  1. W Zhao, R Chellappa, PJ Phillips, A Rosenfeld, Face recognition: a literature survey. ACM Computing Surveys 35(4), 399–458 (2003). Publisher Full Text OpenURL

  2. R Brunelli, D Falavigna, Person identification using multiple cues. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(10), 955–966 (1995). Publisher Full Text OpenURL

  3. K Chang, K Bowyer, P Flynn, Face recognition using 2D and 3D facial data. Proceedings of ACM Workshop on Multimodal User Authentication, December 2003, Santa Barbara, Calif, USA, 25–32

  4. KI Chang, KW Bowyer, PJ Flynn, An evaluation of multimodal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(4), 619–624 (2005). PubMed Abstract | Publisher Full Text OpenURL

  5. F Tsalakanidou, D Tzovaras, MG Strintzis, Use of depth and colour eigenfaces for face recognition. Pattern Recognition Letters 24(9-10), 1427–1435 (2003). Publisher Full Text OpenURL

  6. J-G Wang, H Kong, R Venkateswarlu, Improving face recognition performance by combining colour and depth fisherfaces. Proceedings of 6th Asian Conference on Computer Vision, January 2004, Jeju, Korea, 126–131

  7. J-G Wang, K-A Toh, R Venkateswarlu, Fusion of appearance and depth information for face recognition. Proceedings of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '05), July 2005, Rye Brook, NY, USA, 919–928

  8. KW Bowyer, K Chang, P Flynn, A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Computer Vision and Image Understanding 101(1), 1–15 (2006). Publisher Full Text OpenURL

  9. N Mavridis, F Tsalakanidou, D Pantazis, S Malassiotis, MG Strintzis, The HISCORE face recognition application: affordable desktop face recognition based on a novel 3D camera. Proceedings of International Conference on Augmented, Virtual Environments and Three Dimensional Imaging (ICAV3D '01), May-June 2001, Mykonos, Greece, 157–160

  10. C Beumier, M Acheroy, Automatic face authentication from 3D surface. Proceedings of British Machine Vision Conference (BMVC '98), September 1998, Southampton, UK, 449–458

  11. GG Gordon, Face recognition based on depth maps and surface curvature. Geometric Methods in Computer Vision, July 1991, San Diego, Calif, USA, Proceedings of SPIE 1570, 234–247

  12. X Lu, AK Jain, Deformation analysis for 3D face matching. Proceedings of the 7th IEEE Workshop on Applications of Computer Vision / IEEE Workshop on Motion and Video Computing (WACV/MOTION '05), January 2005, Breckenridge, Colo, USA, 99–104

  13. PJ Philips, P Grother, RJ Micheals, DM Blackburn, E Tabassi, M Bone, Face recognition vendor test 2002. in Tech, ed. by . Rep. NIST IR 6965 (National Institute of Standards and Technology, Gaithersburg, Md, USA, 2003)

  14. SA Rizvi, PJ Phillips, H Moon, The FERET verification testing protocol for face recognition algorithms. in Tech, ed. by . Rep. NIST IR 6281 (National Institute of Standards and Technology, Gaithersburg, Md, USA, 1998)

  15. PJ Phillips, H Moon, SA Rizvi, PJ Rauss, The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000). Publisher Full Text OpenURL

  16. KI Chang, KW Bowyer, PJ Flynn, X Chen, Multi-biometrics using facial appearance, shape and temperature. Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), May 2004, Seoul, Korea, 43–48

  17. C Beumier, M Acheroy, Face verification from 3D and grey level clues. Pattern Recognition Letters 22(12), 1321–1329 (2001). Publisher Full Text OpenURL

  18. JC Lee, EE Milios, Matching range images of human faces. Proceedings of the 3rd International Conference on Computer Vision (ICCV '90), December 1990, Osaka, Japan, 722–726

  19. Y Yacoob, LS Davis, Labeling of human face components from range data. CVGIP: Image Understanding 60(2), 168–178 (1994). Publisher Full Text OpenURL

  20. C-S Chua, F Han, YK Ho, 3D human face recognition using point signature. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG '00), March 2000, Grenoble, France, 233–238

  21. V Blanz, T Vetter, Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(9), 1063–1074 (2003). Publisher Full Text OpenURL

  22. V Blanz, T Vetter, A morphable model for the synthesis of 3D faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH '99), August 1999, Los Angeles, Calif, USA, 187–194

  23. G Pan, Y Wu, Z Wu, Investigating profile extracted from range data for 3D face recognition. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, October 2003, Washington, DC, USA 2, 1396–1399

  24. CW Urquhart, JP McDonald, JP Siebert, RJ Fryer, Active animate stereo vision. Proceedings of the 4th British Machine Vision Conference, September 1993, Guildford, UK (University of Surrey), pp. 75–84

  25. K Liu, Y-Q Cheng, J-Y Yang, Algebraic feature extraction for image recognition based on an optimal discriminant criterion. Pattern Recognition 26(6), 903–911 (1993). Publisher Full Text OpenURL

  26. J Yang, D Zhang, AF Frangi, J-Y Yang, Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(1), 131–137 (2004). PubMed Abstract | Publisher Full Text OpenURL

  27. J Ye, R Janardan, Q Li, Two-dimensional linear discriminant analysis. Proceedings of Neural Information Processing Systems (NIPS '04), December 2004, Vancouver, British Columbia, Canada, 1569–1576

  28. H Kong, L Wang, EK Teoh, J-G Wang, R Venkateswarlu, A framework of 2D fisher discriminant analysis: application to face recognition with small number of training samples. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 2, 1083–1088

  29. J Yang, D Zhang, X Yong, J-Y Yang, Two-dimensional discriminant transform for face recognition. Pattern Recognition 38(7), 1125–1129 (2005). Publisher Full Text OpenURL

  30. M Visani, C Garcia, J-M Jolion, Two-dimensional-oriented linear discriminant analysis for face recognition. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG '04), September 2004, Warsaw, Poland, 1008–1017

  31. Videre Design, MEGA-D Megapixel Digital Stereo Head (http://users), . rcn.com/mclaughl.dnai/sthmdcs.htm webcite

  32. K Messer, J Matas, J Kittler, J Luettin, G Maitre, XM2VTSDB: the extended M2VTS database. Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '99), March 1999, Washington, DC, USA, 72–77

  33. EE Catmull, in A subdivision algorithm for computer display of curved surfaces, Ph, ed. by . D. thesis (Department of Computer Science, University of Utah, Salt Lake City, Utah, USA, 1974)

  34. J-G Wang, E Sung, Frontal-view face detection and facial feature extraction using color and morphological operations. Pattern Recognition Letters 20(10), 1053–1068 (1999). Publisher Full Text OpenURL

  35. RI Jenrich, Stepwise discriminant analysis. in Statistical Methods for Digital Computers, ed. by Enslein K, Ralston A, Wilf HS (John Wiley & Sons, New York, NY, USA, 1977), pp. 76–95

  36. J Kittler, M Hatef, RPW Duin, J Matas, On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998). Publisher Full Text OpenURL

  37. T Choudhury, B Clarkson, T Jebara, A Pentland, Multimodal person recognition using unconstrained audio and video. Proceedings of the 2nd International Conference on Audio- and Video-Based Person Authentication (AVBPA '99), March 1999, Washington, DC, USA, 176–181

  38. DA Socolinsky, A Selinger, JD Neuheisel, Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding 91(1-2), 72–114 (2003). Publisher Full Text OpenURL

  39. PN Belhumeur, JP Hespanha, DJ Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997). Publisher Full Text OpenURL