A Gabor Wavelet Based Multichannel Approach to a Multimodal Face Verification System

Abstract

Gabor filters have received a special attention in the face biometrics community due to their resemblance to the simple cells in the mammalian visual cortex. They can extract information corresponding to the intrinsic spatial frequencies and orientations in a visual scene. Since Lades el al. (1993) and Wiskott el al. (1997), a now "classical" bank of 40 filters (5 frequencies times 8 orientations) has become a standard. Their Elastic Bunch Graph Matching method constitutes an example of the "analytical" paradigm, where Gabor features are extracted only on a reduced set of facial features. On the other hand, Liu and Wechsler (2002) suggested a "holistic" approach, where Gabor channels are computed globally, concatenated in a mosaic style and then reduced to a manageable size ("downsampling trick").

As several authors have decided not to rely on the classical bank, we have performed a statistical study of the influence of the 7 parameters that specify the filters contained in a bank, including the number of frequencies and orientations and the Gaussian width. We present the results of an analysis of variance of the success rate as a function of these parameters for a face recognition experiment with 5 face databases (FERET, FRAV2D, FRAV3D, FRGC, and XM2VTS). In particular, we show that the classical bank is located at the top 8% end of a ranking of 486 banks. However, the best bank was found to make use of 6 spatial frequencies (instead of 5) and a narrower Gaussian envelope.

Spurred by Liu and Wechsler's downsampling trick, we have also presented a brand-new holistic multichannel strategy for a face verification system, where each Gabor channel is processed separately, including a non-downsampling-based dimensionality reduction with Principal Component Analysis (PCA) or its 2D bilateral version (B2SPCA), as well as a classification stage. The scores of the different classiffers are then fused to provide a final decision. Our experiments with FRAV2D and XM2VTS databases have shown that our Multichannel Gabor Principal Component Analysis (MCGPCA) outperformed other algorithms from the literature, including a feature-fusion-. Based Downsampled Gabor PCA (DGPCA), a voting-based fusion of one-channel Gabor decisions, and an analytical Feature-bases Gabor PCA (FGPCA).

Finally, oru multichannel methods have been tested with two multimodal databases: FRAV3D (2D,2.5D, and curvature images) and FRGC (2D, 2.5D, curvature, and infrared images). Both MCGPCA and MCGB2DPCA outperformed DGPCA and the voting strategy where each modality casts a vote. As far as we know, this has been the first fusion of four facial modalities.