metrics to enhance the accuracy of
current unimodal biometrics-based
authentication on mobile devices;
moreover, according to how quickly
the system is able to identify a legitimate user, the Proteus approach
is scalable to consumer mobile devices. This is the first attempt at
implementing two types of fusion
schemes on a modern consumer
mobile device while tackling the
practical issues of user friendliness.
It is also just the beginning. We are
working on improving the performance and efficiency of both fusion
schemes, and the road ahead promises endless opportunity.
Multimodal biometrics is the next
logical step in biometric authentication for consumer-level mobile devices. The challenge remains in making multimodal biometrics usable for
consumers of mainstream mobile devices, but little work has sought to add
multimodal biometrics to them. Our
work is the first step in that direction.
Imagine a mobile device you can
unlock through combinations of face,
voice, fingerprints, ears, irises, and
retinas. It reads all these biometrics
in one step similar to the iPhone’s
TouchID fingerprint system. This
user-friendly interface utilizes an
underlying robust fusion logic based
on biometric sample quality, maximizing the device’s chance of correctly identifying its owner. Dirty
fingers, poorly illuminated or loud
settings, and damage to biometric
sensors are no longer showstoppers;
if one biometric fails, others function as backups. Hackers must now
gain access to the many modalities
required to unlock the device; because these are biometric modalities, they are possessed only by the
legitimate owner of the device. The
device also uses cancelable biometric templates, strong encryption, and
the Trusted Execution Environment
for securely storing and processing
all biometric data.
The Proteus multimodal biomet-
rics scheme leverages the existing
capabilities of mobile device hard-
ware (such as video recording), but
mobile hardware and software are
not equipped to handle more so-
phisticated combinations of bio-
metrics; for example, mainstream
consumer mobile devices lack
sensors capable of reliably acquir-
ing iris and retina biometrics in
a consumer-friendly manner. We
are thus working on designing and
building a device with efficient,
user-friendly, inexpensive soft-
ware and hardware to support such
combinations. We plan to inte-
grate new biometrics into our cur-
rent fusion schemes, develop new,
more robust fusion schemes, and
design user interfaces allowing the
seamless, simultaneous capture of
multiple biometrics. Combining a
user-friendly interface with robust
multimodal fusion algorithms may
well mark a new era in consumer
mobile device authentication.
1. Aronowitz, H., Min L., Toledo-Ronen, O., Harary, S.,
Geva, A., Ben-David, S., Rendel, A., Hoory, R., Ratha, N.,
Pankanti, S., and Nahamoo, D. Multimodal biometrics
for mobile authentication. In Proceedings of the 2014
IEEE International Joint Conference on Biometrics
(Clear water, FL, Sept. 29–Oct. 2). IEEE Computer
Society Press, 2014, 1–8.
2. Avila, C.S., Casanova, J.G., Ballesteros, F., Garcia,
L.R. T., Gomez, M.F.A., and Sierra, D.S. State of the
Art of Mobile Biometrics, Liveness and Non-Coercion
Detection. Personalized Centralized Authentication
System Project, Jan. 31, 2014; https://www.pcas-
3. Belhumeur, P. N., Hespanha, J.P., and Kriegman, D.
Eigenfaces vector vs. FisherFaces: Recognition using
class-specific linear projection. Pattern Analysis and
Machine Intelligence, IEEE Transactions on Pattern
Analysis and Machine Intelligence 19, 7 (July 1997),
4. Bonnington, C. The trouble with Apple’s Touch ID
fingerprint reader. Wired (Dec. 3, 2013); http://www.wired.
5. Dalal, N. and Triggs, B. Histograms of oriented
gradients for human detection. In Proceedings of the
IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (San Diego, CA,
June 20–25). IEEE Computer Society Press, 2005,
6. Daugman, J.G. Two-dimensional spectral analysis of
cortical receptive field profiles. Vision Research 20, 10
(Dec. 1980), 847–856.
7. Devine, R. Face Unlock in Jelly Bean gets a ‘liveness
check.’ AndroidCentral (June 29, 2012); http://www.
8. Duchnowski, P., Hunke, M., Busching, D., Meier, U., and
Waibel, A. Toward movement-invariant automatic lip-reading and speech recognition. In Proceedings of the
1995 International Conference on Acoustics, Speech,
and Signal Processing (Detroit, MI, May 9–12). IEEE
Computer Society Press, 1995, 109–112.
9. Hansen, J.H.L. Analysis and compensation of speech
under stress and noise for environmental robustness
in speech recognition. Speech Communication 20, 1
(Nov. 1996), 151–173.
10. Hsu, D., Kakade, S.M., and Zhang, T. A spectral
algorithm for learning hidden Markov models. Journal
of Computer and System Sciences 78, 5 (Sept. 2012),
11. Jain, A.K., Nandakumar, K., and Ross, A. Score
normalization in multimodal biometric systems.
Pattern Recognition 38, 12 (Dec. 2005), 2270–2285.
12. Kisku, D.R., Gupta, P., and Sing, J.K. Feature-level
fusion of biometrics cues: Human identification with
Doddingtons Caricature. Security Technology (2009),
13. Kuncheva, L.I., Whitaker, C.J., Shipp, C.A., and Duin,
R.P. W. Is independence good for combining classifiers?
In Proceedings of the 15th International Conference
on Pattern Recognition (Barcelona, Spain, Sept. 3–7).
IEEE Computer Society Press, 2000, 168–171.
14. Lee, C. Automatic recognition of animal vocalizations
using averaged MFCC and linear discriminant analysis.
Pattern Recognition Letters 27, 2 (Jan. 2006), 93–101.
15. M2SYS Technology. SecuredPass AFIS/ABIS
Immigration and Border Control System; http://
16. Milner, B. and Xu, S. Speech reconstruction from mel-
frequency cepstral coefficients using a source-filter
model. In Proceedings of the INTERSPEECH Conference
(Denver, CO, Sept. 16–20). International Speech
Communication Association, Baixas, France, 2002.
17. Nasrollahi, K. and Moeslund, T.B. Face-quality
assessment system in video sequences. In
Proceedings of the Workshop on Biometrics and
Identity Management (Roskilde, Denmark, May 7–9).
Springer, 2008, 10–18.
18. Parala, A. UAE Airports get multimodal security.
FindBiometrics Global Identity Management (Mar. 13,
19. Rathgeb, C. and Andreas U. A survey on biometric
cryptosystems and cancelable biometrics. EURASIP
Journal on Information Security (Dec. 2011), 1–25.
20. Ross, A. and Govindarajan, R. Feature-level fusion
of hand and face biometrics. In Proceedings of the
Conference on Biometric Technology for Human
Identification (Orlando, FL). International Society
for Optics and Photonics, Bellingham , WA, 2005,
21. Ross, A. and Jain, A. Multimodal biometrics: An
overview. In Proceedings of the 12th European Signal
Processing Conference (Sept. 6–10). IEEE Computer
Society Press, 2004, 1221–1224.
22. Sacco, A. Fingerprint faceoff: Apple TouchID vs.
Samsung Finger Scanner. Chief Information Officer
(July 16, 2014); http://www.cio.com/article/2454883/
23. Tapellini, D.S. Phone thefts rose to 3. 1 million last
year. Consumer Reports finds industry solution falls
short, while legislative efforts to curb theft continue.
Consumer Reports (May 28, 2014); http://www.
24. Viola, P. and Jones, M. Rapid object detection using a
boosted cascade of simple features. In Proceedings of
the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (Kauai, HI, Dec. 8–14).
IEEE Computer Society Press, 2001.
25. Vondrasek, M. and Pollak, P. Methods for speech
SNR estimation: Evaluation tool and analysis of VAD
dependency. Radioengineering 14, 1 (Apr. 2005), 6–11.
26. Zorabedian, J. Samsung Galaxy S5 fingerprint reader
hacked—It’s the iPhone 5S all over again! Naked
Security (Apr. 17, 2014); https://nakedsecurity.sophos.
Mikhail I. Gofman ( firstname.lastname@example.org) is an
assistant professor in the Department of Computer
Science at California State University, Fullerton, and
director of its Center for Cybersecurity.
Sinjini Mitra ( email@example.com) is an assistant
professor of information systems and decision sciences
at California State University, Fullerton.
Copyright held by authors.
Publication rights licensed to ACM. $15.00