NOVEMBER 2018 | VOL. 61 | NO. 11 | COMMUNICATIONS OF THE ACM 165
accuracy and usability. Further than an in-ear bio-sensing
wearable, we view LIBS as a key enabling technology for concealed head-worn devices for healthcare and communication
applications, especially for personalized health monitoring,
digital assistance, and the introduction of socially-aware
We thank LifeLines Neurodiagnostic Systems Inc. for their
support in providing the gold-standard PSG device and
thank Yiming Deng and Titsa Papantoni for their valuable
feedback at the early stages of this work. This material is
based in part upon work supported by the National Science
Foundation under Grant SCH-1602428.
Child’s interest assessment. With LIBS, child’s interest
assessment can be done less obtrusively and yield more
accurate outcomes. Moreover, from that, the parents will
be able to orient them accordingly so that they can learn what
they like the most. Clinically, kids from the age of 0–2yrs
don’t have the ability to express their interest. More precisely, the only way to express their interest is their crying. As a result, the conventional gold-standard device
(i.e., PSG) is usually used to read their biosignals, which
relatively reflect their interest in what they are allowed to
do. However, it is not comfortable for them to wear and
do activities during the assessment. Hence, by leveraging
LIBS to read the signal from their ears and at the same
time letting them play different sports or learn different
subjects, LIBS should be able to infer what the level of
their interest is with high comfort.
Human-computer interaction. In a broader context, LIBS
can be used as a form of Human Computer Interaction (HCI),
which can especially benefit users with disability. In stead of
using only the brain signal as found in many HCI and brain-to-computer systems today, LIBS can combine the information extracted from the three separated signals to enrich
commands the user can build to interact with the computer
in a more reliable way. This gives users more choices for
integration with computing systems in a potentially more
precise and convenient manner.
In this paper, we enabled LIBS, a sensing system worn
inside human ear canals, that can unobtrusively, comfortably, and continuously monitor the electrical activities of
human brain, eyes, and facial muscles. Different from existing hi-tech systems of measuring only one specific type
of the signals, LIBS deploys a NMF-based signal separation algorithm to feasibly and reliably achieve three individual signals of interest. Through one-month long user
study of collecting the in-ear signals during sleep and scoring them into appropriate sleep stages using a prototype,
LIBS itself demonstrated a promising comparison to the
existing dedicated sleep assessment systems in term of
Figure 12. Potential applications of LIBS.
Child’s interest assessment
In-home sleep monitoring
Autism onset prediction
Eating habit monitoring
Autonomous audio steering
1. Alloway, C., et al. The alpha
attenuation test: assessing excessive
daytime sleepiness in narcolepsy-cataplexy. Sleep 20, 4 (1997), 258–266.
2. Damon, C., et al. Non-negative matrix
factorization for single-channel EEG
artifact rejection. In ICASSP (2013),
3. Essid, S. A single-class SVM based
algorithm for computing an identifiable
NMF. In Proceedings of 2012 IEEE
ICASSP (2012), 2053–2056.
4. Févotte, C., Idier, J. Algorithms for
nonnegative matrix factorization with
the beta-divergence. Neural. Comput.
23, 9 (2011), 2421–2456.
5. Gillingham, G. Autism, Handle with
Care!: Understanding and Managing
Behavior of Children and Adults with
Autism. Future Education, Texas
6. Goverdovsky, V., et al. In-ear EEG
from viscoelastic generic earpieces:
Robust and unobtrusive 24/7
monitoring. IEEE Sens. J. (2015).
7. Hallez, H., et al. Review on solving
the forward problem in EEG source
analysis. J. Neuroeng. Rehabil. 4, 1
8. Kalantarian, H., et al. Monitoring
eating habits using a piezoelectric
sensor-based necklace. Comput. Biol.
Med. 58, Supplement C (2015), 46–55.
9. Kushida, C., et al. Practice
parameters for the indications
for polysomnography and related
procedures: an update for 2005. Sleep
28, 4 (2005), 499–521.
10. Lee, J.H., et al. CNT/PDMS-based
canal-typed ear electrodes for
inconspicuous EEG recording. J.
Neural Eng. 11, 4 (2014).
11. Motamedi-Fakhr, S., et al. Signal
processing techniques applied to
human sleep EEG signals—A review.
Biomed. Signal Process. Control 10
12. Nawrocka, A., Holewa, K. Brain—
Computer interface based on
Steady—State Visual Evoked
Potentials (SSVEP). In Proceedings of
14th ICCC (2013), 251–254.
13. Nguyen, A., Alqurashi, R., Raghebi, Z.,
Banaei-kashani, F., Halbower, A. C.,
Vu, T. A lightweight and inexpensive
in-ear sensing system for automatic
whole-night sleep stage monitoring.
SenSys ‘ 16 (2016), 230–244.
14. Oliveira, R. The dynamic ear canal.
Ballachandra B, ed. The Human Ear
Canal. San Diego: Singular Publishing
Group (1995), 83–111.
15. Oliveira, R., et al. A look at ear canal
changes with jaw motion. Ear Hear
13, 6 (1992), 464–466.
16. OpenBCI. http://openbci.com/.
17. Polysmith–NIHON KOHDEN.
18. Ronzhina, M., et al. Sleep scoring
using artificial neural networks. Sleep
Med. Rev. 16, 3 (2012), 251–263.
19. Sen, B., et al. A Comparative Study on
Classification of Sleep Stage Based on
EEG Signals Using Feature Selection
and Classification Algorithms. J. Med.
Syst. 38, 3 (2014), 1–21.
20. Silber, M.H., et al. The visual scoring
of sleep in adults. J. Clin. Sleep Med.
3, 02 (2007), 121–131.
21. Trackit Mark III—LifeLines
22. Tse, D., Viswanath, P. Fundamentals
of Wireless Communication.
Cambridge University Press,
New York (2005).
23. R.A.U., et al. Non-linear analysis of
EEG signals at various sleep stages.
Comput. Methods and Programs
Biomed. 80, 1 (2005), 37–45.
24. van der Reijden, C., et al. Signal-to-noise ratios of the auditory steady-state response from fifty-five EEG
derivations in adults. J. Am. Acad.
Audiol. 15, 10 (2004), 692–701.
25. Virtanen, T., et al. Compositional
Models for Audio Processing:
Uncovering the structure of sound
mixtures. IEEE Signal Processing
Mag. 32, 2 (2015), 125–144.
26. Zoubek, L., et al. Feature selection for
sleep/wake stages classification using
data driven methods. Biomed. Signal
Process. Control 2, 3 (2007), 171–179.
Anh Nguyen and Tam Vu, University of
Colorado, Boulder, CO, USA.
Raghda Alqurashi, Zohreh Raghebi,
and Farnoush Banaei-Kashani,
University of Colorado, Denver, CO, USA.
Ann C. Halbower, University of Colorado,
School of Medicine, Aurora, CO, USA.
© 2018 ACM 0001-0782/18/11 $15.00