SEPTEMBER 2018 | VOL. 61 | NO. 9 | COMMUNICATIONS OF THE ACM 93
is modulated by breathing and heartbeats. 6, 9, 37 We refer the
reader to Ref. Adib et al. 6 for a detailed description of such
methods, and summarize below the basic information relevant to this paper.
The radio transmits a low power signal and measures its
reflection time. It separates RF reflections from different
objects/bodies into buckets based on their reflection time.
It then eliminates reflections from static objects which do
not change across time and zooms in on human reflections.
It focuses on time periods when the person is quasi-static.
It then looks at the phase of the RF wave which is related to
the traveled distance as follows45: , where φ(t) is
the phase of the signal, l is the wavelength, d(t) is the traveled distance, and t is the time variable. The variations in
the phase correspond to the compound displacement caused
by chest expansion and contraction due to breathing, and
body vibration due to heartbeats.
The phase of the RF signal is illustrated in the top graph
in Figure 1. The envelop shows the chest displacements as
the inhale-exhale process. The small dents are due to minute skin vibrations associated with blood pulsing. EQ-Radio
operates on this phase signal.
5. BEAT EXTRACTION ALGORITHM
A person’s emotions are correlated with small variations in
her/his heartbeat intervals; hence, to recognize emotions,
EQ-Radio needs to extract these intervals from the RF phase
signal described above.
The main challenge in extracting heartbeat intervals
is that the morphology of heartbeats in the reflected RF
signals is unknown. Said differently, EQ-Radio does not
know how these beats look like in the reflected RF signals.
Specifically, these beats result in distance variations in the
reflected signals, but the measured displacement depends
on numerous factors including the person’s body and her
exact posture with respect to EQ-Radio’s antennas. This
is in contrast to ECG signals where the morphology of
heartbeats has a known expected shape, and simple peak
detection algorithms can extract the beat-to-beat intervals. However, because we do not know the morphology of
these heartbeats in RF a priori, we cannot determine when
a heartbeat starts and when it ends, and hence we cannot
obtain the intervals of each beat. In essence, this becomes
a chicken-and-egg problem: if we know the morphology of
the heartbeat, that would help us in segmenting the signal; on the other hand, if we have a segmentation of the
reflected signal, we can use it to recover the morphology of
the human heartbeat.
This problem is exacerbated by two additional factors.
First, the reflected signal is noisy; second, the chest displacement due to breathing is orders of magnitude higher
than the heartbeat displacements. In other words, we are
operating in a low Signal-to-Interference-and-Noise Ratio
(SINR) regime, where “interference” results from the chest
displacement due to breathing.
To address these challenges, EQ-Radio first processes the
RF signal to mitigate interference from breathing. It then
formulates and solves an optimization problem to recover
the beat-to-beat intervals. The optimization neither assumes
activities, 46 and vital signs. 6, 15 Our work is closest to prior art
that uses RF signals to extract a person’s breathing rate and
average heart rate. 6, 15, 19, 23, 33 In contrast to this past work,
which recovers the average period of a heartbeat (which
is of the order of a second), emotion recognition requires
extracting the individual heartbeats and measuring small
variations in the beat-to-beat intervals with millisecond-
scale accuracy. Unfortunately, prior research that aims to
segment RF reflections into individual beats either cannot
achieve sufficient accuracy for emotion recognition11, 20, 31
or requires the monitored subjects to hold their breath. 41
In particular, past work that does not require users to hold
their breath has an average error of 30–50ms, 11, 20, 31 whereas
EQ-Radio achieves average accuracy of 3.2ms.
3. EQ-RADIO OVERVIEW
EQ-Radio is an emotion recognition system that relies
purely on wireless signals. It operates by transmitting an
RF signal and capturing its reflections off a person’s body.
It then analyzes these reflections to infer the person’s emotional state. It classifies the person’s emotional state according to the known arousal-valence model into one of four
basic emotions26, 30: anger, sadness, joy, and pleasure (i.e.,
EQ-Radio’s system architecture consists of three com-
ponents that operate in a pipelined manner, as shown in
• A radio sensor that transmits RF signals and captures
their reflections off a person’s body.
• A beat extraction algorithm, which takes the captured
reflections as input and returns a series of signal segments that correspond to the person’s individual
• An emotion-classification subsystem, which computes
emotion-relevant features from the captured physiological signals – that is, the person’s breathing pattern
and heartbeats – and uses these features to recognize
the person’s emotional state.
4. CAPTURING THE RF SIGNAL
EQ-Radio operates on RF reflections off the human body.
To capture such reflections, EQ-Radio uses a radar technique called FMCW. 5 There is a significant literature on
FMCW radios and their use for obtaining an RF signal that
Figure 2. EQ-Radio Architecture. EQ-Radio has three components: a
radio for capturing RF reflections (Section 4), a heartbeat extraction
algorithm (Section 5), and a classification subsystem that maps the
learned physiological signals to emotional states (Section 6).