the pdf for the possible values of x2. If
the random variables are only uncorrelated, knowing x1 might give us new
information about x2 such as restricting
its possible values but the mean of x2|x1
will still be µ2. Using expectations, this
can be written as E[x2|x1] = E[x2], which is
equivalent to requiring that E[(x1−µ1)(x2−
µ2)], the covariance between the two variables, be equal to zero. This is obviously
a weaker condition than independence.
Although the discussion in this section has focused on measurements,
the same formalization can be used for
estimates produced by an estimator.
Lemma 1(i) shows how the mean and
variance of a linear combination of pair-wise uncorrelated random variables can
be computed from the means and variances of the random variables.
mean and variance can be used to quantify bias and random errors for the estimator as in the case of measurements.
An unbiased estimator is one whose
mean is equal to the unknown value
being estimated and it is preferable to a
biased estimator with the same variance.
Only unbiased estimators are considered
in this article. Furthermore, an unbiased
estimator with a smaller variance is preferable to one with a larger variance as we
would have more confidence in the estimates it produces. As a step toward generalizing this discussion to estimators that
produce vector estimates, we refer to the
variance of an unbiased scalar estimator
as the mean square error of that estimator
or MSE for short.
Lemma 1(ii) asserts that if a random
variable is pairwise uncorrelated with
a set of random variables, it is uncorrelated with any linear combination of
Lemma 1. Let
be a set of pairwise uncorrelated
random variables. Let be a
random variable that is a linear combi-
nation of the xi’s.
(i) The mean and variance of y are:
(ii) If random variable xn+ 1 is pair-wise
uncorrelated with x1,..,xn, it is
uncorrelated with y.
application of these general concepts.
First, the informal ideas discussed here
are formalized using the notions of distributions and random samples from distributions. Confidence in estimates is
quantified using the variances and covariances of these distributions.b Two algorithms are described next. The first one
shows how to fuse estimates (such as core
temperature measurements) optimally,
given a reasonable definition of optimality. The second algorithm addresses a
problem that arises frequently in practice:
estimates are vectors (for example, the
position and velocity of a robot), but only a
part of the vector can be measured
directly; in such a situation, how can an
estimate of the entire vector be obtained
from an estimate of just a part of that
vector? The best linear unbiased estimator (BLUE) is used to solve this problem.
16, 19, 26 It is shown that the Kalman
filter can be derived in a straightforward way by using these two algorithms
to solve the problem of state estimation
in linear systems. The extended Kalman
filter and unscented Kalman filter,
which extended Kalman filtering to nonlinear systems, are described briefly at
the end of the article.
Scalar estimates. To model the behav-
ior of devices producing noisy tempera-
ture measurements, we associate each
device i with a random variable that has
a probability density function (pdf) pi(x)
such as the ones shown in Figure 1 (the
x-axis in this figure represents tempera-
ture). Random variables need not be
Gaussian.c Obtaining a measurement
from device i corresponds to drawing a
b Basic concepts such as probability density func-
tion, mean, expectation, variance and covari-
ance are introduced in the online appendix.
c The role of Gaussians in Kalman filtering is
discussed later in the article.
random sample from the distribution
for that device. We write to
denote that xi is a random variable with
pdf pi whose mean and variance are µi
and , respectively; following conven-
tion, we use xi to represent a random
sample from this distribution as well.
Means and variances of distribu-
tions model different kinds of inaccura-
cies in measurements. Device i is said to
have a systematic error or bias in its
measurements if the mean µi of its dis-
tribution is not equal to the actual tem-
perature xc (in general, to the value being
estimated, which is known as ground
truth); otherwise, the instrument is unbi-
ased. Figure 1 shows pdfs for two devices
that have different amounts of systematic
error. The variance on the other hand
is a measure of the random error in the
measurements. The impact of random
errors can be mitigated by taking many
measurements with a given device and
averaging their values, but this approach
will not reduce systematic error.
In the formulation of Kalman fil-
tering, it is assumed that measuring
devices do not have systematic errors.
However, we do not have the luxury of
taking many measurements of a given
state, so we must take into account the
impact of random error on a single
measurement. Therefore, confidence
in a device is modeled formally by the
variance of the distribution associated
with that device; the smaller the vari-
ance, the higher our confidence in the
measurements made by the device. In
Figure 1, the fact we have less confi-
dence in the first device has been illus-
trated by making p1 more spread out
than p2, giving it a larger variance.
The informal notion that noise should
affect the two devices in “unrelated
ways” is formalized by requiring that
the corresponding random variables be
uncorrelated. This is a weaker condition
than requiring them to be independent,
as explained in our online appendix
3363294&picked=formats). Suppose we
are given the measurement made by
one of the devices (say x1) and we have
to guess what the other measurement
(x2) might be. If knowing x1 does not give
us any new information about what x2
might be, the random variables are inde-
pendent. This is expressed formally by
the equation p(x2|x1) = p(x2); intuitively,
knowing the value of x1 does not change
Figure 1. Using pdfs to model devices with
systematic and random errors. Ground truth
is 60°C. Dashed lines are means of pdfs.
58 60 63