Homework 4 (Due: February 23)
ECE 630: Statistical Communication Theory
Prof. B.-P. Paris
Due: February 23, 2017
- Madhow: Section 3.3.
- Let x and y be elements of a normed linear vector space.
- Determine whether the following are valid inner products for the
- ⟨x,y⟩ = xT Ay, where A is a nonsingular,NxN matrix and
x, y are elements of the space of N-dimensional vectors.
- ⟨x,y⟩ = xyT , where x and y are elements of the space of
N-dimensional (column!) vectors.
- ⟨x,y⟩ = ∫
0T x(t)y(T-t)dt, where x and y are finite energy
signals defined over [0,T].
- ⟨x,y⟩ = ∫
0T w(t)x(t)y(t)dt, where x and y are
finite energy signals defined over [0,T] and w(t) is a
- E[XY ], where X and Y are real-valued random variables
having finite mean-square values.
- Cov(X,Y ), the covariance of the real-valued random
variables X and Y . Assume that X and Y have finite
- Under what conditions is
a valid inner product for the space of finite-energy functions
defined over [0,T]?
- Let x(t) be a signal of finite energy over the interval [0,T]. In other
words, x(t) is a vector in the Hilbert space L2(0,T). Signals
may be complex values, so that the appropriate inner product
Consider subspace of L2(0,T) that consists of signals of the
where Xn may be complex valued.
- Find the signal ŷn(t) that best approximates the signal x(t),
i.e., ŷn(t) minimizes ∥x - yn∥ among all elements of .
Hint: Find the best complex amplitude n.
- Now define the error signal z(t) = x(t) -ŷn(t). Show that
z(t) is orthogonal to the subspace , i.e., it is orthogonal to
all elements of .
- How do the above results illustrate the projection theorem?
- The price of a stock fluctuates during the day while the unobservable,
“true” value is rising or falling. To facilitate investment decisions,
a financial engineer decides to use random process process
and signal space theory. The price Pt of a stock is described
where K is the constant the knowledgeable investor is seeking and
Nt is a random process describing the random fluctuations.
Specifically, Nt is a white, Gaussian process having spectral
height . The investor decides to estimate K using the inner
where the “best” function g(t) is to be found.
- Find a general expression for the probability density function
of the estimate for any g(t) the investor might choose.
- A simple-minded estimate of is to use simple averaging
(i.e., set g(t) = constant). Find the value of the constant that
results in E = K. What is the resulting percentage error
as expressed by ?
- Let g(t) = at and choose a to yield E = K. How much
better is this choice than simple averaging in terms of percent
- Do you think g(t) = at is the best possible choice? Use the
terms subspace and projection theorem in your explanation.
- Linear Regression
The elements of a vector of random variables follow the
where xn are known and Nn are zero mean, iid Gaussian noise samples
with variance σ2. The parameters a and b are to be determined. We can
think of the solution to this problem as the projection of onto the
subspace spanned by a + b
- Determine the least-squares estimates for a and b, i.e., find
- What are the expected values of these estimates, E[â] and
- Compute â and , when data are given by the (xn,Y n) pairs
- Is it true that the least-squares estimates for a and b are given
by the inner products
denotes a vector of 1’s. Explain why or why not?