Homework 4 (Due: February 23)

[PDF]

ECE 630: Statistical Communication Theory
Prof. B.-P. Paris
Homework 4
Due: February 23, 2017

Problems
1. Let x and y be elements of a normed linear vector space.
1. Determine whether the following are valid inner products for the indicated space.
1. x,y= xT Ay, where A is a nonsingular,NxN matrix and x, y are elements of the space of N-dimensional vectors.
2. x,y= xyT , where x and y are elements of the space of N-dimensional (column!) vectors.
3. x,y= 0T x(t)y(T-t)dt, where x and y are finite energy signals defined over [0,T].
4. x,y= 0T w(t)x(t)y(t)dt, where x and y are finite energy signals defined over [0,T] and w(t) is a non-negative function.
5. E[XY ], where X and Y are real-valued random variables having finite mean-square values.
6. Cov(X,Y ), the covariance of the real-valued random variables X and Y . Assume that X and Y have finite mean-square values.
2. Under what conditions is

a valid inner product for the space of finite-energy functions defined over [0,T]?

2. Let x(t) be a signal of finite energy over the interval [0,T]. In other words, x(t) is a vector in the Hilbert space L2(0,T). Signals may be complex values, so that the appropriate inner product is

Consider subspace of L2(0,T) that consists of signals of the form

where Xn may be complex valued.

1. Find the signal ŷn(t) that best approximates the signal x(t), i.e., ŷn(t) minimizes x - ynamong all elements of .
Hint: Find the best complex amplitude n.
2. Now define the error signal z(t) = x(t) -ŷn(t). Show that z(t) is orthogonal to the subspace , i.e., it is orthogonal to all elements of .
3. How do the above results illustrate the projection theorem?
3. The price of a stock fluctuates during the day while the unobservable, “true” value is rising or falling. To facilitate investment decisions, a financial engineer decides to use random process process and signal space theory. The price Pt of a stock is described by

where K is the constant the knowledgeable investor is seeking and Nt is a random process describing the random fluctuations. Specifically, Nt is a white, Gaussian process having spectral height . The investor decides to estimate K using the inner product:

where the “best” function g(t) is to be found.

1. Find a general expression for the probability density function of the estimate for any g(t) the investor might choose.
2. A simple-minded estimate of is to use simple averaging (i.e., set g(t) = constant). Find the value of the constant that results in E[] = K. What is the resulting percentage error as expressed by ?
3. Let g(t) = at and choose a to yield E[] = K. How much better is this choice than simple averaging in terms of percent error?
4. Do you think g(t) = at is the best possible choice? Use the terms subspace and projection theorem in your explanation.
4. Linear Regression
The elements of a vector of random variables follow the model

where xn are known and Nn are zero mean, iid Gaussian noise samples with variance σ2. The parameters a and b are to be determined. We can think of the solution to this problem as the projection of onto the subspace spanned by a + b

1. Determine the least-squares estimates for a and b, i.e., find

2. What are the expected values of these estimates, E[â] and E[ ]?
3. Compute â and , when data are given by the (xn,Y n) pairs

4. Is it true that the least-squares estimates for a and b are given by the inner products

denotes a vector of 1’s. Explain why or why not?