Homework 2

  1. Include your python code in the following answers.

    1. Make a \(10 \times 10\) diagonal matrix with three \(1\)s on the diagonal, and all other entries zero.

    2. What is the rank of this matrix? Find the eigenvalues and eigenvectors.

    3. Now add a small random number to each entry of size. What is the rank of this matrix? How do the eigenvalues and eignevectors change?

    4. What happens if you start with the zero matrix instead?

  2. Consider the polynomial \(p(x) = (x - 2)^9 = x^9 - 18 x^8 + 144 x^7 -672x^6+2016x^5-4032x^4+5376x^3-4608x^2+2304x-512\).

    1. Plot \(p(x)\) for \(x=1.920,1.921,1.922,\ldots,2.080\) evaluating \(p(x)\) via its coefficients \(1,-18,144, \ldots\) (The numpy method linspace will be useful for this)

    2. Plot the same plot again, now evaluating \(p(x)\) via the expression \((x-2)^9\).

    3. Explain the difference.

  3. Consider the linear map \(f \colon \mathbb{R}^m \to \mathbb{R}^n\) given by \(f(x) = A x\) for some fixed matrix \(A\).
    Show that the derivative of \(f\) is equal to \(A\).

  4. Let \(x\) be a vector. You can compute \(||x||^4\) by \(x^T x x^T x\). Note that \(( x^T x )( x^T x) = x^T (x x^T) x\), which is likely to be more efficient to compute?

  5. Let \(\text{vec}(A)\) be the operation which converts a matrix into a vector. Show that \(\text{vec}(AB) = (I_m \otimes A) \text{vec}(B) = (B^T \otimes I_k) \text{vec}(A)\).

    1. Show that the map \(f \colon X \mapsto XA\), with \(A\) a fixed matrix, is linear. What is \(\nabla f\)?

    2. Show that the map \(f \colon X \mapsto AX\), with \(A\) a fixed matrix, is linear. What is \(\nabla f\)?

    3. Let \(F\) be the scaled Frobenius norm \(F(A) = \tfrac{1}{2} \text{tr}(A^T A)\). What is \(\nabla F\)?

    4. Let \(G(W, H) = F(X - WH)\). Fix \(H\) and show that if \({\nabla}_W G = 0\) then \(W H H^T - X H^T = 0\).

    5. If you fix \(W\), show a similar result for \(\nabla_H G\)?