If A= […] and f(x)=x^2-2x-3 , then f(A) is

The correct answer is Option (2) → Zero matrix

For a matrix, the constant term becomes a multiple of I: \( f(A)=A^2-2A-3I \).

    \( \displaystyle A^2=\begin{bmatrix}1 & 2\\2 & 1\end{bmatrix}\begin{bmatrix}1 & 2\\2 & 1\end{bmatrix} \)

    \( \displaystyle =\begin{bmatrix}5 & 4\\4 & 5\end{bmatrix} \)

    \( \displaystyle f(A)=\begin{bmatrix}5 & 4\\4 & 5\end{bmatrix}-\begin{bmatrix}2 & 4\\4 & 2\end{bmatrix}-\begin{bmatrix}3 & 0\\0 & 3\end{bmatrix} \)

    \( \displaystyle =\begin{bmatrix}0 & 0\\0 & 0\end{bmatrix} \)

So f(A) is the zero matrix.

Shortcut: By the Cayley-Hamilton theorem, a 2 × 2 matrix satisfies

    \( \displaystyle A^2-(\mathrm{tr}\,A)A+|A|\,I=O \)

Here tr A = 2 and |A| = 1 − 4 = −3, so \( A^2-2A-3I=O \) at once.

Why the other options are incorrect

• Option (1) \( \begin{bmatrix}3 & 0\\0 & 3\end{bmatrix} \) — Incorrect. This is \( A^2-2A=3I \); the constant term −3I has not been subtracted.

• Option (3) \( \begin{bmatrix}5 & 4\\4 & 5\end{bmatrix} \) — Incorrect. This is \( A^2 \) alone; the terms −2A and −3I have not been subtracted.

• Option (4) Identity matrix — Incorrect. \( A^2-2A-3I \) is O, not I.