[…] is equal to:

The correct answer is Option (1) → \( (a+b+c)(a^2+b^2+c^2) \)

Let \( \Delta \) be the determinant. Take a, b, c non-zero; both sides are polynomials in a, b, c, so the result then holds for all values. Multiply \( C_1 \) by a, \( C_2 \) by b, \( C_3 \) by c, and divide by abc:

    \( \displaystyle \Delta=\frac{1}{abc}\begin{vmatrix}a^2 & b^2-bc & c^2+bc\\a^2+ac & b^2 & c^2-ac\\a^2-ab & ab+b^2 & c^2\end{vmatrix} \)

Apply \( C_1\to C_1+C_2+C_3 \); every entry of \( C_1 \) becomes \( a^2+b^2+c^2 \), which is taken out:

    \( \displaystyle \Delta=\frac{a^2+b^2+c^2}{abc}\begin{vmatrix}1 & b^2-bc & c^2+bc\\1 & b^2 & c^2-ac\\1 & ab+b^2 & c^2\end{vmatrix} \)

Apply \( R_2\to R_2-R_1 \) and \( R_3\to R_3-R_1 \):

    \( \displaystyle \Delta=\frac{a^2+b^2+c^2}{abc}\begin{vmatrix}1 & b^2-bc & c^2+bc\\0 & bc & -ac-bc\\0 & ab+bc & -bc\end{vmatrix} \)

Expand along \( C_1 \):

    \( \displaystyle \begin{vmatrix}bc & -ac-bc\\ab+bc & -bc\end{vmatrix}=-b^2c^2+(ac+bc)(ab+bc) \)

    \( =-b^2c^2+a^2bc+abc^2+ab^2c+b^2c^2 \)

    \( =abc(a+b+c) \)

So

    \( \displaystyle \Delta=\frac{a^2+b^2+c^2}{abc}\cdot abc(a+b+c) \)

    \( =(a+b+c)(a^2+b^2+c^2) \)

Shortcut: Put a = 1, b = 0, c = 0:

    \( \displaystyle \begin{vmatrix}1 & 0 & 0\\1 & 0 & -1\\1 & 1 & 0\end{vmatrix}=1(0+1)=1 \)

Options (2), (3) and (4) all give 0 for these values; only Option (1) gives \( (1)(1)=1 \).