The function \(f(x) = (x - 2)^{2}(x + 1)^{3}\), \(x \in R\), is strictly decreasing in
Answer & explanation
Correct answer: option 3
The correct answer is Option (3) → \(\left(\displaystyle\frac{4}{5}, 2\right)\)
By the product rule:
\( f'(x) = 2(x - 2)(x + 1)^{3} + 3(x - 2)^{2}(x + 1)^{2} \)
Taking out the common factor \((x - 2)(x + 1)^{2}\):
\( f'(x) = (x - 2)(x + 1)^{2}\left[2(x + 1) + 3(x - 2)\right] \)
\( = (x - 2)(x + 1)^{2}(5x - 4) \)
The factor \((x + 1)^{2}\) is never negative, so it does not change the sign at x = −1. The sign of f′(x) is the sign of \((x - 2)(5x - 4)\):
\( f'(x) \gt 0 \) for \( \displaystyle x \lt \frac{4}{5} \) (x ≠ −1)
\( f'(x) \lt 0 \) for \( \displaystyle \frac{4}{5} \lt x \lt 2 \)
\( f'(x) \gt 0 \) for \( x \gt 2 \)
So f is strictly decreasing exactly in \(\left(\frac{4}{5}, 2\right)\).
Why the other options are incorrect
Option (1) \(\left(-1, \displaystyle\frac{4}{5}\right)\) — Incorrect. At x = 0, \(f'(0) = (-2)(1)(-4) = 8 \gt 0\), so f is increasing there.
Option (2) \((-\infty, -1) \cup \left(\displaystyle\frac{4}{5}, 2\right)\) — Incorrect. This treats x = −1 as a change of sign; but at x = −2, \(f'(-2) = (-4)(1)(-14) = 56 \gt 0\).
Option (4) \((-1, 2)\) — Incorrect. It contains \(\left(-1, \frac{4}{5}\right)\), where f′(x) > 0.