If \(\cot^{-1} x = \displaystyle\frac{5\pi}{6}\), then x is equal to
Answer & explanation
Correct answer: option 2
The correct answer is Option (2) → \(-\sqrt{3}\)
Since \(\frac{5\pi}{6}\) lies in \((0, \pi)\), the principal value branch of \(\cot^{-1}\), the definition of \(\cot^{-1}\) gives
\( \displaystyle x = \cot\frac{5\pi}{6} \)
Cotangent is negative in the second quadrant:
\( \displaystyle \cot\left(\pi - \frac{\pi}{6}\right) = -\cot\frac{\pi}{6} \)
\( \displaystyle = -\sqrt{3} \)
Why the other options are incorrect
Option (1) \(\sqrt{3}\) — Incorrect. \(\cot^{-1}\left(\sqrt{3}\right) = \frac{\pi}{6}\), not \(\frac{5\pi}{6}\).
Option (3) \(-\frac{1}{\sqrt{3}}\) — Incorrect. This is \(\tan\frac{5\pi}{6}\); \(\cot^{-1}\left(-\frac{1}{\sqrt{3}}\right) = \frac{2\pi}{3}\).
Option (4) \(\frac{1}{\sqrt{3}}\) — Incorrect. \(\cot^{-1}\left(\frac{1}{\sqrt{3}}\right) = \frac{\pi}{3}\).