The principal value of \(\cot^{-1}\left(-\displaystyle\frac{1}{\sqrt{3}}\right)\) is
Answer & explanation
Correct answer: option 1
The correct answer is Option (1) → \(\displaystyle\frac{2\pi}{3}\)
The principal value branch of \(\cot^{-1}\) is \((0, \pi)\). For a negative number the angle lies in the second quadrant, between \(\frac{\pi}{2}\) and \(\pi\).
Since \(\cot\frac{\pi}{3} = \frac{1}{\sqrt{3}}\),
\( \displaystyle \cot\left(\pi - \frac{\pi}{3}\right) = -\cot\frac{\pi}{3} = -\frac{1}{\sqrt{3}} \)
Hence \(\cot^{-1}\left(-\frac{1}{\sqrt{3}}\right) = \frac{2\pi}{3}\).
Why the other options are incorrect
Option (2) \(-\frac{\pi}{3}\) — Incorrect. \(\cot\left(-\frac{\pi}{3}\right) = -\frac{1}{\sqrt{3}}\), but negative angles are outside \((0, \pi)\).
Option (3) \(\frac{\pi}{3}\) — Incorrect. \(\cot\frac{\pi}{3} = +\frac{1}{\sqrt{3}}\); the sign is wrong.
Option (4) \(\frac{5\pi}{6}\) — Incorrect. \(\cot\frac{5\pi}{6} = -\sqrt{3}\), not \(-\frac{1}{\sqrt{3}}\).