The value of cot^-1≤ft(tan≤ft(-(π)/5)) is

The correct answer is Option (2) → \(\displaystyle\frac{7\pi}{10}\)

The tangent has to be written as the cotangent of an angle in \((0, \pi)\), the principal value branch of \(\cot^{-1}\).

Tangent is odd, so

    \( \displaystyle \tan\left(-\frac{\pi}{5}\right) = -\tan\frac{\pi}{5} \)

Using \(\cot\left(\frac{\pi}{2} + \theta\right) = -\tan\theta\) with \(\theta = \frac{\pi}{5}\):

    \( \displaystyle -\tan\frac{\pi}{5} = \cot\left(\frac{\pi}{2} + \frac{\pi}{5}\right) = \cot\frac{7\pi}{10} \)

Since \(\frac{7\pi}{10}\) lies in \((0, \pi)\),

    \( \displaystyle \cot^{-1}\left(\tan\left(-\frac{\pi}{5}\right)\right) = \frac{7\pi}{10} \)

Why the other options are incorrect

• Option (1) \(-\frac{7\pi}{10}\) — Incorrect. Negative angles are outside \((0, \pi)\).

• Option (3) \(\frac{3\pi}{10}\) — Incorrect. \(\cot\frac{3\pi}{10} = \tan\frac{\pi}{5}\) is positive, but \(\tan\left(-\frac{\pi}{5}\right)\) is negative.

• Option (4) \(-\frac{\pi}{5}\) — Incorrect. It is outside \((0, \pi)\), and \(\cot\left(-\frac{\pi}{5}\right)\) is not equal to \(\tan\left(-\frac{\pi}{5}\right)\).