If $cot(cos^{-1}x) = sec \begin{Bmatrix}tan^{-1}\left(\frac{a}{\sqrt{b^2-a^2}}\right)\end{Bmatrix}$, then x equals

$\frac{b}{\sqrt{2b^2-a^2}}$

We have,

$cot(cos^{-1}x) = sec \begin{Bmatrix}tan^{-1}\left(\frac{a}{\sqrt{b^2-a^2}}\right)\end{Bmatrix}$

$  ⇒cot\begin{Bmatrix}cot^{-1}\left(\frac{x}{\sqrt{1-x^2}}\right)\end{Bmatrix} = sec \begin{Bmatrix}sec^{-1}\left(\frac{b}{\sqrt{b^2-a^2}}\right)\end{Bmatrix}$

$  ⇒ \frac{x}{\sqrt{1-x^2}}=\frac{b}{\sqrt{b^2-a^2}}$

$  ⇒ \frac{x^2}{1-x^2}=\frac{b^2}{b^2-a^2}$

$  ⇒ \frac{1-x^2}{x^2}=\frac{b^2-a^2}{b^2}$

$  ⇒ \frac{1}{x^2} -1 = 1 -\frac{a^2}{b^2} ⇒ \frac{1}{x^2} =\frac{2b^2-a^2}{b^2} ⇒ x = - \frac{b}{\sqrt{b^2-a^2}}$