Differentiate the function $\tan^{-1} \left( \frac{a \cos x - b \sin x}{b \cos x + a \sin x} \right), \quad -\frac{\pi}{2} < x < \frac{\pi}{2} \text{ and } \frac{a}{b} \tan x > -1$ with respect to $x$.

$-1$

The correct answer is Option (2) → $-1$ ##

Let $y = \tan^{-1} \left( \frac{a \cos x - b \sin x}{b \cos x + a \sin x} \right)$

On dividing by $b \cos x$ in both numerator and denominator, we get

$y = \tan^{-1} \left[ \frac{\frac{a \cos x}{b \cos x} - \frac{b \sin x}{b \cos x}}{\frac{b \cos x}{b \cos x} + \frac{a \sin x}{b \cos x}} \right] = \tan^{-1} \left[ \frac{\frac{a}{b} - \tan x}{1 + \frac{a}{b} \tan x} \right]$

$= \tan^{-1} \frac{a}{b} - \tan^{-1}(\tan x) \quad \left[ ∵ \tan^{-1} x - \tan^{-1} y = \tan^{-1} \left( \frac{x - y}{1 + xy} \right) \right]$

$= \tan^{-1} \frac{a}{b} - x$

On differentiating w.r.t. $x$, we get

$\frac{dy}{dx} = \frac{d}{dx} \left( \tan^{-1} \frac{a}{b} \right) - \frac{d}{dx} (x) = 0 - 1 = -1$