A right angled triangle has its longest side BC that is diameter of a circle of radius r and a vertex A lying on the circle. Maximum area that the triangle ABC can enclose is

$r^2$

The correct answer is Option (1) - $r^2$

let $AB=x$

$AC=\sqrt{(2r)^2-x^2}=\sqrt{4r^2-x^2}$

area = $\frac{1}{2}x\sqrt{4r^2-x^2}=\frac{1}{2}\sqrt{4r^2x^2-x^4}$

To increase area we maximise $f(x)=4r^2x^2-x^4$

$f'(x)=8r^2x-4x^3=0$

$x=0$ (not possible), $x=r\sqrt{2}$

$f''(x)=8r^2-4×3x^2$

$f''(r×\sqrt{2})=-16r^2<0$

so $r\sqrt{2}$ (point of increase)

so area = $\frac{1}{2}r\sqrt{2}\sqrt{4r^2-2r^2}=r^2$

max area = $r^2$