The differential equation, representing family of curves y=ae^2x+be^-…

The correct answer is Option (2) → \(\displaystyle\frac{d^2y}{dx^2}+\frac{dy}{dx}-6y=0\)

There are two arbitrary constants, so the equation comes from differentiating twice and eliminating a and b:

    \(y=ae^{2x}+be^{-3x}\)
    \(y'=2ae^{2x}-3be^{-3x}\)
    \(y''=4ae^{2x}+9be^{-3x}\)

Adding the last two lines:

    \(y''+y'=6ae^{2x}+6be^{-3x}\)
    \(y''+y'=6y\)

So

    \( \displaystyle \frac{d^2y}{dx^2}+\frac{dy}{dx}-6y=0 \)

Shortcut: putting \(y=e^{mx}\) into \(y''+py'+qy=0\) gives \(m^2+pm+q=0\). Both \(e^{2x}\) and \(e^{-3x}\) must satisfy the equation, so this quadratic has roots 2 and −3:

    \((m-2)(m+3)=m^2+m-6\)

This gives p = 1, q = −6, which is Option (2).