The set of values of a for which the function $f(x)=\left(\frac{a^2-1}{3}\right) x^3+(a-1) x^2+2 x+1$ increases on R, is

$R-[-3,1]$

If $f(x)=\left(\frac{a^2-1}{3}\right) x^3+(a-1) x^2+2 x+1$ increases on $R$, then

$f^{\prime}(x)>0$ for all $x \in R$

$\Rightarrow \left(a^2-1\right) x^2+2(a-1) x+2>0$ for all $x \in R$

$\Rightarrow a^2-1>0$ and $4(a-1)^2-8\left(a^2-1\right)<0$

$\Rightarrow a^2-1>0$ and $a^2+2 a-3>0$

$\Rightarrow a \in(-\infty,-1) \cup(1, \infty)$ and $a \in(-\infty,-3) \cup(1, \infty)$

$\Rightarrow a \in(-\infty,-3) \cup(1, \infty)$