f(x)=exp(-x^2/2)(x^2-1) using PGFPlots

What I get is an error, actually. When plotting a function of x you should do

addplot {<function>};

or

addplot [<option>] {<function>};

if you're adding any options to the plot. You have done just addplot [...] <function>; without the {}.

In addition, note that multiplication has to be specified explicitly with *, so you need

addplot[smooth, thick,blue,] {exp(-x^2/2)*(x^2-1)};

Looking at the screenshot from Overleaf, I'd suspect you just forgot the *. With the above addplot, I get

By default the domain plotted is -5:5, that's why the graph stops where it does. Add domain=-8:8 to either the axis options or the addplot options if you want to plot the function to the axis limits you have specified.

documentclass[border=5mm]{standalone}
usepackage{pgfplots}
begin{document}   
begin{tikzpicture}
begin{axis}[xmin = -8, xmax = 8,
ymin = -2, ymax = 3,xtick distance = 2.5,
ytick distance = 0.5,
grid = both,
minor tick num = 1,
major grid style = {lightgray},
minor grid style = {lightgray!25},
width = textwidth,
height = 0.5textwidth,
]
addplot[smooth,
thick,
blue,] {exp(-x^2/2)*(x^2-1)};
end{axis}
end{tikzpicture}
end{document}