pgfplots - beamer reveals not working with "set layers"

I am creating a pgfplot in beamer where visual elements are revealed step-by-step. In particular, I have added extra tick marks which appear in a later frame, using the visible on style described here.

I also have TikZ shapes which intersect with the axis, and I would like these to be displayed underneath the axis. I have tried using the set layers option of pgfplots, but this seems to ruin the reveal of the aforementioned tick marks – they just appear on the first frame, rather than a later frame. How can I ensure that both:

• the extra tick marks don’t appear on the first frame; and
• shapes drawn inside the axis environment are placed underneath the axis?

Here is a MWE:

documentclass{beamer}
usepackage{pgfplots}
tikzset{
    invisible/.style={opacity=0},
    visible on/.style={alt={#1{}{invisible}}},
    alt/.code args={<#1>#2#3}{%
        alt<#1>{pgfkeysalso{#2}}{pgfkeysalso{#3}} % pgfkeysalso doesn't change the path
    },
}

begin{document}

begin{frame}
begin{center}
begin{tikzpicture}
begin{axis}[
    clip=false,xtick={0,1},
    extra x ticks={0.5},extra x tick style={visible on=<2->,grid=none},
    extra x tick labels={to appear later},
    xmin=-0.05,xmax=1.05,ymin=0,
    axis line style=semithick,
%   set layers=axis on top   % Uncommenting this ruins the reveal
]
    addplot[domain=0:1,samples=300,visible on=<2->,color=blue,line width=1pt]{0.3-0.1*x} node(endofplot)[anchor=west]{$f$};
    draw[fill=blue!50!white,opacity=0.95] (axis cs:0.3,-0.05) rectangle (axis cs:0.6,0.1) {};
end{axis}
end{tikzpicture}
end{center}
end{frame}

end{document}

documentclass{beamer}
usepackage{pgfplots}
pgfplotsset{compat=newest}
tikzset{
    invisible/.style={opacity=0},
    visible on/.style={alt={#1{}{invisible}}},
    alt/.code args={<#1>#2#3}{%
        alt<#1>{pgfkeysalso{#2}}{pgfkeysalso{#3}} % pgfkeysalso doesn't change the path
    },
}

begin{document}

begin{frame}
begin{tikzpicture}
begin{axis}[
    % set layers=true,
    axis on top=true,
    clip=false,
    xtick={0,1},
    extra x ticks={0.5},
    extra x tick style={visible on=<2>},
    extra x tick labels={to appear later},
    xmin=-0.05,
    xmax=1.05,
    ymin=0,
    ymax=0.3,
    axis line style=semithick,
]
only<1->{draw[fill=blue!50!white,opacity=0.95] (axis cs:0.3,-0.05) rectangle (axis cs:0.6,0.1) {};}
only<2->{addplot[domain=0:1,samples=300,visible on=<2->,color=blue,line width=1pt]{0.3-0.1*x} node(endofplot)[anchor=west]{$f$};}

end{axis}
end{tikzpicture}
end{frame}

end{document}