Kyle - I was able to reproduce the same behaviour with R2014a on OS X 10.8.5. Reducing the number of points that were being displayed by scatter to about 1000 at any one time (from the 26000) the software to update on each call to Update Plots. Why this matters, I don't know.
Like Adam, I don't know why increasing the minimum of x to 700 should cause it to be slower. Though it does seem to work the other way too: if you decrease the maximum to 4000, the same slow behaviour is observed. It could have something to do with the smaller range in the data and so contourf has to work harder (?). Reducing the number of contour levels from 50 to 25 does seem to allow the software to continue without any noticeable lag in performance
Though this doesn't really explain why if scatter is never called, then there is no performance lag with contourf. I wondered about the graphics created by scatter so called cla(handles.scatterplot,'reset') to reset the scatter plot axes, but this didn't have the desired (positive) effect.
I replaced the call to scatter with
and this did work. So the same amount of data is being drawn on the axes, but it is being drawn differently.
So I think you have a couple of workarounds. And this may be a good example to bring to the attention of The MathWorks to get a better understanding of what is going on behind the scatter function.