getappdata() requires a function call, as does set(). That is always going to be slower than a non-object variable access, even a global access. (For objects, there are different cases, one of which would be like accessing a struct but the others would involve function calls.)
Note that every call to guidata() involves climbing the ancestry of the given object to find the figure and then doing a getappdata() on it. GUIDE does this automatically for every callback, in order to supply the handles struture to the callback.
When you have something like the handles struct, then every call to getappdata() takes a copy of the struct. This suggests that it is often not a good idea to store large data in the handles struct; this would not be a problem if you were going to use the large data in every callback but most callbacks use only a small number of handle fields. Thus, if you have large data that you would normally consider storing in the handles struct, it is usually better to use setappdata() / getappdata() for it, so that you can take the hit of copying in the data only when you specifically need it.
For your purposes, you should probably be considering either using shared variables or else using handle objects: handle objects act like pointers, not needing to copy in the contents of the object.