Yes, you're looking at the times indicated. Normally they should be well under a hundredth of a second. If there is a specific query that is taking up a chunk of the time then that would help pinpoint it.
If you turn on debugging in the config file then it will also give you a total, for how long the page took to generate and how much of that was taken up by mysql. That would help to figure out if it's mysql or the webserver causing the slowdown.
No table changes were made between 6.3.2 and 6.4. Same design structure is intact.
Enabling zlib compression might be something you can try. It will tax your server a little bit more but it will make the page sizes smaller as it will compress the page before delivering it to the browser. You'll want to turn on the debugging in the config file before trying this so you can note any time changes.
