The normal worry is the database tables. Just in case any of the upgrades with altertables mess them up, so it should be common practice to do a database backup before hand.
It's also handy to backup your includes, styles, and cache_builders directories. You're never supposed to overwrite these unless specifically told to do so, but some people have on accident. So backing up these directories is a good safeguard as well.
