When I download the zip file do I have to upload each and every single file separately to my server one at a time?

Yes, you will need to upload all files to the server for a new install. For an upgrade you'd upload all but the cache_builders/custom, styles, and includes folders (though you'd also upload styles/common.css).

Most FTP clients will allow you to just select the files to upload and move them so they all upload one after another, without having to do them one at a time.

When I click on my file manager to upload to my publichtml I can only upload three at a time and there appear to be thousands of files. I KNOW this is operator error/stupidity.

This would be your webhosts control panel; you should use an FTP client.