If it's a VPS you should have root access; the exact error should be in your web logs when an error occurs, it should mention some security issue and note the variable that's causing the problem; it's how I've found it before anyhow.
You could just do as I mentioned and have the ubb in a folder that isn't 777 and have the admin folder chmodded 755 and everything should run (the only thing that HAS to be 777 or 666 is if the content in them will be edited from the control panel (such as a language file or template) or will interract with the system (like attachments, galleries, includes folders).
Last edited by Gizmo; 10/20/2017 6:39 PM.