How do you have your vote prefences setup? Is it where all users can vote or only registered users? If all users can vote then it tracks via IP and if your IP changes then you can vote more than once.
The options are rather confusing though asthe Allow Poll Creation gives the Administrators & Moderators and the All Users options, and the Allow Poll Voting gives the All Users and Registered Members options, which makes it confusing as to who the All Users are in each one.
Not sure if this was reported already, but if the poll is in a board, that has a specific group requirement to write access - it will allow votting to all users, no matter which group they are members of.