Probably a caching issue. Were you using IE6 or IE7 to make the change? Give Firefox a try or maybe a CTRL-F5 in IE to see if it's changed.
IE has automatic cache enabled by default to allow faster browsing. You can enable it to check the page every time which may help overcome this issue.