This sounds about right - the cache will build up, then something can cause it to be cleared out. (Many control panel actions can clear part or all of the cache.)
In general, your board will need about 150% more disk space than it uses with an empty cache, at most.