Ok, I think I have this figured out. I was tracking parent posts afterall, but there was a bit of a bug in several places so it was just setting everything to 0.