Hillary Duff is proof some girls don’t lose the baby weight quickly