That sounds like it's losing gases during the baking then. Shorten the rise time and get it in a hot oven to solidify that crust. I presume you're brushing the crust with milk?