I think I have it figured out now.
I picked up a flash hole uniformer and a COL gauge.

Flash Hole Uniformer:
The Winchester brass was pretty clean but the Hornady brass was very jagged around the flash hole.

COL Gauge:
When I measured my chamber I was surprised to see the measurement was 3.440" .2" beyond the recommended COL for the load. Even if I backed off of this measurement by a couple thousandths it would have left very little bullet left to seat in the brass. Is this common? I expected a few hundredths to the lands, not .2"

I decided to seat the bullet to 3.340" -- the Hornady manual max COL listed for the 30-06 -- and still .1" off the lands.

My best 4 shot group was .5" using a 168 grain Amax in front of 48 grains of Varget.
49 grains of varget would have been a good group (<3/4") but I forgot to take a fouling shot first first thing in the morning

I will now work up a few loads using hunting bullets, knowing that I need to seat them long.