I owned my Wrangler X with the hard top and full hard doors for 18.5 years and 278K miles without taking any of them off. It's final act of devotion was saving my life when I was rear-ended at high speed on the Turnpike just east of Pecos. The frame, body, seat belt, doors, hard top and roll bar all worked in conjunction during the collision and post-impact dynamics such that I was able walk away with only minor assistance. I ended-up with a broken collar bone and aggravated ventral hernia, but it easily could have been a fatality.
Use a hard top and full hard doors.