@johns put his finger on the biggest culprit, your polygons are being rasterized. Check to see if you have any layers in the Table of Contents above the polygon layer which are:
- have transparency set
- or are using bitmap symbology (picture fills)
If any of these are true, move your vector layer above them.
After that, ways you can further improve PDF and print export are:
Summary: best quality, jpeg 85%, vectorize markers
Plus: PDF layers only (no attributes) if you wish readers to have ability to toggle layers on/off, or no layers if you need smaller file size.
The export Output Image Quality slider (normal, normal + 1, best) affects jpeg artifacting around transparent colour fills
Jpeg quality slider has little observable effect above 95% while the file size jumps 25% at 100 quality. Sweet spot is 85-95%.
Converting bitmap markers/fills to vectors instead of raster is a no-brainer. The raster markers are horrible no matter what quality is chosen on the first option page.
Fills and transparencies will always look better in Arcmap than in the exported pdf. Text and fine lines on the other hand can be darker and crisper in Adobe Reader.
The PostScript Printer engine is the only one which maintains crisp and smooth curves on the text. All others have little bumps and dips, which won't make much difference in hardcopy but are visible at 200% zoom.
Top left: Normal+2 quality, jpg 85%, vector markers
Top right: Normal+1 quality, jpg 90%, vector markers
Bottom left: Normal+1 quality, jpg 85%, vector markers
Bottom right: Normal+1 quality, jpg 85%, raster markers
The jaggedness seen in the rose coloured buffer is because a transparency effect is being applied to that layer. Remove the effect, remove the jaggies.
Missing: the effect "convert marker symbols to polygons" has.