The simple answer is that you need to export the geometry in the correct projection for your needs. If you're using meters then you need to use a projected CRS that uses meters as its unit (i.e. UTM...). If you're using FT then you need to use a projected CRS that uses FT as the unit — and naturally a CRS that is in the vicinity of your data.
In QGIS I would recommend setting the PROJECT CRS to the projection you need and use Project -> Import/Export -> Export project to DXF
then be sure only the layer(s) you need is(are) checked. And most importantly that the correct CRS is chosen (it should be the project CRS). Symbology can be tricky as you need to define a good scale but straight geometry — that is, the geometry data and not the QGIS generated symbology — is transferred without issue. If you're exporting MText be warned that it is hardcoded and can't be easily manipulated once in CAD. A workaround for this is a simple edit (find and replace) of the ASCII DXF before importing into CAD.
When importing into CAD (I usually insert it as a block) you need to be sure that your drawing is based on either unspecified units or either Meters or Feet — not millimeters or inches! Then when you insert it define its insertion point as 0,0. It will then lay in the cartesian plane (yes, it is cartesian) relative to the CRS's origin, which will be CAD's world origin. And that means that it could be very, very far away from CAD's world origin. Zoom to extents to find it and then leave it where it is to work on it.
This method makes the back and forth between CAD and QGIS relatively painless.
NOTE: I've had issues using a Layer's [right click] "Save As..." for the DXF operation but never using the project export. All other Layer "Save As..." functions seem to work well for CAD (i.e. clipping a raster for a CAD underlay — make sure the correct CRS is chosen!)