You must increase either the page size, the export resolution, or both.
To change the page size right-click the layout --> go to Page Properties --> select or define a larger page dimension in the Item Properties panel (screen-right by default).
The export resolution may be changed at the time of (image) export, or within the Layout panel under Export Settings.
QGIS 3.2 (older versions as well) fetches images from a tile server at the time of screen render as well as at export/print. Screen-space tiles are fetched for the layout preview while page-space tiles are fetched at the time of export. Therefor, the layout preview will not reflect the higher-zoom map.
There are upper bounds to the exportable image size and QGIS will warn you if the operation will result in a memory overflow. Additionally, the process of fetching a large number of tiles can take a long time.
Below are the results of an OSM tile map preview and export of an Arch E @ 450dpi.