When double clicked on tools of arctoolbox, no dialog box is opening, for the "buffer" or "intersect", or for any of the tools in the ArcToolBox.
I faced the same problem in both ArcGIS version 10.5 and 10.6.
How can I solve this?
I would recommend deleting your default map template, called Normal.mxt. What can often happen (especially if you're regularly switching displays on a laptop/extra screen/projector) is the tools open, but they open off-screen. This location that they open in is saved in your map template, and is not easy to fix within ArcMap.
If you're in Windows 10, close ArcMap/Catalog, and navigate to:
There should be a file called Normal.mxt in that folder. Delete it, or move it to a backup location. When you restart ArcMap, a new version of the file is created, which should reset the UI to factory default, and tools should open in the main ArcMap window.
A more thorough walk-through of the process and other fundamentals on template/UI customization is found here on ESRI's site.