Step 1 : set the processing extent. Open Raster calculator go to the environment tab and set the processing extent equivalent to the bigger raster. Then use one of the following options.
OPTION 1 [taken from user FelixIP's comments]
Con(isnull (RasterB),RasterA) This answer assumes that raster A is bigger than raster b, as stated by OP. I believe this is the most straight forward answer so I've put mine as option 2 below. I'd go with this option.
This is assuming both your rasters contain only positive integers. if that is not the case and they include 0 change > into >=, if they both include signed integers use the minimum of the rasters in each case.
Con(((rasterA > 0) & (raster B >0)), -9999, rasterA)
Let's say you named the output for the above as RasterC
Open a new raster calculator and use SetNull to set all the values where rasterB overlapped with RasterA to Null (in the above operation they were set to -9999)
SetNull(RasterC, RasterC, "Value = -9999) <- Output RasterD
Your RasterD should have all the areas covered by Raster A but not Raster B. If you don't mind the -9999 value you can stop at Raster C.