Try to just use ESRI's API, just check to see the extent of the map and move it back if its outside of the bounding box.
Try adding this:
var previousExtent = map.extent.getExtent();
let currentExtent = map.extent.getExtent();
// Update previous extent
So, assuming you have a point (geocoded coordinates of building address).
Now you want to get the building polygon.
There are several problems with this.
First, nominatim/OSM is crap in geocoding, so you have to use the google geocoding API, which is not free anymore. Alternatively you could use the yandex geocoding API.
Second, google ...
Google Maps Platform has extensive documentation on using GeoJSON data.
Although it is not a Google map, you could try looking into the Leaflet package which has support for GeoJSON objects. You can then build the appropriate AJAX calls around it to get the data you need.
If you just want to look at the data visually (and are not really building a web ...