I have a button created using Python Add-Ins which pops up a dialog box which is built using wxpython. I am wondering if I can place a tool (Select Layer by Location) within that dialog box to get location coordinates of layer (basically select that layer). Once a layer is selected, it will fire back some values to the already open dialog box. Any word of advice on whether it is possible and how to create a class for tool would be very helpful here.
No, you cannot. A GP Tool dialog is not an embeddable wx control.
I think the key here is the custom dialog box, if you're writing your own you could put all kinds of stuff on it. I've not done any python addins but have done a great many in C# that do all sorts of useful things.
The python form is a wxDialog http://www.wxpython.org/ which looks to operate in a similar manner to QT for QGIS.