Unfortunately, it is not possible to "reload" the Python add-in while having the ArcMap session open. This is because the add-in is loaded into the program cache. So, the only way to see the changes done is to close ArcMap, install a new add-in and then start ArcMap (as you approach it now).
You should spend most of your time debugging and developing the add-in in your IDE, not in ArcMap. There are just some GUI components you could test in ArcMap; all the code can be tested fairly well within the IDE without involving the ArcMap.
To automate the process of ArcMap restart, I've used to execute an os command directly from the my IDE (I use Wing IDE). The command will close an ArcMap session, run the makeaddin.py and then start ArcMap session. Provided that you have a decent machine, the whole operation shouldn't take more than a couple of seconds (on mine it is about 5 sec or so). If ArcMap starts slowly, consider speeding it up by working with a new template or disabling external extensions and modules.