You need the application to test and debug. When you open the .Net project on a machine without licensed ArcGIS installed, all the references to ArcObjects will be missing. You can probably edit the code itself and save the project but you will not be able to compile or debug. Basically, to create an ArcGIS addin you need ArcObjects libraries which are a part of ArcGIS as well as the appropriate licensing. (some ArcObjects require the advanced license, some standard but most only the Basic) To test and debug you need the application since the debugger runs simultaneously along with the application. You may also want to use the .Net SDK which contains the API that helps you to develop the Desktop Addin among other things.
I am not sure how much the Home Use ArcGIS license is now but ESRI used to offer this for $100 / year for strictly educational (positively non-commercial) use. It is my understanding the Home Use License is an Advanced License with all and full functionality including all extensions. Since only the free Express version of Visual Studio is needed to build AddIns, this would allow you to try and learn building/debugging addins. http://www.esri.com/software/arcgis/arcgis-for-home