It is not possible to use Esri Addins, regardless of whether you are using Python Addins or .NET Addins, to extend a custom application. You can only use Esri Addins to extend ArcGIS for Desktop applications (ArcMap, ArcCatalog, ArcScene, ArcGlobe).
The error you are receiving is likely the result of your target name and/or version value not being a legal entry in your Addin's config.xml.
As indicated by the ArcObjects SDK 10 .NET Framework document on building addins for ArcGIS Desktop:
- The Targets element specifies the most compatible version and product of ArcGIS for the add-in. The Add-in installation utility will install the add-in only to the product and version specified in the Targets element. At present, the only legal values for the name and version attributes are Desktop and 10.0, respectively.
This is true of the name attribute even in the latest version of the ArcObjects .NET SDK (10.4) - the only legal value is 'Desktop' although you can use different versions now. This means you can only build Addins for ArcGIS Desktop applications. They cannot be used to extend custom applications, even if those applications were built using ArcEngine or the .NET Runtime SDK.
The developer likely included the capability to extend the toolbar using built-in or custom tools with the intent of allowing a user who had ArcGIS for Desktop installed to add tools installed with ArcGIS for Desktop to the custom application you are trying to extend.
For example, if you installed ArcGIS for Desktop on your VM, you would get access to a whole bunch of tools that are built into Desktop applications like ArcMap. These are referred to as 'Built-In' tools and you can add them to an addin by referencing their ProgID. Esri's Jason Pardy demonstrated this in a blog post detailing how to do this with a Python Addin but it works with .NET Addins as well. Next, if you developed a custom Addin, using .NET or Python, and specified the target as Desktop, and in your case specifically built the addin for ArcMap, that addin would install to ArcGIS for Desktop and be available within the ArcMap application. From there, you'd likely be able to add that custom Addin tool to the application.
Again, it all hinges on you having ArcGIS for Desktop installed though, since that is the core dependency of all Esri Addins.