I'd second Mapperz suggestion of the Revo-Unistaller, as with the defunct COM Explorer, the freeware version will get the job done. But the Pro version gets routinely updated and while it will work in trial mode, it is such a useful tool you should probably purchase it to have available. http://www.revouninstaller.com/download-professional-version.php
If the 3rd party extensions with ArcGIS version dependencies that you are interested in removing laid down registry entries correctly you should be able to track the components manually in the registry edits.
Do a registry backup first and then start by disabling any orphaned services from the extensions.
I'd start looking the HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall
You want the Product Code CSLIDs listed as Product Code or in the UnistallString, and make a note of any other associated CSLIDs
If the package install was MSI based try to uninstall with the indicated msiexec /x for the Product Code's CSLID. If the installer is intact--everything should come off cleanly.
If extension was not MSI based, see if installer executable is intact and when executed has an uninstall option and run it.
Unfortunately, if either installer is corrupted (missing components from the extension or from the ArcGIS uninstall/upgrade) you'll only achieve a partial removal and with the list of CSLIDs you'll need to chase things down manually in the registry and the file system and doing deletions as you go.
The Revo-Unistaller reliably automates most of this.