First off, I have not tried using Config.esriaddinx for this purpose, but I wouldn't recommend it. It is meant for the configuration of the add-in itself, not necessarily user data, and you probably don't want to mix the two.
It has been a while since I've dealt with this myself so I may be a little fuzzy on the details, but there are multiple issues with using configuration files in ArcGIS add-ins: ESRI ArcMap Add-in with app.settings not recognizing app.config changes
In particular, the directory the add-in is extracted to is overwritten each time the application is started, so you can't really persist changes to settings there. If your settings never change or only change with each new version of your add-in then this is probably not a concern.
If, however, you want to make your add-in configurable by the end user then you need to store the user-configurable information elsewhere so they aren't overwritten. I would suggest using the user's Application Data folder, whose path you can determine programmatically like so:
I would also suggest putting it in a subfolder named after your add-in. But essentially you would be loading from and saving to a file in that location instead of reading settings from your add-in's
Settings class or the configuration file in the add-in's directory. If you want to use .NET configuration for this then I suggest reading up on Application Settings and
The other problem I have had is with using custom configuration sections when using .NET configuration. Using
Assembly.LoadFrom and handling the
AssemblyResolve event was the solution to that particular problem, although in that case I ended up not using .NET configuration for that and other reasons.
Depending on the complexity of your configuration scenario, you could, like I did, eschew using the .NET configuration system entirely and instead use some other method of reading and writing configuration information. I ended up using
SerializableAttribute-marked classes or classes which implemented
IXmlSerializable for this purpose in one of the more complex add-ins I created which included user-configurable settings such as lists of layers, connection settings, etc. I would recommend reading Object Serialization in .NET, Introducing XML Serialization and How to Implement IXmlSerializable Correctly if you are interested in that approach.
It sounds like yours is along the same lines so it is up to you, but I found the XML serialization approach to be preferable to .NET configuration for all but the simplest of configuration scenarios (simple data types, no hierarchies/collections).