I found one reason and it relates to when ArcMap crashes.
ArcMap writes to the AddInFoldersList.xml on shutdown. If ArcMap crashes on exit while writing to the AddInFoldersList.xml and results in an incomplete XML file, no AddIns will be present the next time ArcMap is started.
Then if ArcMap is shutdown again(successfully), then the default xml file is written:
<?xml version="1.0"?>
<ESRI.ArcGIS.AddInFolders ver="1"><ARCMAP/></ESRI.ArcGIS.AddInFolders>
While testing, I also noticed that having 2 ArcMap instances can potentially affect the file because it will be the last ArcMap instance's AddIn configuration that will live on to live another day.
So if you add a bunch of Addin folders to ArcMap #1, shut down ArcMap. The go to ArcMap #2, do nothing and shutdown, you'll end up with whatever AddInFolderList.xml ArcMap #2 sucked in when it started.