In QGIS, besides the regular plugins, there are so-called core-plugins that can't be de-installed. So from a user point of view it's clear what is the difference: you don't have to install (just activate) them. However, less clear to me is what from a technical point of view is the difference: what is the advantage of implementing some of the plugins as core plugins?
The big difference is that the Core Plugins are "are maintained by the QGIS Development Team and are automatically part of every QGIS distribution"
External Plugins are developed and maintained by external developers.
Another difference between them is that for the Core Plugins you will have an Official QGIS.org documentation, whilst is not produced for the external plugins.
More info in the official QGIS.org doc
Not a direct answer to your question, more an addition to answer about the way plugins are loaded.
You may look at this current QEP (QGIS Enhancement Proposals) "QGIS Enhancement Proposal: Turning Plugin Management into Actual Package Management" to better understand how plugins are loaded and the languages involved as there is a state of the current plugin management implementation https://github.com/qgist/pluginmanager-qep/blob/master/QEP.md#analysis-of-current-implementation