I am doing some research and trying to understand how Mapbox compares to GeoServer (or any GIS server software), and whether or not they are used together or if using one precludes using the other. Can you/should you load services stored in GeoServer or similar in a Mapbox map? If you would not use them together, why would you use one vs the other. I'm trying to understand the main differences - it seems to be that Mapbox is more than just an API, perhaps a hosted mapping platform, but how does it interact with GIS servers/databases?
It depends a bit on what your goal is, with Geoserver (and other mapservers) you are in control of your data, as you provide the services. They do not cost anything (beside the infrastructure). But you have to care about the underlying server (hardware or platform as a service and software, security and so on), probably need an Apache/Ils/Nginx whatever configured, need to care about client side visualisation (JScript/Openlayers, PHP or others, as well as Symbology).
With services like MapBox you have less work to do but need to pay for it (beside the free version with several restrictions). There you have lots of stuff preconfigured.
That is in general the difference. For small / fast deployment a service like MapBox is probably nice (never used one though). If it gets bigger those services tend to become expensive, but you need some know-how and time to run services on your own.
And of course you can use them in parallel(if i understand you correct on this point). Its just the underlying data that needs to be yours and you can put them online in several ways.