I'm a software developer. In my world, when you bundle up a set of scripts or binaries to be run on another machine, your goal is usually to publish it to multiple environments. Sometimes the intention is that someone else will use your scripts or binaries inside of an application (libraries), and sometimes you're actually trying to publish an application. In the former case, you generally do all you can to make your tool as environment agnostic as possible, allowing users to pass in information about what resources to use (files, databases, web services). In the latter, you give the user some ability to configure the application so that it can connect to the correct resources. Making an application configurable like this still has uses even when you're only giving it to one client. In particular, it makes it simpler to have multiple testing copies of the application before you actually push to production. (E.g., one for developers to experiment on without bothering anybody, one for internal QA, and of course production.)
When I first heard about service definition files (.sd files) for geoprocessing services, that's what I thought they were supposed to do: let you bundle your code up so you could deploy it to different environments. However, I have found that when you use them, there are zero options for configuring anything at publish time. (See Specify geoprocessing service connection at publish and Prevent parameter defaults in service definition file). This means that a service definition is locked to a single environment, and you have to create a new one for each environment. As a result, I cannot see any way in which service definitions are useful for geoprocessing services since you must recreate them in each environment you want to deploy to.
What is their intended purpose?
What problem do they solve?
When should I be using them?