blah238's very nice answer to this question reminded me of all the time I've spent over the past two years battling against the dreaded geoprocessor memory leak in ArcGIS Desktop.
I only have personal experience of problems with version 9.3.1, but there are numerous threads on the ESRI support pages (e.g. here, here) which suggest that the problem goes back to at least v9.1. blah238's answer and this thread indicate that the problem still isn't fixed at v10.
I know that workarounds are possible, usually involving the use of subprocesses to force the gp to release the memory it's hogging, but they're clunky and often very slow.
This question isn't meant as an attack on ESRI, but I'd be really interested to know:
Why has this problem proved so difficult to fix?
For my work at least, it's a major issue and one that's pushed me away from using ArcGIS for spatial modelling. This has been a great learning experience, but I'm surprised that fixing the problem isn't higher on ESRI's priority list.