You can think of ArcSDE as a spatial data abstraction library used by the GeoDatabase. It has both a client-side library and a server-side component. It stays in between your ArcGIS and Oracle software.
The client-side code always runs, well, in-proc to the client. It is shipped/built-in with all the ArcGIS software you may run - no separate installation required.
The server side portion has different ways of running. It can run as a separate process (i.e as a service) in whatever machine you want to run it (e.g same machine as Oracle, a third machine, whatever), or it can run in-proc to the ArcGIS software, this is known as direct connect.
If you want to understand a bit more about the GeoDatabase and ArcSDE, I would recommend reading a previous question/answer in this site titled "What is the are ESRI GeoDatabases?".