I have a Google Maps compatible tileset running on a local server and I can browse this in a browser with the Google Maps API and OpenLayers. Can I force Google Earth to use this tileset so I can navigate it in 3D or will it be incompatible? If so, where do I set the function or string that takes a zoom,row,column index and turns that into a URL? Is there some KML format somewhere?
Unfortunately your existing tileset is not compatible with Google Earth, as it's been rendered in a different projection. There are some hacks to sort of force the tiles onto Earth, but it never looks great, especially near the poles.
Your best bet is to render a different set of tiles for 2d browsers (often using 'spherical mercator' projection that is epsg:3785) and 3d browsers (generally the basic 'unprojected' or cylindrical projection that is epsg:4326).
MapTiler / gdal2tiles does all this for you nicely if you have the source rasters still available.
I think what you're in the market is a "reflector" script that works with NetworkLink KML to automagically pull tiles in a fixed directory structure with a fixed filename convention and overlay them correctly.
Good news, it's do-able. Bad news, you have to roll up your sleeves a bit.
All of the math you need to figure out the bounding box for any given tile is on this page--
From there, it's a matter of setting up your NetworkLink structure and getting the tag to trigger the pulling of appropriate tiles as the user moves around.