How does ArcGIS determine which transformation to use when projecting on the fly? Where this would come in would be going from NAD 83 to WGS 84. Multiple transformations could be considered so how is one picked?
Note/Disclaimer: I work for Esri.
Prior to ArcGIS for Desktop 10.1 service pack 1 (10.1.1), we loaded one geographic/datum transformation in ArcMap: NAD_1927_To_NAD_1983_NADCON (using the conus grid files). For 10.1.1, that was removed. All other transformations must be user-selected using the Transformations dialog. You can launch this dialog from the data frame properties, coordinate system tab or from the geographic coordinate system warning dialog.
The transformations dialog will show all unique geographic coordinate reference systems. Starting at version 10.1, the list of transformations is sorted by comparing the map's area of use and the transformations' areas of use plus their accuracies. It's still very easy to 'fool' the tool though! If you define a custom transformation, it is sorted to the top of the list because we don't have any area of use/accuracy information for it.
Transformations are stored in a look-up table (SDK: GeoTransformationOperationSet) per pair of geographic coordinate systems. It's not possible to have a transformation active for the US and a different one for Canada, for instance.
There are some 'default' transformations occurring with web service layers IF you don't specify something else.
According to this
(ArcMap automatically loads one geographic transformation. It's designed for the lower 48 states of the United States and converts between NAD 1927 and NAD 1983.)
For datum transformations, you need to pick the transform yourself. This article goes into NAD83 <-> WGS84 choices. Projection changes can be done on the fly automatically because the projection doesn't need a transformation.
The first layer added defines the data frame’s coordinate system. This is true whether the data is projected or geographic. For example, if the first layer added contains a Lambert Conformal Conic projected coordinate system, all other layers will project on the fly to match this. Similarly, if the first layer added to the data frame contains data that uses a WGS84 geographic coordinate system, all other layers will adjust to match this. Even data that uses a projected coordinate system will unproject on the fly.