If you know the exact position of the receiver then it is possible to do timing with only one satellite. Essentially GPS is solving four unknowns (ECEF position, and receiver clock error) with four (pseudo-range) distance measurements, so if you can provide three unknowns, you only need one measurement.
Not all receivers support this though - it was a standard feature in timing receivers the last time I did a survey, several years ago. If your receiver (or integrated implementation) does not support it, you will need at least four satellites (three for the ECEF unknowns, and one for the receiver clock error).
The biggest contributions to timing error are the same as the contributions to distance errors (since its the same measurement set) - multipath and ionospheric variation. A good antenna, good antenna placement and increasing the masking angle will all help to reduce error.
More GPS measurements (including knowledge of your antenna location, which is really what you are measuring with GPS) can give you an over-determined solution, which can reduce error using least-squares fit. However at this level, you need to work out how much you are really gaining in output (Allen variance), and whether you see time steps when you swap between satellites - that can mess up the integration.