So far i have done lot of changes on the process side
In my experience, the users I see with slow performance, nearly always have a huge room for improvement at the service design end.
This video is the best I have seen on best practise for best practise around your map design at the MXD level.
Lets assume your services are as optimal as you can get them.
the best hardware configuration that will provide fast map services.
This is too hard to answer without having a better understanding of your setup, your users requirements, how many users you have, are they power-users or casual, how many services will they need, will there be peak times, are their network bottlenecks with bandwidth, etc etc etc.
I would start off by using the Capacity Planning Tool to start plotting in your existing infrastructure/requirements. This takes into account nearly every factor and will give you a traffic light diagram (Excel spreadsheet) on if your existing hardware will cope, or if you need to get additional SOC machines to spread the load or bump the hardware up on those machines.
Are you looking at doing this on a Virtual Environment? Virtual does have an overhead.
The capacity planning tool can factor this in, but please also consult this whitepaper.
I generally find looking at some of Esri's hardware partners useful, to gain an understanding on the sort of hardware that Esri recommends you putting ArcGIS Server on.
Take a look at Dell:
A Dell PowerEdge R610 Server Including:
- One (1) Quad Core Intel® Xeon® X5687 3.60 GHz Processor, 12 MB Cache or Two (2) Quad Core Intel Xeon X5687 3.60 GHz Processors, 12 MB Cache
- 12 GB RAM (3 x 4 GB Dual Ranked LV RDIMMs) for Four Core Server or 24 GB RAM (6 x 4 GB Dual Ranked LV RDIMMs) for Eight Core Server
- 2 x 146 GB 15000 rpm Serial Attached SCSI (SAS) Disk Drives with RAID 1
- PERC H700 Integrated RAID Controller 512 MB Cache
- 2 x 502W Power Supplies for Four Core Server and 2 x 717W Power Supplies for Eight Core Server
- Windows® Server 2008 R2, Standard x64 with 5 CALs
Other vendors available, and you do get discounts through buying ArcGIS Server with these bundles. I just use them to get an idea of the recommended hardware in more detail than the web-help.