The important difference is the way interop information is stored; either in your assembly, or in an external PIA. Because ArcGIS already provides PIA's, you should always set 'Embed Interop Types' to false for ArcGIS solutions.
The following paragraphs are taken from http://resources.arcgis.com/en/help/arcobjects-net/conceptualhelp/index.html#//000100000nzv000000
Using the Embed Interop Types property on referenced assemblies
Embed Interop Types is a new feature of the .NET 4.0 Framework that allows you to include the attribute information that is normally stored in the Primary Interop Assembly (PIA) in the executable or dynamic-link library (DLL) instead. This was introduced because deployment of managed solutions built on certain COM-based technologies was difficult because the deployment and verification of the PIA was also done on the target machine.
By default, when you add a PIA to a Visual Studio 2010 .NET 4.0 project without using the Add ArcGIS Reference plug-in, the PIA's Embed Interop Types property is set to true. The best practice is to change this setting to false when deploying customizations for ArcGIS, because the ArcGIS installer lays down the appropriate PIAs and assemblies on the system; you should not include any with your solution.