To do this, you need a separate raster where each group has a unique value (ie, each cell within a group shares the same value and no other cells have that value). If you do not have this already, you can create it with the region group tool.
One thing I noticed from your screenshot is that there are some pixels that are not part of any of the large groups. Each of these pixels will become their own regions in the output. Likewise, any of the groups that are not completely contiguous will be broken into multiple regions.
Once you are happy with your region group raster, use the zonal statistics tool to calculate the mean of each region group. The raster you showed above will be your value raster input and your region group raster will be your zone raster input. Statistics type should be mean (or whatever other statistics you want to calculate for your groups).