I know that we could set "Define classes of features and label each class differently" for a point feature class but I prefer to label the features in an annotation layer. We have to set a reference scale when we create an annotation layer. My question is that how to fix the problem that the labels do not scale up or down very well?
You can have multiple annotation feature classes with different reference scales.
When you create your annotation, set up a label, set your scale, and create Annotation. It will create that annotation at that scale. Your next annotation you set the label and your new scale, and create annotation again. The second annotation will be created at that new scale.
Annotation are essentially features linked to your spatial features. They are not labels. As a result the features appear to get bigger or smaller as you zoom in or out.
As they are features, in a layer in your Table of Contents, you can set a reference scale on the layer to make it visible or not visible at specified scales. You can combine this with a second Annotation layer at another scale.
I set the reference scale for the 1:25000 Annotation to not show when zoomed out past 37500
And I set the reference scale for the 1:50000 Annotation to not show when zoomed in past 37500