How do I delete an individual label without converting the entire data frame into annotation?
You can't really control (or delete) an individual label, as they're dynamic. The best you can do is work with the placement properties in either the Standard or Maplex engine to try and get them where you want. In this case, those won't really help. There are a couple of solutions, one of which is the label classes GISKid mentions. But this requires data modification in some way.
Another option that doesn't is to create a label suppression mask. Either with the Draw tools to create a graphic shape, or create a new feature class and draw a shape around the area you don't want labels to appear. Then change the symbology of the mask to no outline and no fill, or make it 100% transparent. With the correct labeling settings (ie conflict resolution between labels and your suppression layer and/or graphic - this should work by default without changing anything), labels will not be drawn under the mask shape because it's a conflict, but you won't see the shape and will see everything under it.
In the above example I use a graphic. If you have a lot of these you might be better off with a masking feature class. The square is visible because I have it selected, but if it weren't you wouldn't see anything. I won't get into label weights and such, which you may need to read about if you go the feature class route. Reading through all of the Labeling Help section might be of value as well.