You can use (Calculate Polygon Main Angle)
First you need to add numerical field to your shapefile (call it e.g angle)
Then open the (Calculate Polygon Main Angle) and define you dataset, and the angle field.
Then, select by attribute, select when Angle = 0,90,-90
"Angle" = 0 OR "Angle" = 90 OR "Angle" =-90
And the result:
PS: this method in not totally accurate because the computed angle (The dominant angle of a polygon is the angle of longest collection of segments that have similar orientation. This angle will be stored in the specified field in decimal degrees from true north)[ARCGIS].