There is a right triangle: the plane is at one vertex (A), the center of the earth is at another (O), and the most distant visible point on the horizon is the third (B), where the right angle occurs.

That point on the horizon is about 6,378,140 meters = 20.9362 million feet from the center of the earth (the earth's radius)--that's one leg--and you are between 25,000 and 41,000 feet feet further from the center--that's the hypotenuse. A little trigonometry does the rest. Specifically, let R be the earth's radius (in feet) and h be your altitude. Then the angle from the horizontal down to the horizon (alpha) equals
Angle = ArcCos(R / R+h).
Note that this is purely a geometric solution; it is not the line of sight angle! (The earth's atmosphere refracts the light rays.)
For R = 20.9362 million feet and heights in 1000's of feet between 25000 and 41000 I obtain the following angles (in degrees) with this formula:
2.8, 2.85, 2.91, 2.96, 3.01, 3.07, 3.12, 3.17, 3.21, 3.26, 3.31, 3.36, 3.4, 3.45, 3.49, 3.54, 3.58
You could just linearly interpolate within this interval if you prefer, using a formula like
Angle = 1.5924 + 0.048892 (h / 1000)
for heights h in feet. The result will typically be good to 0.01 degree (except at the extremes of 25,000 and 41,000 feet, where it's off almost 0.02 degrees). E.g., with h = 33,293 feet, the angle should be around 1.5924 + 0.048892 * (33.293) = 3.22 degrees. (The correct value is 3.23 degrees.)
For all heights less than 300 miles, an acceptably accurate approximation (i.e., to 0.05 degree or better) is to compute
Angle = Sqrt(1 - (R / (R + h))^2).
This is in radians; convert it to degrees by multiplying by 180/pi = 57.296.
The ellipsoidal flattening of the earth won't make much difference. Because the flattening is only about 1/300, that ought to introduce only around 0.01 degree of error or so in these results.