What QGIS is referring to in terms of the layer boundary is actually the envelope or bounding box of the geometry.
I've documented a solution which is probably the closest you will get to automating this is (short of filing a bug / feature request).
The process works based on the ratio between area of the two objects: the feature geometry and it's bounding box).
Check the following steps below:
Select the layer you want to work on.
Select the feature you wish to work on. Make sure nothing else is selected.
Run the following snippet within the Python Console.
layer = qgis.utils.iface.activeLayer()
features = layer.selectedFeatures()
feature = features
geom = feature.geometry()
env_rect = geom.boundingBox()
env_geom = QgsGeometry.fromRect(env_rect)
env_geom.area() / geom.area()
Be sure to push enter as the last line will not automatically return.
Regular Points tool again, but enter the number of points that you want inside the box multiplied by the output given by your script.
In my sample below, the result I received from the output was a value of ~2. I requested 20 dots to be output by the process where I actually only wanted 10. I was given 8 in return, which is fairly close. Depending on the actual shape you're using the results will vary.
If you were so inclined, you could automate (recursively) the Regular Points process modifying the variables until you arrived at your desired number.