I would expect the option "filter legend by map content" to cause the "filter count" to be limited to the features within the map extent. But you're right, the "feature count" is not filtered by the map extent.
This seems like a bug, so I made a bug report. Visit the bug report page to check on the status of having this bug fixed, but don't expect it to happen immediately. This is a pretty minor issue and the QGIS devs have a lot on their plates. In the meantime, try out the workaround below.
How to use expressions in a label item to display a count of features within the map extent:
This expression returns the map extent as a polygon:
map_get(item_variables( 'Map1'),'map_extent'), where
Map1 is the Id of the map. You have to define the Id of the map item in the Item properties panel.
aggregate(count) function to count the points in a given category that intersect the map extent. The following example will count the points in the map canvas that have the value 'Group A' in their "category" field.
aggregate( 'New scratch layer','count',"category","category"='Group A' AND intersects($geometry,map_get(item_variables( 'Map1'),'map_extent')))
Repeat for each category.
Note: If you want a simple (uncategorized) count of all the features on your map, use this expression:
aggregate('New scratch layer','count',"category",intersects($geometry,map_get(item_variables( 'Map1'),'map_extent')))