Agree that csv might cause problems. Another hint would be to check if your polygon layer has no topological inconsistencies (invalid geometries, i.e. duplicate consecutive points, self intersections, ...) You can check this under Vector > Topology Checker (under Plugins, active the Topology Checker Plugin). This shows you in the map where possible topo issues occur.
For further reading about repairing invalid geometries i.e. cp.
Fixing geometry validity errors in QGIS?
It's not totally clear to me, whether you only want to select (or extract) the points WITHIN your polygon, or if you want to calculate statistics for the points within your polygons (counting them would be such statistics). In the first case you can use Select by location or Extract by location algorithms in the processing toolbox, in the latter case you can use algorithms like GRASS 7 v.vect.stats (cp. Populating polygon with most frequent point attribute using QGIS?) or SAGA point statistics for polygons (each found in the processing toolbox). I observed, that QGIS Join attributes by location algorithm produces weird results in some situations (cp. QGIS 'join attributes by location' algorithm produces strange results for min, max), but this might have been fixed meanwhile.
Here is a screenshot where to find the algorithms, in each you can select 'Save to file', where you can even choose csv format: