Its a bit tricky to do but you get your result.
First select your CSV and create a join to your shapefile. You will get all results but without geometry.
To fix that you can now create a new virtual field in your CSV.
Open your attribute table and start the field calculator. Choose
unlimited length (text), as field name choose
wkt_geom ideally and enter the following code:
point2 is my shapefile layer; replace it with the name of your shapefile layer.
id is the matching field in my shapefile layer; replace it with your ID in your shapefile layer.
attribute($currentfeature,'id') is the matching field in my CSV layer; replace it with yours.
After you've done this, save your CSV as a new CSV. (I'm not aware of a method to change geometry source of a already added CSV file; please correct me if there is one)
Now you can import your new CSV with geometry. Do whatever you want with it, e.g. save it as shapefile or something else.