Based on Eric's answer, this is an approach using the SQLite command instead of csvkit.
If you're not already starting from a GeoPackage, convert to it first:
ogr2ogr -f GPKG myjoinshp.gpkg myjoinshp.shp
Then, exploiting the fact that a GeoPackage is actually an SQLite database, import the CSV as a new table (named
myjoincsv), which modifies the existing GeoPackage:
sqlite3 myjoinshp.gpkg --cmd '.mode csv' '.import myjoincsv.csv myjoincsv'
This step could also be done in a GUI, e.g. DB Browser for SQLite supports importing a CSV into the loaded database by drag and drop.
Finally, make a new GeoPackage or shapefile, using a join, as in Eric's answer:
ogr2ogr -f GPKG joined_output.gpkg myjoinshp.gpkg -sql "SELECT shp.*, csv.* FROM myjoinshp shp JOIN myjoincsv csv ON shp.joinfield = csv.joinfield"