I recommend QGIS as a credible alternative to ArcGIS. There are others too such as SAGA GIS, which is also very popular. Both of these packages are free and contain functions like ArcGIS. You could also use PostGIS or Spatialite (free spatially enabled databases) but for this solution I will limit myself to QGIS.
To join your CSV to your SHP in QGIS, load the SHP layer and then use the
Add Delimited Text Tool (button icon is a blue inverted comma) to load the CSV data. If it does not contain coordinates then make sure you check that box otherwise the OK button will remain greyed out. Next double-click the shapfile layer to open its properties dialog box. Select the
Joins tab and then click the green plus button at the bottom to join the CSV data to the SHP.
You can also perform the join with a scripted solution using OGR. So basically, part of this answer is to say that you have a plethora of alternatives to commercial GIS solutions like ArcGIS.
If you are of a delicate disposition do not read below here:
There is another method you could use to manually perform the join. This is not the recommended approach. It is a nasty hack but it would work. You could open the DBF file of your shapefile in Excel or OpenOffice Calc (other spreadsheets are available), sort the data by your join field. Do the same for your CSV and then cut and paste the CSV into the DBF. You may have to edit the field names to avoid breaking the DBF (depending on your package - stripping out the information on field type usually does the trick where you can see it). This is a hack and requires care and a bit of knowlegde of how your favourite spreadsheet package handles DBF files (OO Calc does it a bit differently to Excel for instance). Purists will not like this approach and I only mention it in case you need a fix to get you out of a hole.