Here are relevant excerpts from pages 8 and 9 of the Shapefile Technical Description (emphasis mine):
A polygon consists of one or more rings. A ring is a connected sequence of four or more points that form a closed, non-self-intersecting loop.
And:
So, as far as the shapefile specification is concerned, the answer is no, polygons may not intersect themselves.
This is a conceptual rule, not a limitation of the file format. It is quite possible to construct shapefiles containing self-intersecting polygons. However, since self-intersecting geometry is invalid according to the specification, applications may differ in how they handle such cases: some may attempt to clean up the features, while others may [justifiably, if less helpfully] consider the shapefile malformed. Most problematic may be cases where the software assumes a shapefile conforms to the specification (ie, contains no self-intersections) but does not validate it before performing geometric operations, which may yield wrong results.
It's unclear exactly what you want to do, but if you're generating shapefiles, I'd advise against outputting polygons with self-intersections, as it may cause difficulty for your data users. Likewise, if you're writing some sort of algorithm to process shapefiles, you do not need to support self-intersecting polygons to comply with the specification - but if possible, I'd advise you check the input files are not malformed before proceeding.