At first, you should distinguish "georeferenced rasters" from "non georeferenced raster".
The second group includes all the raw image acquisitions from remote sensing (aerial photographs, satellite images, UAV images) or scanners. Those raw images will be captured according to the sensor orientation, which is rarely North up. They are affected by geometric distortions and the coordinates of the pixels are relative to the sensor.
In the first group, the coordinates of the pixels relate to an absolute position on the ground in a given coordinate system. Most coordinate systems are North up (according to Wikipedia, this convention was established by Ptolemy), so you will find North up coordinate systems if you work at low latitude. However, north up coordinate systems are not good at the poles, so polar coordinates systems are not North up (see an example here). Once you have a coordiante system, you need an affine transform based on one corner coordinate (most of the time the upper left corner, but not always, it can depend on the software), the size in X and Y, and sometimes a rotation. The rotation is rare because it often increases the processing times, but it can help to avoid increasing the size of a raster by unnecessary NoData pixels, so you can still have some "non north up" rasters with a "north up" coordinate system.