What is the best possible way to create a cloud mask of a Landsat 8 scene without using band 9? I think the best way will be to determine the cloud cover by thresholding in the Panchromatic band and verify the output with the Thermal band (clouds are cold...) - but I am open for any other ways.
There is actually no need to develop a cloud mask from scratch (unless you feel the absolute need of doing so). If you download the Landsat 8 surface reflectance data, there would be a quality file associated with it, which contains several cloud-related bands. If you are not happy with them, you can modify them based on your needs. It would be a much better starting point than trying to develop one completely by yourself.
You can try using QA band. The leftmost 4 bits of the 16 bits indicate cloud and cirrus cloud presence.
00 -> Not determined 01 -> No (0 to 33% probability) 10 -> Maybe (34 to 66% probability) 11 -> Yes (67 - 100% probability)