I have a random forest model that works pretty well, taking a bunch of vanilla remote sensing raster data as input. I think it could be improved with addition of some information that I currently have stored as categorical variables (for example: geological substrate, landform, etc.). Because my RF model is built using sklearn, I need to encode these categorical variables numerically.
I've been convinced after reading this and this that one-hot encoding is a bad idea with tree-based methods because it can reduce a single important feature into lots and lots of less important features--bloating the model and reducing performance at the same time. Plus it sounds like a nightmare to add 20+ new raster bands to my image stack. But I'm struggling with which of the alternative encoding techniques to use.
Label Encoding: I understand the critique that this introduces the "dog > cat > horse" problem, but it seems like decision trees are designed such that this isn't actually a problem.
Hashing Trick: As far as I can tell, this is functionally equivalent to assigning largish random numbers to each class. It's not really clear to me how this doesn't suffer from the same ordinality problem as label encoding, but the internet seems to think it's great.
My question is this: If I have a moderate number of categorical variables (less than 40), what is the best way to encode them specifically for use in a random forest?
---EDIT--- I made a vocabulary error in the original post. I am not trying to include a large number of categorical variables/features. Instead, I am trying to include a very small number of categorical variables/features (for example: lithology, landforms), each of which has a small-to-moderate number of classes/levels within my ROI (e.g. "sandstone, granite, limestone"). So the "less that 40" statement above applies to the number of classes/levels within each of a very small number of variables/features, rather than to variables per se. So that is what I am trying to figure out how to encode.
