I recommend reading Russell Congalton's book Assessing the Accuracy of Remotely Sensed Data: Principles and Practices for a comprehensive analysis of the subject.
Looking at your accuracy assessment, I see two red flags suggesting that the accuracy of the classified product is poor. First, an overall accuracy of 0.4 can be interpreted as saying 6 times out of 10, the class will not correspond to reference data. Second, the Kappa Index of Agreement (KIA) of 0.17 indicates (very) poor agreement between the classified results and reference data
To fully answer your question regarding thresholds for accuracy assessments, Congalton's book references work from Landis and Koch (1977) stating the following (Note that KHAT is comparable to KIA):
Landis and Koch (1977) characterized the possible ranges for KHAT into
three groupings: a value greater than 0.80 (i.e., 80%) represents
strong agreement; a value between 0.40 and 0.80 (i.e., 40–80%)
represents moderate agreement; and a value below 0.40 (i.e., 40%)
represents poor agreement.
Congalton, R. G., & Green, K. (2008). Assessing the accuracy of remotely sensed data: principles and practices. CRC press.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, 159-174.