Is there a way (e.g. package) to perform an unsupervised classification using the ISODATA clustering algorithm in R?
There is a GitHub page on ISODATA clustering in R that may be worth a check.
I believe that closest you are going to come is the unsuperClass function in the RStoolbox package (http://bleutner.github.io/RStoolbox/rstbx-docu/unsuperClass.html). This function implements a k-means and not ISOCLUSER but, it is an unsupervised approach that is, honestly, preferable.