2

I didn't know which method to use cross-validation or bootstrap in maxent so I ran both and my AUC values for each method are quite different for bootstrap I have an AUC of 0.758 and for cross validation I have an AUC of 0.737. I have quite a large dataset 3000 + occurrence points, and have tested different combinations of feature classes and regularisation multipliers to find that the best for my data is LQHPT and RM of 1.5.

Can anyone explain why using each of these methods results in such a difference in AUC values and which one would be the best to use?

Improve this question

1 Answer 1

Reset to default
4

Cross validation splits your training data into a number of groups. Maxent then calibrates a model on a number of those groups and tests it on the the groups left out (say 2/3rd's of the folds as calibration and 1/3rd as validation).

Bootstrapping is where a new sample is selected from your calibration data by randomly selecting observations WITH replacement. This means each bootstrap sample may have repeated values and is tested on the observations not selected.

They are just two different methods for estimating the uncertainty in your evaluation of predictive power of your model. Given you have a lot of occurrence observations cross validation is probably a reliable method as bootstrapping is often preferred for small sample sizes (where folds in CV are very small).

Also, the differences between their estimates are actually very similar and are telling you they have essentially the same discriminatory power.

Improve this answer
1
  • 2
    I would point out that you can use a bootstrap in a cross validation and they are not, in fact, two different methods. The concept of CV is to derive a performance statistic by comparing a model to data that was not included in fitting said model. This can be done iteratively, via bootstrap, jackknife, etc..., to evaluate an error distribution by using different realizations of the data in refitting the model. Splitting in to groups for a cross validation is an n-fold CV.
    – Jeffrey Evans
    Jul 6, 2020 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.