I am trying to run a PCA on a set of normalized raster bands within ArcGIS. I would like to calculate the PCA on the correlation, not the covariance, matrix of the PCA, but Arc as far as I can tell only calculates PCs on the covariance matrix of the input bands. It seems the only fix to this is to feed the tool pre-normalized band values in the form of a z-score. I z-scored all of the input raster bands before feeding them into the Principle Components tool in Arc 10.1. The function is completing nicely, but the output is producing bands with mean values of anywhere between 9 and 34, even though they should be producing bands with a mean of 0, due to the nature of a PCA. What am I missing? Has anyone has experience with this issue before?

As requested, , here is a snapshot of my study site with a zscore raster overlaid: you can't see it well: enter image description here

Here is a zoom-in to a New Jersey site showing PCA bands: enter image description here

Finally, here is a screenshot of the first 2 PCA band output properties:enter image description here

  • Could you tell us what the output is supposed to represent? Normally the output of a principal components analysis will include the components, their eigenvalues, and ancillary information useful for interpreting the result, none of which would be raster bands. The only thing that seems to make sense would be that these "bands" represent the principal components themselves, but normally these are standardized to unit L^2 length and typically do not have zero means. – whuber Oct 24 '13 at 21:27
  • As I understand PCAs calculated on correlation (not covariance, as Arc does unless you Z-score the values before they are input) matrices, their mean should be 0 since they are standardized around the axis and no longer include information about units, only about variation around the trend line. The value of the correlation matrix PC indicates how far away from the axis of variation the value is, but these values should all mean to 0. I believe the first PC indicates amount of water present within tidal marsh - the images I am processing are of tidal marsh along the coast. Thanks for the help! – Mo Correll Oct 25 '13 at 2:38
  • You might want to do some reading about PCA on the stats site. – whuber Oct 25 '13 at 2:41
  • 1
    After fiddling with R for several hours, I am going to embarrass myself and ask if you have any code examples for this. I have exported each individual raster band into an ASCII file from Arc, and have tried to upload them into arc using b1=raster(file.choose()), then concatenating, then running them through a PCA, however the error "error is is.finite: default method not implemented for type "list" comes up. I have tried converting the loaded rasters into arrays and matrices using as.array and as. matrix, but it overloads R, several error messages appear saying the data files are too large. – Mo Correll Oct 27 '13 at 22:14
  • 2
    Triumph! I subset my data by extracting z-score band values to a bunch of random points, then ran those values through princomp using princomp(SSU, cor=F) to simulate what should be happening within Arc. All of the output PC scores mean to 0. So, this tells me there is either a problem with the NAs in the raster in Arc, or something else is going wrong. Thoughts? I can't thank you enough for batting these ideas around with me, I appreciate it. – Mo Correll Oct 28 '13 at 0:58

A quick test with a pair of random rasters is informative. (I love using random rasters because they are so useful for experimenting with and testing procedures.) The large ones I generated had means very close to the theoretical ideal of 1/2 and standard deviations very close to the ideal of 1/sqrt(12) for uniform distributions. Because these rasters were supposed to be independent, the principal components would be multiples of their sum and difference, suitably normalized.

After standardization (that is, converting them to their "z scores"), these two random rasters had means of zero, unit standard deviations, and extreme values close to sqrt(3) = 1.732051 in size. Their sums and differences should therefore have a mean of zero and SD of 1. Their extremes should be about sqrt(12/2) = sqrt(6) = 2.45 from the mean, for a total range of almost 4.90.

Instead, the two components of the output had means near 2.45, standard deviations of 1, and extremes from 0 to 4.87 (similar to the patterns shown in the question, but the question has larger means, maxima, and SDs). These incorrect means, correct SDs, correct ranges, and exactly zero minima strongly suggest that the bands have been additively shifted to make all their values nonnegative. (Asking for just one principal component does not change this: the additive shift still occurs.)

The workaround, then, is to subtract its mean from each of the PCA bands to shift it back to a zero mean. As a check, the standard deviation of each band should equal the square root of its eigenvalue reported in the PCA output file. In my test (because I computed only approximate z scores) the eigenvalues were 1.00047 and 0.99932. Sure enough, the standard deviations reported in the Layer Properties dialog are 1.000234515307069 and 0.9996583939019468, respectively, and their squares agree perfectly with the eigenvalues (to the limited precision reported).

A quick hunt through the ArcGIS 10.1 help pages produced no information at all about the PCA output. If it cannot be found, we have to rely on tests like this to inform our understanding and hope we are correct.

  • this post made me look for my old university books. – nickves Oct 28 '13 at 21:56
  • @nickves I would recommend instead searching StackExchange :-). – whuber Oct 28 '13 at 22:16
  • whuber- I compared the SDs of my subsetted PCs (produced in R) with the full PCs calculated in Arc, and they are VERY similar, which makes me agree with you in that subtracting the means from the PCs produced in Arc is a reliable work around. I am compiling the final files right now. Thank you very much for the help on this! A final question - for those having a similar issue with PCA in Arc, is there anywhere else I should post this besides here, and in ArcGIS 10.1 help? I would love to save those having similar confusion the time it takes to figure it out. Thanks again! – Mo Correll Oct 29 '13 at 12:41
  • You could look on the ESRI support forums. – whuber Oct 29 '13 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.