# What is math behind entropy calculation in ArcGIS Voronoi diagram?

ArcGIS for Desktop 10.3.1, Geospatial Analyst Toolbar, Voronoi Diagrams

How is the entropy value calculated? What is the math?

I have gone through ESRI's E-learning exercise. The exercise has a data set that contains temperature for various location.

``````temp 1, at (x1,y1)
temp 2, at (x2,y2)
temp 3, at (x3,y3)
etc.
``````

One temperature reading is an outlier. I can see the 'variation' in the Voronoi diagram with the outlier (70 deg), but if I change the outlier data to be similar to the cells nearby (20 deg C), I cannot understand the diagram. I have similarly changed the values of other temperature readings, and cannot interpret the Voronoi results.

With the semivariogram, once I found out the equation, it was very simple to understand the result. Hoping for the same with the entropy Voronoi method.

I found this link, but can't quite figure out how I do the calculations. How to find Entropy in land use (Voronoi polygons)?

I have attached the two Voronoi diagrams and the data behind the one with the outlier.  The "entropy" is an attempt to characterize the distribution of values within each Voronoi cell and its neighbors. The ESRI calculation makes some simplifications (which are statistically dubious, but this is what it does):

All cells are placed into five classes based on a natural grouping of data values.

Apparently this is done at the outset for the entire dataset. Think of painting the Voronoi cells with five colors.

Focus now on a particular cell. When it has n neighbors (and n varies from one cell to the next), the relative frequency of occurrence of each class counts the number of times it appears among the cell and its neighbors, divided by n. This gives five relative frequencies, one for each class, and those five sum to unity. By definition, the entropy of any set of non-negative numbers adding up to 1 is obtained by multiplying each nonzero number by its logarithm and adding those up.

Which base of logs to use doesn't really matter, so long as you are consistent. The ESRI calculation uses base 1/2. (That's the same as taking the base-2 log and negating the result. Changing to any other base merely multiplies all your entropies by a constant value.)

Let's do some examples. Consider a cell with just three neighbors, so there are n = four cells total. There are only five possible distinct configurations of the relative frequencies. To indicate the frequencies, I will write the number of times each class appears in braces.

• All four cells have the same class: {4}. There is just one nonzero relative frequency, 4/4 = 1. Its logarithm (to any base) is zero, so 1*log(1) = 0 is the entropy.

• One class appears in three cells and another class is in the fourth: {3,1}. The two nonzero relative frequencies are 3/4 and 1/4. The entropy is 3/4 * log(3/4) + 1/4 * log(1/4). This is approximately 0.811278.

• One class appears in two cells and another class appears in the other two cells: {2,2}. The two nonzero relative frequencies are 2/4 and 2/4. The entropy is 2/4 * log(2/4) + 2/4 * log(2/4) = 2/4 * 1 + 2/4 * 1 = 1.

• One class appears in two cells and two other classes appear in the remaining two cells: {2,1,1}. The three nonzero relative frequencies are 2/4, 1/4, and 1/4. The entropy is 2/4 * log(2/4) + 1/4 * log(1/4) + 1/4 * log(1/4) = 2/4 * 1 + 1/4 * 2 + 1/4 * 2 = 3/2. (Why do we know log(1/4) = 2? It's simple: the base of our logarithms is 1/2, and 1/2^2 = 1/4. By definition, the logarithm is this power, 2, to which you must raise 1/2 to get 1/4.)

• All four cells are in four distinct classes: {1,1,1,1}. The entropy is 1/4 * log(1/4) + 1/4 * log(1/4) + 1/4 * log(1/4) + 1/4 * log(1/4) = 4(1/4 * 2) = 2.

We can reverse-engineer your output to understand it better:

• All the entropies of 0 correspond to situations where the cell and all its neighbors fall into a single class. The relative frequency is n/n = 1 and its entropy is 1*log(1) = 0. This is the only way you can get an entropy of 0.

• 0.986427 is an approximation to 7/9*log(7/9) + 1/9*log(1/9) + 1/9*log(1/9). That suggests this is occurring at cells with eight neighbors, seven of them are in one class, and the other two cells are in two other distinct classes: {7,1,1} is the notation.

• 0.591673 approximates 6/7*log(6/7) + 1/7*log(1/7). This most likely is the configuration {6,1} (although, just possibly, it could be {12,2} for instance).

• 0.954434 approximates the entropy for {5,3}: 5/8*log(5/8) + 3/8*log(3/8).

And so on. Doing some of these computations will help give you a feel for what different values of these entropies mean.

All the equations about entropy is located in Esri Help page about Voronoi maps calculations, and it says:

``````Entropy = - Σ (p i * Log p i ),
``````

where `p i` is the proportion of cells that are assigned to each class. For more details, please have a look at the help page above.

• I have read that page and tried to actually follow the logic, but could not figure it out. So I have 431 rows of data, do I still make five classes? Is distance between cells not a factor in the calculation? Is it simply how many polygons touch the center polygon? Also, in this equation – am95405 Mar 10 '17 at 7:11
• You requested the equation and I showed you the source page. Now if your question is about statistics, I think you need to ask this question in different way or different SE site, or someone with more statistical background can help you. – ahmadhanb Mar 10 '17 at 7:18
• Thank you for the equation. Modifying my question, does the entropy equation take into account distance between cells? – am95405 Mar 10 '17 at 7:24
• Thank you for your help. I figured out the calculations. I will post later the response in case others have the same question. – am95405 Mar 10 '17 at 15:18
• I can't add attachments, and I did the calculations in Excel, so email me if you want the Excel file. The key point is that the data needs to get classified based on the smart quantiles or geometric classification. – am95405 Mar 24 '17 at 2:31