1

Im using QGIS 3.4.2 to create a heatmap to plot the source place where a network message is generated, in a vehicular network.
I separated the messages in two groups, one in each layer: delivered and not delivered.
So far is working 100%.

Now I want to combine both layers in order to generate a single heatmap, which must show the % rate of delivery combined.
For instance, if in a determined region 4 messages were generated, and only one of them could be delivered, the % rate should indicate 25%...

Currently I'm using two separated and very basic CSV files, one for delivered messages and another for not_delivered messages:

Longitude,Latitude
-5483420,-2938848
-5490938,-2938382
-5480800,-2953976

But I can easily merge them and create a single CSV similar to this:

Longitude,Latitude,Delivered
-5483420,-2938848,Y
-5490938,-2938382,N
-5480800,-2953976,N

I cannot find a single example online.

  • Welcome to GIS SE! Your source data, the messages, are point features with the delivered / not delivered value in an attribute, or two rasters with the values in their pixels, or any other type? I cannot figure the source data type from the question. – Gabriel De Luca Nov 19 at 1:59
  • Hi Gabriel, Currently i'm using two separated and very basic csv files, one for delivered messages and another for not_delivered messages: Longitude,Latitude -5483420,-2938848 -5490938,-2938382 -5480800,-2953976 But i can easily merge them and create a single csv similar to this: Longitude,Latitude,Delivered -5483420,-2938848,Y -5490938,-2938382,N -5480800,-2953976,N Thank you very much – Rafael Cordeiro Nov 19 at 3:23
  • Hi Rafael, you are welcome. Now I see that you have a CSV layer with a list of features, each representing a message, with an attribute Y or N according to its delivery status. At an intermediate step, you must calculate, for a given region or point, how many messages were delivered and how many were not delivered. That is, I think you will group them spatially in some way. In what way do you plan to group them in order to determine the percentage of delivery within each group? – Gabriel De Luca Nov 19 at 3:40
  • Im not sure if i want to group the messages per region.... My idea is to let the heatmap do its work, coloring regions according to their delivery rate. But regions are not predefined, they are dynamically formed according to their delivery rate. the heat map would then show, a scale ... for example ... 0-20% - Red 21-40% - Orange 42-60% - Yellow 61-80% - Green 81-100% - Blue – Rafael Cordeiro Nov 19 at 16:57
  • I think I can understand it. It seems to me that two heatmaps could be done with the same parameters, one that shows the density of delivered messages and another that shows the total message density. Then divide the values of one raster over the other, as long as the total message density is greater than zero. In the next few days I could create a set of test data and try it. – Gabriel De Luca Nov 19 at 19:57
1

Your coordinates do not look like latitudes and longitudes. I assumed they are X and Y coordinates projected in Web Mercator. So I started a new project in QGIS, loaded the OpenStreet Maps basemap and checked those coordinates. I saw that they were from Curitiba.

I started looking for open geospatial data from Curitiba to be able to use real information in this answer. I found the following geographic data site: https://ippuc.org.br/geodownloads/geo.htm

From there I downloaded the file of transport terminals, the data are referred to the SIRGAS 2000 system and projected in UTM 22S: https://ippuc.org.br/geodownloads/SHAPES_SIRGAS/TERMINAL_DE_TRANSPORTE_SIRGAS.zip

Looking at the table of attributes, it occurred to me that I could assign Y value to those entities that had inauguration data prior to 1983, and N value to the others. I called the field: "older_than_1983".

1

I removed other fields and extracted two vector layers, one with all the stations, and another with only the old ones.

2

At this moment I have some test data to try to make a heatmap of proportion, which indicates: In what proportion, the density of transport stations in an area of Curitiba was inaugurated before 1983, with respect to the total density of transport stations for that sector. The radius to consider was 5000m.


The first thing I did was generate two heatmaps, with the HeatMap (Kernel Density Estimation) tool, one for each vector layer. Both with the same following parameters:

3

The resulting values in the rasters were not exactly the same, as expected, but were approximately in the range between 0 and 4 (terminals in 5000m round). I gave the same style to both raster:

4

In order to obtain the ratio between the values of the layers, I performed a raster algebra operation, with the Raster Calculator, and the following expression:
("Terminales_Total_HeatMap@1" > 0) * 100 * "Terminales_Yes_HeatMap@1" / "Terminales_Total_HeatMap@1"

5

The logic of that expression I would describe in the following way: When the pixel value of the band 1 of the "Terminals_Total" layer is greater than zero, the condition between parentheses returns a 1, indicating true, if the condition results false, returns a 0; to the result of that condition, multiply it by the percentage of the proportion between the pixel values of the "Terminals_Yes" layer and the "Terminals_Total" layer.

Surprisingly, the output raster had values between 0 and 440:

6

That is, in some places there were 4 times terminals older than 1983, with respect to the total. This is obviously false. Seeing the image, it is difficult to find which are the pixels with high values (they should be white). I suppose that since the layers of origin have a different extension, some estimate at the edges of one raster already gave a value close to 5 while in the other raster it still gave a value close to 1.

Anyway, one more algebraic operation could be done to get rid of values greater than 100. I leave you restless. I solved it only for the visualization, in the style of the layer, stretching the color ramp between 0 and 100:

7


Let's analyze a little the problem of percentages greater than 100.

At first I thought that the problem was due to an incorrect alignment of the pixels between both input rasters, because they have different lengths. However, when I reviewed the raster information, I saw that both had their pixels in multiple coordinates of 10 meters, so they were correctly aligned.

I decided to make a new algebra to know which were the pixels that gave a percentage greater than 100. I renamed the heatmap of percentages as: Percentage_Greater_than_100_Heatmap. Ok, assigning good names to layers seems not to be among my virtues. But I can deal with that.

I made a new raster algebra with the following expression:

("Percentage_Greater_than_100_Heatmap@1" > 100)

The output is a raster that contains value 1 for the source pixels that have a value greater than 100, and zero value for the others.

I added an assignment of the value 0 as NoData in the Output layer style to see only the pixels of value 1:

8

The location of the black pixels is totally strange to me, I expected to see only a few points.
I selected any black pixel with the identify tool, to understand what the wrong value was due to:

9

Indeed, the value of the "Yes" heatmap is greater than the "Total" heatmap value, against the logic.

I consider it an indeterminacy of the estimation of the algorithm that generates the heatmaps. That is, the term "estimation" makes sense in the name of the algorithm (HeatMap (Kernel Density Estimation)).

I think those pixels should be worth 100%. I correct them with the following expression in the raster calculator:

("Percentage_Greater_than_100_Heatmap@1" > 100) * 100 +
("Percentage_Greater_than_100_Heatmap@1" <= 100) * "Percentage_Greater_than_100_Heatmap@1"

This expression returns value 100 for pixels that meet the condition of having a value greater than 100 at the input, and retains the value of the input pixel for others.

The output is a raster that has values greater than zero and less than or equal to 100.

  • Wow, thanks for all the explanation, you put a lot of effort on this. I understand almost all that you did, and did the same by myself to practice, but my output show values above 100 (that is the part i didnt understand). My main question is.... this output should be in a % scale, so, in theory, all raster values ​​should be between 0 and 100, right? Can i check this values by any means? Also, just confirming, this heat map generated would only show the message delivery rate, it does not show the locations with the highest absolute amount of messages, correct? – Rafael Cordeiro Nov 22 at 0:20
  • And, last question... Stretching the color ramp have any other implications? Is it right to do this to force the map to show the right colors? Or should I necessarily perform algebraic operations to eliminate values ​​above 100? – Rafael Cordeiro Nov 22 at 0:20
  • @RafaelCordeiro Hi, about the "greater than 100%" problem: If both source heatmaps don't have the same extent, their pixels are not exactly one over the other. They partially overlaps. In that case the calculator assumes the nearest neighbour to perform the operation and I think that is causing the error. – Gabriel De Luca Nov 22 at 0:51
  • You can check the values in many ways: From the identify tool, pixel by pixel. Or you can do a new algebra: ("Output" > 100). If the value is greater than 100, returns 1. If not, 0. And you will see in the new output what are the conflictive pixels. But I will recover the project and try to avoid that problem using the same extent for both heatmaps. I did not realize to register the value of the extent of the "Total" heatmap, and write the same extent when performing the heatmap Yes. – Gabriel De Luca Nov 22 at 0:56
  • Yes, it is only showing the delivery rate. The one that shows the highest absolute amount is the first heatmap generated ("Total"). – Gabriel De Luca Nov 22 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.