Question: How do you take a heatmap generated using a GIS software (arcgis, qgis...) and convert it into a uniform grid array of points which can be used to regenerate the heatmap (although with lesser accuracy)?

Initial to final product:

(Initial)Millions of Datapoints in MySQL -> (... some processing, maybe GIS software) -> (Final)Hundreds of Datapoints in a grid array -> Render density map in browser

How do I get from Millions of Datapoints in MySQL to Hundreds of Datapoints in a grid array?

enter image description here

Purpose: Transferring millions of points from database to browser where the heatmap is created will cause heavy database load and network traffic and strain the browser. So instead of transferring millions of points to the browser, it is more optimal to simplify the millions of points to 100s of points in a grid arrangement which will be easier for the browser to handle.

  • What data type is your heatmap of? Is it a raster or a polygon? – Ryan Garnett Mar 22 '13 at 17:05
  • The heatmaps (density map and choroplet) has not been created yet. It will be created from a dataset of lat/lng pair and its value. – Nyxynyx Mar 22 '13 at 17:11
  • 2
    Maybe I am misunderstanding the question, but I thought you were asking about how to make it into an array of points. All the heat based maps I have made have been rasters. I was asking if your outputs will be the same (rasters). If this is the case, you can do a raster to point conversion. Because they are rasters, the generated points will be in a uniform array. – Ryan Garnett Mar 22 '13 at 17:30
  • Sorry I was confused about the process. Updateed the quesotin. If the raster was to be converted to a uniform array of points (without being too dense, that will defeat the purpose), is it still possible to reconstruct a heatmap somewhat similar to the original? – Nyxynyx Mar 22 '13 at 17:56
  • 2
    What about doing something like a dissolve to a polygon approach? You will have to pick a tolerance for the dissolve (ie: integer, one decimal point, etc.) – Ryan Garnett Mar 22 '13 at 18:02

The way that I have approached such problems in sampling across a landscape is to define a uniform grid using the 'create fishnet' tool in ArcMap. Once you have a established a grid you can perform a spatial join and average (or min/max/var) the point data plus you can get a count that you can use as the parameter by which to establish a heatmap using graduated symbology. The polygon grid can be converted into a raster file but I will leave that part out for now. What is the extent of the area you are working with? You ought to be able to establish a average distance between points and then create a grid area based upon a desired range of density.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.