Setting fill to a variable based on data within SpatialPolygonsDataFrame?

Question

I'm currently on the cusp of getting done what I want to get done - but having a slight problem with referring to my data from within R's sp SpatialPolygonDataFrame object. The issue is how ggplot2 deals with spatial polygons, and factors that define or are mapped to them. I have my polygons, and in the @data element, I have the relevant data I'm interested in mapping/visualising, in a SpatialPolygonDataFrame called plotData. The following code;

p = ggplot(plotData, aes(x = long, y = lat, group = id))

p + geom_polygon(fill=plotData@data$total,color='black')

yields an error about the number of elements within the aesthetic call:

Error: Aesthetics must be either length 1 or the same as the data (1651526): colour, fill

I've tried a number of different ways of accessing these data - but am not able to actually figure out how to colour it based on anything within the @data element. The difference is something to do with how ggplot2 treats polygons - (my data has only 2208 unique elements, and 2208 unique polys, not 1.6 million), but I can't seem to identify how this is put together.

Answer 1

Okidokey.

So - it turns out that even though ggmap2 can manage SpatialDataFrames, assigning values to it is pretty tricky. The issue I was having is a mismatch between each element of the polygon (so for a simple rectangular poly, there'd be 4 points), and the values associated with it. Fortifying the data as suggested by @Spacedman yields a basic data frame type (or using broom::tidy as suggested by internal package details), with groups or ID's assigned as expected.

What needs to happen is tidy() is called on the SpatialDataFrame - after which the data is joined using left.join() from base R. The concern is largely that this creates a great deal of redundant data - one polygon or statistical area needs only one record of the variable of interest - but the way the aes() function in ggplot2 works requires that each row has this value. Essentially for a polygon with 100 'corners', the same value needs to be stored 100 times.

After creating the SpatialPolygonDataFrame (which will soon if not already be deprecated by the sf package... but that's another story), it is fortified with broom::tidy as follows;

spdf = SpatialPolygonsDataFrame(input variables)
tidy_spdf = tidy(spdf)
plotData = left_join(tidy_spdf,df_with_variables,by='id')

Then there is some simple renaming of columns in plotData for the sake of clarity, and the following line yields (a basic) version of what I wanted, where 'wmHrs' is the variable of interest.

p = ggplot() # Simply initialise an empty plot.
p + geom_polygon(data=plotData2, aes(fill=wmHrs,x=long,y=lat,group=Suburb)) # populate it!

Again - this is a solution, but perhaps not the cleanest one. It works, but from a programming perspective seems needlessly heavy on memory, particularly when your analysis grows! I'm marking this as the answer, and hopefully can inform others who are having the same stumbling blocks when starting in this sort of environment. Also - as @Spacedman rightly pointed out this linked question was greatly informative - once you start figuring out how things are done (rather than asking why!).