Complex Shapefile Compression Advice

Question

I'm looking for some advice on how I can simplify this very detailed shapefile to ultimately be used for the web (as topojson) as a interactive heatmap. It was originally created from a raster file (geotiff - http://ecotope.org/anthromes/v2/data/). I believe it's going to take a little more massaging before using a Douglas-Peucker or Visvalingam algorithm. I believe the best way to achieve this would be to intelligently merge features based on the attribute value each contains. I'm just not sure the best way to achieve this. I have a sneaking suspicion that "dissolve" is what I need, but QGIS just freezes anytime I try that.

I'm working with command-line gdal utils and QGIS.

I have attached some screenshots below to give you a better idea of what I'm dealing with.

Answer 1

That would be a huge dissolve. You could try first simplifying on the raster version, then converting that to vector and doing further simplification. http://docs.qgis.org/2.6/en/docs/training_manual/rasters/terrain_analysis.html#moderate-fa-simplifying-the-raster

A couple ways to simplify the vectors via gdal: in ogr2ogr you can use the -simplify # command on any data, and the -lco COORDINATE_PRECISION=# command for converting to geojson

Answer 2

Update

I'd just like to expand on my solution a bit here for anyone who may be having a similar issue. @neuhausr was dead on. I realized that setting the threshold to a low number, say 10 or 50 wasn't quite enough, while a higher number would compress it too much while leaving some areas untouched. Strangely enough, I found that if you compress it with the same low threshold over and over again it will actually have a much more consistent result. Basically, giving the file multiple passes. So I wrote a little bash script to help me out and I am getting great results despite how hacky it may be. Tweak the COMPRESSION and THRESHOLD to your liking.

    #!/bin/bash

    # Which raster to compress.
    ORG_FILE=./raw/anthromes/1700/anthro2_a1700.tif

    # Where to output the new file.
    TMP_DIR=./tmp

    # Total amount of compression that should be done.
    COMPRESSION=800

    # Threshold for each iteration.
    THRESHOLD=50

    # Process...
    rm -rf $TMP_DIR
    mkdir -p $TMP_DIR
    gdal_sieve.py -st 50 -4 $ORG_FILE $TMP_DIR/output-"$THRESHOLD".tiff
    _CUR=$THRESHOLD
    while [ $_CUR -le $COMPRESSION ]; do
        let _PREV=$_CUR
        let _CUR=$_CUR+$THRESHOLD
        echo "Compressing output-$_PREV.tiff into $_CUR.tiff"
        gdal_sieve.py -st $THRESHOLD -4 "$TMP_DIR/output-$_PREV.tiff" \
            "$TMP_DIR/output-$_CUR.tiff"
    done

Results below:

Original (no compression):

Basic Sieve with 2100 thresold:

Using bash script 21 passes of threshold 100 (2100):