within a python application a uge PostGIS geometry table (Polygon) is polled for certain processing. Yesterday we have set our testing system back to an earlier snapshot and failed to notice that this application still used the table on our testing system. So we have lost some million records of much more million records. No we want to visualize the remaining 26 million records to evaluate the 'damage'. We tried PostGIS Layer in QGis (no chance), export to shape or GML (no chance, 28GB GML file), export to GTiff (aborted), we tried to union to reduce number of geometries (query is still running after 2 days) ... Is there a method out there to 'quick' visualize such an amount of data? Our testing system is running on a server so we can add additional processors and RAM if needed.

Any help is welcome.

  • 1
    It takes time to select 26 million features. Could you do with a well defined sample, like every 1000th feature at small scale? – user30184 Jan 19 '17 at 9:47
  • Probably the easiest and most straight forward method. Why didn`t I have this idea? thanks! – Olle Onkel Jan 19 '17 at 12:40
  • There's 1M pixels in a 1000x1000 screen window. You have 26x that. Even if you've lost half your data you might well not visually be able to see the difference, depending on the pattern in which it was gathered in the first place. – Paul Ramsey Jan 19 '17 at 13:16
  • 1
    A very fast way to do this is CREATE VIEW my_view AS SELECT * FROM my_table TABLESAMPLE SYSTEM (sample_fraction) or, better yet, grab just ST_Envelope(geom) instead of *. – dbaston Jan 19 '17 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.