4

Is there a way to have the QGIS attribute table viewer/editor not load the entire dataset when opening? I'm working with a shapefile of 100,000 grid references and this is rendering the application unusable.

I have 12GB RAM. Experiencing this behaviour on both Windows 7 and OSX.

Running latest QGIS stable version 2.18

3

See the available options under Options -> Data Sources -> Feature attributes and table. You can choose to only show the currently selected features or the ones in the current map extents.

  • This could solve the issue however if I happen to be zoomed to the full extent of the map or have not selected everything, then the application freezes while loading all the attributes (very very slowly). The abort button is non-functional. – Alasdair Nov 16 '16 at 17:16
  • Shape file is on my local hard drive, an SSD. – Alasdair Nov 16 '16 at 17:18
  • This is perfect to keep qgis from crashing when you have a huge dataset loaded. Just make sure you don't query the table when zoomed all the way out. – jamierob Aug 9 '17 at 17:42
3

If you only want to load (say) the first 1000 rows to look at the data, you can apply a SQL query using DB Manager (tested in 2.18). This works even for shapefiles, and is more flexible than applying Filters.

  • open up Database > DB Manager
  • go to Virtual Layers and choose your shapefile with lots of rows
  • click on the 'sql query' button

apply a SQL query such as

select * from my_layer limit 1000;

or for a random sample of 1 in 1000 rows..

select * from newyork where randf(0.0,1.0)>0.999;

you can choose to load this in as a new layer:-

enter image description here

Alternatively, as you have lots of RAM, looks at Settings > Options > Data Sources, you can increase the size of the Attribute Table Row Cache. It won't speed up loading initially, but should speed up subsequent openings of the attribute table.

2

Shapefiles are not the most efficient format for storing data with many features and attributes: http://resources.esri.com/help/9.3/ArcGISDesktop/com/Gp_ToolRef/geoprocessing_tool_reference/geoprocessing_considerations_for_shapefile_output.htm

For improved efficiency consider transitioning the data to a geodatabase feature class or an external spatial database such as PostGIS.

0

I have found significant improvements loading attribute data from large shapefiles by using older versions of QGIS, either v2.14 or 2.16. For some reason QGIS v2.18 is a significantly slower (like a 100 times slower) at loading attribute data on large shapefiles on machines in this office (QGIS v2.18.6 - v2.18.5, Windows 7 64Bit, 8GB RAM).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.