3

I have a shapefile containing about 240,000 features. I want to split this into five files with no overlaps. My current approach is to write a non-trivial expression that defines the first subfile, save the selection, then delete the selected features. I then repeat this process. The problem is that deleting the records is taking 10-15 minutes for each step. In principle I could skip the deletion at each step, and then select the final subset as the negation of the OR of the earlier queries, but the likelihood of writing that query correctly seems quite low.

I suppose I could create a new field corresponding to each subset and set the value to 1 or 0 by expression, then set the last field as the negation of the OR of the fields. But I'm nearly done, so I'm not sure I'll be trying this.

For future reference, is there a better way to do this?

  • 1
    What do you use to define what goes in what set? – Nathan W Jan 27 '14 at 6:33
  • 1
    How about importing it into a database like spatialite, then exporting different shapefiles for each set? – BradHards Jan 27 '14 at 6:36
  • I appreciate the suggestions provided in the answers below, but the real challenge here is in insuring the sets are mutually exclusive, and none of the answers addresses that point. I think the pragmatic solution, whether is uses qgis expressions of OGR2OGR with a WHERE clause is to rely on a plain vanilla text editor to write the clauses and then cut-and-paste. More specifically, write clause1 which defines the first subset. Then write clause2 and set the WHERE clause to "clause2 AND NOT clause1". Next "clause3 AND NOT (clause1 OR clause)". Final case is the negation of the preceding clauses. – Llaves Jan 28 '14 at 5:06
  • Nathan: I am using select by expression in QGIS where the expression uses field values as the criteria. – Llaves Jan 28 '14 at 5:07
  • I've refined my answer in order to include the mutually exclusive logic. – Antonio Falciano Jan 28 '14 at 9:10
2

OGR2OGR allows you to define a where clause when converting. This approach requires that you can write a mutually exclusive expression.

| improve this answer | |
1

Like my good friend Uffe states OGR could be a solution. If you have the five areas as geometry like a shapefile (mask), you can use the ogr2ogr -clipsrc option: ogr2ogr -f "ESRI Shapefile" -clipsrc mask.shp outputDir sourceDir

(The param '-f "ESRI Shapefile"' is not needed when shapefile is used since shp is default)

ogr2ogr -clipsrc D:\data\mask1.shp D:\data\output D:\data\source
ogr2ogr -clipsrc D:\data\mask2.shp D:\data\output D:\data\source
ogr2ogr -clipsrc D:\data\mask3.shp D:\data\output D:\data\source
ogr2ogr -clipsrc D:\data\mask4.shp D:\data\output D:\data\source
ogr2ogr -clipsrc D:\data\mask5.shp D:\data\output D:\data\source

If you can identify the five areas by an attribute or expression check the option: -clipsrcsql instead of -clipsrc

More info: http://www.gdal.org/ogr2ogr.html

| improve this answer | |
0

IMHO a good approach consists into using the OGR VRT format. In detail, you can apply different queries on your input datasource, taking advantage of OGR SQL and eventually SQLite SQL dialect (for spatial queries), as documented in OGR VRT documentation. For instance:

query1.vrt (example of OGR SQL)

<OGRVRTDataSource>
    <OGRVRTLayer name="output1">
        <SrcDataSource>input1.shp</SrcDataSource>
        <SrcSQL>SELECT * FROM input1 WHERE condition1</SrcSQL>
    </OGRVRTLayer>
    <OGRVRTLayer name="input2">
        <SrcDataSource>input.shp</SrcDataSource>
        <SrcSQL>SELECT * FROM input1 WHERE NOT condition1</SrcSQL>
    </OGRVRTLayer>
</OGRVRTDataSource>

query2.vrt (example of SQLite SQL dialect)

<OGRVRTDataSource>
    <OGRVRTLayer name="output2">
        <SrcDataSource>query1.vrt</SrcDataSource>
        <SrcSQL dialect="SQLite">SELECT Simplify(geometry,10) from input2 WHERE condition2</SrcSQL>
    </OGRVRTLayer>
    <OGRVRTLayer name="input3">
        <SrcDataSource>query1.vrt</SrcDataSource>
        <SrcSQL dialect="SQLite">SELECT Simplify(geometry,10) from input2 WHERE NOT condition2</SrcSQL>
    </OGRVRTLayer>
</OGRVRTDataSource>

...etc.

query5.vrt (final query)

<OGRVRTDataSource>
    <OGRVRTLayer name="output5">
        <SrcDataSource>query4.vrt</SrcDataSource>
        <SrcSQL>SELECT * FROM input5 WHERE condition5</SrcSQL>
    </OGRVRTLayer>
</OGRVRTDataSource>

Finally you have simply to convert every OGR VRT file as SHP, considering only the outputs:

ogr2ogr output1.shp query1.vrt -sql "SELECT * FROM output1"
ogr2ogr output2.shp query2.vrt -sql "SELECT * FROM output2"
ogr2ogr output3.shp query3.vrt -sql "SELECT * FROM output3"
ogr2ogr output4.shp query4.vrt -sql "SELECT * FROM output4"
ogr2ogr output5.shp query5.vrt -sql "SELECT * FROM output5"

IMHO another advantage of this approach is that it's possible to control and keep track of all the queries with a text editor, because OGR VRTs are simply XML files.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.