Skip to main content
Geographic Information Systems

Return to Question

edited title
Link
GeoJim
GeoJim
  • 81
  • 1
  • 7

Borehole What geospatial table and indexing functions are suitable for a borehole database design for large volume of serialised constant increment data?

added 24 characters in body
Source Link
GeoJim
GeoJim
  • 81
  • 1
  • 7

Borehole database design for large volume of serialised, incremented constant increment data

Pretty common use case I suspect, but want to make sure I optimise from the get go. PostGIS I'm suspecting is the weapon of choice, and one way I'll be interrogating data is QGIS. Large part of data set originally sourced from LAS files.

Raw Data comes as typically 3 tables:

  1. collars/wellhead - point data of well head xyz ,
  2. wellpath - linestring data, dip/azimuth surveys at regular intervals, puts well/borehole in 3d space
  3. geophysics - 1d data, regular intervals along welpath like M values, up to say 15 parameters per depth depth step. Typically all 1520 params at same intervals.

Size is an issue (well, for me). Given most of the data is by definition at regular intervals, say 1cm, I thought there must be some standard functions that could make the storage retrieval efficient. I'm aware of things like ST_LineLocatePoint

  1. collars - 3000 xyz points
  2. wellpath - 30 dip/azi points per collar, regular 10m intervals
  3. geophysics - 50000 rows per collar, regular 1cm intervals (CWS Las 1.0), 20 single precision observations per intervals.

A typical csv containing the above weighs in at 10gb. Nice to have appropriately indexed, hence the question. Its the regular 1cm interval data that is the bulk that I'd like to have indexed appropriately.

Borehole database design for large volume of serialised, incremented data

Pretty common use case I suspect, but want to make sure I optimise from the get go. PostGIS I'm suspecting is the weapon of choice, and one way I'll be interrogating data is QGIS. Large part of data set originally sourced from LAS files.

Raw Data comes as typically 3 tables:

  1. collars/wellhead - point data of well head xyz ,
  2. wellpath - linestring data, dip/azimuth surveys at regular intervals, puts well/borehole in 3d space
  3. geophysics - 1d data, regular intervals, up to say 15 parameters per depth step. Typically all 15 params at same intervals.

Size is an issue (well, for me). Given most of the data is by definition at regular intervals, say 1cm, I thought there must be some standard functions that could make the storage retrieval efficient. I'm aware of things like ST_LineLocatePoint

  1. collars - 3000 xyz points
  2. wellpath - 30 dip/azi points per collar, regular 10m intervals
  3. geophysics - 50000 rows per collar, regular 1cm intervals (CWS Las 1.0), 20 single precision observations per intervals

A typical csv containing the above weighs in at 10gb. Nice to have appropriately indexed, hence the question. Its the regular 1cm interval data that is the bulk that I'd like to have indexed appropriately.

Borehole database design for large volume of serialised constant increment data

Pretty common use case I suspect, but want to make sure I optimise from the get go. PostGIS I'm suspecting is the weapon of choice, and one way I'll be interrogating data is QGIS. Large part of data set originally sourced from LAS files.

Raw Data comes as typically 3 tables:

  1. collars/wellhead - point data of well head xyz ,
  2. wellpath - linestring data, dip/azimuth surveys at regular intervals, puts well/borehole in 3d space
  3. geophysics - 1d data, regular intervals along welpath like M values, up to say 15 parameters per depth step. Typically all 20 params at same intervals.

Size is an issue (well, for me). Given most of the data is by definition at regular intervals, say 1cm, I thought there must be some standard functions that could make the storage retrieval efficient. I'm aware of things like ST_LineLocatePoint

  1. collars - 3000 xyz points
  2. wellpath - 30 dip/azi points per collar, regular 10m intervals
  3. geophysics - 50000 rows per collar, regular 1cm intervals (CWS Las 1.0), 20 single precision observations per intervals.

A typical csv containing the above weighs in at 10gb. Nice to have appropriately indexed, hence the question. Its the regular 1cm interval data that is the bulk that I'd like to have indexed appropriately.

Source Link
GeoJim
GeoJim
  • 81
  • 1
  • 7

Borehole database design for large volume of serialised, incremented data

Pretty common use case I suspect, but want to make sure I optimise from the get go. PostGIS I'm suspecting is the weapon of choice, and one way I'll be interrogating data is QGIS. Large part of data set originally sourced from LAS files.

Raw Data comes as typically 3 tables:

  1. collars/wellhead - point data of well head xyz ,
  2. wellpath - linestring data, dip/azimuth surveys at regular intervals, puts well/borehole in 3d space
  3. geophysics - 1d data, regular intervals, up to say 15 parameters per depth step. Typically all 15 params at same intervals.

Size is an issue (well, for me). Given most of the data is by definition at regular intervals, say 1cm, I thought there must be some standard functions that could make the storage retrieval efficient. I'm aware of things like ST_LineLocatePoint

  1. collars - 3000 xyz points
  2. wellpath - 30 dip/azi points per collar, regular 10m intervals
  3. geophysics - 50000 rows per collar, regular 1cm intervals (CWS Las 1.0), 20 single precision observations per intervals

A typical csv containing the above weighs in at 10gb. Nice to have appropriately indexed, hence the question. Its the regular 1cm interval data that is the bulk that I'd like to have indexed appropriately.