3

I'm working with spatial/temporal data, stored in s3 (AWS). The data is in the format:

ID,    Date,                 Lat,    Lon
23424  2017-02-03T23:15:15Z  23.434  -5.2343
23424  2017-02-03T23:20:26Z  23.4356 -5.2637
etc...

What I want to do is import this data into QGIS. It is very large, several gb.

My current method is to use an Athena query to download the relevant data, as a csv. I then import the csv into a PostgreSQL database on my local machine, change the lat/long columns into a point geometry column, and import into QGIS as a POSTGIS layer.

This is time consuming, and also uses up space on my local machine.

Is there a way of connecting to the data stored externally on s3 instead?

I am pretty new to python, and I haven't used the python console much.

1 Answer 1

2

This question/answer explains how to use a .vrt file to create a link to the URL that corresponds to the S3 file/bucket. This assumes that you are trying to read the entire S3 file, not a subset of it (though it's probably possible to do that, but it's not explained in the referenced question).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.