Questions tagged [amazon-web-services]
Amazon Web Services (AWS), is a collection of cloud computing services that make up the on-demand computing platform offered by Amazon.com.
99
questions
0
votes
0
answers
5
views
AWS Lambda layer in eu-west-2 containing gdal & python-gdal bindings
I need to write a Lambda that uses GDAL. I've tried using https://github.com/developmentseed/geolambda, but I need the layer from eu-west-2 region due to other dependencies. I've tried other Lambda ...
- 1
1
vote
1
answer
78
views
gdal /vsis3/ on NetCDF from Earthdata
I'm trying to access data from NASA's Earthdata S3 buckets, but I get a "<filename> does not exist in the file system, and is not recognized as a supported dataset name." error after ...
- 1,861
1
vote
2
answers
202
views
Converting shapefiles from S3 to GeoJSON in JavaScript for web map
I am working on a web map using OpenLayers and have encountered a challenge in converting shapefiles stored in an S3 bucket to GeoJSON using JavaScript. My goal is to display the converted shapefiles ...
- 125
0
votes
0
answers
37
views
Enabling outdb rasters in PostgreSQL
I have an AWS RDS server connected to PostgreSQL, and I also have some rasters in the cloud (s3 bucket). I want to remotely query them using outdb rasters but I'm not sure how to set ...
0
votes
0
answers
26
views
How to add custom basemap to QGIS with missing tiles
I am new to GIS. I have a custom basemap in S3 and I'm trying to import it into QGIS. It's timing out on some tiles that do not exist in my map:
WARNING Network request s3://mybucket/14/4384/6146.jpg ...
2
votes
0
answers
36
views
Performance issues with Open Data Cube when `find_datasets` with a lot of results
Our ODC instance is deployed on AWS Aurora RDS and is populated with approximately 200 million datasets. The Aurora RDS is configured as Serverless v2 (64 - 128 ACUs) and has 4 replicas (1 writer + 3 ...
- 51
0
votes
0
answers
121
views
Is there a way to add XYZ Tile Layer to QGIS from an AWS bucket?
I am trying to add Map tiles from Pix4D to QGIS. Has anyone successfully accomplished this? I have loaded XYZ tiles to an Amazon AWS bucket and things seem to work fine with leaflet (LINK). The ...
- 838
0
votes
1
answer
68
views
Can't create GeoServer data store from RDS PostGIS
I'm trying to add a PostGIS data store to my Geoserver but it is returning the below error:
Error creating data store, check the parameters. Error message: Unable to obtain connection: Cannot create ...
- 23
0
votes
0
answers
158
views
Bulk download from USGS The National Map (TNM)
I am looking for a way to bulk download data from the USGS The National Map (TNM). I found approximately 150 small (10MB) DEM tiles. The official USGS instructions are to download an open source ...
0
votes
0
answers
52
views
QGIS cache S3 files once imported
I'm considering a solution to store raster files in AWS S3.
However QGIS is installed outside of AWS. Hence egress costs could play a substantial role in determining this solution.
So if I store the ...
1
vote
3
answers
315
views
GeoServer WMS is slow when using OpenLayers and OSM
I am new to GIS, and attempting to create a map with PostgreSQL/PostGIS,GeoServer, OpenLayers, and OSM data.
I am only serving the NA continent, but it is taking approximately 20 seconds to render the ...
4
votes
0
answers
173
views
Reducing loading time of QGIS project
I have saved a couple of QGIS projects in a PostGIS DB, which is hosted in the cloud. The projects consist of different layers. When I open the QGIS projects, the loading time is very long. That is a ...
- 2,217
2
votes
2
answers
588
views
"S3 Support for GeoTiff" vs "COG (Cloud Optimized GeoTIFF) Support" Geoserver Community Extensions
I have large raster datasets on AWS S3 (~100gb each) that I would like to serve as WMS through Geoserver.
Has the "S3 Support for GeoTiff" extension been replaced by the "COG" one?
...
- 300
0
votes
0
answers
172
views
QGIS PostgreSQL Amazon RDS AWS - id issue
This is a novice question. I created a database using Amazon RDS AWS. Then I connected it to my PostgreSQL.
I am planning to utilize this for multi-user access to digitize blighted areas through QGIS.
...
- 139
2
votes
1
answer
2k
views
GeoServer j_spring_security_check on Login
I have a GeoServer container that I am trying to push up to the AWS Cloud Environment. Things work as usual (default settings ) for my 19.2.2 version of GeoServer on my localhost.
When I push the ...
- 183
0
votes
1
answer
220
views
Hosting Multiple GeoServer instances on the Same DNS
I have 2 instances of GeoServer that I am trying to run in the AWS Cloud environment on the same DNS. I am using Tomcat.
https://testhost.com/geoserver
https://testhost.com/geoserver-archive
The load ...
- 183
0
votes
0
answers
149
views
Public AWS URL to Landsat-8 images - error 404
I'm not the only one having trouble downloading Landsat images from aws server using a URL like this: https://s3-us-west-2.amazonaws.com/landsat-pds/L8/017/008/LC80170082015211LGN00/index.html
As in ...
0
votes
1
answer
526
views
Connecting ArcGIS Online to PostgreSQL on Amazon Web Server
I have some key data sets stored on PostgreSQL on an Amazon Web Server.
Is there any way, (direct or work around), that I can connect to this data in ArcGIS Online.
I can connect to it with ArcGIS Pro,...
- 93
1
vote
0
answers
233
views
Registering raster layers in Amazon RDS PostgreSQL using enable_out_db parameter
I have an AWS RDS PostgreSQL instance (v.11) that I'm currently storing vector data in. I also have a lot of raster layers in AWS S3 that are COG geotiffs. I'd like to register them in the database to ...
1
vote
1
answer
3k
views
Error trying to open NetCDF file with xarray from s3 bucket
I'm trying to open a .nc file from an S3 bucket using xarray, but I'm getting an error. Here's the method I'm using:
import xarray as xr
aws_url = 's3://nasanex/NEX-GDDP/BCSD/rcp85/day/atmos/tasmax/...
- 524
1
vote
1
answer
331
views
Getting 404 response while fetching Landsat-8 data from AWS S3
I am trying to fetch Landsat-8 satellite data from AWS S3 and for getting details, I am using gdalinfo command:
gdalinfo /vsicurl/https://s3-us-west-2.amazonaws.com/landsat-pds/c1/L8/144/048/...
- 11
0
votes
1
answer
299
views
Protecting Vector Tile Server Data
I am deploying a Vector Tile Server on AWS with data I don't want to make publicly available and there are two main elements I want to cover regarding the security of the data
The tile server needs ...
- 13
1
vote
1
answer
550
views
Create STAC catalog file for my s3 bucket
I have s3 private bucket which contains .tif files.
Now, I need to create STAC file like the following:
https://earth-search.aws.element84.com/v0
https://earthengine-stac.storage.googleapis.com/...
- 77
0
votes
1
answer
393
views
Does QGIS support AWS Redshift connection?
I am planning to connect AWS Redshift using QGIS for work purposes. However, I am not sure whether I can connect or not.
- 31
1
vote
1
answer
80
views
Finding libraries for GRASS Installation Linux
I am installing GRASS on Linux. I have gotten as far as running Make in these instructions: https://grass.osgeo.org/grass78/binary/linux/snapshot/INSTALL
I get this for multiple libraries:
error while ...
- 77
0
votes
1
answer
117
views
Installing GRASS on Linux
I am installing GRASS on Linux using source code and the ./configure file to create a standalone python script. I have been following instructions and installing relevant code like proj4 and GDAL and ...
- 77
3
votes
2
answers
640
views
How to get Copernicus DEM GeoTIFFs for a bounding box using Python
Background
According to AWS's Copernicus Digital Elevation Model page,
The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, ...
- 1,308
1
vote
1
answer
428
views
How to avoid (occasional) 404 errors when trying to access Sentinel-2 images using pystac-client
I use pystac-client to access some sentinel images. In Google Colab, I install pystac-client and rasterio by:
%%shell
pip install pystac-client
pip install rasterio
Import both modules:
from ...
- 1,308
0
votes
1
answer
187
views
AWS GeoServer not responding
I made a WMS host with AWS. The WMS work with link below
localhost:8080/geoserver/OSM/wms?
But I'd like to reach it out of the network with this link and QGIS:
http://ec2-xxx-xxx-xxx-xxx.us-east-2....
- 73
1
vote
0
answers
182
views
QGIS Server Not Rendering VRT Layer that is accessing COGS on S3
I have deployed a QGIS server following steps in the user manual on an AWS EC-2 instance. https://docs.qgis.org/3.16/en/docs/server_manual/containerized_deployment.html#docker-stacks
I am serving ...
- 1,688
2
votes
1
answer
339
views
Intake-STAC: KeyError: 'open_stac_item_collection'
I downloaded the AWS Earth search notebook from the intake-stac examples and whithout changing anything. Just run the cells...
But I've got everytime the error: KeyError: 'open_stac_item_collection'
...
- 153
7
votes
1
answer
1k
views
Trouble writing .tif from memory directly into S3 Bucket
I've used question on the topic as initial help and structured my code after it. I'm warping a raster file using these kwargs
rast = gdal.Open(/vsicurl/raster_url)
wkt = rast.GetProjection()
outbounds ...
- 524
0
votes
1
answer
453
views
Licensing ArcGIS Enterprise when scaling in cloud environment [closed]
Esri's ArcGIS Enterprise licenses are for use on machines with "up to 4 cores". I understand that in cloud environments (such as AWS) that available machines have an equivalent number of ...
- 11
1
vote
0
answers
205
views
DBSCAN with huge number of points
We have a dataset of approximately 30 million new points per month (latitude, longitude, and a measurement associated with that point).
So far, we have applied DBSCAN to generate a monthly map such ...
- 11
2
votes
1
answer
292
views
Synchronizing on-prem PostgreSQL raster database with AWS RDS
I have a folder of rasters that updates daily that I load into a PostgreSQL database using the raster2pgsql command line tool. I want to make this database outward facing so that I can connect to it. ...
- 524
3
votes
2
answers
4k
views
HTTP request -to- AWS Lambda Python function -to- AGOL
I want to make an HTTP GET request to AGOL, but I want to route the request through a Python function.
Since I can't host the Python function on AGOL (more info here), I'd like to use AWS-Lambda ...
- 1,114
2
votes
0
answers
135
views
Unable to list Sentinel l1c file using AWS CLI
Following Setting for AWS CLI using AWS configure
AWS Access Key ID : **********
AWS Secret Access Key : ********
Default region name : eu-central-1
which I get from IAM user
After setup I try to ...
- 21
1
vote
0
answers
138
views
Unable to execute QGIS algorithm while taking Input raster directly from AWS S3 account
I have a raster file (.TIF) in my AWS S3 account. I am using GDAL to open that .TIF file and to save the reprojected .TIF file to my local computer using QGIS GDAL 'Warp (reproject)' algorithm as ...
- 729
0
votes
0
answers
124
views
Why is spatialite query taking so long (only) on first execution?
I have an AWS lambda function deployed with a spatialite table. When queried, a point lookup is done on the table to find the locations in the table containing the point.
The first execution of the ...
- 121
1
vote
0
answers
162
views
Publishing PostGIS layer where PostgreSQL in Amazon RDS
I am trying to publish a PostGIS layer in GeoServer from my local machine. PostgreSQL is in AWS RDSs server. I am unable to see any features in layer preview. AWS RDS is SSL-enabled. it is worknig ...
- 11
5
votes
1
answer
3k
views
QGIS Read COG from S3
I am trying to read a Cloud Optimised GeoTiff (COG) from AWS S3. I can successfully run
gdalinfo /vsis3/cog-testing/HP_COG.tif
And see the details of the file and it says Layout: COG
But if I ...
- 3,280
1
vote
0
answers
6k
views
Writing Python list data into CSV file in multiple columns? [closed]
I am trying to write data into csv file from python list.
I am using aws lambda for this......
My code
fields = ['SnapshotId', 'StartDate']
csvio = io.StringIO()
writer = csv.writer(csvio,delimiter=...
- 111
1
vote
1
answer
687
views
Spatial queries with PostgreSQL on AWS
I have some data in PostgreSQL which I am exporting to Amazon RDS service.
Is it possible to use PostGIS extension in AWS?
(And will I need the mandatory geom column or I could run queries with only ...
- 11
2
votes
1
answer
802
views
How to download tile from Amazon’s Sentinel-2 service
How do I access Sentinel-2 imagery via Amazon?
I'm new to GIS and have been relying on USGS earth explorer and EO Browser but would now like to access satellite imagery with higher resolution. I ...
- 449
1
vote
0
answers
181
views
Convert a AWS EC2 URL to WMS URL
I have built my tile server on AWS EC2 instance and want to use it as a base map in QGIS.
When I try to add it as a WMS layer on QGIS, I get an error saying The server you are trying to connect to ...
1
vote
1
answer
964
views
Build tile server manually with PostgreSQL db on a different server
I have set up my map tile server and have used the following components:
Mod_tile
osm2pgsql
postgres
Mapnik
open street map carto
In my configuration, postgres is on a different server. when I ...
1
vote
0
answers
106
views
Point density from XY table in ArcGIS Pro
I have a large 1.1gb CSV table of points in Lat/Long WGS84 for which I'd like to generate a point density raster. I'm taking the following steps but the process is extremely slow. I welcome any advice ...
- 631
3
votes
1
answer
333
views
Sentinel 2 Atmospheric Correction Inconsistencies
I'm having some peculiar issues with Sentinel 2 data. I've downloaded two coterminous tiles (one L1C tile from EarthExplorer and one L1C/L2A tile from AWS) taken at the same time and date. I have done ...
1
vote
0
answers
126
views
Load unsafe script error: http vs https
We have an ArcGIS server hosted on Amazon. The maps and layers on that server are used to build a website (i.e. xxxx.com/map. The problem is, that page always shows an error "load unsafe scritps". If ...
0
votes
0
answers
204
views
Moving GIS Data to AWS?
I am an ArcGIS Desktop user and a layman to cloud computing. Our GIS data is currently hosted on a server and recently our IT dept started to look at possibilities to get rid of this server and move ...
- 11