2

I've downloaded the Global Human Settlement Layer data from here. They are huge files, too big for ArcMap or QGIS to handle comfortably so for that reason I decided to mask them. I was only interested in Ireland so I made sure my coordinate system was the same for both the tif and the polygon (Ireland-admin area) before I begun.

I tried to the following code on one of the tifs that I'm interested in.

import fiona
import rasterio
from rasterio import mask

with fiona.open(r"D:\GIS\Global Human Settlement Layer\Built_Eire\IRL_adm0_WEB.shp", "r") as shapefile:
    geoms = [feature["geometry"] for feature in shapefile]

with rasterio.open(r"D:\GIS\Global Human Settlement Layer\Built_Eire\GHS_BUILT_LDS2000_GLOBE_R2016A_3857_38_v1_0\GHS_BUILT_LDS2000_GLOBE_R2016A_3857_38_v1_0_p1.tif") as src:
    out_image, out_transform = mask(src, geoms, crop=True)
    out_meta = src.meta.copy()

out_meta.update({"driver": "GTiff",
                 "height": out_image.shape[1],
                 "width": out_image.shape[2],
                 "transform": out_transform})

with rasterio.open(r"D:\GIS\Global Human Settlement Layer\Built_Eire\GHS_BUILT_LDS2000_GLOBE_R2016A_3857_38_v1_0\GHS_BUILT_LDS2000_GLOBE_R2016A_3857_38_v1_0_p1_Masked.tif", "w", **out_meta) as dest:
    dest.write(out_image)

The original .tif is 358.5MB, as soon as I try and run this code it started creating a huge file that even after only 9% completion it is 50GB.

Can anybody point me in the right direction as to what I may be doing wrong? I presume the file I downloaded was compressed?

  • I'm surprised that a 350Mb raster is too large for QGIS or ArcMap – Ian Turton May 18 '17 at 19:45
  • in fact I can open it fine in QGIS and it's only using 3 cores to render – Ian Turton May 18 '17 at 19:57
1

Dunuts, yes, your source could be compressed. As a sanity check, make sure the footprint of your out_image (out_image.shape[1:]) is smaller than the footprint of src. The mask() function should only read pixels from src that are within the cumulative spatial extent of your shapes and if your output footprint is larger than you expect there may be some odd features in the shapefile or a bug in rasterio.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.