Questions tagged [data-quality]

for questions related to implementing best practices to ensure good quality in gespatial datasets; including quality assurance (QA) and quality control (QC)

Filter by
Sorted by
Tagged with
0 votes
0 answers
50 views

Formula to use in respect to spatial resolution, to calculate the appropriate amount of random samples for QA of raster data

I am trying to write a script that will automatically calculate the amount of random samples needed to compare my derived DEM data against my truth datasets. The DEM data can vary in resolution from ...
Noah P's user avatar
  • 59
1 vote
1 answer
38 views

Working with 8-bit image in Earth Engine

I'm working with the MODIS Terra Continuous Fields 250m product. The data include several bands, including a "Quality" band. var geometry = ee.FeatureCollection("FAO/GAUL/2015/level0&...
user157564's user avatar
1 vote
1 answer
42 views

Remove bad quality pixels from daily PROBA-V images

In a previous question of mine, I exported all images from PROBA-V satellite for a particular month without removing the bad quality pixels. Now I want to remove the bad quality pixels before ...
Nikos's user avatar
  • 1,033
1 vote
1 answer
384 views

Download NASA's Black Marble VNPA2 nighttime light data using Google Earth Engine

I want to download images from NASA's Black Marble nighttime light (NTL) using Google Earth Engine. The product can be found here. Before downloading them I need to check the pixel's quality using the ...
Nikos's user avatar
  • 1,033
0 votes
1 answer
56 views

Landsat 8 Collection 2 Surface Temperature QA data exhibits strange clustering

I'm working with both Collection 1 and 2 data to explore Surface Temperature. Collection 2 data has a new product that enumerates the uncertainty in calculated surface temperature. The data seems ...
strangecharm's user avatar
1 vote
0 answers
230 views

Landsat 8 - Collection 1 BQA vs Collection 2 QA_PIXEL

I am working on updating a Python program for atmospherically correcting Level 1 Landsat 8 data from Collection 1 to Collection 2, as Collection 1 data is discontinued and will no longer be available ...
bashful_creature's user avatar
2 votes
0 answers
120 views

Checking quality assurance of MOD10A1 NDSI for snow fraction retrieval

I have recently retrieved some snow fraction data over Fennoscandia from GEE for the first time. I realise the script has no quality assurance that is a built-in feature in MODIS imagery normally. ...
Thomas's user avatar
  • 905
1 vote
1 answer
212 views

Obtain product quality for the (Sentinel-5P OFFL NO2: Offline Nitrogen Dioxide) mean image

I am unable to obtain this property of image: PRODUCT_QUALITY STRING - (Indicator) that specifies whether the product quality is degraded or not. Allowed values are "Degraded" and "...
No Name's user avatar
  • 21
0 votes
1 answer
537 views

Problems filtering MOD09A1 in GEE using the QA layer

I'm trying to mask each modis image using the QA layer. As far as I understand, the part .select("QA").eq(0) should remove all pixels with no ideal quality for all bands. However, when I ...
jose maria's user avatar
1 vote
0 answers
406 views

Using MOD16A2 in GEE to extract best quality of PET

I am beginner with Google Earth Engine programming. I want to know how I can use MOD16A2 to extract good quality values of potential evapotranspiration data (MOD16A2)? because I extract the PET of ...
Soukaina 's user avatar
0 votes
0 answers
109 views

Spike and crater detection in DSM using machine learning / deep learning / computer vision techniques

I'd like to try machine learning / computer vision / deep learning approaches for detecting artifacts in Digital Surface Models, and am specifically interested in doing this for common artifacts like ...
Lakshay Sharma's user avatar
1 vote
1 answer
463 views

Landsat 8 extracted pixel values for up North: 'pixel_qa' value of 322

I am trying to get landsat reflectance values corresponding to field plots spread all over Finland using the Google Earth Engine. As you know, Finland is up North, between latitudes of ~60 and 70 ...
user7831861's user avatar
1 vote
0 answers
494 views

How to covert a GeoTIFF DEM to a standard raster format without losing quality or resolution

I'm working on a project, a spin-off of my graduate dissertation, to digitize LiDAR data of a portion of Tampa Bay (20x20 km^2 at 1:1 scale) into a Minecraft map. Ideally I want not just accurate ...
qu4ntumrush's user avatar
2 votes
2 answers
922 views

When working with MOD09GA product, is supposed QC bitmask for high aerosol quantity to return 4 values?

it's the first time I'm working with MODIS data and I am a bit confused about how bits work and how masks are created. After a bit of research I've been able to understand, more or less, how bit ...
Juanma Cintas's user avatar
1 vote
1 answer
1k views

Google Earth Engine Mask application

I'm using GEE in order to extract evapotranspiration (ET) data from MODIS (MOD16A2v6). I want to extract only ET band in order to make some further process. To do so, I apply some filters (like ...
baptelet's user avatar
1 vote
0 answers
80 views

What is Kappa coefficient in HSI classification? [closed]

What is the Kappa coefficient and how it is calculated in the HSI classification process?
Daniel's user avatar
  • 11
2 votes
0 answers
68 views

Download QA band of Landsat 1-8

In one of my work, I just need the QA band of Landsat, but USGS Earth Explorer does not provide band by band data download opportunity. As I am interested to get all the available Landsat QA data from ...
M. M. Farhad's user avatar
2 votes
1 answer
983 views

NDVI index based on QA bands for MCD43A4 , in GEE

I want to calculate the NDVI index based on the MODIS/006/MCD43A4 collection and based on quality bands, but as a new GEE user I have no idea If I am on the rght way to do this. So far I have the ...
foo's user avatar
  • 186
0 votes
1 answer
847 views

Select observations based on Quality Bands , Google Earth Engine

From the following NDVI collection, I want to keep only the observations with Good Quality (equal to zero). I try to do this with the SummaryQA but I get "Pattern 'SummaryQA' did not match any bands"...
foo's user avatar
  • 186
6 votes
1 answer
2k views

What are the other bits in Sentinel's QA60 band?

According to this page on Google Earth Engine documentation bit 10 is 1 for dense clouds, and bit 11 is 1 for cirrus. The documentation doesn't say anything about all the other 14 bits that can be set,...
Andrei's user avatar
  • 766
1 vote
1 answer
72 views

Does the MODIStsp package computes the QA's and eventually corrects the bad quality pixels?

I am using R's MODIStsp package for downloading MOD11A2 time series products. While executing the command it was written that QA's were computed. Several files were created, LST_Day_1km, QC_Day, ...
Vasilis Ntoufas's user avatar
6 votes
1 answer
189 views

Hansen loss and lossyear bands do not always match in Earth Engine

I'm looking at the Hansen forest loss v1.5 data in Earth Engine. According to the Hansen documentation, In addition, to reduce confusion, beginning with version 1.4 we are no longer releasing loss ...
svangordon's user avatar
6 votes
1 answer
3k views

LiDAR returns over water bodies?

I have obtained LiDAR data from my environmental agency in order to obtain the relative heights of each watercourse station. A far as I know, LiDAR signals do not get returned over water bodies. ...
Robert Buckley's user avatar
6 votes
3 answers
315 views

How to check accuracy of Roads

We've been doing QA work on a roads dataset and I realized that many of the roads don't overlay correctly with their counterparts in Google Maps or OSM. Currently we compare our roads with the ...
Hasan Mustafa's user avatar
1 vote
0 answers
83 views

Geospatial quality standards

I am reviewing the geodata quality standards within my organization. During my review I keep coming across the ISO 19100 series. I am wondering if anyone has used these standards and if they have any ...
Ryan Garnett's user avatar
  • 9,479
1 vote
1 answer
563 views

How to extract MODIS QA informaton

I want to use the LDOPE tool unpack_sds_bits to extract the bit information from the State QA layer of the MYD09GA Aqua Surface Reflectance product. However, I have never run a command line before ...
Anna's user avatar
  • 11
1 vote
1 answer
117 views

Discrepancy between shape area and data from municipality [closed]

I am attempting to map several parcels just to show very basic dimensions (frontage, etc.) and parcel area. I have the parcel area from the municipal assessor's office in acres. And I have a parcel ...
cmann's user avatar
  • 11
1 vote
0 answers
293 views

MODIS Quality Control in QGIS

I have to process 46 MODIS MOD15A2H images (leaf area index) in QGIS 2.12 Lyon. I have to clean the images from bad pixels using MODIS Quality Control. Is there any method to automate the process for ...
okapina's user avatar
  • 103
4 votes
1 answer
1k views

MOD09A1 QC layer

Does anyone know how to handle MOD09A1 Quality flags (QC) in in R? I was trying to use raster package but unable to unlock the QC band.
uday's user avatar
  • 61
3 votes
1 answer
281 views

How can I select ways that have missing access parameters "[key]=Not Specified"?

Follow up to How can I programmatically identify missing sidewalks in OpenStreetMap data?. I've noticed that a number of streets and roads in a region I'm interested in are missing pedestrian access ...
raphael's user avatar
  • 3,407
2 votes
3 answers
795 views

Identifying missing sidewalks in OpenStreetMap data [closed]

I have a web map that is using the pedestrian routing from Mapbox's directions API. However, users have seen very long circuitous paths in certain locations because it appears that sidewalks (and/or ...
raphael's user avatar
  • 3,407
5 votes
2 answers
917 views

Parsing MODIS QA layers [closed]

I'd like to correctly parse the MOD13Q1 VI Quality layer (see Table 2 in "layers" here). For this, I'm trying to follow Example 1 from this tutorial (page 10). I'll illustrate my problem ...
Jon Snow's user avatar
  • 295
1 vote
0 answers
95 views

Changes in MOD09A1 quality mask as processed with GDAL

I'm analyzing MOD09A1 spectral data using GDAL/Python. The quality mask is a uint32 dataset and want to pull out the value of the first bit to mask the spectral data (1 == poor data; 0 == good data). ...
Emily's user avatar
  • 769
0 votes
4 answers
3k views

Raster images losing quality when clipped?

I'm using the ArcGIS data management clip tool for raster, a jpg file. It is being clipped with a polygon. After the clip is successful, the clipped image is appearing in lower quality and the colors ...
Ben's user avatar
  • 1
1 vote
2 answers
110 views

Implementing transactions with Collector for ArcGIS?

Currently I am looking into using ArcGIS Collector for field use. I would like for there to be a QA process done before final submission. Is there a way to implement an approval process for the ...
user avatar
0 votes
2 answers
79 views

Ensuring data quality from organizations

You are working for an organization that receives data from other organizations in different data formats (geospatial files, spreadsheets, text files, etc.) Is there specific things you would ask ...
Toronto23's user avatar
  • 123
7 votes
1 answer
2k views

How can I parse modis MOD13Q1 quality layers in R?

I'd like to parse MOD13Q1 VI quality layers in R. My original code must have been incorrect, because it gave lots of "deep ocean" flags in the middle of Brazil. After the changes suggested by Mikkel,...
Adrian's user avatar
  • 309
1 vote
0 answers
72 views

Collecting shapefiles from different users - any rules for quality data/quality control?

Apologies in advance for this very basic question! My company collects market information from forestry projects in an annual survey. This year, we are trying to add a spatial element to our analysis....
Kelley Hamrick's user avatar
2 votes
2 answers
603 views

Why OSM data has different attribute values from the same element and same time frame?

Here is my question: Does anyone know why OSM has different attribute values from the same element and same frame time? The example explains itself, when I try the Shapefile from different mirrors ...
juasmilla's user avatar
  • 479
2 votes
2 answers
232 views

What are the ways we can control the quality of a DTM?

I imagine that for DTM obtained from Stereoscopic procedures or interpolation from a set of topographic points a comparaison with the original data can help, but what for DTM obtained from Remote ...
Akheloes's user avatar
  • 688
10 votes
1 answer
314 views

What PostGIS [topology] provided tools may be used for data quality assurance?

I'm learning about techniques to ensure GIS data quality. Googling around, I've found a few papers about theory and best practices (here and here), and a couple of short presentations about the topic ...
Jorge Arévalo's user avatar
0 votes
1 answer
460 views

Utility GIS - Type of data cleanup needed & Effort

I am trying to get into Utility GIS and I have been asked by my boss to size up the effort of what it takes to clean up Pipeline, Utility asset GIS data and figure out if there is a way think of a ...
ved's user avatar
  • 307
5 votes
2 answers
514 views

How to check if Waste-Water Manhole connect to lines with the right flow direction?

I have a waste-water geometric network with flow direction throw digitize direction. I want to check if each manhole have a line flowing in and a line flowing out. Is there an attribute of line vertex ...
Dror Har Gil's user avatar
  • 2,524
2 votes
1 answer
507 views

Using line of sight or something similar to identify where lines go through hills

When QCing power infrastructure I am having to zoom into every gap and check to make sure the line makes sense (not going through the mountain). I typically check for odd spacing between the pylons ...
MjonesGEO's user avatar
  • 791
2 votes
1 answer
903 views

Validating Values in Multiple Fields in an Attribute Table

I've been playing around with 'Maintaining Attribute Integrity' functions found here: http://resources.arcgis.com/en/help/...000000n000000/ I see how I can set domains and subtypes to control the ...
Mike's user avatar
  • 2,281
1 vote
1 answer
256 views

Running a QC check for gaps between vertices on a polyline

I'm checking polylines to make sure power pylons were correctly extracted from imagery. Is there a way to find all of the gaps or missed pylons in the database? Currently, I'm manually making a ...
MjonesGEO's user avatar
  • 791
4 votes
1 answer
2k views

Automated quality control and assurance - geodata?

I've recently started looking at the issue of automated quality control and assurance of geographic data. What software exists that can be used for accomplishing this? Esri has got their product '...
Vincent's user avatar
  • 41
12 votes
2 answers
10k views

Finding differences between shapefiles using ArcGIS for Desktop?

Every month I have to QA/QC some streets data for an entire county (over 47,000 records). The first step is to compare the two files. Currently, I'm using a tool that was created in house via VBA ...
Craig's user avatar
  • 4,580
3 votes
0 answers
384 views

Is there a tool for automated quality assurance for vector data captured on raster data? [closed]

I wonder whether is any tool(or script) in ArcGIS or QGIS which allows to make an automated quality assurance for vector data captured on raster data? vector data on binary raster:
tetik's user avatar
  • 31
10 votes
3 answers
3k views

How do I compare two maps to determine which is better?

I'm trying to do an objective comparison of two different maps for the same region. At the moment, I'm struggling with defining criteria that will allow me to do a dispassionate evaluation. Does ...
Styp's user avatar
  • 1,473