13

I am building a QGIS plugin that connects to a MySQL database in the local network, and then adds a subset of one of the tables to an in-memory layer; the subset is based on data currency (only taking the most recent observation for each location where measurements are made). This memory layer is successfully created.

However I then want to run some geoprocessing algorithms, and I'm having trouble using the in-memory layer in any of them.

    self.stationuri = "point?crs=epsg:4326&field=id:integer&field={}:double&index=yes".format(self.cb_field.currentText())
    self.vlayer = QgsVectorLayer(self.stationuri,"scratch","memory")
    if not self.vlayer.isValid():
        raise Exception("Failed to create in-memory layer")
    self.vlayer.startEditing()
    for i,r in enumerate(result): # Result is row-by-row result of SQL query
        # Add features
        ...
    self.vlayer.commitChanges()
    self.vlayer.updateExtents()
    # Add layer to map
    QgsMapLayerRegistry.instance().addMapLayer(self.vlayer)
    # Layer is successfully added to map with all features and geometry
    # BELOW IS WHERE IT FALLS APART
    try:
        processing.runandload("gdalogr:gridinvdist",self.vlayer,self.cb_field.currentText(),2,0,0,0,0,0,0,0,'Float32',None) # None = in-memory output; I get the same error if I specify a string path and filename.
    except Exception, e:
        raise e

No exception is raised, but no output is produced or added to the TOC, but the following log is made in processing.log:

INFO|Mon May 04 2015 11:28:23|GDAL execution console output|/bin/sh: 1: /tmp/processing/bbebe7599c83446d9c2b03a251879657/OUTPUT.tif: not found|/bin/sh: 1: -zfield: not found||FAILURE: Source datasource is not specified.|Usage: gdal_grid [--help-general] [--formats]|    [-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/|          CInt16/CInt32/CFloat32/CFloat64}]|    [-of format] [-co "NAME=VALUE"]|    [-zfield field_name] [-z_increase increase_value] [-z_multiply multiply_value]|    [-a_srs srs_def] [-spat xmin ymin xmax ymax]|    [-clipsrc <xmin ymin xmax ymax>|WKT|datasource|spat_extent]|    [-clipsrcsql sql_statement] [-clipsrclayer layer]|    [-clipsrcwhere expression]|    [-l layername]* [-where expression] [-sql select_statement]|    [-txe xmin xmax] [-tye ymin ymax] [-outsize xsize ysize]|    [-a algorithm[:parameter1=value1]*]    [-q]|    <src_datasource> <dst_filename>||Available algorithms and parameters with their's defaults:|    Inverse distance to a power (default)|        invdist:power=2.0:smoothing=0.0:radius1=0.0:radius2=0.0:angle=0.0:max_points=0:min_points=0:nodata=0.0|    Moving average|        average:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|    Nearest neighbor|        nearest:radius1=0.0:radius2=0.0:angle=0.0:nodata=0.0|    Various data metrics|        <metric name>:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|        possible metrics are:|            minimum|            maximum|            range|            count|            average_distance|            average_distance_pts|

The important part seems to be FAILURE: Source datasource is not specified. However self.vlayer.isValid() == True, so I don't see what is wrong with my input. I have tried substituting self.vlayer with 'memory:scratch' in the call to processing.runandload, but then I get the following error printed to the console (but not raised): Error: Wrong parameter value: memory:scratch.

I get the same issue when running this through the QGIS GUI, and using the dropdown menu to select my scratch layer which is in the TOC. This occurs whether I specify the output raster as in-memory or specify a location on disk.

This question seems similar, but their solution was to add the memory layer to the TOC before using it. I am already doing that and yet the error persists.

I thought that this was a general issue with memory layers and the QGIS geoprocessing algorithms, but the following works without issue:

processing.runandload("qgis:fixeddistancebuffer",self.vlayer, 500, 5, True, "output_buffer.shp")

What am I doing wrong? Why can't my memory source dataset "be specified" in some of the processing algorithms?

EDIT: here's the source code of gdalogr:gridinvdist if that's useful.

3 Answers 3

4

It seems like memory layers cannot be used as input for GDAL/OGR processing scripts because Processing fails to properly prepare the data for use with ogr2ogr. That's why, for example, the QGIS buffer tool works but the GDAL/OGR buffer tool fails:

Algorithm Buffer vectors starting...
GDAL command:
cmd.exe /C ogr2ogr.exe "C:\Users\anita\AppData\Local\Temp\processing70e5e0852cb9456ba2e3780f8386122e\86d237c8f41443f58a230a8133172047\OUTPUTLAYER.shp" point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} -dialect sqlite -sql "SELECT ST_Buffer( geometry , 1000 ),* FROM 'point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea}' " 
GDAL command output:
FAILURE: 
Unable to open datasource `point?crs=EPSG:4326' with the following drivers. 
-> JP2ECW 
-> OCI 
-> SOSI 
...

Processing would have to somehow prepare the data (save it to a file) and then feed it to the GDAL/OGR tool.

I opened a ticket: Cannot use memory layers with OGR tools

4

That is the correct way, as explained in the documentation http://docs.qgis.org/2.14/es/docs/user_manual/processing/console.html

the next code work with in memory all except the last that it is load

MDT=path/mdt.tif
drain=processing.runalg("grass:r.drain",MDT,"",(pun),False,False,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,-1,0.00100,None)
vect=processing.runalg("grass:r.to.vect",drain['output'],0,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,None)
bu=processing.runalg("qgis:fixeddistancebuffer",vect['output'],Metros_afecta,1,False,None)
buf=bu['OUTPUT']
bufe= QgsVectorLayer(buf,"area", "ogr")
#the last load the layer 
QgsMapLayerRegistry.instance().addMapLayers([bufe])

the processing.runalg return a dictionary in this case bu['OUTPUT'] OUTPUT IS THE KEY, and the value is a temp path you can see the key with processeing.alghelp("name processing") as processing,alghelp("grass:r.drain")

return

processing.alghelp("grass:r.drain")
ALGORITHM: r.drain - Traces a flow through an elevation model on a raster map.
input <ParameterRaster>
coordinate <ParameterString>
vector_points <ParameterMultipleInput>
-c <ParameterBoolean>
-a <ParameterBoolean>
-n <ParameterBoolean>
GRASS_REGION_PARAMETER <ParameterExtent>
GRASS_REGION_CELLSIZE_PARAMETER <ParameterNumber>
GRASS_SNAP_TOLERANCE_PARAMETER <ParameterNumber>
GRASS_MIN_AREA_PARAMETER <ParameterNumber>
output <OutputRaster>

in this case the key is output , take care with capital letter you must writte in capital or with out capital, in this case not capital

1
  • Memory layers are a special thing and I think you are talking about TEMPORARY_OUTPUT files here, so files written to a temporary directory as files. Memory layers do not exist as files. Feb 9 at 11:41
1

And what about to look at GDAL Virtual File Systems (http://www.gdal.org/gdal_virtual_file_systems.html)

There is vsimem prefix for in-memory file-like paths...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.