# Is it possible to use memory layers in QGIS processing algorithms?

I am building a QGIS plugin that connects to a MySQL database in the local network, and then adds a subset of one of the tables to an in-memory layer; the subset is based on data currency (only taking the most recent observation for each location where measurements are made). This memory layer is successfully created.

However I then want to run some geoprocessing algorithms, and I'm having trouble using the in-memory layer in any of them.

    self.stationuri = "point?crs=epsg:4326&field=id:integer&field={}:double&index=yes".format(self.cb_field.currentText())
self.vlayer = QgsVectorLayer(self.stationuri,"scratch","memory")
if not self.vlayer.isValid():
raise Exception("Failed to create in-memory layer")
self.vlayer.startEditing()
for i,r in enumerate(result): # Result is row-by-row result of SQL query
...
self.vlayer.commitChanges()
self.vlayer.updateExtents()
# Layer is successfully added to map with all features and geometry
# BELOW IS WHERE IT FALLS APART
try:
processing.runandload("gdalogr:gridinvdist",self.vlayer,self.cb_field.currentText(),2,0,0,0,0,0,0,0,'Float32',None) # None = in-memory output; I get the same error if I specify a string path and filename.
except Exception, e:
raise e


No exception is raised, but no output is produced or added to the TOC, but the following log is made in processing.log:

INFO|Mon May 04 2015 11:28:23|GDAL execution console output|/bin/sh: 1: /tmp/processing/bbebe7599c83446d9c2b03a251879657/OUTPUT.tif: not found|/bin/sh: 1: -zfield: not found||FAILURE: Source datasource is not specified.|Usage: gdal_grid [--help-general] [--formats]|    [-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/|          CInt16/CInt32/CFloat32/CFloat64}]|    [-of format] [-co "NAME=VALUE"]|    [-zfield field_name] [-z_increase increase_value] [-z_multiply multiply_value]|    [-a_srs srs_def] [-spat xmin ymin xmax ymax]|    [-clipsrc <xmin ymin xmax ymax>|WKT|datasource|spat_extent]|    [-clipsrcsql sql_statement] [-clipsrclayer layer]|    [-clipsrcwhere expression]|    [-l layername]* [-where expression] [-sql select_statement]|    [-txe xmin xmax] [-tye ymin ymax] [-outsize xsize ysize]|    [-a algorithm[:parameter1=value1]*]    [-q]|    <src_datasource> <dst_filename>||Available algorithms and parameters with their's defaults:|    Inverse distance to a power (default)|        invdist:power=2.0:smoothing=0.0:radius1=0.0:radius2=0.0:angle=0.0:max_points=0:min_points=0:nodata=0.0|    Moving average|        average:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|    Nearest neighbor|        nearest:radius1=0.0:radius2=0.0:angle=0.0:nodata=0.0|    Various data metrics|        <metric name>:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|        possible metrics are:|            minimum|            maximum|            range|            count|            average_distance|            average_distance_pts|


The important part seems to be FAILURE: Source datasource is not specified. However self.vlayer.isValid() == True, so I don't see what is wrong with my input. I have tried substituting self.vlayer with 'memory:scratch' in the call to processing.runandload, but then I get the following error printed to the console (but not raised): Error: Wrong parameter value: memory:scratch.

I get the same issue when running this through the QGIS GUI, and using the dropdown menu to select my scratch layer which is in the TOC. This occurs whether I specify the output raster as in-memory or specify a location on disk.

This question seems similar, but their solution was to add the memory layer to the TOC before using it. I am already doing that and yet the error persists.

I thought that this was a general issue with memory layers and the QGIS geoprocessing algorithms, but the following works without issue:

processing.runandload("qgis:fixeddistancebuffer",self.vlayer, 500, 5, True, "output_buffer.shp")


What am I doing wrong? Why can't my memory source dataset "be specified" in some of the processing algorithms?

EDIT: here's the source code of gdalogr:gridinvdist if that's useful.

It seems like memory layers cannot be used as input for GDAL/OGR processing scripts because Processing fails to properly prepare the data for use with ogr2ogr. That's why, for example, the QGIS buffer tool works but the GDAL/OGR buffer tool fails:

Algorithm Buffer vectors starting...
GDAL command:
cmd.exe /C ogr2ogr.exe "C:\Users\anita\AppData\Local\Temp\processing70e5e0852cb9456ba2e3780f8386122e\86d237c8f41443f58a230a8133172047\OUTPUTLAYER.shp" point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} -dialect sqlite -sql "SELECT ST_Buffer( geometry , 1000 ),* FROM 'point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea}' "
GDAL command output:
FAILURE:
Unable to open datasource point?crs=EPSG:4326' with the following drivers.
-> JP2ECW
-> OCI
-> SOSI
...


Processing would have to somehow prepare the data (save it to a file) and then feed it to the GDAL/OGR tool.

I opened a ticket: Cannot use memory layers with OGR tools

That is the correct way, as explained in the documentation http://docs.qgis.org/2.14/es/docs/user_manual/processing/console.html

the next code work with in memory all except the last that it is load

MDT=path/mdt.tif
drain=processing.runalg("grass:r.drain",MDT,"",(pun),False,False,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,-1,0.00100,None)
vect=processing.runalg("grass:r.to.vect",drain['output'],0,False,"%f,%f,%f,%f"% (xmin, xmax, ymin, ymax),0,None)
bu=processing.runalg("qgis:fixeddistancebuffer",vect['output'],Metros_afecta,1,False,None)
buf=bu['OUTPUT']
bufe= QgsVectorLayer(buf,"area", "ogr")


the processing.runalg return a dictionary in this case bu['OUTPUT'] OUTPUT IS THE KEY, and the value is a temp path you can see the key with processeing.alghelp("name processing") as processing,alghelp("grass:r.drain")

return

processing.alghelp("grass:r.drain")
ALGORITHM: r.drain - Traces a flow through an elevation model on a raster map.
input <ParameterRaster>
coordinate <ParameterString>
vector_points <ParameterMultipleInput>
-c <ParameterBoolean>
-a <ParameterBoolean>
-n <ParameterBoolean>
GRASS_REGION_PARAMETER <ParameterExtent>
GRASS_REGION_CELLSIZE_PARAMETER <ParameterNumber>
GRASS_SNAP_TOLERANCE_PARAMETER <ParameterNumber>
GRASS_MIN_AREA_PARAMETER <ParameterNumber>
output <OutputRaster>


in this case the key is output , take care with capital letter you must writte in capital or with out capital, in this case not capital

And what about to look at GDAL Virtual File Systems (http://www.gdal.org/gdal_virtual_file_systems.html)

There is vsimem` prefix for in-memory file-like paths...