1

I'm attempting to iterate through several shape files within a folder and only move some of them to a new folder. I'm doing this based on contents of the DBF file where I'm using dbfread to determine whether is should be moved or not. I have this all working as expected except it's only moving the DBF file and not the .shp, .shx, and .prj. files since the logic is only on the DBF. One way I was thinking I could move all files would be to add the basename of the ones I want to move to a List but I seem to be failing at that. Here is the code I'm using.

from dbfread import DBF
MoveFolder_path = r"C:\Data\Move"
move_list = []
for file in directory_list:
    if file.split('.')[1] == 'dbf':
        file_basename = file.split('.')[0]
        table = DBF(file, load=True)
        if table.records[0]['name'] == None:
            move_list.append(file_basename)
for movefile in move_list:
    for file in directory_list:
        file_basename = file.split('.')[0]
        if file_basename == movefile:
            shutil.move(file, MoveFolder_path)
  • I copied your code and filled directory_list with a list of filenames of existing shapefiles. The code worked as expected. Thus I suppose that directory_list contains dbf files only. Unfortunately it is not clear from your code snipped how the list is filled. – Detlev Dec 25 '15 at 20:47
1

Check this out:

import shutil, os
from dbfread import DBF

MoveFolder_path = r"C:\Data\Move"
move_list = []

for i in path: #path to your directory_list
    if i.endswith('.dbf'):
        dbf = os.path.join(path, i)
        table = DBF(dbf, load=True)
        if table.records[0]['name'] == None:
            move_list.append(dbf[:-4])

for i in path:
    name = i[:-4]
    if name in move_list:
        src = os.path.join(path, i)
        shutil.move(src, MoveFolder_path)
  • Thanks Digd. For better or worst I wanted to stay away from the shp libraries and just work with the dbf. I didn't end up using the os.path.join part of your script since it was giving me trouble but the other syntax I did. – wilbev Dec 26 '15 at 18:41
4

With the solution to consider only the content of a dbf files you move only the dbf file.

If you want to move all the files of a "shapefile" (.shp,.shx,.dbf, .proj, ...) you need to use Python modules processing all the files at the same time as osgeo (GDAL/OGR), Fiona or Pyshp (shapefile) and copy the resulting shapefiles to the new directory

A solution with Fiona (all the results are Python dictionaries) for example

import glob, os
import fiona
MoveFolder_path = r"C:\Data\Move"
for file in glob.glob("*.shp"):
    shp = fiona.open(file)
    # first feature of the shapefile
    if shp.next()['properties']['name'] == None:
        # preserve the schema of the original shapefile, including the crs
        meta = shp.meta
        # copy the shapefile to the new directory
        newPath = os.path.join(MoveFolder_path, shp.path)
        with fiona.open(newPath, 'w', **meta) as output:
            for feature in shp:
                output.write(feature)

If you want to delete the original files after copy

1) with Fiona >= 1.7

fiona.remove(shp.path, driver='ESRI Shapefile')

2) with osgeo

from osgeo import ogr
driver = ogr.GetDriverByName("ESRI Shapefile")
driver.DeleteDataSource(shp.path)

New

The results of dbfread and Fiona for the dbf files are the same (dictionaries) but the result of Fiona is an generator/iterator with the Iteration protocol (next(),...) and all the all the advantages of the method for large files.

import fiona
shp = fiona.open('test.shp')
shp.next()['properties'] # only first element is loaded
OrderedDict([(u'dip_dir', 165), (u'dip', 67), (u'name', u'')])

In contrast, when you use dbfread

By default records will streamed directly from disk. If you pass load=True they will instead be loaded into lists and made available as the records and deleted attributes.

from dbfread import DBF
table = DBF("test.dbf", load = True) # all the elements are loaded in a list
table[0]
OrderedDict([(u'dip_dir', 165), (u'dip', 67), (u'name', u'')])

If you want an generator/iterator as in Fiona

If the table is not loaded, the records and deleted attributes return RecordIterator objects.

table = DBF("test.dbf")
iter(table.records).next()  # only first element is loaded
OrderedDict([(u'dip_dir', 165), (u'dip', 67), (u'name', u'')])

The geometric part of the shapefile (.shp) is minimal in large files and working only with the dbf file is not necessarily more efficient and faster ( and 40 MB is not large)

  • with shutils, you need to know all the files associated with the dbf file (.shp, ...) and move them after your script.
  • with Fiona it is automatic
  • Thanks gene. I wanted to stay away from using any of the shp packages since I was concerned they would add more overhead. Not sure if that would be the case or not but seemed like only working with the dbf may make the script more efficient and faster. Some of the shp/dbf files are rather larger, like over 40MB in size. – wilbev Dec 26 '15 at 18:44
  • 1
    look above for an explanation – gene Dec 26 '15 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.