7

I would like to know whether there are Python 3.x-compatible (preferably Python 3.3) modules to read the kind of .dbf files produced by ArcGIS for Desktop.

All the modules I found were only compatible with Python 2.x. I may have missed something, but I really couldn't find any which was compatible with Python 3.x.

3
  • Recognize that ArcGIS does not officially support Python 3.x (as of ArcGIS 10.1). The final release of 10.1 includes Python 2.7.2. Apr 8, 2013 at 21:07
  • @RyanDalton Actually, I just need to import some big bases of legacy ArcGIS-produced .dbf and will not need to use ArcGIS anymore.
    – Morwenn
    Apr 8, 2013 at 22:11
  • PRO supports py3+ not arcmap.
    – NULL.Dude
    Dec 21, 2017 at 13:18

4 Answers 4

13

As all Python programmers, I will

1
  • 1
    Well, GDAL might be a little bit too much just for reading such files. But thank you anyway, I had not found dbfread before but it worls fine :)
    – Morwenn
    Apr 8, 2013 at 18:01
2

I adapted this function for python 2,7 to python 3,x, works for me, but no warranty ... :)

def dbfreader(f): #Found on http://code.activestate.com/recipes/362715-dbf-
reader-and-writer/ - by Raymond Hettinger
"""Returns an iterator over records in a Xbase DBF file.

The first row returned contains the field names.
The second row contains field specs: (type, size, decimal places).
Subsequent rows contain the data records.
If a record is marked as deleted, it is skipped.

File should be opened for binary reads.
Adapted for python 3,5 20-09-2017
"""
# See DBF format spec at:
#     http://www.pgts.com.au/download/public/xbase.htm#DBF_STRUCT

numrec, lenheader = struct.unpack('<xxxxLH22x', f.read(32))
numfields = (lenheader - 33) // 32

fields = []
for fieldno in range(numfields): #PYTHON3
    name, typ, size, deci = struct.unpack('<11sc4xBB14x', f.read(32))
    name=name.decode('utf-8') #PYTHON3
    typ=typ.decode('utf-8') #PYTHON3
    name = name.replace('\0', '')       # eliminate NULs from string
    fields.append((name, typ, size, deci))
yield [field[0] for field in fields]
yield [tuple(field[1:]) for field in fields]

terminator = f.read(1)
terminator=terminator.decode('utf-8') #PYTHON3
assert terminator == '\r'

fields.insert(0, ('DeletionFlag', 'C', 1, 0))
fmt = ''.join(['%ds' % fieldinfo[2] for fieldinfo in fields])
fmtsiz = struct.calcsize(fmt)
for i in range(numrec):
    recordb = struct.unpack(fmt, f.read(fmtsiz))
    record=[]
    for j in range(0,len(recordb)):
        record.append(recordb[j].decode('utf-8')) #PYTHON3
    if record[0] != ' ':
        continue                        # deleted record
    result = []
    for (name, typ, size, deci), value in zip(fields, record):
        if name == 'DeletionFlag':
            continue
        if typ == "N":
            value = value.replace('\0', '').lstrip()
            if (value == '') or "*" in value:
                value = -99
            elif deci:
                value = decimal.Decimal(value)
            else:
                value = int(value)
        elif typ == 'D':
            y, m, d = int(value[:4]), int(value[4:6]), int(value[6:8])
            value = datetime.date(y, m, d)
        elif typ == 'L':
            value = (value in 'YyTt' and 'T') or (value in 'NnFf' and 'F') or '?'
        elif typ == 'F':
            value = float(value)
        result.append(value)
    yield result
1

Pyshp is the library you are looking for. It's well documented, step by step. Personally have found it great after osgeo ogr.

0

Geopandas is definately the way to go! Objects are brought in pandas like structure. Super easy to use + you can run all the typical geospatial operations if needed.

# just the dbf
dbf = geopandas.GeoDataFrame.from_file(u"./name.shp")
# whole shapefile
shp  = geopandas.read_file(u"./name.shp")
5
  • I would expect the dbf call to have a faster run time than the whole shapefile, but I'm seeing no difference.
    – jsta
    Mar 12, 2021 at 15:50
  • The question asks for how to read the .dbf file, but your example points to a .shp file. Maybe your answer can include why this is the same thing?
    – Olsgaard
    Jun 2, 2022 at 7:51
  • .dbf files are part of a relational database that always include a .shp .proj and a few other files that make up a "shapefile"
    – mmann1123
    Jun 6, 2022 at 14:39
  • WARNING: geopandas relies on GDAL and fiona, both of which are compiled and can introduce complications with deploying to the cloud or working with different platforms. Sep 23, 2022 at 19:34
  • Gotta disagree. The cloud runs on Linux where compiling is the norm, installing gdal etc is pretty trivial at this point. The only problem with scale/cloud is anything ESRI.
    – mmann1123
    Sep 24, 2022 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.