3

I used the Python Module Pyshp to select particular census blocks from the Census's shapefiles and package these into a single shapefile. However when I add this data into ArcMap and right click to identify blocks, I get attributes for multiple blocks nowhere near the intended block. At first I assumed I had written my script in a way that misaligned the records and shapes objects in pyshp, but when I read this shapefile back into python, everything appears to be aligned correctly (i.e. I can look up the record I want and find the correct shape at the same index and vice versa).

Does anyone have a suggestion as to the next place I can look to try and solve my problem?

Here is the class I built to do this. The order of operations is to call load to get a particular zip file from the census, use add_blocks to collect any that match my list and add them to the writer object, and then write_shapefile to save whatever I've collected.

class CensusConnection:

    def __init__(self):
        self.file_name = ''
        self.ftp_path = 'ftp://ftp2.census.gov/geo/tiger/TIGER2010/TABBLOCK/2010/'
        self.reader = None
        self.writer = shapefile.Writer()

    def load(self,state_fips, county_fips = ''): 
        '''Use me to load all blocks into a folder in pwd that match state FIPS, or state FIPS plus county FIPS.'''
        #import data and set up reader object
        self.file_name = 'tl_2010_' + state_fips + county_fips + '_tabblock10' 
        tempdir = tempfile.mkdtemp()
        if not os.path.exists('tl_2010_' + state_fips + county_fips + '_tabblock10'):
            print 'Retrieving ' + self.ftp_path + self.file_name + '.zip...'
            sock = urllib2.urlopen(self.ftp_path + self.file_name + '.zip')
            memfile = io.BytesIO(sock.read())
            ZipFile(memfile, 'a').extractall(tempdir + "/" + self.file_name)
        self.reader = shapefile.Reader(tempdir + "/" + self.file_name + "/" + self.file_name)
        self._dump()
        #set up writer object to match reader
        self.shapes = self.reader.shapes()
        self.fields = self.reader.fields
        self.records = self.reader.records()
        self.writer.fields = self.fields

    def _dump(self):
        if (os.path.exists(self.file_name)):
            shutil.rmtree(self.file_name)

    def add_blocks(self, block_keys): 
        '''Take a block key, find that block and add shapefiles to the writer object'''
        '''Block Key = StateFIPS + CountyFIPS + TractID + GroupID + BlockID (15 digit)'''
        fields = self.reader.fields
        records = self.reader.records()
        for index in range(len(records)):
            if(records[index][4] in set(block_keys)):
                self.writer.shapes().append(self.reader.shapes()[index])
                self.writer.records.append(self.reader.records()[index])

    def write_shapefile(self,dir_name):
        if(len(self.writer.records)>0):
            if(os.path.exists(dir_name)):
                shutil.rmtree(dir_name)
            os.mkdir(dir_name)
            self.writer.save(dir_name+"/"+dir_name)
  • Sorry to say, but the most likely explanation is that your script is wrong. Can you post it? – phloem Jan 15 '15 at 19:55
  • just posted it. I appreciate the help. – logan Jan 15 '15 at 20:02
  • Just curious if you need to add '()' to records: self.writer.records().append(self.reader.records()[index]) – phloem Jan 15 '15 at 20:58
  • It looks like a quirk of this module that writers are accessed directly as a list rather than through a method like you do with shapes(). A tutorial I found seems to do the same thing (see the code from that tutorial here: cimms.ou.edu/~lakshman/spatialprogramming/chapter03_basicgis/… ) – logan Jan 15 '15 at 21:59
  • 1
    When you get a chance, I am hoping you can write up what you did to help future readers encountering the same problem with PyShp. It is fine to answer your own question here. – PolyGeo Jan 17 '15 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.