Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

An arcpy merge process is failing due to the size of the datasets and I need to just merge every 100 tiles before it goes on to the next 100 tiles.

I wrote the following snippet but it is just putting the same tif into the list. It's probably something simple that I am doing wrong but would appreciate your advice....

arcpy.env.workspace=root+"\\Stage1\\"+name+"\\GeoTif"
tifList=arcpy.ListRasters('*.tif')
tifListCount=len(tifList)

tempTifList=[]
x=0
oldx=0
while x <= tifListCount:
    for tif in tifList:
        while x<(oldx+100):
            tempTifList.append(tif)
            x=x+1

        print "List "+str(oldx)+" is"+ str(tempTifList)
        # Add merge process on tempTifList
        tempTifList=[]
        oldx=x

Gives

, u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif', u'PFTI_Roads2_Z31.tif'] List 75900 is[u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif', u'PFTI_Roads2_Z32.tif

share|improve this question
add comment

2 Answers

up vote 6 down vote accepted

You have erroneously nested a loop that is iterating over the same tif - i.e., basically on the innermost you have: while x is less than 100, loop on the tif list, increment x by 1

So the while loop executes inside your 'for' loop, starting with the 1st tif. The intended mechanism of the 'for' loop doesn't hand off successive list tifs because x has to run until reaching its limit of 100.

Then, probably do something like this instead, since ListRasters returns a Python list-- EDIT: The 1st code I posted was untested and didn't function as I expected. Properly working code is posted below (and there is likely a more straightforward way, but at any rate it works):

x = 1
for i in range(0, tifListCount, 100):
    for j in range(i, 100*x):
        if j + 1 <= tifListCount:
            tempTifList.append(tifList[j])
    print tempTifList
    # do your processing here
    tempTifList = []
    x += 1
share|improve this answer
    
Thanks. It almost works but if tifListCount is 760 it is only creating 6 lists with records upto 699. Have been trying to resolve this without success. BTW/ you should use the code tags on the top of the screen when you enter code as it is easier to read/copy paste. –  GeorgeC Nov 3 '12 at 13:21
    
The new version with "j" works fine. Thanks mate... –  GeorgeC Nov 3 '12 at 13:30
    
George - try the edited code above that I reposted. Sorry, the previous code didn't do the job, so try the code I actually tested above. Thanks for your patience. -Wayne –  T. Wayne Whitley Nov 3 '12 at 13:31
add comment

This StackOverflow answer has a more concise, Pythonic suggestion:

>>> print list(izip_longest(*(iter(range(10)),) * 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

You could incorporate it like this:

from itertools import izip_longest
for i, group in enumerate(izip_longest(*(iter(tifList),) * 100)):
    group = filter(None, group) # Remove None objects from group
    #Merge group here
share|improve this answer
    
Wow, fantastic, I can't wait to try that! I've been trying to write tighter code, so thanks for that lead. –  T. Wayne Whitley Nov 4 '12 at 13:02
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.