3

I have multiple ‘pairs of images’ that need to be mosaicked but I cannot find how to automate selection of the 'paired' file names as inputs into a model. Each has a file name such as LS7_ETM_NBAR_2011-01-22T23-47-13.tif (top) LS7_ETM_NBAR_2011-01-22T23-47-37.tif (bottom). a top image and a bottom image taken 24 secs apart!

At the moment, I have to physically select every second or alternate file to input into the model. But I have to process 1000s of images.

enter image description here

Ideally I would like to use an iterator but I don’t think I can because of the double model input.

I have been scouring the forums, Perhaps there is a bulk rename utility method that would be able to ‘match’ the first 35 characters? But surely someone has had this problem before with having to match cloud masks to reflectance grids?

Any ideas?

  • why mosaic? A mosaic dataset is much quicker to make and offers all the benefits of a single raster and some more without having to physically mosaic anything. Do you have ArcGis 10.2+? – Michael Stimson Jul 5 '15 at 7:39
  • Have you found a solution yet? If so, could you please update your question with the answer. Otherwise I have an option to propose. – fatih_dur Jul 6 '15 at 3:35
  • No I have not found a solution but have a friend looking into some AML or bulk utility rename code for python. My issue is that I am not yet great at coding and love the model builder interface both for ease for myself but explaining it to others... – JkSmits Jul 6 '15 at 8:47
  • Michael - Yes I have ArcGIS 10.3 - I thought that was what I was doing, is there an easier way? – JkSmits Jul 6 '15 at 8:48
  • 1
    Perhaps you can create an old school image catalog - paths to rasters and sort this table. Select pairs using iterator with step 2 and provide them as inputs to your model. From the picture provided it is what you are doing at the moment – FelixIP Jul 6 '15 at 9:36
2

The following is a Python script that uses itertools.groupby for a very efficient grouping approach. itertools creates a list of lists (i.e. grouped pairs), that is then iteratively passed to the Mosaic To New Raster (Data Management) tool.

import arcpy, itertools, os

# Define the input and output workspaces
arcpy.env.workspace = r'C:\path\to\your\workspace'
outws = r'C:\path\to\your\output\workspace'

# List the rasters in your workspace
rasters = arcpy.ListRasters()

# Group the items based on first 35 characters
grouped = [list(g) for _, g in itertools.groupby(sorted(rasters), lambda x: x[:35])]

# Loop over the groups and perform the mosaic operation
for group in grouped:
    outname = group[0][:35] + "_mosaic.img"   # Name the output mosaic here
    arcpy.MosaicToNewRaster_management(group, outws, outname, number_of_bands = 1)
  • So this was a winner! In the end I used: import arcpy, itertools, os ... ... arcpy.env.workspace = r'E:\test' ... outws = r'E:\test\output' ... ... rasters = arcpy.ListRasters() ... ... grouped = [list(g) for _, g in itertools.groupby(sorted(rasters), lambda x: x[:35])] ... ... for group in grouped: ... outname = group[0][:35] + "_mosaic.tif" ... arcpy.MosaicToNewRaster_management(group, outws, outname, pixel_type = "32_BIT_SIGNED", number_of_bands = 6, mosaic_method = "FIRST") ... – JkSmits Jul 8 '15 at 8:22
  • 1
    +1 itertools has some very useful functions! – Paul Jul 8 '15 at 15:27
  • Would this work for the 9th to 35th character something like x[9:35]? – JkSmits Jul 9 '15 at 1:11
  • Yes, you can test it out for yourself by slicing the x variable as you mentioned (e.g. x[9:35]) within the list comprehension (i.e. grouped). Then simply use print grouped to see which items are grouped before you proceed to the for loop. – Aaron Jul 9 '15 at 1:24
1

This is a Model Builder solution by using Calculate Value tool (this is how Python scripting starts for many model builders :)) by assuming your rasters are in one workspace and their names are generated by using time stamps (which allows sorting, i.e., ascending order will yield correct TOP and BOTTOM parts):

enter image description here

Here are the specifics of each Calculate Value tools:


  • Calculate Value

Expression:

my_rasters

Code Block:

arcpy.env.workspace=r"C:\Temp\YOUR_RASTER_WORKSPACE"

my_rasters= str(sorted(arcpy.ListRasters()))


  • Calculate Value (4)

Expression:

len(%my_rasters_output%)-1


  • Calculate Value (2)

Expression:

%my_rasters_output%[%Value%]


  • Calculate Value (3)

Expression:

%my_rasters_output%[%Value%+1]


EDIT: Lastly, For loop parameters:

For Value=0
To Value=len
By Value=2

  • I really REALLY appreciate this. I will try it overnight and let you know! Thanks again. – JkSmits Jul 7 '15 at 12:28
  • Thank you faith_dur - could i ask for the For iterator settings To Value From Value By Value? Also, are the output type the default 'variant?' – JkSmits Jul 7 '15 at 21:38
  • Sorry @JkSmits, now they are in the EDIT part. – fatih_dur Jul 7 '15 at 22:19
  • Thanks so much - so I got it to run, was very excited that it ran all the processes but then was confused how to harness this new list in my model? Not a big deal because Aaron posted some code that I was able to get working, not that I have EVER used python before (except in uni for 2 mins) – JkSmits Jul 8 '15 at 8:29
0

If the first 35 characters are always the same for a pair of bottom/top images, then the following is a good starting place. Regex would be useful for more advanced matching techniques:

#Note that the list of files doesn't have to be in order.
files = ["LS7_ETM_NBAR_150_-027_2011-01-22T23-47-13.tif",
         "LS7_ETM_NBAR_150_-027_2011-01-22T23-47-37.tif",
         "LS7_ETM_NBAR_150_-027_2011-04-28T23-47-58.tif",
         "LS7_ETM_NBAR_150_-027_2011-08-22T21-52-09.tif",
         "LS7_ETM_NBAR_150_-027_2011-04-28T23-47-29.tif",
         "LS7_ETM_NBAR_150_-027_2011-08-22T21-52-45.tif"]

#We're only interested in the first 35 characters for matching
stripped = [x[:35] for x in files]

indices = []

#Create list of tuples containing similar names.
for name in stripped:
    #This is a simple way to get the index locations of similar names.
    indices.append(tuple((i for i,j in enumerate(stripped) if j == name)))    

#Since links are both-ways, remove duplicate entries.
for i,(top,bot) in enumerate(list(set(indices))):
    #Use files[top] and files[bot] as inputs to mosaic, etc.
    print "TOP{}\t{}\nBOT{}\t{}\n".format(i, files[top], i, files[bot])

Output:

TOP0    LS7_ETM_NBAR_150_-027_2011-01-22T23-47-13.tif
BOT0    LS7_ETM_NBAR_150_-027_2011-01-22T23-47-37.tif

TOP1    LS7_ETM_NBAR_150_-027_2011-04-28T23-47-58.tif
BOT1    LS7_ETM_NBAR_150_-027_2011-04-28T23-47-29.tif

TOP2    LS7_ETM_NBAR_150_-027_2011-08-22T21-52-09.tif
BOT2    LS7_ETM_NBAR_150_-027_2011-08-22T21-52-45.tif
  • Thank you, I will try my best to use this code (I am not very good at it ...yet! – JkSmits Jul 7 '15 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.