Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

My script takes separate 3D feature classes, calculates a field, appends to a target feature class, and then deletes the originals. This is a huge data processing task of over 4700 mass point feature classes appending to a single one. I know it was a big mistake to put so many feature classes into SDE, but given they're in there now, I've got to deal with them there (yes, the sheer fc count is bringing SDE performance to its knees--hence my urgency).

Early on it worked great, in that the merged result showed valid z-values. I was able to build a terrain and it looked great. I'd been running this in the background for weeks, and have just noticed z-values of 0 in the output, where the originals were not 0! Oh no!!! I'm certainly glad I still have the file-based originals!

Pseudocode:

An in_memory feature class is created from a z-aware template

For each fc in the feature dataset:

It is appended to the empty in_memory fc

The TILE_ID field is calculated using a portion of the name of the source fc

It is then appended to the target SDE fc

The in_memory feature class is emptied

Each processed fc is then deleted

EDIT: When I initially posted, I was curious if it was my use of in_memory that caused this trouble. I had not initially been using it, and only switched to it because performance was far too slow otherwise, and WAY faster in_memory. To find where trouble started, I ran the Add XY Coordinates tool, which proved the trouble started with in_memory use. At that time I had also switched from CalculateField_management to da.UpdateCursor, so that's likely the bigger reason for the performance enhancement. I will now change my code to avoid in_memory.

Thank you to eric and PolyGeo.

share|improve this question
    
I've never had a problem retaining z values using in_memory (although this was in model builder, not in straight python). One thing that comes to mind, if you haven't already thought of it, is your z-aware template field the same type as the z field that it is being populated with? Seeing as how this was working for you before, I doubt this is the problem, but it's worth a shot. Also, can you try running the process on a sample set with proper results? –  eric Mar 8 '13 at 18:35
    
Thanks for the reply, @eric. I am suspicious of the in_memory use only because Esri is very non-specific in what they describe being supported in-memory (i.e. no extended geodatabase parameters...). The z-aware template field is simply the Shape field, because these are 3D features. The rows in the target fc all have Shape as Point ZM, as they do in the source FCs. So I'm not sure how the template could be different. I will try on Monday to test on a small sample set and see what happens. By then I'll have the results of the Add XY Coordinates tool to see where the 0 z-values began. Thanks –  Justin Mar 8 '13 at 21:27
    
It would not come as a total surprise to me if processing in_memory does not provide access to z values because I know it does not calculate Shape_Area while in memory - see here - you may or may not be encountering (something like) NIM063795 –  PolyGeo Mar 9 '13 at 6:31
    
@PolyGeo, you are correct about in_memory z values, because after Add XY Coordinates, POINT_Z values of 0 begin right where I changed my code to write to in_memory. I was then astounded by the speed increase... so much so that I didn't think to check for z values in the output. That was three weeks ago. If only I'd run Add XY Coordinates then! It's lousy if I cannot use in_memory, because CalculateField_management on normal fc took ~25 minutes, but dropped to <5 in_memory (mind you I also switched to using da here, so I'll see what happens with da on normal fc). Thank you to PolyGeo and eric. –  Justin Mar 11 '13 at 14:08
2  
@Justin, can you post this as an answer for other users who may encounter this same problem? –  scw Jul 5 '13 at 8:09
show 2 more comments

1 Answer

It would not come as a total surprise to me if processing in_memory does not provide access to z values because I know it does not calculate Shape_Area while in memory - see here - you may or may not be encountering (something like) NIM063795

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.