2

I want to find number of records synchronized while using arcpy.SynchronizeChanges_management tool. Right now, I'm getting only messages like process is Executing, Synchronize successfully completed... and Succeeded at some time.

One way I thought of get count of all feature classes in dataset before and after the tool run and then compare the both counts. In this process I can only get the newly added but not the modified record or one record is added and other existing record is deleted from feature class then too the count will be same.

# sync parameters
#Path to source sde connection file
geodatabase_1 = source_sde 
in_replica = "replica_name" 
#Path to target sde connection file
geodatabase_2 = target_sde 
in_direction = "FROM_GEODATABASE1_TO_2"
conflict_policy = "IN_FAVOR_OF_GDB1"
conflict_defin = "BY_OBJECT"
reconcile = "DO_NOT_RECONCILE"

# call sync tool
sync_result = arcpy.SynchronizeChanges_management(geodatabase_1, in_replica,
                                           geodatabase_2, in_direction,
                                           conflict_policy, conflict_defin,
                                           reconcile)
# want to print synchronized records
print sync_result

How can I find the number of records(newly added and updated) synchronized?

  • I am not an expert on data replication, but have you looked at Export Data Change Message GP tool? – Alex Tereshenkov Dec 12 '14 at 9:39
  • @AlexTereshenkov: Hi Alex, would you please add your suggestion in answer. This worked for me. – Surya Dec 25 '14 at 5:23
1

Export Data Change Message GP tool could be used which creates a delta file with changes to synchronize (number of records could be obtained from the result).

3

There is only one way to efficiently do this that I know of. Load the replica database fields into a dictionary using a searchcursor with the GlobalID as the dictionary key, then loop through a searchcursor on the origin database and attempt to retrieve the GlobalID dictionary key that matches each record. Set up four counters: count, added, deleted, and changed to represent what is different going from the original to the replica. Then compare each item. Items in the original not found in the dictionary are being added. The deleted items will equal the total count of the replica minus (the count of the original minus the counted added items). Changed items will be detected on each matched record by comparing each field value (except ObjectID) until a change is detected. A change will be correctly detected even if all attribute values are the same and the only difference is found in the Shape field of the two matching features.

The code below works provide all fields and field names match in the replica and original and that the GlobalID field is named either "GlobalID" or "GUID". The code has been tested for the commented replicaFC and originalFC path assignments, but not the source_sde and target_sde connection path assignments you are using. In my test a replica feature class with 38,127 features was compared to an origin feature class with 38,126 features and correctly identified that there was 1 added feature in the original, 2 deleted features in the replica, and 1 changed feature with a change only to its shape. The comparison test took just over 24 seconds to complete.

import datetime
from time import strftime  

startTime = datetime.datetime.now()  
print "Start script: " + startTime.strftime("%Y-%m-%d %H:%M:%S")  

import arcpy  

# replicaFC = r"C:\Users\OWNER\Documents\ArcGIS\Centerline_Edit.gdb\CENTERLINE_ROUTES_1"  
replicaFC = source_sde + r"\replica_name"  

fields = arcpy.ListFields(replicaFC)
replicaFieldsList = []
for field in fields:
    if field.name in ["GlobalID", "GUID"]:
        replicaFieldsList.insert(0, field.name)
        print 'Key field is "' + field.name + '"'
    elif not field.type in ["OID"]:
        replicaFieldsList.append(field.name)

print ' '
print 'Comparing fields: ' + str(replicaFieldsList)

# Use list comprehension to build a dictionary from a da SearchCursor  
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(replicaFC, replicaFieldsList)}  

currentTime = datetime.datetime.now()  
elapsedTime = currentTime - startTime
print ' '
print "Dictionary created: " + currentTime.strftime("%Y-%m-%d %H:%M:%S")  
print 'Total Time Elapsed: ' + str(elapsedTime)
print ' '

# originalFC = r"C:\Users\OWNER\Documents\ArcGIS\Centerline_Edit.gdb\CENTERLINE_ROUTES"    
originalFC = target_sde + r"\orignal_name"    

originalFieldsList = replicaFieldsList  

count = 0
added = 0
deleted = 0
changed = 0  
with arcpy.da.SearchCursor(originalFC, originalFieldsList) as originalRows:  
    for originalRow in originalRows:
        count += 1  
        # store the Join value of the row being originald in a keyValue variable  
        keyValue = originalRow[0]  
         # verify that the keyValue is in the Dictionary  
        if not keyValue in valueDict:  
            added += 1
            print(str(keyValue) + ' was Added')
        else:
            for n in range (1,len(replicaFieldsList)):  
                if originalRow[n] != valueDict[keyValue][n-1]:
                    changed += 1
                    print(str(keyValue) + ' was Changed in field "' + replicaFieldsList[n] + '"')
                    break

deleted = len(valueDict) - (count - added)

print ' '
print('Added to Original = {0}; Deleted from Replica = {1}; Changed = {2}; Replica Count: {3}; Original Count: {4}'.format(added, deleted, changed, len(valueDict), count))

del valueDict  

currentTime = datetime.datetime.now()  
elapsedTime = currentTime - startTime
print ' '
print "Finished script: " + currentTime.strftime("%Y-%m-%d %H:%M:%S")
print 'Total Time Elapsed: ' + str(elapsedTime)

See my blog post on Turbo Charging Data Manipulation with Python Cursors and Dictionaries for other ways to do data transfers and summaries using this technique.

  • Hi Richard, thanks for the help. This is good if I have 1 or 2 layers with 30-35K records. But, my problem is I have very large number of layers in the dataset(like 20-25) and each layer contains more than 50000 records. The suggested process will take too much time to execute. Also, the memory footprint of the valueDict dictionary could be very large if records increases in future. – Surya Dec 24 '14 at 7:29
  • I have used dictionaries with 800,000 records, so you are no where near close to exceeding the memory footprint. The 20 to 25 datasets may be as simple as a for loop, since the code is designed for all data inputs provided the schemas are the same between replicas. As far as speed, I guarantee there is nothing faster than this code. Did you try it? 50,000 records to 50,000 record should finish in 30 seconds or less, with most of that being to load your datasets and only a fraction of that required for the actual comparisons. It is the only thing fast enough to get you what you need. – Richard Fairhurst Dec 24 '14 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.