I have a large network dataset and want to perform an analysis on it which would output the start/end point, the distance between them and whether they're first, second or third (etc.) nearest neighbours. There are approximately 30,000 nodes in the dataset, so I thought it might be possible to process 1,000 at a time (still comparing to all 30,000 nodes, but only doing the analysis on part of the dataset). I have an idea of how to do it (using a for loop, creating new files for each part of the analysis and putting it together) but I'm not certain as to how to create a for loop to use only part of the data and to go through and eventually finish analysis on all data.

Does anyone know if this is possible?

I have a large network dataset and want to perform an analysis on it which would output the start/end point, the distance between them and whether they're first, second or third (etc.) nearest neighbours. There are approximately 30,000 nodes in the dataset, so I thought it might be possible to process 1,000 at a time (still comparing to all 30,000 nodes, but only doing the analysis on part of the dataset). I have an idea of how to do it (using a for loop, creating new files for each part of the analysis and putting it together) but I'm not certain as to how to create a for loop to use only part of the data and to go through and eventually finish analysis on all data.

Does anyone know if this is possible?

Edit: I've been having thoughts about the partitioning by rowID and was wondering then if I were to be making a service area analysis layer if then the layer would be impossible to create due to the partitioning or otherwise incorrect because of it?