Tell me more ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.
up vote 1 down vote favorite

When working with large amounts of raw input data for your analysis, how do you track data from different sources and covering different geographic extents to determine when you can proceed with the next step in your workflow?

My company works with LiDAR data for large electrical transmissions systems (thousands of miles in one project). We model the transmission lines and do analysis using CAD and GIS workflows. We break the system into lines or circuits (1-150 miles each) in order to process the data in smaller units. The LiDAR data itself is broken up into blocks (less than 1sq mile). We always struggle with tracking which blocks have been delivered and which lines are ready for the next step in our workflow.

Has anyone had any experience tracking input data like this? Do you know of any software platforms out there designed for this particular task?

I appreciate any advice you can throw my way.

share|improve this question
1  
This is all a bit general, isn't it? Lacking specifics, the best anyone could do would probably be to suggest that your "workflow" should automatically send back a signal of some kind when a processing job is complete. You could have a database somewhere with a row for every dataset and a (boolean) column for every stage of completion, and check against that. Or you could include the stages of completion in the metadata. Of course this is more difficult if your workflow is manually driven, because then it depends on whether the operator remembers to update the table/metadata/etc. – R Thiede Nov 1 '11 at 9:10
@RTheide - I realize this question is broad and it is meant to be. We are less concerned with signaling when a process is complete and more concerned with determining if all of the necessary raw LiDAR data has been delivered from our collection partner. I know there are all sorts of business intelligence software platforms out there for managing workflows and documents and other contents, but I'm curious if anyone knows of an existing platform for managing lots of raw input data related to the workflow. Thanks – Brian Nov 1 '11 at 14:10
thanks for the explanation, that narrows things down a bit. Personally I'm unaware of what's going on with this type of software (in fact it's news to me that such things exist), but if you more specifically outline your use-case, you may get better help. Also consider reposting this question on Stack Overflow (even though your question has a GIS component) to improve your chances of getting a good answer. Good luck hey – R Thiede Nov 1 '11 at 14:52

closed as not a real question by iant♦, whuber♦ Jul 18 '12 at 15:20

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, see the FAQ.