3

Not exactly sure how to word this, but hopefully I do it in a way that accurately conveys what it is that I'm trying to do...

I'm using Manifold GIS for this particular project, but I'm open to any methods that will achieve the desired result (less ArcGIS as I do not have any licensed ESRI software).

I started with a dataset that contained a series of points. Using a script posted by a Manifold user, I was able to convert these points into line segments (which may or may not be necessary to do what I need).

I have data like the following:

ID      US_NUMBER    Absolute_Distance_ft   DESCRIPTION    FEATURE_TYPE_ID   Line id
365694    5          0.059                  GWD            30                5
365695    0          0.059                  Weld Start     77                0
365696    0          1.814                  VLV 1          7                 0
365697    7          3.56                   GWD            30                7
365698    0          4.177                  Offtake        78                0
365699    10         4.783                 GWD            30                10
365700    0          5.774                  Weldolet       80                0
365701    20         6.709                 GWD            30                20
365702    0          7.602                                 5                 0
365703    0          9.209                  Bend           2                 0
365704    30         11.683                GWD            30                30
365705    40         20.705                GWD            30                40

Each line segment starts where the "FEATURE_TYPE_ID" is 30 and has a "US_NUMBER" of n. The next line starts at the next 30 and has a "US_NUMBER" n1, and so on...

As you can see in the table, "US_NUMBER" 5 is the start of one line segment. The next relevant line segment is "US_NUMBER" 7. The two segments between 5 and 7 are 0 and 0.

I want to go through the entire line and update each segments "Line ID" so that every segment that has a Line ID of 0 is set to the "US_NUMBER" where the previous "FEATURE_TYPE_ID" was 30...

So I would have something like...

ID      US_NUMBER    Absolute_Distance_ft   DESCRIPTION    FEATURE_TYPE_ID   Line id
365694    5          0.059                  GWD            30                5
365695    0          0.059                  Weld Start     77                5
365696    0          1.814                  VLV 1          7                 5
365697    7          3.56                   GWD            30                7
365698    0          4.177                  Offtake        78                7
365699    10         4.783                 GWD            30                 10
365700    0          5.774                  Weldolet       80                10
365701    20         6.709                 GWD            30                 20
365702    0          7.602                                 5                 20
365703    0          9.209                  Bend           2                 20
365704    30         11.683                GWD            30                 30
365705    40         20.705                GWD            30                 40

Does that make sense?

  • You could use pythons csv module (you would need to convert the data to csv first) to read each line capturing the Line ID. I would create 2 variables Pre_Feat_ID and Current_Feat_ID to store the current and previous feature id. Then I would use an if statement to check to see if the Current_Feat_ID == 0, if true, update Current_Feat_ID with Pre_Feat_ID value. I am not really familiar with Manifold, but this was the first thing that came to mind – TsvGis Apr 14 '16 at 0:17
  • Sounds simple enough. So simple it just may work!! I'll give it a try and see if I can make it work. Thanks! – Fernando Vega Apr 14 '16 at 15:37
  • I'm running into some issues with my logic on this one... Would you be able to post an example to illustrate your thought process? – Fernando Vega Apr 19 '16 at 22:24
1

This would have been my solution:

import csv

results_csv = r"D:\results.csv"
# This is the location of the csv containing your results

modified_results_csv = r"D:\modified_results.csv"
# This is the location of the csv that will contain the modified results

header = ["US_NUMBER","Absolute_Distance_ft","DESCRIPTION","FEATURE_TYPE_ID",
            "Line id","ID"]
# This is the header information for the columns in the csv

Pre_Feat_ID = 0
# This variable will hold the line id number if it is not = 0

with open(modified_results_csv,"w") as csv_file:
# This creates the modified_results.csv and writes in the column headings

            writer = csv.DictWriter(csv_file, fieldnames = header,
                                    lineterminator='\n')

            writer.writeheader()

with open(results_csv,"r") as results_csv_file:
# This reads the results.csv

    reader = csv.DictReader(results_csv_file)

    for row in reader:
        # loops through rows and assigns row attributes to variable

        row_id = row["ID"]

        row_us_number = row["US_NUMBER"]

        row_abs_dist = row["Absolute_Distance_ft"]

        row_desc = row["DESCRIPTION"]

        row_feat_type_id = row["FEATURE_TYPE_ID"]

        row_line_id = row["Line id"]

        if row_line_id  != "0":
        # If the Line ID  is not = 0, then assign the Line Id value to Pre_Feat_ID
        # and append  the row to modified_results.csv

            Pre_Feat_ID = int(row_line_id)

            with open(modified_results_csv,"a") as mod_results_csv_file:

                writer = csv.DictWriter(mod_results_csv_file, fieldnames = header,
                                        lineterminator='\n')

                writer.writerow({"ID": row_id,"US_NUMBER": row_us_number,
                                "Absolute_Distance_ft": row_abs_dist,
                                "DESCRIPTION": row_desc,
                                "FEATURE_TYPE_ID": row_feat_type_id,
                                "Line id":row_line_id})

        else:
        # If the Line ID is 0, then assign the Pre_Feat_ID to the Line
        # ID value and append the row to modified_results.csv

             with open(modified_results_csv,"a") as mod_results_csv_file:

                writer = csv.DictWriter(mod_results_csv_file, fieldnames = header,
                                        lineterminator='\n')

                writer.writerow({"ID": row_id,"US_NUMBER": row_us_number,
                                "Absolute_Distance_ft": row_abs_dist,
                                "DESCRIPTION": row_desc,
                                "FEATURE_TYPE_ID": row_feat_type_id,
                                "Line id":Pre_Feat_ID})

print("Done")

The out line of the code is as follows:

  1. Create a new csv (modified_results.csv) to contain the new results using the header information from the original results
  2. Read the results from the original results.csv and assign the row values to variables
  3. Use a conditional statement to see if the Line ID value is not equal to 0. If true, assign the Line ID value to the Pre_Feat_ID variable and then write the row to modified_results.csv. If the condition is false, then the Pre_Feat_ID variable is assigned to the Line ID and then the row is written to modified_results.csv

Here are the results of the script: original results.csv original results.csv

modified_results.csv modified_results.csv

Note: In the modified_results.csv I have moved the ID column to the end to avoid excel popping up an error regarding the csv being a SYLK file.

  • That's it! I was trying to do with lists and was running into issues twith my poor looping logic... Dicts much better idea and much faster! Thank you, sir! – Fernando Vega Apr 20 '16 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.