I'm organizing a large amount of data for a project and the data is stored in .csv format, with a code to relate each row to a polygon in a .shp. I was planning on converting the .csv data into .dbf and importing it into a geodatabase, all using ESRI tools. Is this the best way of organizing such data? (The alternative I thought of was to join the .shp and the .csv and to export the data so I have a permanent link, but it seems like I would have a lot of duplicated data)

I'm interested in learning how best to improve my work flow and organize this data. I am planning on doing it all using ArcGIS 10 and using python where possible.

  • 2
    What is the cardinality of the data to the geography? One-to-one? Many-to-one? – Sean Dec 14 '10 at 18:28
  • 2
    You should give some workflow background. do the CSV's get updated regularly? Is it possible to maintain both sources as one? Can you migrate to fbgd or even pgdb? – Brad Nesom Dec 14 '10 at 18:42
  • It's a once off issue of organizing the data - the csv files will not be update. The relationship between the data is one to one. – djq Dec 14 '10 at 18:57

joining .csv to .shp will be slow. but the best way will be to append the converted .csv file to .dbf and merged based on a unique attribute. You should be able to use model builder to automate most of this. Finally convert the shapefile to file geodatabase.

  • 1
    Converting to dbf is not without its problems. You are converting between two dubious formats. Much better to import the csv into Access first. – Matthew Snape Dec 16 '10 at 12:49
  • 1
    .csv to .txt [dos batch] open in Arcview 3.2 [very fast] save as .dbf works very well for the past 10+years.. – Mapperz Dec 16 '10 at 15:35

As Mapperz so well put it, the Join Operation between CSV and Shapefile is slow. Shapefile is a useful format but does have it's limitations, depending on your required output and operational constraints may I suggest you look at one of the Geodatabase formats available. There are 3 which may assist you at ArcGIS Desktop 10.

1) File Geodatabase - no logisitcal size limit, stored in the operating system folder structure, quicker and for non-edited data can be easilly compressed. Does have draw backs that you have to access in ArcGIS, so has no Textual only viewer. For the workflow identified you can then set up individual feature classes with the connections already done

2) Personal Geodatabase - 2GB size limit (4GB with options changes), stored in Access MDB so does run a bit slower due to the Jet Engine overhead, easilly import data as can use Microsoft Access to view and import the csv data. It is slower than File Geodatabase but could be an option

3) Personal SDE - if you have an ArcEditor License or higher you can install SQL Server Express and run Personal SDE (on the ArcGIS Desktop installation Media). This is a good option if you are looking at the possibility of expansion as gives you the full SDE capability which can be administered in Arccatalog. Has a size limit of 4GB specified by the database, and does require you to do more set up and installations.

Based on this I would say import all of your CSV data into an Access ACCDB (Access 2007 native Database Type) In ArcCatalog Create a new File Geodatabase, and import your shapefiles. Also create a new Personal Geodatabase, load all of your ACCDB Data into the Personal Geodatabase using an OLEDB Connection in ArcCatalog, and then import the Feature classes from the File Geodatabase. Append the fields and then save the data.

I hope this helps, CDB

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.