Hot answers tagged csv
15
Here's a simple script that uses the OGR python bindings:
import ogr,csv,sys
shpfile=r'C:\Temp\test.shp' #sys.argv[1]
csvfile=r'C:\Temp\test.csv' #sys.argv[2]
#Open files
csvfile=open(csvfile,'wb')
ds=ogr.Open(shpfile)
lyr=ds.GetLayer()
#Get field names
dfn=lyr.GetLayerDefn()
nfields=dfn.GetFieldCount()
fields=[]
for i in range(nfields):
...
12
I once had the same question.
The answer is that you put a file in the same folder with the ending csvt.
For example your file name is xyzdata.csv you add the file xyzdata.csvt
This one you can edit with editor for example. And in it you set the data type like this.
"Integer","String","Integer","Integer","String","Real"
Integer is now the variable for the ...
11
Since the Lat and Long field values "32.363544","-110.969778" have quotes around them, they aren't really treated as numeric.
The display x-y data needs the x and y fields to be numeric.
Remove the surrounding quotes and it should work. If you have access to a unix machine grep/sed sure would make this job easier. There are also windows ports of these ...
8
The "Add delimited text layer" tool does not interpret the CSVT. Only "add vector layer" needs a CSVT. Basically, "add delimited text layer" tries to make a best guess and it usually does very well recognizing numeric values. One of the reasons to use "add vector layer" is that "add delimited text layer" requires a geometry column.
Does your CSV contain a ...
8
My quick fix is to create the first row all with dummy values, and then delete this row/record after bringing into in ArcGIS.
This first row contains representative values or often wildly different values (e.g. alphabetic characters even if the column contains numbers that I want to be text data type) and with the largest number of characters needed for ...
7
There are 2 immediate options that would best get you where you are trying to go.
Your first, and probably easiest option, would be to download
Quantum GIS, using the OSGeo4W Installer.
Once you have installed that program, follow this tutorial, which
covers Importing a .csv file to QGIS.
Once you have imported the file, simply right click on the layer ...
7
If you convert your shapefile to spatialite, you should be able to do the following:
1) Experiment with SQL to test the output:
ex.
SELECT col1, col2, col3, AsKml(geometry_column)
FROM tab
2) Once you are satisfied with the result, you can export it to CSV format:
http://stackoverflow.com/questions/5776660/export-from-sqlite-to-csv-using-shell-script
...
7
Ths may be a bit more complicated, but if my two-cents are worth anything (and if you're using MS Office products), I'd recommend importing your .csv into an MS Access .mdb database as a table. (Note: there is a 2GB size limit for an .mdb database).
You can then add that Access .mdb table into your ArcMap document and do a right-click > "Display x,y data":
...
7
Rather than messing with the input file, trying to get everything picture perfect, just use the field calculator to convert the text to a number / real. See the examples below:
7
PostgreSQL has a column limit of between 250 and 1600 "depending on column types", and supports spatial data and queries with the PostGIS extension. So I would be inclined to do two things:
First, where a column represents a category rather than free text, create a separate table with those categories, and replace the column with an integer ID and foreign ...
7
I recently dealt with the exact same issue with Statistics Canada census profile CSV files containing 2172 columns. You can import your csv into an ESRI File Geodatabase (FGDB) if you have access to ArcGIS. According to ESRI, the FGDB format can handle 65,534 fields in a feature class or table.
In my case, I was able to import my 2172 column wide CSV file ...
7
I have simplified your code and corrected the error by using the da module introduced in 10.1. It greatly streamlines the reading of data using cursors, and used in conjunction with the with command this code should be more stable than if it used the the older method of file access.
It works by making a list of all the fields and then removing the fields ...
7
A slight departure from the traditional CSV->ArcGIS layer could be to use ogr2ogr to create your shapefile from CSV BEFORE loading it into ArcMap.
When using OGR, you can manually create a CSVT file that describes your column types, similar to what ArcGIS attempts to do with the schema.ini file.
This post by @underdark on How to Specify Data Types of CSV ...
7
First, you have to load the data into QGIS, and tell it that the data should be treated as EPSG:31467. Normally you do that with right-click -> Set CRS for layer.
Then, save the data as shapefile in EPSG:31467, and add the file to the canvas. Check if QGIS has chosen the right CRS: The new layer should align with the old, and rightclick -> Set CRS should ...
6
I created the following data.csv file:
AttributeID
101
101A
101B
Then I imported the data using the Table to Table tool:
Open ArcMap
Expand Catalog window
Located data.csv
Right click and select Export > To Geodatabase (single)...
Specified Output Location to be my Geodatabase (I'm using SDE on SQL Server)
Specified Output Table
Note that the field map ...
6
Yet another option, this is more of a theory and programmatic one, using arcpy.
A polygon can consist not only of a single outer ring with a single inner donut hole -- they can be nested to an arbitrary number of levels.
Consider the following:
A topologically correct polygon's rings are ordered according to their containment relationship (source). ...
6
You will need to do some formatting of your data to make them into polygons. Merely labeling a field as WKT will not help. If you have a lot of files (which it sounds like you do), the most effective way will be to automate your solution by writing a script.
I was going to explain how when I decided that the simplest way would be to write an example script ...
5
Please have a look at the Copy Rows geoprocessing tool.
The code can be as simple as the following:
import arcpy
arcpy.CopyRows_management("C:/data/vegtable.csv", "C:/output/output.gdb/vegtable")
5
Qgis 1.8.0 has a bug in correct encoding of shapefiles. Until this is solved, you can try this workaround in Windows:
Browse to C:\Programs\Quantum GIS Lisboa\bin on Windows XP or C:\Programs (x86)\Quantum GIS Lisboa\bin on Windows 7. Look for qgis.bat and open it with a suitable editor (I have installed notepad++ for such things). After line 1, insert:
...
5
the attributes of a shapefile are stored in an extra dbase-file. for example: mypolygons.shp contains geometries, mypolygons.dbf contains attributes. via an id the attributes are connected to the polygons. so just simple load your dbf file into capable software (e.g libreoffice) , edit attributes and your done
5
I do this using R. There is a package called foreign which enables reading and writing of dbf files.
library(foreign) # load the package
data <- read.dbf('myfile.dbf') # read the dbf as a dataframe
In this case I just wanted to change the rounding of a column pop_den:
data$pop_den <- round(data$pop_den) # trim data (removing significant figures)
...
5
I have seen this behaviour in other software when the number of entries in a row doesn't match the number of field headers, or the data type changes at one line (ie a string field finding a numeric entry or visa versa). You would be advised to examine the rows carefully, particularly if you are using a spreadsheet to create the *.csv since mixed data types ...
5
With ArcGIS 10 you can create the feature class and set all field types first, then load the .csv file using ArcCatalog. Simply right-click the feature class and choose Load > Load Data. Then follow the Simple Data Loader dialog to match fields, etc. It is a simple loader in that there are not many options but it works for well-formatted data. I'm not sure ...
4
I think you were missing closing your output text file - until you do that, you won't get anything written to the text file. Below code tested and works:
import arcpy
arcpy.env.workspace = r"C:\Users\chad\Documents\ArcGIS\Default.gdb"
arcpy.BuildRasterAttributeTable_management("raster17", "Overwrite")
outfile = open(r"D:\temp\raster17.txt", "w")
rows = ...
4
You're looking at the right question, just not the right answer! The reason that that worked for the answer you were looking at is that the data was only in points. Since your data is in polygons, it won't work.
BWill's answer further down is what you need. It will copy all the co-ordinates describing each polygon into WKT format, which is a standard ...
4
if u use python, you can handle csv file for generating geojson form it....
the basic way:
import csv, json
file = open( 'test.csv', 'r' )
reader = csv.DictReader( file , fieldnames = ( "id","name","attribute","lat","lng" ) )
outfile = json.dumps( [ row for row in reader ] )
print outfile
out = open("output.txt", "w")
out.write(outfile)
out.close()
...
4
You can use Layer > Add delimited text layer
For more about this, you can read: Visualising WKT geometry string in QGIS
4
On a simple level, if you define a bounding box for the Czech Republic you can construct an SQL query to test if the X and Y coordinates are within the maximum and minimum coordinates of the bounding box.
The query would be something like:
SELECT * FROM [TABLE] WHERE
[X_FIELD] BETWEEN [BOUND_XMIN] AND [BOUND_XMAX]
AND
[Y_FIELD] BETWEEN [BOUND_YMIN] AND ...
4
I was able to export to CSV, using other than a comma, by separating the layer creation options in the Save As.. dialog with linebreaks.
Neither comma, nor space-separating them (even when they were in quotes) worked, but the linebreaks did the trick. To emphasize..
THIS APPROACH WORKED (linebreak-separated):
GEOMETRY=AS_WKT
SEPARATOR=SEMICOLON
...
3
Oddly enough, you are better off in the FOSS world than proprietary software. I don't know why. Anyway, some geospatial tools that support export to CSV include OGR2OGR, PostGIS, Spatialite, QGIS etc.
Of course, it is simple enough to write your own exporter using Python (or ArcPy in your case) so maybe ESRI haven't felt the need to give full support.
...
Only top voted, non community-wiki answers of a minimum length are eligible
