0

So, I have tested adding columns with the WKT format into Carto, and it has always worked great (I tried 1 row, then 1000 random rows from our data). I geocoded it on load by naming the column with the WKT the_geom. When I uploaded all the data, some of the data was corrupted (random numbers and letters, no format). After testing, I have found the following:

The csv being used is normal and fine.

When the data is mixed (some the_geom values are blank, some are in WKT format), less than 40,000 rows can be added (40K doesn't work, 39.5K does)

The data that is corrupted is the first 24,000-25,000 rows (it varies) that have a value in the_geom column (only those first ones, no matter whether its 40K rows or 1.1 million).

If I upload 39K rows, the same rows that were corrupted before are now fine and geocoded.

I can load as many rows as I want, as long as the_geom is in WKT format OR blank, but not mixed.

All of this to say, I found a work around (upload 2 separate tables [one with the_geom WKT format, the other blank], then using sql union to combine them). Unfortunately, I cannot make a table out of these 2 because I have reached the size limits for our account (Xentity). Any ideas what is causing this? If not, is there any way we could do a one time extension of the DB size, in order to combine the tables (then delete the 2 halves)?

Improve this question
1

There is a lot going on here, so I'm going to try to break it into a segmented set of answers.

I geocoded it on load by naming the column with the WKT the_geom

This is not advised. It is always better to update the exiting the_geom column with values from any other column. The reason is that the_geom column has a number of triggers and dependancies. So renaming another column to the_geom will create broken tables.

less than 40,000 rows can be added (40K doesn't work, 39.5K does)

Have you tried completely different datasets with the same result? I mean, totally different, not the same source or shared rows or anything? There is no row number limit in CartoDB currently, so this is likely only about table size or an error in the data format/encoding. At least as far as I know. Or perhaps there is a funny line break in the mix orrrr a row with too many columns. Often times, it helps to open a file in OpenOffice or Excel and resave it as a totally new CSV.

Any ideas what is causing this?

Could you provide a link to a file that fails?

Is there any way we could do a one time extension of the DB size

If we can't figure it out, then you'll have to email support@cartodb.com and mention this thread and that no fix was found.

Improve this answer
2
  • So, how would you reference the geometry from a separate column? For instance, I have a column named polygon, and an example entry is "POLYGON ((-85.68288600036 35.7669269996909 -85.68288600036 36.0820520002434, -85.2131150000475 36.0820520002434, -85.2131150000475 35.7669269996909, -85.68288600036 35.7669269996909))". Once this is loaded from a csv into Carto, what needs to be done to georeference the data?
    – James Brown
    Apr 30 '14 at 22:18
  • 1
    I would write it into the_geom, UPDATE table_name SET the_geom = ST_SetSRID(polygon::geometry, 4326) WHERE polygon != '' and polygon NOT NULL
    – andrewxhill
    Apr 30 '14 at 22:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.