Slow FileGDB Performance in GDAL

Question

I am attempting to write many ASPRS LAS point files into an ESRI File Geodatabase using my compilation of GDAL 1.9.2. The FileGDB driver for GDAL/OGR appears to be incredibly slow when writing large files, taking as much as 45 minutes to write just 8 million point records. Write speeds of FileGDB using GDAL on a SATA3 drive are somewhere on the order of 200 kilobytes per second, which is unacceptably slow when I'm trying to convert terrabytes of data.

I noticed in the FileGDB documentation that defining the FGDB_BULK_LOAD macro should improve performance for large datasets, but I noticed no change in performance when I wrote a line into the "nmake.opt" file with the text "FGDB_BULK_LOAD = YES" immediately after the FGDB_LIB line.

Admittedly, a FileGDB is not the ideal way to store billions of point data records, but that's a gripe for another time. Have I correctly used the FGDB_BULK_LOAD feature? Is that supposed to be in my source code, not the GDAL build?

Thanks.

UPDATE: Proper Usage: (Answered in chat)

The FGDB_BULK_LOAD setting is properly stored as an environment variable for the GDAL/OGR process. This is set on the command line during the ogr exe call as shown by Ragi. Using the GDAL functions, it can be set programmatically for the entire process with

 CPLSetConfigOption("FGDB_BULK_LOAD", "YES");

or just for the current thread using

 CPLSetThreadLocalConfigOption("FGDB_BULK_LOAD", "YES");

FGDB_BULK_LOAD must be set prior to calling FGdbDataSource::CreateLayer(). It was unclear whether OGRCleanupAll() unset this variable, but it is safe to call multiple times to be sure.

Using that option boosted performance to be around 5.5x faster for writing millions to tens of millions of points.

Answer 1

FGDB_BULK_LOAD is not a compilation setting, it is a configuration option for the command line tools (can also be done programmatically).

ogr2ogr --config FGDB_BULK_LOAD YES -f "FileGDB" MyFileGDB.gdb myKML.kml

Would create a filegdb and load the KML vector data to it. Let me know if your performance still sucks. By the way, what platform are you on?

Update:

Just to summarize what was discussed in chat and IRC:

For programmatic access:

 OGRDataSource* datasource = driver->Open("c:/myfilegdb.gdb", TRUE); 
 char* options[] = { "FGDB_BULK_LOAD=YES" , NULL }; 
 OGRLayer newlayer = datasource->CreateLayer("tiny", NULL, wkbPoint25D, options);

should be the equivalent.