I'm in the process of adding automatic CRS conversion to a Spark application and running into some problems with the implementation.

Each row in a dataframe that is being converted will contain columns Latitude, Longitude, and EPSGCode and will result in new columns for the converted Lat/Long in WGS84.

Spark itself isn't the issue (included for context only), as wiring this up to run within a UDF is fairly simple.

Right now, I'm running into issues with a common conversion, given EPSGCode 4267 and targeting 4326, I either get an error or just get back the original coordinates.

I've tried two libraries...

OrbisGIS CTS: https://github.com/orbisgis/cts (throws error)

Proj4j: https://github.com/locationtech/proj4j (returns same points)

Here's a sample of my scala code using CTS:

class DatumConversionSpec extends FunSpec {

  val cRSFactory = new CRSFactory()
  val registryManager: RegistryManager = cRSFactory.getRegistryManager
  registryManager.addRegistry(new EPSGRegistry)

  def getCRSFromEpsgCode(code: Int): GeodeticCRS = cRSFactory.getCRS(s"EPSG:$code").asInstanceOf[GeodeticCRS]

  it("handles NAD27 to WGS84 conversion") {

    val source = getCRSFromEpsgCode(4267)
    val target = getCRSFromEpsgCode(4326)


    // throws exception here!!!
    val ops = CoordinateOperationFactory.createCoordinateOperations(source, target)

    val op = CoordinateOperationFactory.getMostPrecise(ops)


which throws the exception java.lang.IllegalArgumentException: No transformation found from EPSG:4267 to EPSG:4326.

When using proj4j I just got back the original coordinates.

Is there another library that could recommended for this or a different approach I should be taking?

note: our Spark infrastructure runs Scala 2.11.8 currently against Java 8. Upgrading to a newer Java version to use another library is not an option right now

  • Turns out I get the same failure going from NAD83 to WGS84 as well. – Ron DeFreitas Sep 4 at 16:00
  • With proj4j, does the nad27 / 4267 definition include a +towgs84 parameter? CTS says for NAD27 that it needs file-based tfm and that's not implemented yet. It might be possible to add a three or seven parameter tfm instead into its definition but it won't be very accurate. – mkennedy Sep 4 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.