2

I am creating a FME script that will accept to databases (MDB file and SDE Database) and will update the SDE database if there are any changes to the MDB file. I have a CRCCalculator for both SDE and MDB file to determine if there are any changes and a UniqueIdentifier is present on both.

Its currently working but I'm noticing that when FME writes the features to the SDE database (e.g. FMEMultiArea), in some cases it changes the sequence of the parts therefore changing the area value. This will in turn change the CRC value calculated when i run the FME script again so everytime it thinks there is an update.

I'm wondering if i need to add something to prevent the writer from changing the sequence.

FME Workflow: enter image description here

Sample MDB Feature: enter image description here

Sample SDE Feature: enter image description here

I am using SQL Server 2012 to store the data.

  • 1
    Are you sure that it's FME which is changing the order? The SE_shape object constructor of ArcSDE will repair ring order when incorrect vertex sequencing is detected (PGDB does not care). You've also got serious issues with running a CRC on coordinates that are encoded through the SE_shape API, since integer encoding/decoding may shift each coordinate +/- 1/(xyscale*2) (which in your case doesn't seem well-defined). – Vince Feb 24 '15 at 21:51
  • 2
    It's probably also important to note that data is never stored "in SDE" -- ArcSDE is only the access protocol; the data is stored in an RDBMS (which you haven't specified). – Vince Feb 25 '15 at 5:37
4

There's a piece of the puzzle that your workflow does not consider:

  • Enterprise geodatabase shapes all pass though the ArcSDE API, which performs standardization of geometries (integer encoding, dropping duplicate vertices, reordering incorrectly wound rings, merging inversions into their parent ring, etc)

This impacts you in two ways:

  1. The vertex ordering may be different (personal geodatabase and shapefile don't care), and
  2. The vertex values exported may be shifted by up to one-half the inverse of the xyscale, zscale, and mscale associated with the feature class coordinate reference.

In order to overcome these differences, you'll need to be ready to export geometries back to personal geodatabase as part of a validation pass.

It would also help if you used standard whole number (preferably power of 10) values for xyscale (10000000 vice 4167839431.4937829), zscale (1000 vs 778658.748954) and mscale, which will round coordinates to a fixed number of places, and will make computing a CRC easier.

  • Hi Vince, you're right i didn't have any values set on the XYSCALE field for any of my SDE Writers as i was not sure what to put in them. My MDB reader also has no settings on its min/max x and y. I am not sure how to "export geometries back to personal geodatabase". Is there a specific FME transformer that i can use? Thanks – Juno Feb 24 '15 at 23:13
  • 1
    These are really two or three additional questions, one of which is at least partially answered by a 30-page whitepaper. If you don't know how coordinate references work in ArcSDE, you really aren't likely to have much success using enterprise geodatabases. – Vince Feb 25 '15 at 1:09
  • Thanks Vince. I'm seeing that i don't know enough about coordinate references than i thought. I will go through your advices. – Juno Feb 25 '15 at 16:10
1

Those are great answers by Vince. I think he's spot on with how, for SDE, FME will need to possibly make changes to meet SQL Server specifications (like vertex ordering, removing duplicate points, etc).

I did want to suggest, re the grid/coordinate issue, that you try the ArcSDEGridSnapper transformer. It simulates the effects of writing to ArcSDE (in terms of grids). The help doc says:

Simulates the ArcSDE conversion on a feature by performing ArcSDE translation, scaling, and coordinate snapping. Also removes duplicate vertices resulting from snapping multiple formerly separate vertices to the same grid point. These coordinates will also be recorded in a list attribute.

So running your MDB data through that transformer will, I think, ensure it has the same settings as the SDE version, in terms of coordinate precision.

Alternatively you could try the CoordinateRounder transformer. It will obviously reduce your precision, but I'm thinking for gross matching of features it might not be an issue. The ArcSDEGridSnapper still should be the better solution.

Sadly I can't think of a simple fix for the vertex order issue. One extreme solution is writing a duplicate of all multi-part features to a separate (non-SDE) dataset, just for the purposes of later comparisons! Or you could try calculating CRC values on coordinate values (ie extract them as attributes) in which case order might not be an issue. Or round-trip all your MDB data through SDE and back, just to have it go through the same standardization process.

  • Thanks Mark. I did add a bandaid solution for now by adding an area calculator for both data since i noticed that even though the have different area values in each individual feature, when they pass through the area calculator they are equal. Of course this is not a definite solution and i am still familiarizing myself on the conversion. I will also try your advice. Thanks again. – Juno Feb 26 '15 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.