0

I have a live data feeds as an XML file from the Internet. I need to use Python and ArcPy lib to store each feed as a point feature in a feature class of an ArcSDE datbase.

The xml looks like:

<array-list>
    <spider-pos-wSVO brigade-no="36145" heading="280" altitude="302" longitude="146.09046833333332" latitude="-37.054605" personnel-no="0" speed="13">
        <attrA>12</attrA>
        <attrB>0</attrB>
    </spider-pos-wSVO>
    <spider-pos-wSVO brigade-no="36145" heading="280" altitude="302" longitude="146.09046833333332" latitude="-37.054605" personnel-no="0" speed="13">
        <attrA>12</attrA>
        <attrC>0</attrC>
    </spider-pos-wSVO>
    <spider-pos-wSVO brigade-no="36145" heading="280" altitude="302" longitude="146.09046833333332" latitude="-37.054605" personnel-no="0" speed="13">
        <attrA>12</attrA>
        <attrD>0</attrD>
    </spider-pos-wSVO>
    ....
</array-list>

Obviously each element has a fixed set of attributes (heading, altitude, longitude and latitude, etc etc) and a various set of child-elements (attrA, attrB, attrC and attrD, etc). I am able to store the fixed attributes as attribute fields in the SDE feature class but have no idea how and where to store the variable ones.

Has anyone got a best practise for it? Thanks

1

This calls for a Entity–attribute–value (EAV) data model.

There are many negatives to using this data model, but in cases like this, it is the only option.

The basic theory is that you have a table with the fixed attributes, and you have an additional table to store the additional(variable) attributes.

In your case, the main table cane be something like:
spider_pos(spider_id, brigade_no, heading, altitude, longitude, latitude, personnel_no, speed)

where spider_id is the primary key of the table. Make sure you don't use ObjectIDs for this, since you have no control over them.

Your variable data table will be something like this:

spider_data(spider_id, attribute_name, value)

Please note that I don't know about your data, so the names might not be really appropriate. Secondly, I don't know if time is involved. If it is, then you need to store that as well.

Additionally, read this wonderful answer by @underdark, which talks about how such a system us used for a temporal Meteorological database.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.