3

I have an FME process that is duplicating some values in an attribute. When I try to use regex in FME to clean it up it is deleting the separator (see screengrab below). Any ideas on how I can change the regex or process in FME to just get the required values.

So from

164511|119958|164512|164511|119958|164512|164511|119958|164512
161634|161635|161636|161634|161635|161636|161634|161635|161636|161634|161635|161636|161634|161635|161636|161634|161635|161636
1114|1114|1114|1114|1114

I just want

164511|119958|164512
161634|161635|161636
1114

The value in AttributeManager from FME is @ReplaceRegEx(@Value(Name_Ids),"(?:^|\G)(\b\w+\b),?(?=.*\1)\|",\1)

I have tried to replace \b),?(? with \b)\|(?to use the | seperator but that doesn't work either. This is based on https://superuser.com/a/1343524/639219

The regex tester is https://regex101.com/r/1YhC59/1

enter image description here

3 Answers 3

2

You are almost there, instead of

@ReplaceRegEx(@Value(Name_Ids),"(?:^|\G)(\b\w+\b),?(?=.*\1)\|",\1)

you need

@ReplaceRegEx(@Value(Name_Ids),"(?:^|\G)(\b\w+\b),?(?=.*\1)\|","")
2
  • Thanks but for my future knowledge can you let me know what the ` \1` at the end does as against ""? That's the only difference I noticed.
    – GeorgeC
    Aug 26, 2020 at 22:54
  • 1
    I don’t know, fiddled around and it just worked :) Aug 26, 2020 at 23:08
1

You could use a PythonCaller as well to remove your duplicate values:

def processFeature(feature):
    ids = feature.getAttribute('Name_Ids')
    feature.setAttribute('Name_Ids', '|'.join(list(dict.fromkeys(ids.split('|')))))

enter image description here

1

Without regex you could always use:

  • An AttributeSplitter transformer to turn the attribute into a list
  • A ListDuplicateRemover transformer to remove duplicate list entries
  • A ListConcatenator transformer to turn the list back into an attribute

But also, could we stop the duplication happening from the very start? Can you clarify how that string is being created?

2
  • Wow, 3 transformers for a simple task ;-) ... would be nice if Attribute Manager could support Python!
    – christoph
    Aug 25, 2020 at 15:58
  • 1
    Or maybe we could ask Don and Dale for an expression engine with custom Python functions like QGIS offers us... only thinking loud.
    – christoph
    Aug 25, 2020 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.