6

I'm having trouble getting the qgis field calculator to validate regular expressions that work on regex101.com and also getting the same results between the two.

I'm trying to extract the digits and letters into separate fields.

example string in column ref:

CR 267;SR 1025
  1. In qgis, I can extract 267 with:

    regexp_substr( "ref",'(^\d[0-9]+)')
    

    regex101.com will only validate this if I remove the ^:

    \d[0-9]+
    

    and then returns 267 and 1025.

  2. now to extract letters. In qgis:

    regexp_substr( "ref",'(^\D[a-z]+)')
    

    does not validate. On regex101.com:

    ^\D[a-z]+
    

    returns CR

So, is there something fundamental that I'm missing about the way regular expressions work in qgis?

  • What do you want to extract? Your question is not clear. – Evan Carroll Nov 11 '16 at 3:39
10

QGIS is simple PCRE syntax -- what most libraries and languages have moved to use. In PCRE regex's ^ anchor the pattern at the beginning of a string.

/foo/

matches anywhere

/^foo/

Matches only at the beginning of the string. However (^\\d[0-9]+) shouldn't work at all unless MultiLineOption is set in the QGIS source -- and it's not so a ^ anywhere but at the first character should be taken literally.

So I have no idea what you're doing that's working but rather than trying to figure that out I'll teach you about regexes.

  • if you don't care where your pattern is anchored don't use ^ at all.
  • also \\d[0-9] is very awkward. That is essentially saying give me a digit, and then any character in the range of 0-9 (also likely just a digit). Which is the same as \\d\\d, also the same as \\d{2}

Perhaps this will help you learning regexes, but I have no idea what you're trying to do or how what you've done is working.

If you want to extract two groups of one or more digits in the above pattern perhaps you want either

/(\\d+)/g

which reads capture a group (one or more) of digits, globally across the regex.

/(\\d+).+?(\\d+)/

which reads capture a group of digits followed by an ignored group of characters and then capture another group of digits.

Update

So if you want to match, CR 267;SR 1025 and extract CR, 267, SR, and 1025, you likely want

/([A-Z]+) (\\d+);([A-Z]+) (\\d+)/

There are shorter ways to write this in QGIS 3.0 (with regexp_matches), but for 2.18 that's probably the best you'll get. With 3.0, you could write.

/([^ ;]+)/g

If you wish to learn PCREs you can read the Perl docs on regexes.

  • 1
    "a ^ anywhere but at the first character should be taken literally." And other than negating character groups such as [^a-f] which matches any character except a-f. – Phil G Nov 11 '16 at 7:50
  • @PhilG I had mention of the character group when I first wrote this answer, but I factored it (and other stuff) out to keep it simple. – Evan Carroll Nov 11 '16 at 8:43
  • 1
    Sorry if I wasn't clear--I have a string in the field 'ref' of: 'CR 267;SR 1025' I want to turn it into 4 new fields: field1: CR, field2: 267, field3: SR, field4: 1025. regexp_substr( "ref",'(^\d[0-9]+)') DOES work to return '267'. regexp_substr( "ref",'((\d+)/g)') and regexp_substr( "ref",'((\d+).+?(\d+)/)' do not validate. – jamierob Nov 11 '16 at 15:57
  • @jamierob updated. – Evan Carroll Nov 11 '16 at 16:10
  • @EvanCarroll thanks! your update does work on regex101.com to extract all the values into groups as desired. My problem of translating these working expressions to qgis remains. Maybe i'm just not typing things correctly? regexp_substr( "ref",'([A-Z]+) (\d+);([A-Z]+) (\d+)') does not validate in qgis 2.16.3. – jamierob Nov 11 '16 at 16:17
1

Don't forget to double escape your slashes - eg:

regexp_substr( "ref",'(^\d[0-9]+)')

should be

regexp_substr( "ref",'(^\\d[0-9]+)')

and

regexp_substr( "ref",'(^\D[a-z]+)')

should be

regexp_substr( "ref",'(^\\D[a-z]+)')

It's also important to note that QGIS < 3.0 does not support non-greedy regex, so something like (\\d*?) will not work. In QGIS 3.0 the regex engine has been upgraded substantially and supports these.

  • For the first one, adding the escape slash before \d returns no matches. Without the escape slash, returns '267'. For the second example, in qgis, both return no matches, instead of 'CR'. on regex101.com, adding the excape slash returns no matches instead of 'CR' with ^\D[a-z]+ i'm confused. – jamierob Nov 11 '16 at 15:48
  • i figured out what I was missing here to make these work in qgis. the ^ character needed to be removed and additional parens around the regex (in addition to the parens required by the qgis syntax. so: regexp_substr( "ref",'((\\d[0-9]+))') and regexp_substr( "ref",'((\\D[A-Z]+))') – jamierob Nov 11 '16 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.