Why do regular expressions that work on regex101.com not work in qgis?

Question

I'm having trouble getting the qgis field calculator to validate regular expressions that work on regex101.com and also getting the same results between the two.

I'm trying to extract the digits and letters into separate fields.

example string in column ref:

CR 267;SR 1025

1. In qgis, I can extract 267 with:

   regexp_substr( "ref",'(^\d[0-9]+)')
   

   regex101.com will only validate this if I remove the ^:

   \d[0-9]+
   

   and then returns 267 and 1025.

2. now to extract letters. In qgis:

   regexp_substr( "ref",'(^\D[a-z]+)')
   

   does not validate. On regex101.com:

   ^\D[a-z]+
   

   returns CR

So, is there something fundamental that I'm missing about the way regular expressions work in qgis?

Answer 1

QGIS is simple PCRE syntax -- what most libraries and languages have moved to use. In PCRE regex's ^ anchor the pattern at the beginning of a string.

/foo/

matches anywhere

/^foo/

Matches only at the beginning of the string. However (^\\d[0-9]+) shouldn't work at all unless MultiLineOption is set in the QGIS source -- and it's not so a ^ anywhere but at the first character should be taken literally.

So I have no idea what you're doing that's working but rather than trying to figure that out I'll teach you about regexes.

• if you don't care where your pattern is anchored don't use ^ at all.
• also \\d[0-9] is very awkward. That is essentially saying give me a digit, and then any character in the range of 0-9 (also likely just a digit). Which is the same as \\d\\d, also the same as \\d{2}

Perhaps this will help you learning regexes, but I have no idea what you're trying to do or how what you've done is working.

If you want to extract two groups of one or more digits in the above pattern perhaps you want either

/(\\d+)/g

which reads capture a group (one or more) of digits, globally across the regex.

/(\\d+).+?(\\d+)/

which reads capture a group of digits followed by an ignored group of characters and then capture another group of digits.

Update

So if you want to match, CR 267;SR 1025 and extract CR, 267, SR, and 1025, you likely want

/([A-Z]+) (\\d+);([A-Z]+) (\\d+)/

There are shorter ways to write this in QGIS 3.0 (with regexp_matches), but for 2.18 that's probably the best you'll get. With 3.0, you could write.

/([^ ;]+)/g

If you wish to learn PCREs you can read the Perl docs on regexes.

Answer 2

Don't forget to double escape your slashes - eg:

regexp_substr( "ref",'(^\d[0-9]+)')

should be

regexp_substr( "ref",'(^\\d[0-9]+)')

and

regexp_substr( "ref",'(^\D[a-z]+)')

should be

regexp_substr( "ref",'(^\\D[a-z]+)')

It's also important to note that QGIS < 3.0 does not support non-greedy regex, so something like (\\d*?) will not work. In QGIS 3.0 the regex engine has been upgraded substantially and supports these.