3

I solved my own question a couple weeks ago on how to create percentile breaks in QGIS.

Creating percentile class breaks in vector data QGIS using numpy.percentile

However, this code is unbelievably slow. Does anyone have any recommendations for speeding things up? See code below:

case
    when ag_ppm <= q1(ag_ppm)
        then '<25th: ' || minimum("ag_ppm") || ' - ' || q1(ag_ppm)
    when ag_ppm < median(ag_ppm) AND ag_ppm > q1(ag_ppm)
        then '25th to 50th: ' || (q1(ag_ppm)+0.001) || ' - ' || median(ag_ppm)
    when ag_ppm <= q3(ag_ppm) AND ag_ppm > median(ag_ppm)
        then '50th to 75th: ' || (median(ag_ppm)+0.001) || ' - ' || q3(ag_ppm)
    when ag_ppm <= percentile(90,array_agg(ag_ppm,project)) AND ag_ppm > q3(ag_ppm)
        then '75th to 90th: ' || (q3(ag_ppm)+0.001) || ' - ' || percentile(90,array_agg(ag_ppm,project))
    when ag_ppm <= percentile(95,array_agg(ag_ppm,project)) AND ag_ppm > percentile(90,ag_ppm)
        then '90th to 95th: ' || (percentile(90,array_agg(ag_ppm,project))+0.001) || ' - ' || percentile(95,array_agg(ag_ppm,project))
    when ag_ppm <= percentile(98,ag_ppm) AND ag_ppm > percentile(95,ag_ppm)
        then '95th to 98th: ' || (percentile(95,array_agg(ag_ppm,project))+0.001) || ' - ' || percentile(98,array_agg(ag_ppm,project))
    when ag_ppm <= maximum(ag_ppm) AND ag_ppm > percentile(98,array_agg(ag_ppm,project))
        then '>98th: ' || (percentile(98,array_agg(ag_ppm,project))+0.001) || ' - ' || maximum(ag_ppm)
    else 'NULL'
end
Improve this question
5
  • 1
    You can change the case when to several if statements, then you compute and store the variables that are reused (ex: start by computing valq1 = q1(ag_ppm) and then use valq1 in the comparisons). It should cut the processing time in half at least. Then you can reorder the comparisons, moving away from the median (most likely bin if you have a normal distribution) to avoid computing/comparing stats of outliers first
    – JGH
    Commented Nov 25, 2021 at 18:18
  • thanks, I'll give that a try. Although it seems that even a single if statement with percentile takes forever with 1000+ records. I'm wondering if the numpy percentile function is a bit chuggy
    – jrowley
    Commented Nov 26, 2021 at 23:19
  • UPDATE: but even a single if statement takes 5+ minutes to classify. eg. if(ag_ppm <= percentile(90,array_agg(ag_ppm)),'<25th: ' ,NULL)
    – jrowley
    Commented Nov 26, 2021 at 23:48
  • 1
    As it is, you are computing the classification and the percentiles for every feature, so more than 1000 times. Try using the graduated symbology, or the rule based symbology, and modify your python function so you call it once only and it populates the class breaks. Once the breaks have been defined, the classification is done comparing ag_ppm to static values only so it will be much faster
    – JGH
    Commented Nov 29, 2021 at 13:54
  • I have tried using rule-based which is just as slow. I didn't think you could use a formula for breaks in graduated symbols. I have no idea how to do this if it's possible
    – jrowley
    Commented Nov 29, 2021 at 16:16

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.