0

I need to use the pyarrow package on QGIS 3 (using QGIS 3.32.2 'Lima') on Windows 11, and install it in OSGeo4W shell using pip:

pip install pyarrow,

which installs 13.0.0 by default as I'm writing this. The installed numpy of 1.25.2 satisfies the requirements of numpy>1.16.6 in pyarrow.

However, when trying to import pyarrow, I get the error message:

""" Traceback (most recent call last): File "C:\PROGRA~1\QGIS33~1.1\apps\Python39\lib\code.py", line 90, in runcode exec(code, self.locals) File "", line 1, in File "C:\PROGRA~1/QGIS33~1.1/apps/qgis/./python\qgis\utils.py", line 888, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:\PROGRA~1\QGIS33~1.1\apps\Python39\lib\site-packages\pyarrow_init.py", line 65, in import pyarrow.lib as _lib File "C:\PROGRA~1/QGIS33~1.1/apps/qgis/./python\qgis\utils.py", line 888, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ImportError: DLL load failed while importing lib: The specified procedure could not be found. """

Some comments regarding variations I've tried:

  1. Upgrading pyarrow and numpy with pip from the original installation did not help:

pip install pyarrow upgrade (same w./ numpy),

nor did upgrading pip.

  1. The error is the same regardless of whether the entire package is imported:

import pyarrow,

or individual functions:

import pyarrow.parquet as pq

  1. I don't think it's a python or pip issue, because about a dozen other packages are installed and used without any problem whatsoever

  2. I've tried downgrading to earlier pyarrow versions (e.g., pip install pyarrow==12.0.0), and tried 12, 11, etc, but to no avail.

  3. conda is widely recommended in python environments outside QGIS and I generally prefer it over pip, but I was not successful in installing it within OSGeo4W (using pip).

pip install conda

gives:

""" ERROR: Could not find a version that satisfies the requirement conda (from versions: 3.0.6, 3.5.0, 3.7.0, 3.17.0, 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 4.0.7, 4.0.8, 4.0.9, 4.1.2, 4.1.6, 4.2.6, 4.2.7, 4.3.13, 4.3.16) ERROR: No matching distribution found for conda """

Installing miniconda through git has also not worked:

pip install git

gives: """ ERROR: Could not find a version that satisfies the requirement git (from versions: none) ERROR: No matching distribution found for git """

I would prefer to stay within the QGIS Linux - has anybody encountered similar issues and/or found a reasonably easy way to get pyarrow to work?

Knowing that it hasn't worked for others is also useful, so feedback other than solutions is welcome.

3
  • Can't answer your question, just noting you can't mix conda and osgeo4w. If you want to use conda, install miniconda and then create a QGIS conda environment using the conda-forge channel which has a QGIS package.
    – user2856
    Commented Aug 30, 2023 at 0:46
  • 1
    user2856: Thanks for the feedback! With "QGIS Linux" I meant "QGIS Python", meaning the Python packaged with QGIS. Thanks also for the feedback RE: miniconda on osgeo4w. Resolution of this matter is not urgent, I will wait for feedback on possible other ways to get pyarrow to work in osgeo4w. The preferred product here is a plugin for QGIS on a Windows machine, it's a pity that pyarrow is the only package that doesn't work! Commented Aug 31, 2023 at 19:45
  • 1
    For the record: "QGIS Linux" was a brainfart, I meant "QGIS Python". Commented Aug 31, 2023 at 19:46

2 Answers 2

0

I also encountered this problem today and my QGIS version is 3.32.3. I then tried to resolve the "DLL load failed" error using PyCharm IDE and successfully executed "import pyarrow" when using python-qgis.bat as the Python interpreter.

import pyarrow error

only import pyarrow

However, if pyqgis and pyarrow are operated at the same time, the latter being executed will cause a dll error. Using one of them alone will not cause problems.

error 1

error 2

only import qgis.core

I think this pyarrow error is related to QGIS's own environment configuration, and this problem should probably be solved by QGIS developers.

Or you can use other table processing libraries, such as pandas, polars, etc., instead of pyarrow, if there is such a demand.

0

Check the default installation version for pyarrow in the OSGeo4w setup.

QGIS version 3.34.6-Prizren solution:

pip install pyarrow==15.0.2

Using QGIS version 3.34.6-Prizren and python version 3.12.3, I ran into the same issue for several pyarrow versions. Using OSGeo4W setup I discovered the default instalation version for pyarrow is 15.0.2.

1
  • pip install pyarrow==15.0.2 ! Thank you ajruben, you save me !
    – Lionel
    Commented May 26 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.