Error loading psycopg2 module dll load failed while importing psycopg

I created a project yesterday evening with 3.8.2 and ran into this problem. Error can be found below.
This morning I did a new venv with 3.7.6, exact same setup besides that, and there was no issue.

One remark… When installing the package on python38 (the -binary), it seems to compile it… But not on python37…
Installing psycopg2-binary on python37: https://gist.github.com/tobiasgardner/57ac84e45c8c5502ea5682852e300a31
Installing psycopg2-binary on python38: https://gist.github.com/tobiasgardner/7f93fcd22e413b525833adfe75c54d92

C:UserstobbeAppDataLocalJetBrainsToolboxappsPyCharm-Pch-0193.6494.30binrunnerw64.exe C:UserstobbeDocumentsgit-reposecombooster.venvScriptspython.exe C:/Users/tobbe/Documents/git-repos/ecombooster/manage.py runserver 8000 Watching for file changes with StatReloader Exception in thread django-main-thread: Traceback (most recent call last): File "C:Program FilesPython38libthreading.py", line 932, in _bootstrap_inner self.run() File "C:Program FilesPython38libthreading.py", line 870, in run self._target(*self._args, **self._kwargs) File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py", line 53, in wrapper fn(*args, **kwargs) File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocoremanagementcommandsrunserver.py", line 109, in inner_run autoreload.raise_last_exception() File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py", line 76, in raise_last_exception raise _exception[1] File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocoremanagement__init__.py", line 357, in execute autoreload.check_errors(django.setup)() File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py", line 53, in wrapper fn(*args, **kwargs) File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjango__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoappsregistry.py", line 91, in populate app_config = AppConfig.create(entry) File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoappsconfig.py", line 116, in create mod = import_module(mod_path) File "C:Program FilesPython38libimportlib__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 783, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocontribpostgresapps.py", line 4, in <module> from psycopg2.extras import DateRange File "C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagespsycopg2__init__.py", line 50, in <module> from psycopg2._psycopg import ( # noqa ImportError: DLL load failed while importing _psycopg: The specified module could not be found.

I installed psycopg2 using conda on Windows 10.

https://anaconda.org/anaconda/psycopg2

I did it in a clean new conda environment (named wr).

I then tried to run this sample app but I am getting this error (see below).
I have no idea what I might be doing wrong because it was all straightforward and I did it in a clean way.

Any ideas how to solve this?

import psycopg2
try:
    connection = psycopg2.connect(user = "***",
                                  password = "***",
                                  host = "***",
                                  port = "5432",
                                  database = "***")


    cursor = connection.cursor()
    # Print PostgreSQL Connection properties
    print ( connection.get_dsn_parameters(),"n")

    # Print PostgreSQL version
    cursor.execute("SELECT version();")
    record = cursor.fetchone()
    print("You are connected to - ", record,"n")

except (Exception, psycopg2.Error) as error :
    print ("Error while connecting to PostgreSQL", error)
finally:
    #closing database connection.
        if(connection):
            cursor.close()
            connection.close()
            print("PostgreSQL connection is closed")

Error in VS code:

PS C:WorkWRRgittoolsJTunnelTestApp>  cd 'c:WorkWRRgittoolsJTunnelTestApp'; & 'C:ProgramsAnaconda3envswrpython.exe' 'c:Userspetrop01.vscodeextensionsms-python.python-2020.9.114305pythonFileslibpythondebugpylauncher' '56143' '--' 'c:WorkWRRgittoolsJTunnelTestAppmain.py'
Traceback (most recent call last):
  File "c:WorkWRRgittoolsJTunnelTestAppmain.py", line 1, in <module>
    import psycopg2
  File "C:ProgramsAnaconda3envswrlibsite-packagespsycopg2__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: DLL load failed while importing _psycopg: The operating system cannot run %1.
PS C:WorkWRRgittoolsJTunnelTestApp>

EDIT: Seems they had a bug open for this 2 years ago and they just closed it, ignoring it completely.

https://github.com/psycopg/psycopg2/issues/734

Python 3.8: ImportError: DLL load failed while importing _psycopg: The specified module could not be found. #1006

TL; DR: update pip. See this comment

I get this error when trying to import psycopg2 in Python 3.8 on Windows 10. I already tried reinstalling (both psycopg2 and psycopg2-binary) and even reinstalling Postgres. Seems to work fine on Python 3.7.

The text was updated successfully, but these errors were encountered:

I think this is actually a different issue, this is a problem with importing the library, not with installation.

You just have installed Postgres in your os and because library installs successfully.

Even Im facing same issue. Any luck with this ?

Nope, looks like this won’t get fixed until Appveyor adds Python 3.8 support. I had to just downgrade back to Python 3.7 for now. :/

So the fix is waiting for new release to support 3.8 and walkaround is switching to 3.7 ？

FYI, if someone wants to fix the appveyor script and install Python 3.8 there for the build, be my guest.

So correct me if i’m wrong, but what i got from comments in other related issue threads here, the appveyor change/update is required to build and provide the binary package of psycopg2 for Python 3.8.

The problem is that the issue reported here, contrary to #1004, doesn’t seem to be related to the lack of the binary package.

Because of the lack of a binary version, the package seems to be built by pip from sources locally, and i can also build it myself by downloading it manually.
In both cases it doesn’t work when imported (at runtime) in Python 3.8 while it does in Python 3.7.
I clearly do pass the build prerequisites and runtime requirements.
The same error appears in Python 3.7 when i remove psql bin folder from PATH.
Is there a way to get the exact dll file that was not found from the ImportError? maybe that would shine some light here.
Should the appveyor issue also affect building the package from sources locally?

To be honest, psycopg2’s setup.py does a VERY poor job of building from source on end user’s Windows. If someone is interested in improving it.

The windows environment has several challenges when building from source vs LINUX/UNIX, some of which:

• Need to have the correct Visual C compiler installed based upon Python version (how many of us have Visual C++ 14 installed?)
• Detect the location of the PostgreSQL build dependencies (includes/libs)
• Detect if we also need any OpenSSL dependencies based upon the PostgreSQL build and where those lib/include files are located
• If linking against DLL version of libpq, make sure all the appropriate DLLs (and correct versions) are accessible via the path

Also, the PostgreSQL binaries were probably built with a newer version of the Visual C compiler, causing the potential for multiple LIBC libraries being included into the process space when pscopyg2 is imported (This shouldn’t be the case, but past experience is why we build ALL dependencies in Appveyor).

Appveyor has added Python 3.8 support. Could this please be a priority now?

I’m having this same problem.

Traceback (most recent call last):
File «main.py», line 9, in
import kepo
ImportError: DLL load failed while importing kepo: The specified module could not be found.

how do i fix this problem

Hi folks, I had the same problem and using python 3.7.2, I don’t know if it works on python 3.8 or not, but I just downgraded psycopg2-binary to 2.7.* and everything is good.

Febrary 2020 — still the same problem

@vsmelov what pip version are you using? What is the command output, possibly with a -v ?

So, Smelov Vladimir @vsmelov: your observation is perfectly correct: it is February 2020. You know your calendar. Now, in this rainy month, what brings you to dump a passive-aggressive comment on a closed bug? Because this bug has been long closed and last psycopg release is compatible indeed with Python 3.8. Binary packages are tested after build and do work for me.

If you think you have a problem please report details of what you are trying to do and about your environment. Otherwise you are just someone who thinks he is on twitter and want to create toxic atmosphere, which is a disrespectful behaviour for people who have made available for your their work for free, and for which I have no sympathy in my project.

Waiting for your answer.

I created a project yesterday evening with 3.8.2 and ran into this problem. Error can be found below.
This morning I did a new venv with 3.7.6, exact same setup besides that, and there was no issue.

One remark. When installing the package on python38 (the -binary), it seems to compile it. But not on python37.
Installing psycopg2-binary on python37: https://gist.github.com/tobiasgardner/57ac84e45c8c5502ea5682852e300a31
Installing psycopg2-binary on python38: https://gist.github.com/tobiasgardner/7f93fcd22e413b525833adfe75c54d92

C:UserstobbeAppDataLocalJetBrainsToolboxappsPyCharm-Pch-0193.6494.30binrunnerw64.exe C:UserstobbeDocumentsgit-reposecombooster.venvScriptspython.exe C:/Users/tobbe/Documents/git-repos/ecombooster/manage.py runserver 8000 Watching for file changes with StatReloader Exception in thread django-main-thread: Traceback (most recent call last): File «C:Program FilesPython38libthreading.py», line 932, in _bootstrap_inner self.run() File «C:Program FilesPython38libthreading.py», line 870, in run self._target(*self._args, **self._kwargs) File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py», line 53, in wrapper fn(*args, **kwargs) File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocoremanagementcommandsrunserver.py», line 109, in inner_run autoreload.raise_last_exception() File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py», line 76, in raise_last_exception raise _exception[1] File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocoremanagement__init__.py», line 357, in execute autoreload.check_errors(django.setup)() File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoutilsautoreload.py», line 53, in wrapper fn(*args, **kwargs) File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjango__init__.py», line 24, in setup apps.populate(settings.INSTALLED_APPS) File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoappsregistry.py», line 91, in populate app_config = AppConfig.create(entry) File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangoappsconfig.py», line 116, in create mod = import_module(mod_path) File «C:Program FilesPython38libimportlib__init__.py», line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File » «, line 1014, in _gcd_import File » «, line 991, in _find_and_load File » «, line 975, in _find_and_load_unlocked File » «, line 671, in _load_unlocked File » «, line 783, in exec_module File » «, line 219, in _call_with_frames_removed File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagesdjangocontribpostgresapps.py», line 4, in from psycopg2.extras import DateRange File «C:UserstobbeDocumentsgit-reposecombooster.venvlibsite-packagespsycopg2__init__.py», line 50, in from psycopg2._psycopg import ( # noqa ImportError: DLL load failed while importing _psycopg: The specified module could not be found.

@tobiasgardner thank you for the report.

The psycopg packages are available here, this is where pip would fetch them from.

My questions are:

• What is your pip version?
• Pip should download a wheel file. What file does pip download psycopg2 download? The binary package (which I assume psycopg2-2.8.4-cp38-cp38-win_amd64.whl but let me know which) or the .tar.gz ?
• If it downloads the .tar.gz I wonder why, so what is the output of pip download -v psycopg2 ?
• If it downloads the .whl , why it doesn’t install it? So what is the output of pip install -v psycopg2-2.8.4-cp38-cp38-win_amd64.whl ?

On windows the binary package and the non-binary package are the same, but please try the above with psycopg2-binary instead of psycopg2 too, thank you.

@dvarrazzo Hmm, there is something strange here. Noticed now that when I check pip version from PyCharm environment (settings), it says pip version 20.0.2 for Python 3.7 BUT when I run pip list in the terminal, it says 19.0.3. For the python 3.8 environment, it is the same. But when I try to upgrade from command prompt, it says that I already have 20.0.2.

Staying in the same venv from command prompt.

I.e. your assumption that the .whl file is downloaded was wrong, I get the tar.gz files.

Since I did not get the .whl file, it is not possible to run this command: pip install -v psycopg2-2.8.4-cp38-cp38-win_amd64.whl , it results in an error message:

Источник

Python 3.8: ImportError: DLL load failed while importing _psycopg: The specified module could not be found. #1014

Just installed Python 3.8 64-bit and was able to successfully install psycopg2 with pip install psycopg2 . I received this ImportError when trying to run my project.

The text was updated successfully, but these errors were encountered:

You already said it #1006. Please follow #990.

Unlike in #990, I am able to install psycopg2 with Python 3.8 64 on Windows 10

The error indicates it is unable to load the psycopg DLL.

Based upon what I believe your file structure is, the file should exist here:
C:. venvlibsite-packagespsycopg2_psycopg.cp38-win_amd64.pyd

Also, you should be able to look in Windows Event viewer and find an error message that maybe describes the reason a little better why it failed (ie unable to find, corrupt file, etc).

The other thing, with every command prompt (or however you run your project), you need to ‘activate’ the venv, otherwise the paths will not be setup correctly, and it will not be able to find the DLL.

@BrendanMartin, it should be pycharm’s bug, however, you can run through by up pic.
I use python3.8.1-64bit ,win10, pycharm 2019.3.1 version.

Источник

Unable to install python 3.8 windows 10 #990

Windows 10 Python 3.8

Windows 10 python 3.7 — Good

The text was updated successfully, but these errors were encountered:

Appveyor doesn’t support Python 3.8 yet.

Support for Python 3.8 on Windows will be added when appveyor will add it.

When does Appveyor usually add a new version of python?

So . what do you need to fix this?

@FrankyBoy That either appveyor makes 3.8 available on their machine, or that someone contributes a patch to the appveyor.py script to install it ourselves.

Hi! Sorry for potentially stupid questions, new to python and appveyor (mainly did c# on gitlab-ci lately). Regarding that script you mentioned, am I correct that this would be some solution akin to this here: installing pypy on appveyor? Also, if I adapt the script, how can I actually test it before making a PR?

Python 3.8 is now available on AppVeyor (appveyor/ci#3142). (Where, incidentally, I have the same issue as #1006)

pip uninstall psycopg2-binary
pip install psycopg2-binary —no-cache-dir

It’s not clear whether this is closed because it’s believed the problem is fixed, but. it’s not. Trying to install either psycopg2 or psycopg2-binary on Windows via Pip will error out with a message saying «Error: pg_config executable not found.»

@Kylotan, the problem is fixed. My guess is you are inadvertently reinstalling the cached version of the previous package. @hingston provided the correct steps by adding the parameter:
—no-cache-dir
onto the pip command line.

You can verify pip is using a cache file if it displaying the following after running the pip install command:

For me also doesn’t work. For a fresh Python 3.8 installation I have:

As far as I can see psycopg2-2.8.4-cp38-cp38-win_amd64.whl is there, at https://pypi.org/project/psycopg2/#files
Please someone debug why pip decides not to pick it up, thank you.

It works for me with both 32bit and 64bit binaries from Python.org, so there is something with your setup that is forcing it to download the source file package instead of the binary package.

One could try the following pip switch to force not use source packages, tho it doesn’t explain why it is attempting to download the sources, and I half way expect this not to work, tho hopeful the error message may explain more of what is going on:
—only-binary :all:

If the above doesn’t work, issue the following command and copy the output:
pip debug psycopg2-binary —verbose

Источник

Configuring a Django PostgreSQL project is a straight-forward process that involves updating the settings.py file and adding one additional library (psycopg2). Depending on which IDE, OS, and Python version used there are several possible errors that can arise. We’ll address those here as well.

Django comes configured to work with a local SQLite database by default, which is great for early-stage app development. Unfortunately, it’s not great for production use or even some more stringent development tasks like concurrent I/O. Below are the steps needed to configure a new Django project for use with a PostgreSQL database.

Django uses some third-party libraries by default but does not install the required libraries to support postgres. Here’s a look at the packages installed after a clean install of Django on a new virtual environment:

asgiref==3.3.1
Django==3.1.4
pytz==2020.4
sqlparse==0.4.1

Django is designed to use a project-folder sqlite database by default that is configured in the project/project/settings.py and to use the django.db.backends.sqlite3 adapter as such:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}

This tells Django to use a database named BASE_DIR/db.sqlite3 as the project’s database. To use a postgres database, the following steps must happen:

1. Specify the correct Django adapter + credentials for Postgres
2. Install necessary 3rd party library for Postgres
3. Specify credentials to new/existing postgres database

These steps are simple enough but can benefit from some elaboration.

Step 1: Specify Postgres Adapter + Credentials

To make Django work with Postgres, an adapter other than the default sqlite adapter must be specified. I like working with the psycopg2 Postgres adaptor and have shown how to specify that adapter below:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': "db_name",
        'USER': "db_username",
        'PASSWORD': "db_password",
        'HOST': "db_host",
        'PORT': "db_port",  # 5432 by default
    }
}

In older versions of Django, the adapter was named django.db.backends.postgresql_psycopg2. This was changed from release 1.9 forwards and summarized in the official release notes as such:

The PostgreSQL backend django.db.backends.postgresql_psycopg2 is also available as django.db.backends.postgresql. The old name will continue to be available for backward compatibility.

Obviously, you need to have access to a Postgres database to make this work well. You should replace the values in the above code with the database to which your project will connect.  Common values for HOST and PORT are localhost and 5432 respectively, which 5432 being the default Postgres port, as described in the official PostgreSQL documentation.

Step 2: Install Necessary Postgres Adaptor

While Django ships with an “adapter” for Postgres, it still needs a 3rd party library for support. If you simply add the code above and run python manage.py makemigrations you’ll get a stacktrace with the following exception:

django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'

Django’s django.db.backends.postgresql adaptor bridges Django’s own ORM models to Postgres, but doesn’t actually provide the adaptors needed for actual Postgres usage. Rather than re-inventing the wheel, the Django Project has elected to use the psycopg2 library—arguably the most popular python adapter for Postgres.

To make the Django adaptor work, the psycopg2 library needs to be installed with this command: pip install psycopg2. Running the pip freeze command should now produce the following output:

asgiref==3.3.1
Django==3.1.4
psycopg2==2.8.6
pytz==2020.4
sqlparse==0.4.1

Common Issues with psycopg2

Depending on what platform, Python version, or IDE used; psycopg2 has some known-issues when it comes to installation. The most common is a DLL error that will prevent Django from creating the required migrations:

ImproperlyConfigured Exception

django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: DLL load failed while importing _psycopg: The specified module could not be found.

This error indicates that a core DLL file is missing from the psycopg2 installation. It’s important to recognize that psychopg2 is a third-party library and, as such, may not always be compatible with the latest Python version. In some cases, reverting to a previous version of Python may be necessary.

Option 1: Use pyscopg2-binary

Note: This package comes with its own version of some core libraries that may, under certain conditions, cause conflicts with other libraries. See the developer’s notes for more info.

Option 2: Manual Copy from System Install

Some IDEs, at least PyCharm in my experience, may have trouble installing psycopg2. In such cases, a possible workaround is to install psycopg2 to the main system install of python, then copy those files manually into your project’s virtual environment’s site-packages directory. On a Windows machine, the steps are as follows:

1. Using the Windows terminal: pip install psycopg2
2. Copy Program Files/PythonXX/site-packages/psycopg2 to your virtual environments’ site-packages directory.

If installed without elevated privileges, the psycopg2 library may install to the local User’s Python installation. If this is the case (you won’t find the psycopg2 package in the directory above) look in the following directory:

.
└── C:/
    └── Users/
        └── pc/
            └── AppData/
                └── Roaming/
                    └── Python/
                        └── Python38/
                            └── site-packages/
                                └── psycopg2

Migrate Changes

Assuming psycopg2 was configured correctly, your project should be adequately configured for basic Postgres usage now. Assuming your project doesn’t require any changes to models or other areas to work with Postgres, running the makemigrations and migrate commands should give you the confirmation of success you need. Below is what the output should look like—for a blank Django project having just configured Postgres:

(venv) C:pathtoyour-project>python manage.py makemigrations
No changes detected

(venv) C:pathtoyour-project>python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK

These are all the core Django models being migrated to the Postgres database.

Discussion

Django is an incredible framework for building multi-purpose web applications ranging from hobbyist projects to enterprise-caliber endeavors. As we’ve seen here, Django can be configured to work with PostgreSQL relatively easily. There may be some quirks depending on your OS or IDE but I’ve still to see anything that would prevent the setup outlined here from being troubleshot.