From psycopg2 import error

When I try to import psycopg2 it show below log for me: Traceback (most recent call last): File "D:/Desktop/learn/python/webcatch/appserver/testpgsql.py", line 2, in import

When I try to import psycopg2 it show below log for me:

Traceback (most recent call last):
  File "D:/Desktop/learn/python/webcatch/appserver/testpgsql.py", line 2, in <module>
    import psycopg2
  File "D:/Desktop/learn/python/webcatch/appserver/webcatch/lib/site-packages/psycopg2-2.6.1-py3.5-win32.egg/psycopg2/__init__.py", line 50, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: No module named 'psycopg2._psycopg'

How can I solve it?
My platform is win10 (64) and version is python 3.5

Tadhg McDonald-Jensen's user avatar

asked Mar 19, 2016 at 14:38

sappy's user avatar

4

Eureka! I pulled my hair out for 2 days trying to get this to work. Enlightenment came from this SO Question. Simply stated, you probably installed psycopg2 x64 version like I did, not realizing your python version was 32-bit. Unistall your current psycopg2, then:

Download: psycopg2-2.6.1.win32-py3.4-pg9.4.4-release.exe from HERE, then run the following in a Terminal:

C:pathtoproject> easy_install /path/to/psycopg2-2.6.1.win32-py3.4-pg9.4.4-release.exe
C:pathtoproject> python manage.py makemigrations
C:pathtoproject> python manage.py migrate

You may also need to (re)create super user with:

C:pathtoproject> python manage.py createsuperuser

Community's user avatar

answered Jun 14, 2016 at 18:26

Matthew Weber's user avatar

4

I had the same problem, solved it in this way:

Reinstall the package psycopg2 using pip (by default installed with python 3)

On Linux:

pip uninstall psycopg2

Confirm with (y) and then:

pip install psycopg2

On Windows I add the prefix ('python -m') to the commands above.
I think the problem occurs when you change the version of Python. (Even between minor versions such as Python 3.5 and 3.6).

answered Mar 30, 2018 at 15:27

JoDavid's user avatar

JoDavidJoDavid

3632 silver badges8 bronze badges

3

I am using psycopg in an AWS Glue Job, where is harder to follow the instructions listed in the other answers.

What I did is installing psycopg2-binary into a directory and zip up the contents of that directory:

mkdir psycopg2-binary
cd psycopg2-binary
pip install psycopg2-binary -t  .
# in case using python3:
# python3 -m pip install --system psycopg2-binary -t  .
zip -r9 psycopg2.zip *

I then copied psycopg2.zip to an S3 bucket and add it as an extra Python library under «Python library path» in the Glue Spark job.

I then launched the job with the following script to verify if psycopg2 is present (the zip file will be downloaded by Glue into the directory in which the Job script is located)

from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
import sys
import os
import zipfile

## @params: [JOB_NAME]
args = getResolvedOptions(sys.argv, ['JOB_NAME'])

sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)

zip_ref = zipfile.ZipFile('./psycopg2.zip', 'r')
print os.listdir('.')
zip_ref.extractall('/tmp/packages')
zip_ref.close()
sys.path.insert(0, '/tmp/packages')

import psycopg2
print(psycopg2.__version__)

job.commit()

answered Oct 9, 2019 at 13:46

Vzzarr's user avatar

VzzarrVzzarr

4,2301 gold badge40 silver badges71 bronze badges

1

This also happens to me in new Ubuntu 18.04. It is caused by missing one file _psycopg.py in the /usr/local/lib/python3.7/site-packages/psycopg2.

It is fixed by:

  1. remove the old psycopg2 from your machine pip3 uninstall psycopg2.
  2. download new pyscopg2 manually from the official page http://initd.org/psycopg/tarballs/PSYCOPG-2-7/psycopg2-2.7.7.tar.gz
  3. tar xvf psycopg2-2.7.7.tar.gz
  4. python setup.py build
  5. sudo python setup.py install

answered Mar 1, 2019 at 4:58

Saray Chak's user avatar

Saray ChakSaray Chak

591 silver badge6 bronze badges

1

I had this happen in Linux using Python 3.7. It is caused by missing one file _psycopg.cpython-37m-x86_64-linux-gnu.so in the /usr/local/lib/python3.7/site-packages/psycopg2.
I downloaded _psycopg.cpython-37m-x86_64-linux-gnu.so from https://github.com/jkehler/awslambda-psycopg2/tree/master/psycopg2-3.7, and Copied this file into my anaconda lib.

answered Jul 8, 2021 at 7:05

土豆陈's user avatar

I had this happen in Linux using Python 2 because I had accidentally had my PYTHONPATH set to Python 3 libraries, and it was trying to load the python3 version of psycopg2. Solution was to unset PYTHONPATH.

answered Feb 8, 2017 at 2:20

sudo's user avatar

sudosudo

5,3895 gold badges39 silver badges78 bronze badges

I had the same error on Windows, this worked for me:
pip install -U psycopg2

I had an older version installed, must have depreciated

answered Jun 5, 2019 at 13:46

Ryan L's user avatar

Ryan LRyan L

811 silver badge1 bronze badge

I came to know that most times the WINDOWS packaging does not go fine with LAMBDA.

I faced same issue while running LAMBDA with WINDOWS installed 3rd party pscyopg2 packaging.

Solution:

step1>
I installed psycopg2 in Linux.
Copied both the directories psycopg2_binary-2.8.2.dist-info and psycopg2 from Linux to windows.

step2>
Along with source *.py, packaged with copied 3rd party dependencies psycopg2 in windows to *.zip file

step3>
Upload the file to LAMBDA — Here it goes, It runs successfully without any error.

Community's user avatar

answered Apr 15, 2019 at 13:30

Deep's user avatar

0

Windows 10 with conda environment manager (fresh install of Django, wagtail with PostgreSQL), had the same error. Removed psycopg2

conda remove -n myenv psycopg2

it updated some packages, removed others (it also removed django, wagtail…). Then installed psycopg2 back

conda install -n myenv psycopg2

Tested it, import worked

python
>>> import psycopg2

Installed django, wagtail back. python manage.py migrate now populated PostgreSQL.

answered Sep 6, 2022 at 14:19

frmbelz's user avatar

frmbelzfrmbelz

1,86924 silver badges24 bronze badges

In my case, it was other site-packages that was exposed by installing pgcli, uninstalling pgcli resolved the issue for the time being.

This somehow penetrated virtualenv too.

answered Sep 24, 2022 at 0:25

Heechul Ryu's user avatar

Introduction

This article will provide a brief overview of how you can better handle PostgreSQL Python exceptions while using the psycopg2 adapter in your code. Make sure that the psycopg2 package is installed on your machine using the PIP3 package manager for Python 3 using the following command:

We’ll also be building a function from scratch that prints detailed information about the psycopg2 exceptions by accessing several of its exception library attributes. It should be noted, however, that this is mostly for educational and debugging purposes, and it should be noted that, in the implementation phase your website or application, you may want to handle them less explicitly.

Catching and handling exceptions in Python

A Python script will terminate as soon as an exception or error is raised, but there is a try-except block (that works in a similar fashion to the try {} catch(err) {} code block in PHP or JavaScript) that will allow you to catch the exception, handle it, and then respond to it with more code within the except: part of the indentation block.

The following code allows you to catch all exceptions, as a wildcard, and print them out without having to explicitly mention the exact exception:

1
2
3
4
5
6

try:
    a = 1234
    print (a + » hello world»)
except Exception as error:
    print («Oops! An exception has occured:», error)
    print («Exception TYPE:», type(error))

the except Exception as error: bit will allow you to handle any exception, and return the exception information as a TypeError class object using Python’s as keyword.

Screenshot of Python IDLE returning exception information in a try-except indentation block

Exception libraries for the psycopg2 Python adapter

Some of the two most commonly occurring exceptions in the psycopg2 library are the OperationalError and ProgrammingError exception classes.

An OperationalError typically occurs when the parameters passed to the connect() method are incorrect, or if the server runs out of memory, or if a piece of datum cannot be found, etc.

A ProgrammingError happens when there is a syntax error in the SQL statement string passed to the psycopg2 execute() method, or if a SQL statement is executed to delete a non-existent table, or an attempt is made to create a table that already exists, and exceptions of that nature.

Complete list of the psycopg2 exception classes

Here’s the complete list of all of psycopg2 exception classes:

InterfaceError, DatabaseError, DataError, OperationalError, IntegrityError, InternalError, ProgrammingError, and NotSupportedError.

Brief overview of PostgreSQL Error Codes

There is an extensive list of over 200 error codes on the postgresql.org website that describes, in detail, each five-character SQL exception.

In the psycopg2 adapter library you can return the code by accessing the exception’s pgcode attribute. It should be an alpha-numeric string, five characters in length, that corresponds to an exception in the PostgreSQL Error Codes table.

Here’s some example code showing how one can access the attribute for the PostgreSQL error code:

1
2
3
4
5

try:
        cursor.execute(«INVALID SQL STATEMENT»)
    except Exception as err:
        print («Oops! An exception has occured:», error)
        print («Exception TYPE:», type(error))

Import the exception libraries for the psycopg2 Python adapter

You’ll need to import the following libraries at the beginning of your Python script:

1
2
3
4
5
6
7
8

# import sys to get more detailed Python exception info
import sys

# import the connect library for psycopg2
from psycopg2 import connect

# import the error handling libraries for psycopg2
from psycopg2 import OperationalError, errorcodes, errors

Get the psycopg2 version string

Older versions of the psycopg2 adapter may handle some exceptions differently. Here’s some code that imports the __version__ attribute string for the psycopg2 library and prints it:

1
2
3
4
5

# import the psycopg2 library’s __version__ string
from psycopg2 import __version__ as psycopg2_version

# print the version string for psycopg2
print («psycopg2 version:», psycopg2_version, «n«)

Screenshot of Python's IDLE getting the version string for psycopg2

Define a Python function to handle and print psycopg2 SQL exceptions

The code in this section will define a Python function that will take a Python TypeError object class and parse, both the psycopg2 and native Python, exception attributes from it in order to print the details of the exception:

Define the ‘print_psycopg2_exception()’ Python function

Use Python’s def keyword to define a new function and make it accept a TypeError Python object class as its only parameter:

1
2

# define a function that handles and parses psycopg2 exceptions
def print_psycopg2_exception(err):

Use Python’s built-in ‘sys’ library to get more detailed exception information

The next bit of code grabs the traceback information for the exception, including the line number in the code that the error occurred on, by calling the sys library’s exc_info() method:

1
2
3
4
5

   

# get details about the exception
    err_type, err_obj, traceback = sys.exc_info()

    # get the line number when exception occured
    line_num = traceback.tb_lineno

Print the details for the psycopg2 exception

Use Python’s print() function to print the details of the psycopg2 exception that was passed to the function call:

1
2
3
4
5
6
7
8
9
10

   

# print the connect() error
    print («npsycopg2 ERROR:», err, «on line number:», line_num)
    print («psycopg2 traceback:», traceback, «— type:», err_type)

    # psycopg2 extensions.Diagnostics object attribute
    print («nextensions.Diagnostics:», err.diag)

    # print the pgcode and pgerror exceptions
    print («pgerror:», err.pgerror)
    print («pgcode:», err.pgcode, «n«)

Handle psycopg2 exceptions that occur while connecting to PostgreSQL

Now that the function has been defined it’s time to test it out by running some psycopg2 code. The following Python code attempts to make a connection to PostgreSQL in a try-except indentation block, and, in the case of an exception, passes the TypeError Python object to the print_psycopg2_exception() function defined earlier:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# declare a new PostgreSQL connection object
try:
    conn = connect(
        dbname = «python_test»,
        user = «WRONG_USER»,
        host = «localhost»,
        password = «mypass»
    )
except OperationalError as err:
    # pass exception to function
    print_psycopg2_exception(err)

    # set the connection to ‘None’ in case of error
    conn = None

NOTE: The above code will give the connection object a value of None in the case of an exception.

If the username string, passed to the user parameter, doesn’t match any of the users for the PostgreSQL server then the function should print something that closely resembles the following:

1
2
3
4
5
6
7
8

psycopg2 ERROR: FATAL:  password authentication failed for user «WRONG_USER»
FATAL:  password authentication failed for user «WRONG_USER»
 on line number: 43
psycopg2 traceback: <traceback object at 0x7fa361660a88> — type: <class ‘psycopg2.OperationalError’>

extensions.Diagnostics: <psycopg2.extensions.Diagnostics object at 0x7fa3646e1558>
pgerror: None
pgcode: None

Handle psycopg2 exceptions that occur while executing SQL statements

If the code to connect to PostgreSQL didn’t have any problems, and no exceptions were raised, then test out the function again. The following code purposely attempts to use a cursor object to execute() a SQL statement with bad syntax:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# if the connection was successful
if conn != None:

    # declare a cursor object from the connection
    cursor = conn.cursor()
    print («cursor object:», cursor, «n«)

    # catch exception for invalid SQL statement
    try:
        cursor.execute(«INVALID SQL STATEMENT»)
    except Exception as err:
        # pass exception to function
        print_psycopg2_exception(err)

        # rollback the previous transaction before starting another
        conn.rollback()

The print_psycopg2_exception() function should print a response that resembles the following:

1
2
3
4
5
6
7
8
9
10
11
12

psycopg2 ERROR: syntax error at or near «INVALID»
LINE 1: INVALID SQL STATEMENT
        ^
 on line number: 90
psycopg2 traceback: <traceback object at 0x7f58dd244188> — type: <class ‘psycopg2.errors.SyntaxError’>

extensions.Diagnostics: <psycopg2.extensions.Diagnostics object at 0x7f58e0018558>
pgerror: ERROR:  syntax error at or near «INVALID»
LINE 1: INVALID SQL STATEMENT
        ^

pgcode: 42601

The 42601 PostgreSQL code indicates that the exception resulted from a syntax error in the SQL statement:

Screenshot of the PostgreSQL Error Codes web page in a browser tab

Catch ‘InFailedSqlTransaction’ psycopg2 exceptions

This last bit of Python code will raise a InFailedSqlTransaction exception if the last PostgreSQL transaction, with the bad SQL statement, wasn’t rolled back using the connection object’s rollback() method:

1
2
3
4
5
6

# returns ‘psycopg2.errors.InFailedSqlTransaction’ if rollback() not called
    try:
        cursor.execute(«SELECT * FROM some_table;»)
    except errors.InFailedSqlTransaction as err:
        # pass exception to function
        print_psycopg2_exception(err)

Conclusion

The psycopg2 library adapter for PostgreSQL has an extensive list of exception Python classes, and this article only covered a few of them just to give a general idea of how you can handle such exceptions in your own Python script.

Just the Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

# import sys to get more detailed Python exception info
import sys

# import the connect library for psycopg2
from psycopg2 import connect

# import the error handling libraries for psycopg2
from psycopg2 import OperationalError, errorcodes, errors

# import the psycopg2 library’s __version__ string
from psycopg2 import __version__ as psycopg2_version
«`python
#!/usr/bin/python3
# -*- coding: utf-8 -*-

# import sys to get more detailed Python exception info
import sys

# import the connect library for psycopg2
from psycopg2 import connectDoes NOT need TextBroker to re-write

# import the error handling libraries for psycopg2
from psycopg2 import OperationalError, errorcodes, errors

# import the psycopg2 library’s __version__ string
from psycopg2 import __version__ as psycopg2_version

# print the version string for psycopg2
print («psycopg2 version:», psycopg2_version, «n«)

# define a function that handles and parses psycopg2 exceptions
def print_psycopg2_exception(err):
    # get details about the exception
    err_type, err_obj, traceback = sys.exc_info()

    # get the line number when exception occured
    line_num = traceback.tb_lineno

    # print the connect() error
    print («npsycopg2 ERROR:», err, «on line number:», line_num)
    print («psycopg2 traceback:», traceback, «— type:», err_type)

    # psycopg2 extensions.Diagnostics object attribute
    print («nextensions.Diagnostics:», err.diag)

    # print the pgcode and pgerror exceptions
    print («pgerror:», err.pgerror)
    print («pgcode:», err.pgcode, «n«)

try:
    conn = connect(
        dbname = «python_test»,
        user = «objectrocket»,
        host = «localhost»,
        password = «mypass»
    )
except OperationalError as err:
    # pass exception to function
    print_psycopg2_exception(err)

    # set the connection to ‘None’ in case of error
    conn = None

# if the connection was successful
if conn != None:

    # declare a cursor object from the connection
    cursor = conn.cursor()
    print («cursor object:», cursor, «n«)

    # catch exception for invalid SQL statement
    try:
        cursor.execute(«INVALID SQL STATEMENT»)
    except Exception as err:
        # pass exception to function
        print_psycopg2_exception(err)

        # rollback the previous transaction before starting another
        conn.rollback()

    # execute a PostgreSQL command to get all rows in a table
    # returns ‘psycopg2.errors.InFailedSqlTransaction’ if rollback() not called
    try:
        cursor.execute(«SELECT * FROM some_table;»)
    except errors.InFailedSqlTranroughsaction as err:
        # pass exception to function
        print_psycopg2_exception(err)

    # close the cursor object to avoid memory leaks
    cursor.close()

    # close the connection object also
    conn.close()

Pilot the ObjectRocket Platform Free!

Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.

Get Started

New in version 2.8.

Changed in version 2.8.4: added errors introduced in PostgreSQL 12

Changed in version 2.8.6: added errors introduced in PostgreSQL 13

Changed in version 2.9.2: added errors introduced in PostgreSQL 14

Changed in version 2.9.4: added errors introduced in PostgreSQL 15

This module exposes the classes psycopg raises upon receiving an error from
the database with a SQLSTATE value attached (available in the
pgcode attribute). The content of the module is generated
from the PostgreSQL source code and includes classes for every error defined
by PostgreSQL in versions between 9.1 and 15.

Every class in the module is named after what referred as “condition name” in
the documentation, converted to CamelCase: e.g. the error 22012,
division_by_zero is exposed by this module as the class DivisionByZero.

Every exception class is a subclass of one of the standard DB-API
exception
and expose the Error interface.
Each class’ superclass is what used to be raised by psycopg in versions before
the introduction of this module, so everything should be compatible with
previously written code catching one the DB-API class: if your code used to
catch IntegrityError to detect a duplicate entry, it will keep on working
even if a more specialised subclass such as UniqueViolation is raised.

The new classes allow a more idiomatic way to check and process a specific
error among the many the database may return. For instance, in order to check
that a table is locked, the following code could have been used previously:

try:
    cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.OperationalError as e:
    if e.pgcode == psycopg2.errorcodes.LOCK_NOT_AVAILABLE:
        locked = True
    else:
        raise

While this method is still available, the specialised class allows for a more
idiomatic error handler:

try:
    cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.errors.LockNotAvailable:
    locked = True
psycopg2.errors.lookup(code)

Lookup an error code and return its exception class.

Raise KeyError if the code is not found.

try:
    cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.errors.lookup("55P03"):
    locked = True

SQLSTATE exception classes¶

The following table contains the list of all the SQLSTATE classes exposed by
the module.

Note that, for completeness, the module also exposes all the
DB-API-defined exceptions and a few
psycopg-specific ones
exposed by the extensions
module, which are not listed here.

SQLSTATE

Exception

Base exception

Class 02: No Data (this is also a warning class per the SQL standard)

02000

NoData

DatabaseError

02001

NoAdditionalDynamicResultSetsReturned

DatabaseError

Class 03: SQL Statement Not Yet Complete

03000

SqlStatementNotYetComplete

DatabaseError

Class 08: Connection Exception

08000

ConnectionException

OperationalError

08001

SqlclientUnableToEstablishSqlconnection

OperationalError

08003

ConnectionDoesNotExist

OperationalError

08004

SqlserverRejectedEstablishmentOfSqlconnection

OperationalError

08006

ConnectionFailure

OperationalError

08007

TransactionResolutionUnknown

OperationalError

08P01

ProtocolViolation

OperationalError

Class 09: Triggered Action Exception

09000

TriggeredActionException

DatabaseError

Class 0A: Feature Not Supported

0A000

FeatureNotSupported

NotSupportedError

Class 0B: Invalid Transaction Initiation

0B000

InvalidTransactionInitiation

DatabaseError

Class 0F: Locator Exception

0F000

LocatorException

DatabaseError

0F001

InvalidLocatorSpecification

DatabaseError

Class 0L: Invalid Grantor

0L000

InvalidGrantor

DatabaseError

0LP01

InvalidGrantOperation

DatabaseError

Class 0P: Invalid Role Specification

0P000

InvalidRoleSpecification

DatabaseError

Class 0Z: Diagnostics Exception

0Z000

DiagnosticsException

DatabaseError

0Z002

StackedDiagnosticsAccessedWithoutActiveHandler

DatabaseError

Class 20: Case Not Found

20000

CaseNotFound

ProgrammingError

Class 21: Cardinality Violation

21000

CardinalityViolation

ProgrammingError

Class 22: Data Exception

22000

DataException

DataError

22001

StringDataRightTruncation

DataError

22002

NullValueNoIndicatorParameter

DataError

22003

NumericValueOutOfRange

DataError

22004

NullValueNotAllowed

DataError

22005

ErrorInAssignment

DataError

22007

InvalidDatetimeFormat

DataError

22008

DatetimeFieldOverflow

DataError

22009

InvalidTimeZoneDisplacementValue

DataError

2200B

EscapeCharacterConflict

DataError

2200C

InvalidUseOfEscapeCharacter

DataError

2200D

InvalidEscapeOctet

DataError

2200F

ZeroLengthCharacterString

DataError

2200G

MostSpecificTypeMismatch

DataError

2200H

SequenceGeneratorLimitExceeded

DataError

2200L

NotAnXmlDocument

DataError

2200M

InvalidXmlDocument

DataError

2200N

InvalidXmlContent

DataError

2200S

InvalidXmlComment

DataError

2200T

InvalidXmlProcessingInstruction

DataError

22010

InvalidIndicatorParameterValue

DataError

22011

SubstringError

DataError

22012

DivisionByZero

DataError

22013

InvalidPrecedingOrFollowingSize

DataError

22014

InvalidArgumentForNtileFunction

DataError

22015

IntervalFieldOverflow

DataError

22016

InvalidArgumentForNthValueFunction

DataError

22018

InvalidCharacterValueForCast

DataError

22019

InvalidEscapeCharacter

DataError

2201B

InvalidRegularExpression

DataError

2201E

InvalidArgumentForLogarithm

DataError

2201F

InvalidArgumentForPowerFunction

DataError

2201G

InvalidArgumentForWidthBucketFunction

DataError

2201W

InvalidRowCountInLimitClause

DataError

2201X

InvalidRowCountInResultOffsetClause

DataError

22021

CharacterNotInRepertoire

DataError

22022

IndicatorOverflow

DataError

22023

InvalidParameterValue

DataError

22024

UnterminatedCString

DataError

22025

InvalidEscapeSequence

DataError

22026

StringDataLengthMismatch

DataError

22027

TrimError

DataError

2202E

ArraySubscriptError

DataError

2202G

InvalidTablesampleRepeat

DataError

2202H

InvalidTablesampleArgument

DataError

22030

DuplicateJsonObjectKeyValue

DataError

22031

InvalidArgumentForSqlJsonDatetimeFunction

DataError

22032

InvalidJsonText

DataError

22033

InvalidSqlJsonSubscript

DataError

22034

MoreThanOneSqlJsonItem

DataError

22035

NoSqlJsonItem

DataError

22036

NonNumericSqlJsonItem

DataError

22037

NonUniqueKeysInAJsonObject

DataError

22038

SingletonSqlJsonItemRequired

DataError

22039

SqlJsonArrayNotFound

DataError

2203A

SqlJsonMemberNotFound

DataError

2203B

SqlJsonNumberNotFound

DataError

2203C

SqlJsonObjectNotFound

DataError

2203D

TooManyJsonArrayElements

DataError

2203E

TooManyJsonObjectMembers

DataError

2203F

SqlJsonScalarRequired

DataError

2203G

SqlJsonItemCannotBeCastToTargetType

DataError

22P01

FloatingPointException

DataError

22P02

InvalidTextRepresentation

DataError

22P03

InvalidBinaryRepresentation

DataError

22P04

BadCopyFileFormat

DataError

22P05

UntranslatableCharacter

DataError

22P06

NonstandardUseOfEscapeCharacter

DataError

Class 23: Integrity Constraint Violation

23000

IntegrityConstraintViolation

IntegrityError

23001

RestrictViolation

IntegrityError

23502

NotNullViolation

IntegrityError

23503

ForeignKeyViolation

IntegrityError

23505

UniqueViolation

IntegrityError

23514

CheckViolation

IntegrityError

23P01

ExclusionViolation

IntegrityError

Class 24: Invalid Cursor State

24000

InvalidCursorState

InternalError

Class 25: Invalid Transaction State

25000

InvalidTransactionState

InternalError

25001

ActiveSqlTransaction

InternalError

25002

BranchTransactionAlreadyActive

InternalError

25003

InappropriateAccessModeForBranchTransaction

InternalError

25004

InappropriateIsolationLevelForBranchTransaction

InternalError

25005

NoActiveSqlTransactionForBranchTransaction

InternalError

25006

ReadOnlySqlTransaction

InternalError

25007

SchemaAndDataStatementMixingNotSupported

InternalError

25008

HeldCursorRequiresSameIsolationLevel

InternalError

25P01

NoActiveSqlTransaction

InternalError

25P02

InFailedSqlTransaction

InternalError

25P03

IdleInTransactionSessionTimeout

InternalError

Class 26: Invalid SQL Statement Name

26000

InvalidSqlStatementName

OperationalError

Class 27: Triggered Data Change Violation

27000

TriggeredDataChangeViolation

OperationalError

Class 28: Invalid Authorization Specification

28000

InvalidAuthorizationSpecification

OperationalError

28P01

InvalidPassword

OperationalError

Class 2B: Dependent Privilege Descriptors Still Exist

2B000

DependentPrivilegeDescriptorsStillExist

InternalError

2BP01

DependentObjectsStillExist

InternalError

Class 2D: Invalid Transaction Termination

2D000

InvalidTransactionTermination

InternalError

Class 2F: SQL Routine Exception

2F000

SqlRoutineException

InternalError

2F002

ModifyingSqlDataNotPermitted

InternalError

2F003

ProhibitedSqlStatementAttempted

InternalError

2F004

ReadingSqlDataNotPermitted

InternalError

2F005

FunctionExecutedNoReturnStatement

InternalError

Class 34: Invalid Cursor Name

34000

InvalidCursorName

OperationalError

Class 38: External Routine Exception

38000

ExternalRoutineException

InternalError

38001

ContainingSqlNotPermitted

InternalError

38002

ModifyingSqlDataNotPermittedExt

InternalError

38003

ProhibitedSqlStatementAttemptedExt

InternalError

38004

ReadingSqlDataNotPermittedExt

InternalError

Class 39: External Routine Invocation Exception

39000

ExternalRoutineInvocationException

InternalError

39001

InvalidSqlstateReturned

InternalError

39004

NullValueNotAllowedExt

InternalError

39P01

TriggerProtocolViolated

InternalError

39P02

SrfProtocolViolated

InternalError

39P03

EventTriggerProtocolViolated

InternalError

Class 3B: Savepoint Exception

3B000

SavepointException

InternalError

3B001

InvalidSavepointSpecification

InternalError

Class 3D: Invalid Catalog Name

3D000

InvalidCatalogName

ProgrammingError

Class 3F: Invalid Schema Name

3F000

InvalidSchemaName

ProgrammingError

Class 40: Transaction Rollback

40000

TransactionRollback

OperationalError

40001

SerializationFailure

OperationalError

40002

TransactionIntegrityConstraintViolation

OperationalError

40003

StatementCompletionUnknown

OperationalError

40P01

DeadlockDetected

OperationalError

Class 42: Syntax Error or Access Rule Violation

42000

SyntaxErrorOrAccessRuleViolation

ProgrammingError

42501

InsufficientPrivilege

ProgrammingError

42601

SyntaxError

ProgrammingError

42602

InvalidName

ProgrammingError

42611

InvalidColumnDefinition

ProgrammingError

42622

NameTooLong

ProgrammingError

42701

DuplicateColumn

ProgrammingError

42702

AmbiguousColumn

ProgrammingError

42703

UndefinedColumn

ProgrammingError

42704

UndefinedObject

ProgrammingError

42710

DuplicateObject

ProgrammingError

42712

DuplicateAlias

ProgrammingError

42723

DuplicateFunction

ProgrammingError

42725

AmbiguousFunction

ProgrammingError

42803

GroupingError

ProgrammingError

42804

DatatypeMismatch

ProgrammingError

42809

WrongObjectType

ProgrammingError

42830

InvalidForeignKey

ProgrammingError

42846

CannotCoerce

ProgrammingError

42883

UndefinedFunction

ProgrammingError

428C9

GeneratedAlways

ProgrammingError

42939

ReservedName

ProgrammingError

42P01

UndefinedTable

ProgrammingError

42P02

UndefinedParameter

ProgrammingError

42P03

DuplicateCursor

ProgrammingError

42P04

DuplicateDatabase

ProgrammingError

42P05

DuplicatePreparedStatement

ProgrammingError

42P06

DuplicateSchema

ProgrammingError

42P07

DuplicateTable

ProgrammingError

42P08

AmbiguousParameter

ProgrammingError

42P09

AmbiguousAlias

ProgrammingError

42P10

InvalidColumnReference

ProgrammingError

42P11

InvalidCursorDefinition

ProgrammingError

42P12

InvalidDatabaseDefinition

ProgrammingError

42P13

InvalidFunctionDefinition

ProgrammingError

42P14

InvalidPreparedStatementDefinition

ProgrammingError

42P15

InvalidSchemaDefinition

ProgrammingError

42P16

InvalidTableDefinition

ProgrammingError

42P17

InvalidObjectDefinition

ProgrammingError

42P18

IndeterminateDatatype

ProgrammingError

42P19

InvalidRecursion

ProgrammingError

42P20

WindowingError

ProgrammingError

42P21

CollationMismatch

ProgrammingError

42P22

IndeterminateCollation

ProgrammingError

Class 44: WITH CHECK OPTION Violation

44000

WithCheckOptionViolation

ProgrammingError

Class 53: Insufficient Resources

53000

InsufficientResources

OperationalError

53100

DiskFull

OperationalError

53200

OutOfMemory

OperationalError

53300

TooManyConnections

OperationalError

53400

ConfigurationLimitExceeded

OperationalError

Class 54: Program Limit Exceeded

54000

ProgramLimitExceeded

OperationalError

54001

StatementTooComplex

OperationalError

54011

TooManyColumns

OperationalError

54023

TooManyArguments

OperationalError

Class 55: Object Not In Prerequisite State

55000

ObjectNotInPrerequisiteState

OperationalError

55006

ObjectInUse

OperationalError

55P02

CantChangeRuntimeParam

OperationalError

55P03

LockNotAvailable

OperationalError

55P04

UnsafeNewEnumValueUsage

OperationalError

Class 57: Operator Intervention

57000

OperatorIntervention

OperationalError

57014

QueryCanceled

OperationalError

57P01

AdminShutdown

OperationalError

57P02

CrashShutdown

OperationalError

57P03

CannotConnectNow

OperationalError

57P04

DatabaseDropped

OperationalError

57P05

IdleSessionTimeout

OperationalError

Class 58: System Error (errors external to PostgreSQL itself)

58000

SystemError

OperationalError

58030

IoError

OperationalError

58P01

UndefinedFile

OperationalError

58P02

DuplicateFile

OperationalError

Class 72: Snapshot Failure

72000

SnapshotTooOld

DatabaseError

Class F0: Configuration File Error

F0000

ConfigFileError

InternalError

F0001

LockFileExists

InternalError

Class HV: Foreign Data Wrapper Error (SQL/MED)

HV000

FdwError

OperationalError

HV001

FdwOutOfMemory

OperationalError

HV002

FdwDynamicParameterValueNeeded

OperationalError

HV004

FdwInvalidDataType

OperationalError

HV005

FdwColumnNameNotFound

OperationalError

HV006

FdwInvalidDataTypeDescriptors

OperationalError

HV007

FdwInvalidColumnName

OperationalError

HV008

FdwInvalidColumnNumber

OperationalError

HV009

FdwInvalidUseOfNullPointer

OperationalError

HV00A

FdwInvalidStringFormat

OperationalError

HV00B

FdwInvalidHandle

OperationalError

HV00C

FdwInvalidOptionIndex

OperationalError

HV00D

FdwInvalidOptionName

OperationalError

HV00J

FdwOptionNameNotFound

OperationalError

HV00K

FdwReplyHandle

OperationalError

HV00L

FdwUnableToCreateExecution

OperationalError

HV00M

FdwUnableToCreateReply

OperationalError

HV00N

FdwUnableToEstablishConnection

OperationalError

HV00P

FdwNoSchemas

OperationalError

HV00Q

FdwSchemaNotFound

OperationalError

HV00R

FdwTableNotFound

OperationalError

HV010

FdwFunctionSequenceError

OperationalError

HV014

FdwTooManyHandles

OperationalError

HV021

FdwInconsistentDescriptorInformation

OperationalError

HV024

FdwInvalidAttributeValue

OperationalError

HV090

FdwInvalidStringLengthOrBufferLength

OperationalError

HV091

FdwInvalidDescriptorFieldIdentifier

OperationalError

Class P0: PL/pgSQL Error

P0000

PlpgsqlError

InternalError

P0001

RaiseException

InternalError

P0002

NoDataFound

InternalError

P0003

TooManyRows

InternalError

P0004

AssertFailure

InternalError

Class XX: Internal Error

XX000

InternalError_

InternalError

XX001

DataCorrupted

InternalError

XX002

IndexCorrupted

InternalError

Это руководство по PostgreSQL в Python описывает, как использовать модуль Psycopg2 для подключения к PostgreSQL, выполнения SQL-запросов и других операций с базой данных.

Здесь не инструкции по установки локального сервера, так как это не касается python. Скачайте и установите PostgreSQL с официального сайта https://www.postgresql.org/download/. Подойдут версии 10+, 11+, 12+.

Вот список разных модулей Python для работы с сервером базы данных PostgreSQL:

  • Psycopg2,
  • pg8000,
  • py-postgreql,
  • PyGreSQL,
  • ocpgdb,
  • bpsql,
  • SQLAlchemy. Для работы SQLAlchemy нужно, чтобы хотя бы одно из перечисленных выше решений было установлено.

Примечание: все модули придерживаются спецификации Python Database API Specification v2.0 (PEP 249). Этот API разработан с целью обеспечить сходство разных модулей для доступа к базам данных из Python. Другими словами, синтаксис, методы и прочее очень похожи во всех этих модулях.

В этом руководстве будем использовать Psycopg2, потому что это один из самых популярных и стабильных модулей для работы с PostgreSQL:

  • Он используется в большинстве фреймворков Python и Postgres;
  • Он активно поддерживается и работает как с Python 3, так и с Python 2;
  • Он потокобезопасен и спроектирован для работы в многопоточных приложениях. Несколько потоков могут работать с одним подключением.

В этом руководстве пройдемся по следующим пунктам:

  • Установка Psycopg2 и использование его API для доступа к базе данных PostgreSQL;
  • Вставка, получение, обновление и удаление данных в базе данных из приложения Python;
  • Дальше рассмотрим управление транзакциями PostgreSQL, пул соединений и методы обработки исключений, что понадобится для разработки сложных программ на Python с помощью PostgreSQL.

Установка Psycopg2 с помощью pip

Для начала нужно установить текущую версию Psycopg2 для использования PostgreSQL в Python. С помощью команды pip можно установить модуль в любую операцию систему: Windows, macOS, Linux:

pip install psycopg2

Также можно установить конкретную версию программы с помощью такой команды:

pip install psycopg2=2.8.6

Если возникает ошибка установки, например «connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)», то ее можно решить, сделав files.pythonhosted.org доверенным хостом:

python -m pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org psycopg2

Модуль psycopg2 поддерживает:

  • Python 2.7 и Python 3, начиная с версии 3.4.
  • Сервер PostgreSQL от 7.4 до 12.
  • Клиентскую библиотеку PostgreSQL от 9.1.

Проверка установки Psycopg2

После запуска команды должны появиться следующие сообщения:

  • Collecting psycopg2
  • Downloading psycopg2-2.8.6
  • Installing collected packages: psycopg2
  • Successfully installed psycopg2-2.8.6

При использовании anaconda подойдет следующая команда.

conda install -c anaconda psycopg2

В этом разделе рассмотрим, как подключиться к PostgreSQL из Python с помощью модуля Psycopg2.

Вот какие аргументы потребуются для подключения:

  • Имя пользователя: значение по умолчанию для базы данных PostgreSQL – postgres.
  • Пароль: пользователь получает пароль при установке PostgreSQL.
  • Имя хоста: имя сервера или IP-адрес, на котором работает база данных. Если она запущена локально, то нужно использовать localhost или 127.0.0.0.
  • Имя базы данных: в этом руководстве будем использовать базу postgres_db.

Шаги для подключения:

  • Использовать метод connect() с обязательными параметрами для подключения базы данных.
  • Создать объект cursor с помощью объекта соединения, который возвращает метод connect. Он нужен для выполнения запросов.
  • Закрыть объект cursor и соединение с базой данных после завершения работы.
  • Перехватить исключения, которые могут возникнуть в процессе.

Создание базы данных PostgreSQL с Psycopg2

Для начала создадим базу данных на сервере. Во время установки PostgreSQL вы указывали пароль, его нужно использовать при подключении.


import psycopg2
from psycopg2 import Error
from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT try:
# Подключение к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432")
connection.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
# Курсор для выполнения операций с базой данных
cursor = connection.cursor()
sql_create_database = 'create database postgres_db'
cursor.execute(sql_create_database) except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

Пример кода для подключения к базе данных PostgreSQL из Python

Для подключения к базе данных PostgreSQL и выполнения SQL-запросов нужно знать название базы данных. Ее нужно создать прежде чем пытаться выполнить подключение.


import psycopg2
from psycopg2 import Error

try:
# Подключение к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432",
database="postgres_db")

# Курсор для выполнения операций с базой данных
cursor = connection.cursor()
# Распечатать сведения о PostgreSQL
print("Информация о сервере PostgreSQL")
print(connection.get_dsn_parameters(), "n")
# Выполнение SQL-запроса
cursor.execute("SELECT version();")
# Получить результат
record = cursor.fetchone()
print("Вы подключены к - ", record, "n")

except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

После подключения появится следующий вывод:

Информация о сервере PostgreSQL
{'user': 'postgres', 'dbname': 'postgres_db', 'host': '127.0.0.1', 'port': '5432', 'tty': '', 'options': '', 'sslmode': 'prefer', 'sslcompression': '0', 'krbsrvname': 'postgres', 'target_session_attrs': 'any'} 

Вы подключены к -  ('PostgreSQL 10.13, compiled by Visual C++ build 1800, 64-bit',) 

Соединение с PostgreSQL закрыто

Разбор процесса подключения в деталях

import psycopg2 — Эта строка импортирует модуль Psycopg2 в программу. С помощью классов и методов модуля можно взаимодействовать с базой.

from psycopg2 import Error — С помощью класса Error можно обрабатывать любые ошибки и исключения базы данных. Это сделает приложение более отказоустойчивым. Этот класс также поможет понять ошибку в подробностях. Он возвращает сообщение об ошибке и ее код.

psycopg2.connect() — С помощью метода connect() создается подключение к экземпляру базы данных PostgreSQL. Он возвращает объект подключения. Этот объект является потокобезопасным и может быть использован на разных потоках.

Метод connect() принимает разные аргументы, рассмотренные выше. В этом примере в метод были переданы следующие аргументы: user = "postgres", password = "1111", host = "127.0.0.1", port = "5432", database = "postgres_db".

cursor = connection.cursor() — С базой данных можно взаимодействовать с помощью класса cursor. Его можно получить из метода cursor(), который есть у объекта соединения. Он поможет выполнять SQL-команды из Python.

Из одного объекта соединения можно создавать неограниченное количество объектов cursor. Они не изолированы, поэтому любые изменения, сделанные в базе данных с помощью одного объекта, будут видны остальным. Объекты cursor не являются потокобезопасными.

После этого выведем свойства соединения с помощью connection.get_dsn_parameters().

cursor.execute() — С помощью метода execute объекта cursor можно выполнить любую операцию или запрос к базе данных. В качестве параметра этот метод принимает SQL-запрос. Результаты запроса можно получить с помощью fetchone(), fetchmany(), fetchall().

В этом примере выполняем SELECT version(); для получения сведений о версии PosgreSQL.

Блок try-except-finally — Разместим код в блоке try-except для перехвата исключений и ошибок базы данных.

cursor.close() и connection.close() — Правильно всегда закрывать объекты cursor и connection после завершения работы, чтобы избежать проблем с базой данных.

Создание таблицы PostgreSQL из Python

В этом разделе разберем, как создавать таблицу в PostgreSQL из Python. В качестве примера создадим таблицу Mobile.

Выполним следующие шаги:

  • Подготовим запрос для базы данных
  • Подключимся к PosgreSQL с помощью psycopg2.connect().
  • Выполним запрос с помощью cursor.execute().
  • Закроем соединение с базой данных и объект cursor.

Теперь рассмотрим пример.


import psycopg2
from psycopg2 import Error

try:
# Подключиться к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432",
database="postgres_db")

# Создайте курсор для выполнения операций с базой данных
cursor = connection.cursor()
# SQL-запрос для создания новой таблицы
create_table_query = '''CREATE TABLE mobile
(ID INT PRIMARY KEY NOT NULL,
MODEL TEXT NOT NULL,
PRICE REAL); '''
# Выполнение команды: это создает новую таблицу
cursor.execute(create_table_query)
connection.commit()
print("Таблица успешно создана в PostgreSQL")

except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

Вывод:

Таблица успешно создана в PostgreSQL
Соединение с PostgreSQL закрыто

Примечание: наконец, коммитим изменения с помощью метода commit().

Соответствие типов данных Python и PostgreSQL

Есть стандартный маппер для конвертации типов Python в их эквиваленты в PosgreSQL и наоборот. Каждый раз при выполнении запроса PostgreSQL из Python с помощью psycopg2 результат возвращается в виде объектов Python.

Python PostgreSQL
None NULL
bool bool
float real
double
int
long
smallint
integer
bigint
Decimal numeric
str
unicode
varchar
text
date date
time time
timetz
datetime timestamp
timestamptz
timedelta interval
list ARRAY
tuple
namedtuple
Composite types
IN syntax
dict hstore

Константы и числовые преобразования

При попытке вставить значения None и boolean (True, False) из Python в PostgreSQL, они конвертируются в соответствующие литералы SQL. То же происходит и с числовыми типами. Они конвертируются в соответствующие типы PostgreSQL.

Например, при выполнении запроса на вставку числовые объекты, такие как int, long, float и Decimal, конвертируются в числовые представления из PostgreSQL. При чтении из таблицы целые числа конвертируются в int, числа с плавающей точкой — во float, а десятичные — в Decimal.

Выполнение CRUD-операций из Python

Таблица mobile уже есть. Теперь рассмотрим, как выполнять запросы для вставки, обновления, удаления или получения данных из таблицы в Python.


import psycopg2
from psycopg2 import Error

try:
# Подключиться к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432",
database="postgres_db")

cursor = connection.cursor()
# Выполнение SQL-запроса для вставки данных в таблицу
insert_query = """ INSERT INTO mobile (ID, MODEL, PRICE) VALUES (1, 'Iphone12', 1100)"""
cursor.execute(insert_query)
connection.commit()
print("1 запись успешно вставлена")
# Получить результат
cursor.execute("SELECT * from mobile")
record = cursor.fetchall()
print("Результат", record)

# Выполнение SQL-запроса для обновления таблицы
update_query = """Update mobile set price = 1500 where id = 1"""
cursor.execute(update_query)
connection.commit()
count = cursor.rowcount
print(count, "Запись успешно удалена")
# Получить результат
cursor.execute("SELECT * from mobile")
print("Результат", cursor.fetchall())

# Выполнение SQL-запроса для удаления таблицы
delete_query = """Delete from mobile where id = 1"""
cursor.execute(delete_query)
connection.commit()
count = cursor.rowcount
print(count, "Запись успешно удалена")
# Получить результат
cursor.execute("SELECT * from mobile")
print("Результат", cursor.fetchall())

except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

Вывод:

1 запись успешно вставлена
Результат [(1, 'Iphone12', 1100.0)]
1 Запись успешно удалена
Результат [(1, 'Iphone12', 1500.0)]
1 Запись успешно удалена
Результат []
Соединение с PostgreSQL закрыто

Примечание: не забывайте сохранять изменения в базу данных с помощью connection.commit() после успешного выполнения операции базы данных.

Работа с датой и временем из PostgreSQL

В этом разделе рассмотрим, как работать с типами date и timestamp из PostgreSQL в Python и наоборот.

Обычно при выполнении вставки объекта datetime модуль psycopg2 конвертирует его в формат timestamp PostgreSQL.

По аналогии при чтении значений timestamp из таблицы PostgreSQL модуль psycopg2 конвертирует их в объекты datetime Python.

Для этого примера используем таблицу Item. Выполните следующий код, чтобы подготовить таблицу.


import psycopg2
from psycopg2 import Error

try:
# Подключиться к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432",
database="postgres_db")

# Создайте курсор для выполнения операций с базой данных
cursor = connection.cursor()
# SQL-запрос для создания новой таблицы
create_table_query = '''CREATE TABLE item (
item_id serial NOT NULL PRIMARY KEY,
item_name VARCHAR (100) NOT NULL,
purchase_time timestamp NOT NULL,
price INTEGER NOT NULL
);'''
# Выполнение команды: это создает новую таблицу
cursor.execute(create_table_query)
connection.commit()
print("Таблица успешно создана в PostgreSQL")

except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

Рассмотрим сценарий на простом примере. Здесь мы читаем колонку purchase_time из таблицы и конвертируем значение в объект datetime Python.


import psycopg2
import datetime
from psycopg2 import Error

try:
# Подключиться к существующей базе данных
connection = psycopg2.connect(user="postgres",
# пароль, который указали при установке PostgreSQL
password="1111",
host="127.0.0.1",
port="5432",
database="postgres_db")

cursor = connection.cursor()
# Выполнение SQL-запроса для вставки даты и времени в таблицу
insert_query = """ INSERT INTO item (item_Id, item_name, purchase_time, price)
VALUES (%s, %s, %s, %s)"""
item_purchase_time = datetime.datetime.now()
item_tuple = (12, "Keyboard", item_purchase_time, 150)
cursor.execute(insert_query, item_tuple)
connection.commit()
print("1 элемент успешно добавлен")

# Считать значение времени покупки PostgreSQL в Python datetime
cursor.execute("SELECT purchase_time from item where item_id = 12")
purchase_datetime = cursor.fetchone()
print("Дата покупки товара", purchase_datetime[0].date())
print("Время покупки товара", purchase_datetime[0].time())

except (Exception, Error) as error:
print("Ошибка при работе с PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Соединение с PostgreSQL закрыто")

Вывод:

1 элемент успешно добавлен
Дата покупки товара 2021-01-16
Время покупки товара 20:16:23.166867
Соединение с PostgreSQL закрыто

This Python PostgreSQL tutorial demonstrates how to use the Psycopg2 module to connect to PostgreSQL and perform SQL queries, database operations. There are many ways we can connect to a PostgreSQL database from Python, and in this tutorial, we’re going to explore several options to see how to achieve this.

Below is the list of available Python modules to work with the PostgreSQL database server.

  • Psycopg2
  • pg8000
  • py-postgresql
  • PyGreSQL
  • ocpgdb
  • bpgsql
  • SQLAlchemy. SQLAlchemy needs any of the above to be installed separately.

NoteAbove all modules adhere to Python Database API Specification v2.0 (PEP 249). This API is designed to encourage and maintain the similarity between the Python database modules to access databases. In other words, the syntax, method, and way of access the database are the same in all the above modules.

We stick to the Psycopg2 because it is arguably the most popular and stable module to work with PostgreSQL. Also, We are using Psycopg2 to work with PostgreSQL because of the following reasons.

  • It is used in most of the Python and Postgres frameworks.
  • It is also actively maintained and supports Python’s primary version, i.e., Python 3 and Python 2.
  • It is thread-safe and designed for heavily multi-threaded applications. Note, threads can share the connections.

This Python PostgreSQL guide mainly focuses on the followings

  • Installing Psycopg2 and use its API to access the PostgreSQL database
  • Perform data insertion, data retrieval, data update, and data deletion through Python application.
  • Next, it will cover PostgreSQL transaction management, connection pooling, and error-handling techniques to develop robust Python programs with PostgreSQL.

Let’s dive right in.

Table of contents

  • Install Psycopg2 using the pip command
    • Verify Psycopg2 installation
  • Python PostgreSQL database connection
    • Python example to connect PostgreSQL database
    • Important points
  • Create a PostgreSQL table from Python
  • The mapping between Python and PostgreSQL types
    • Constants and numeric conversion
  • Perform PostgreSQL CRUD operations from Python
  • Working with PostgreSQL date and time in Python
  • Call PostgreSQL Function and Stored Procedure from Python
  • Python PostgreSQL Transaction management
  • Python PostgreSQL Connection Pooling
  • Python PostgreSQL Exercise Project

Install Psycopg2 using the pip command

You need to install the current version of Psycopg2 (2.8.6) on your machine to use PostgreSQL from Python. This module is available on pypi.org.

Using the following pip command, you can install Psycopg2 on any operating system, including Windows, macOS, Linux, Unix, and Ubuntu.

pip install psycopg2

You can also install a specific version using the following command.

pip install psycopg2=2.8.6

If you are facing pip install error like “connection error:  [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)”. You can resolve this error by setting pypi.org and files.pythonhosted.org as trusted hosts. If you are facing a pip install error Please try following the command.

python -m pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org psycopg2

The current psycopg2 module supports:

  • Python version 2.7, and Python 3 versions from 3.4 to 3.8
  • PostgreSQL server versions from 7.4 to 12
  • PostgreSQL client library version from 9.1

Verify Psycopg2 installation

You should get the following messages after running the above command.

  • Collecting psycopg2
  • Downloading psycopg2-2.8.6
  • Installing collected packages: psycopg2
  • Successfully installed psycopg2-2.8.6

Please use the following command to install Psycopg2 If you are using anaconda.

conda install -c anaconda psycopg2

Python PostgreSQL database connection

In this section, we will learn how to connect to PostgreSQL through Python using Psycopg2.

Arguments required to connect PostgreSQL database from Python

You need to know the following detail of the PostgreSQL server to perform the connection.

  • Username: The username you use to work with PostgreSQL, The default username for the PostgreSQL database is Postgres.
  • Password: Password is given by the user at the time of installing the PostgreSQL.
  • Host Name: This is the server name or Ip address on which PostgreSQL is running. if you are running on localhost, then you can use localhost, or its IP, i.e., 127.0.0.0
  • Database Name: Database name to which you want to connect. Here we are using Database named “postgres_db“.

How to Connect to PostgreSQL in Python

  1. Install Psycopg2 module

    Install and import psycopg2 module. Import using a import psycopg2 statement so you can use this module’s methods to communicate with the PostgreSQL database.

  2. Use the connect() method

    Use the psycopg2.connect() method with the required arguments to connect MySQL. It would return an Connection object if the connection established successfully

  3. Use the cursor() method

    Create a cursor object using the connection object returned by the connect method to execute PostgreSQL queries from Python.

  4. Use the execute() method

    The execute() methods run the SQL query and return the result.

  5. Extract result using fetchall()

    Use cursor.fetchall() or fetchone() or fetchmany() to read query result.

  6. Close cursor and connection objects

    use cursor.clsoe() and connection.clsoe() method to close PostgreSQL connections after your work completes

Python PostgreSQL connection to access database
Python PostgreSQL connection to access database

Python example to connect PostgreSQL database

To connect the PostgreSQL database and perform SQL queries, you must know the database name you want to connect to, and if you have not created any database, I advise you to create one before proceeding further.

import psycopg2
from psycopg2 import Error

try:
    # Connect to an existing database
    connection = psycopg2.connect(user="postgres",
                                  password="pynative@#29",
                                  host="127.0.0.1",
                                  port="5432",
                                  database="postgres_db")

    # Create a cursor to perform database operations
    cursor = connection.cursor()
    # Print PostgreSQL details
    print("PostgreSQL server information")
    print(connection.get_dsn_parameters(), "n")
    # Executing a SQL query
    cursor.execute("SELECT version();")
    # Fetch result
    record = cursor.fetchone()
    print("You are connected to - ", record, "n")

except (Exception, Error) as error:
    print("Error while connecting to PostgreSQL", error)
finally:
    if (connection):
        cursor.close()
        connection.close()
        print("PostgreSQL connection is closed")

You should get the following output after connecting to PostgreSQL from Python

PostgreSQL server information
{'user': 'postgres', 'dbname': 'python_db', 'host': '127.0.0.1', 'port': '5432', 'tty': '', 'options': '', 'sslmode': 'prefer', 'sslcompression': '0', 'krbsrvname': 'postgres', 'target_session_attrs': 'any'} 

You are connected to -  ('PostgreSQL 12.2) 
PostgreSQL connection is closed

Important points

  • In our example, we are executing a SELECT version(); query to fetch the PostgreSQL version.
  • Using the Error class of Psycopg2, we can handle any database error and exception while working with PostgreSQL from Python. Using this approach, we can make our application robust.
  • The error class helps us to understand the error in detail. It returns an error message and error code if any.
  • We can create as many cursors as we want from a single connection object. Cursors created from the same connection are not isolated, i.e., any changes done to the database by a cursor are immediately visible by the other cursors.
  • Cursors are not thread-safe.
  • We can retrieve query result using cursor methods such as fetchone(), fetchmany(), fetcthall().

try-except-finally block

  • We placed all our code in the try-except block to catch the database exceptions and errors that may occur during this process.

cursor.close()  and connection.close()

  • It is always good practice to close the cursor and connection object once your work gets completed to avoid database issues.

Create a PostgreSQL table from Python

This section will learn how to create a table in PostgreSQL from Python. In this example, we will create a “Mobile” table in PostgreSQL.

import psycopg2
from psycopg2 import Error

try:
    connection = psycopg2.connect(user="postgres",
                                  password="pynative@#29",
                                  host="127.0.0.1",
                                  port="5432",
                                  database="postgres_db")

    cursor = connection.cursor()
    # SQL query to create a new table
    create_table_query = '''CREATE TABLE mobile
          (ID INT PRIMARY KEY     NOT NULL,
          MODEL           TEXT    NOT NULL,
          PRICE         REAL); '''
    # Execute a command: this creates a new table
    cursor.execute(create_table_query)
    connection.commit()
    print("Table created successfully in PostgreSQL ")

except (Exception, Error) as error:
    print("Error while connecting to PostgreSQL", error)
finally:
    if connection:
        cursor.close()
        connection.close()
        print("PostgreSQL connection is closed")

Output

Table created successfully in PostgreSQL PostgreSQL connection is closed
Create a PostgreSQL table from Python
Create a PostgreSQL table from Python

Note: Note: In the end, we are committing our changes to the database using the commit() method.

The mapping between Python and PostgreSQL types

There is default mapping specified to convert Python types into PostgreSQL equivalent, and vice versa. Whenever you execute a PostgreSQL query using Python following table is used by psycopg2 to return the result in the form of Python objects.

Python PostgreSQL
None NULL
bool bool
float real or double
int smallint
integer
bigint
Decimal numeric
str varchar
text
date date
time time
timetz
datetime timestamp
timestamptz
timedelta interval
list ARRAY
tuple Composite types
IN syntax
dict hstore

Constants and numeric conversion

When you try to insert Python None and boolean values such as True and False into PostgreSQL, it gets converted into the proper SQL literals. The same case is with Python numerical types. It gets converted into equivalent PostgreSQL types.

For example, When you execute an insert query, Python numeric objects such as int, long, float, Decimal are converted into a PostgreSQL numerical representation. When you read from the PostgreSQL table, integer types are converted into an int, floating-point types are converted into a float, numeric/Decimal are converted into Decimal.

Perform PostgreSQL CRUD operations from Python

Now, we created a “mobile” table. Now let’ see how to perform insert, select, update, and delete PostgreSQL queries from Python.

In this section, We will learn how to perform PostgreSQL CRUD operations from Python.

Now, Let’s see the example.

import psycopg2

try:
    connection = psycopg2.connect(user="postgres",
                                  password="pynative@#29",
                                  host="127.0.0.1",
                                  port="5432",
                                  database="postgres_db")

    cursor = connection.cursor()
    # Executing a SQL query to insert data into  table
    insert_query = """ INSERT INTO mobile (ID, MODEL, PRICE) VALUES (1, 'Iphone12', 1100)"""
    cursor.execute(insert_query)
    connection.commit()
    print("1 Record inserted successfully")
    # Fetch result
    cursor.execute("SELECT * from mobile")
    record = cursor.fetchall()
    print("Result ", record)

    # Executing a SQL query to update table
    update_query = """Update mobile set price = 1500 where id = 1"""
    cursor.execute(update_query)
    connection.commit()
    count = cursor.rowcount
    print(count, "Record updated successfully ")
    # Fetch result
    cursor.execute("SELECT * from mobile")
    print("Result ", cursor.fetchall())

    # Executing a SQL query to delete table
    delete_query = """Delete from mobile where id = 1"""
    cursor.execute(delete_query)
    connection.commit()
    count = cursor.rowcount
    print(count, "Record deleted successfully ")
    # Fetch result
    cursor.execute("SELECT * from mobile")
    print("Result ", cursor.fetchall())


except (Exception, psycopg2.Error) as error:
    print("Error while connecting to PostgreSQL", error)
finally:
    if connection:
        cursor.close()
        connection.close()
        print("PostgreSQL connection is closed")

Output:

1 Record inserted successfully
Result  [(1, 'Iphone12', 1100.0)]
1 Record updated successfully 
Result  [(1, 'Iphone12', 1500.0)]
1 Record deleted successfully 
Result  []
PostgreSQL connection is closed

Please refer to the following tutorials to have more information on insert, update, and delete data from the PostgreSQL table using Python.

In the following tutorial, we will teach you how to pass parameters to SQL queries.  We will learn how to use a parameterized query to pass Python variables and dynamic data into SQL queries.

  • Insert data into the PostgreSQL Table from Python: Learn how to execute the SQL insert query from a Python application to add a record to the PostgreSQL table.
  • Select data from PostgreSQL Table from Python: Learn how to execute a SQL select query from a Python application to fetch rows from the database table. We will also learn how to use fetchall(), fetchmany() and fetchone() methods to read a limited number of rows from the table.
  • Update data of PostgreSQL table from Python: Learn how to execute SQL update query from Python application to modify a PostgreSQL table’s records.
  • Delete data from PostgreSQL table from Python: Learn how to execute SQL delete query from Python application to delete records from a PostgreSQL table.

Working with PostgreSQL date and time in Python

This section will demonstrate how to work with PostgreSQL date and timestamp data types in Python and vice-versa. Most of the time, we work with date and time data. We insert date and time into the table and also read from it in our application whenever required.

In a usual scenario, when you execute the insert query with the datetime object, the Python psycopg2 module converts it into a PostgreSQL timestamp format to insert it in the table.

And when you execute a SELECT query from Python to read timestamp values from the PostgreSQL table, the psycopg2 module converts it into a datetime object.

We are using the “Item” table for this demo. Please copy and execute the below query on your PostgreSQL query tool to have adequate data for this operation.

CREATE TABLE item ( 
	item_id serial NOT NULL PRIMARY KEY, 
	item_name VARCHAR (100) NOT NULL, 
	purchase_time timestamp NOT NULL,
	price INTEGER NOT NULL
);

Let’s understand this scenario with a simple example. Here we will read purchase_time column from the PostgreSQL table and convert it into a Python datetime object.

import datetime

import psycopg2

try:
    connection = psycopg2.connect(user="postgres",
                                  password="pynative@#29",
                                  host="127.0.0.1",
                                  port="5432",
                                  database="postgres_db")

    cursor = connection.cursor()
    # Executing a SQL query to insert datetime into table
    insert_query = """ INSERT INTO item (item_Id, item_name, purchase_time, price) VALUES (%s, %s, %s, %s)"""
    item_purchase_time = datetime.datetime.now()
    item_tuple = (12, "Keyboard", item_purchase_time, 150)
    cursor.execute(insert_query, item_tuple)
    connection.commit()
    print("1 item inserted successfully")

    # Read PostgreSQL purchase timestamp value into Python datetime
    cursor.execute("SELECT purchase_time from item where item_id = 12")
    purchase_datetime = cursor.fetchone()
    print("Item Purchase date is  ", purchase_datetime[0].date())
    print("Item Purchase time is  ", purchase_datetime[0].time())

except (Exception, psycopg2.Error) as error:
    print("Error while connecting to PostgreSQL", error)
finally:
    if connection:
        cursor.close()
        connection.close()
        print("PostgreSQL connection is closed")

Output:

1 item inserted successfully
Item Purchase date is   2020-12-14
Item Purchase time is   12:47:45.854942
PostgreSQL connection is closed

Call PostgreSQL Function and Stored Procedure from Python

PostgreSQL function and the Stored procedure can perform different operations, such as data manipulation or data retrieval. We can execute such functions from Python.

Learn how to execute the PostgreSQL function and Stored procedure in Python.

Python PostgreSQL Transaction management

In this article, we will see how to manage PostgreSQL transactions from Python using psycopg2.

  • Learn how to use the commit() and the rollback() method of a connection class to manage database transactions and maintain the ACID properties.
  • Also, learn how to change the PostgreSQL transaction isolation level from Python.

Python PostgreSQL Connection Pooling

This section will let you know what a connection pool is and how to implement a PostgreSQL database connection pool using Psycopg2 in Python. Using Psycopg2, we can implement a connection pool for simple as well as multithreaded applications.

Use the Connection pool to increase the speed and performance of database-centric applications.

Python PostgreSQL Exercise Project

Solve our free Python database exercise project to practice and master the PostgreSQL database operations using Python.

In this exercise project, We will implement the Hospital Information System, which covers all database operations. In this Python database exercise, we will do database CRUD operations From Python. This practice exercise also covers transaction management and error-handling techniques.

Reference:

  • Psycopg2
  • PostgreSQL
  • PEP 249

Many developers face the issue of the No module named ‘psycopg2’ when they try to take their project to the production level. With the help of this article, we will understand the cause of the error and the possible solutions to avoid them. Let’s dive in.

What is ‘psycopg2’?

‘psycopg2’ is the most popular database adapter dealing in PostgreSQL. Its core is to completely implement the Python DB API 2.0 specification and the thread-safety. That means it can allow several threads to share a standard connection. It can easily handle concurrent insertion and deletion in an application. It can create or destroy lots of connections simultaneously.

Architechture behind ‘psycopg2’

‘psycopg2’ uses a libpq wrapper, which implements the C programming language interface. It consists of a set of library functions that allow client programs to receive the results of the queries passed to the PostgreSQL backend server.

Cause behind the error: No module named ‘psycopg2’

To execute the program smoothly, some pre-requisites should be met. Failing to meet these requirements will trigger import errors during the compilation.

Pre-requisites are :

  1. Python Version
    • 3.6 to 3.9
  2. PostgreSQL server versions
    • 7.4 to 13
  3. PostgreSQL client library version
    • from 9.1
  4. C compiler
  5. Package such as python-dev or python3-dev to install python header files.
  6. libpq-dev package containing libpq header files.
  7. pg-config file should be present in the PATH file. It compiles ‘psycopg2

If the system does not meet the above requirements, the ‘psycopg2’ module will not be installed, leading to no modules name ‘psycopg2’ error. This error is often viewed by programmers who don’t have a C compiler in their system. As the binaries fail to install, the module will not work.

Resolving the issue: No module named ‘psycopg2’

To resolve the issue, we must satisfy all the pre-requisites laid by the ‘psycopg2’ to meet its build requirements. However, pyscopg2 also provides us with a binary package with its versions of C libraries, libpq, and libssl, which will be used regardless of other libraries available to the client.

Perform these commands to resolve the issue:

pip uninstall psycopg2
pip install psycopg2-binary

Running the above commands will solve the problem, but the installation may fail in a few cases due to a non-supportive environment. Follow these steps to install the precompiled library –

  1. Go to the Precompiled Library Packages list.
  2. Then download the wheel file (.whl) for the psycopg module.
  3. Then use the command pip install <file>.whl to install the library using downloaded wheel file.

Using the above steps will guarantee installing psycopg2 on your computer.

Working On Incorrect Virtual Enviornment?

Many “No module named psycopg2” errors occur due to working on incorrect virtual environments and installing the ‘psycopg2’ on a different environment. Suppose you have two versions of python3 installed, and how will you install ‘psycopg2’ to a specific python?

Use the following command to call your python and install the package in your respective environment –

python3 -m pip install psycopg2

This will ensure that you install the psycopg2 module in the working environment. Hopefully, this resolves the issue. Moreover, if you face the C Compiler issues in this method, use the precompiled method as mentioned last way.

Recommended Reading | [Solved] No Module Named Numpy in Python

Resolving No module named ‘psycopg2’ in AWS EC2 lambda/ Linux OS

However, one cannot rely on binary packages if they are using them in production, and we should build the ‘psycopg2’ from the source. Because upgrading the system libraries will not upgrade the libraries used by ‘psycopg2'. Hence, there might be a dependencies error.

One can perform these commands to solve the problem

sudo apt install gcc g++ build-essential
sudo apt install python3-dev
sudo apt install libpq-dev
python -m pip install psycopg2

How to solve the No module named ‘psycopg2’ Error in Conda/Anaconda?

Conda or Anaconda has its virtual environment. So to work ‘psycopg2’, you have to install psycopg2 on Conda Environment. Use conda install psycopg2 to install psycopg2.

How to solve the No module named ‘psycopg2’ Error in Jupyter?

In most cases, Jupyter Notebook works in the anaconda environment. Use the conda install psycopg2 command to install the package in Jupyter. If it’s not working on Anaconda Environment, you have to find the location of the working environment and install the package there.

Conclusion

So, in this way, one can resolve the import error related to the PostgreSQL connection. A quick tip is to keep in mind the requisites we should follow before executing any program. We can permanently activate a python virtual window and maintain that virtual window according to the project’s needs. Now it’s your time to leverage the DB connection and create fantastic projects with ‘psycopg2’ and PostgreSQL.

Bon Codage!

Other Errors You Might Get

  • “Other Commands Don’t Work After on_message” in Discord Bots

    “Other Commands Don’t Work After on_message” in Discord Bots

    February 5, 2023

  • Botocore.Exceptions.NoCredentialsError: Unable to Locate Credentials

    Botocore.Exceptions.NoCredentialsError: Unable to Locate Credentials

    by Rahul Kumar YadavFebruary 5, 2023

  • [Resolved] NameError: Name _mysql is Not Defined

    [Resolved] NameError: Name _mysql is Not Defined

    by Rahul Kumar YadavFebruary 5, 2023

  • Best Ways to Implement Regex New Line in Python

    Best Ways to Implement Regex New Line in Python

    by Rahul Kumar YadavFebruary 5, 2023

Quick Fix: Python raises the ImportError: No module named 'psycopg2' when it cannot find the library psycopg2. The most frequent source of this error is that you haven’t installed psycopg2 explicitly with pip install psycopg2. Alternatively, you may have different Python versions on your computer, and psycopg2 is not installed for the particular version you’re using.

Problem Formulation

You’ve just learned about the awesome capabilities of the psycopg2 library and you want to try it out, so you start your code with the following statement:

import psycopg2

This is supposed to import the Pandas library into your (virtual) environment. However, it only throws the following ImportError: No module named psycopg2:

>>> import psycopg2
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

Solution Idea 1: Install Library psycopg2

The most likely reason is that Python doesn’t provide psycopg2 in its standard library. You need to install it first!

Before being able to import the Pandas module, you need to install it using Python’s package manager pip. Make sure pip is installed on your machine.

To fix this error, you can run the following command in your Windows shell:

$ pip install psycopg2

This simple command installs psycopg2 in your virtual environment on Windows, Linux, and MacOS. It assumes that your pip version is updated. If it isn’t, use the following two commands in your terminal, command line, or shell (there’s no harm in doing it anyways):

$ python -m pip install --upgrade pip
$ pip install pandas

💡 Note: Don’t copy and paste the $ symbol. This is just to illustrate that you run it in your shell/terminal/command line.

Solution Idea 2: Fix the Path

The error might persist even after you have installed the psycopg2 library. This likely happens because pip is installed but doesn’t reside in the path you can use. Although pip may be installed on your system the script is unable to locate it. Therefore, it is unable to install the library using pip in the correct path.

To fix the problem with the path in Windows follow the steps given next.

Step 1: Open the folder where you installed Python by opening the command prompt and typing where python

Step 2: Once you have opened the Python folder, browse and open the Scripts folder and copy its location. Also verify that the folder contains the pip file.

Step 3: Now open the Scripts directory in the command prompt using the cd command and the location that you copied previously.

Step 4: Now install the library using pip install psycopg2 command. Here’s an analogous example:

After having followed the above steps, execute our script once again. And you should get the desired output.

Other Solution Ideas

  • The ModuleNotFoundError may appear due to relative imports. You can learn everything about relative imports and how to create your own module in this article.
  • You may have mixed up Python and pip versions on your machine. In this case, to install psycopg2 for Python 3, you may want to try python3 -m pip install psycopg2 or even pip3 install psycopg2 instead of pip install psycopg2
  • If you face this issue server-side, you may want to try the command pip install --user psycopg2
  • If you’re using Ubuntu, you may want to try this command: sudo apt install psycopg2
  • You can check out our in-depth guide on installing psycopg2 here.
  • You can also check out this article to learn more about possible problems that may lead to an error when importing a library.

Understanding the “import” Statement

import psycopg2

In Python, the import statement serves two main purposes:

  • Search the module by its name, load it, and initialize it.
  • Define a name in the local namespace within the scope of the import statement. This local name is then used to reference the accessed module throughout the code.

What’s the Difference Between ImportError and ModuleNotFoundError?

What’s the difference between ImportError and ModuleNotFoundError?

Python defines an error hierarchy, so some error classes inherit from other error classes. In our case, the ModuleNotFoundError is a subclass of the ImportError class.

You can see this in this screenshot from the docs:

You can also check this relationship using the issubclass() built-in function:

>>> issubclass(ModuleNotFoundError, ImportError)
True

Specifically, Python raises the ModuleNotFoundError if the module (e.g., psycopg2) cannot be found. If it can be found, there may be a problem loading the module or some specific files within the module. In those cases, Python would raise an ImportError.

If an import statement cannot import a module, it raises an ImportError. This may occur because of a faulty installation or an invalid path. In Python 3.6 or newer, this will usually raise a ModuleNotFoundError.

Related Videos

The following video shows you how to resolve the ImportError:

How to Fix : “ImportError: Cannot import name X” in Python?

The following video shows you how to import a function from another folder—doing it the wrong way often results in the ModuleNotFoundError:

How to Call a Function from Another File in Python?

How to Fix “ModuleNotFoundError: No module named ‘psycopg2′” in PyCharm

If you create a new Python project in PyCharm and try to import the psycopg2 library, it’ll raise the following error message:

Traceback (most recent call last):
  File "C:/Users/.../main.py", line 1, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

Process finished with exit code 1

The reason is that each PyCharm project, per default, creates a virtual environment in which you can install custom Python modules. But the virtual environment is initially empty—even if you’ve already installed psycopg2 on your computer!

Here’s a screenshot exemplifying this for the pandas library. It’ll look similar for psycopg2.

The fix is simple: Use the PyCharm installation tooltips to install Pandas in your virtual environment—two clicks and you’re good to go!

First, right-click on the pandas text in your editor:

Second, click “Show Context Actions” in your context menu. In the new menu that arises, click “Install Pandas” and wait for PyCharm to finish the installation.

The code will run after your installation completes successfully.

As an alternative, you can also open the Terminal tool at the bottom and type:

$ pip install psycopg2

If this doesn’t work, you may want to set the Python interpreter to another version using the following tutorial: https://www.jetbrains.com/help/pycharm/2016.1/configuring-python-interpreter-for-a-project.html

You can also manually install a new library such as psycopg2 in PyCharm using the following procedure:

  • Open File > Settings > Project from the PyCharm menu.
  • Select your current project.
  • Click the Python Interpreter tab within your project tab.
  • Click the small + symbol to add a new library to the project.
  • Now type in the library to be installed, in your example Pandas, and click Install Package.
  • Wait for the installation to terminate and close all popup windows.

Here’s an analogous example:

Here’s a full guide on how to install a library on PyCharm.

  • How to Install a Library on PyCharm

While working as a researcher in distributed systems, Dr. Christian Mayer found his love for teaching computer science students.

To help students reach higher levels of Python success, he founded the programming education website Finxter.com. He’s author of the popular programming book Python One-Liners (NoStarch 2020), coauthor of the Coffee Break Python series of self-published books, computer science enthusiast, freelancer, and owner of one of the top 10 largest Python blogs worldwide.

His passions are writing, reading, and coding. But his greatest passion is to serve aspiring coders through Finxter and help them to boost their skills. You can join his free email academy here.

A common error you may encounter when using Python is modulenotfounderror: no module named ‘psycopg2’.

This error occurs when the Python interpreter cannot detect the Psycopg library in your current environment.

You can install Psycopg2 in Python 3 with python3 -m pip install psycopg2-binary.

This tutorial goes through the exact steps to troubleshoot this error for the Windows, Mac and Linux operating systems.


Table of contents

  • ModuleNotFoundError: no module named ‘psycopg2’
    • What is ModuleNotFoundError?
    • What is Psycopg2?
  • Always Use a Virtual Environment to Install Packages
    • How to Install Psycopg2 on Windows Operating System
      • Psycopg2 installation on Windows Using pip
    • How to Install Psycopg2 on Mac Operating System using pip
    • How to Install Psycopg2 on Linux Operating Systems
      • Installing pip for Ubuntu, Debian, and Linux Mint
      • Installing pip for CentOS 8 (and newer), Fedora, and Red Hat
      • Installing pip for CentOS 6 and 7, and older versions of Red Hat
      • Installing pip for Arch Linux and Manjaro
      • Installing pip for OpenSUSE
      • Psycopg2 installation on Linux with Pip
  • Installing Psycopg2 Using Anaconda
    • Check Psycopg2 Version
  • Summary

ModuleNotFoundError: no module named ‘psycopg2’

What is ModuleNotFoundError?

The ModuleNotFoundError occurs when the module you want to use is not present in your Python environment. There are several causes of the modulenotfounderror:

The module’s name is incorrect, in which case you have to check the name of the module you tried to import. Let’s try to import the re module with a double e to see what happens:

import ree
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
1 import ree

ModuleNotFoundError: No module named 'ree'

To solve this error, ensure the module name is correct. Let’s look at the revised code:

import re

print(re.__version__)
2.2.1

You may want to import a local module file, but the module is not in the same directory. Let’s look at an example package with a script and a local module to import. Let’s look at the following steps to perform from your terminal:

mkdir example_package

cd example_package

mkdir folder_1

cd folder_1

vi module.py

Note that we use Vim to create the module.py file in this example. You can use your preferred file editor, such as Emacs or Atom. In module.py, we will import the re module and define a simple function that prints the re version:

import re

def print_re_version():

    print(re.__version__)

Close the module.py, then complete the following commands from your terminal:

cd ../

vi script.py

Inside script.py, we will try to import the module we created.

import module

if __name__ == '__main__':

    mod.print_re_version()

Let’s run python script.py from the terminal to see what happens:

Traceback (most recent call last):
  File "script.py", line 1, in ≺module≻
    import module
ModuleNotFoundError: No module named 'module'

To solve this error, we need to point to the correct path to module.py, which is inside folder_1. Let’s look at the revised code:

import folder_1.module as mod

if __name__ == '__main__':

    mod.print_re_version()

When we run python script.py, we will get the following result:

2.2.1

Lastly, you can encounter the modulenotfounderror when you import a module that is not installed in your Python environment.

What is Psycopg2?

Psycopg2 is a PostgreSQL database adapter for Python. It provides an API to connect to an external database.

The simplest way to install psycopg2 is to use the package manager for Python called pip. The following installation instructions are for the major Python version 3.

Always Use a Virtual Environment to Install Packages

It is always best to install new libraries within a virtual environment. You should not install anything into your global Python interpreter when you develop locally. You may introduce incompatibilities between packages, or you may break your system if you install an incompatible version of a library that your operating system needs. Using a virtual environment helps compartmentalize your projects and their dependencies. Each project will have its environment with everything the code needs to run. Most ImportErrors and ModuleNotFoundErrors occur due to installing a library for one interpreter and trying to use the library with another interpreter. Using a virtual environment avoids this. In Python, you can use virtual environments and conda environments. We will go through how to install psycopg2 with both.

How to Install Psycopg2 on Windows Operating System

First, you need to download and install Python on your PC. Ensure you select the install launcher for all users and Add Python to PATH checkboxes. The latter ensures the interpreter is in the execution path. Pip is automatically on Windows for Python versions 2.7.9+ and 3.4+.

You can check your Python version with the following command:

python3 --version

You can install pip on Windows by downloading the installation package, opening the command line and launching the installer. You can install pip via the CMD prompt by running the following command.

python get-pip.py

You may need to run the command prompt as administrator. Check whether the installation has been successful by typing.

pip --version

Psycopg2 installation on Windows Using pip

To install psycopg2, first create the virtual environment. The environment can be any name, in this we choose “env”:

virtualenv env

You can activate the environment by typing the command:

envScriptsactivate

You will see “env” in parenthesis next to the command line prompt. You can install psycopg2 within the environment by running the following command from the command prompt.

python3 -m pip install psycopg2-binary

We use python -m pip to execute pip using the Python interpreter we specify as Python. Doing this helps avoid ImportError when we try to use a package installed with one version of Python interpreter with a different version. You can use the command which python to determine which Python interpreter you are using.

How to Install Psycopg2 on Mac Operating System using pip

Open a terminal by pressing command (⌘) + Space Bar to open the Spotlight search. Type in terminal and press enter. To get pip, first ensure you have installed Python3:

python3 --version
Python 3.8.8

Download pip by running the following curl command:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

The curl command allows you to specify a direct download link. Using the -o option sets the name of the downloaded file.

Install pip by running:

python3 get-pip.py

To install psycopg2, first create the virtual environment:

python3 -m venv env

Then activate the environment using:

source env/bin/activate 

You will see “env” in parenthesis next to the command line prompt. You can install psycopg2 within the environment by running the following command from the command prompt.

python3 -m pip install psycopg2-binary

How to Install Psycopg2 on Linux Operating Systems

All major Linux distributions have Python installed by default. However, you will need to install pip. You can install pip from the terminal, but the installation instructions depend on the Linux distribution you are using. You will need root privileges to install pip. Open a terminal and use the commands relevant to your Linux distribution to install pip.

Installing pip for Ubuntu, Debian, and Linux Mint

sudo apt install python-pip3

Installing pip for CentOS 8 (and newer), Fedora, and Red Hat

sudo dnf install python-pip3

Installing pip for CentOS 6 and 7, and older versions of Red Hat

sudo yum install epel-release

sudo yum install python-pip3

Installing pip for Arch Linux and Manjaro

sudo pacman -S python-pip

Installing pip for OpenSUSE

sudo zypper python3-pip

Psycopg2 installation on Linux with Pip

To install psycopg2, first create the virtual environment:

python3 -m venv env

Then activate the environment using:

source env/bin/activate 

You will see “env” in parenthesis next to the command line prompt. You can install psycopg2 within the environment by running the following command from the command prompt.

Once you have activated your virtual environment, you can install psycopg2 using:

python3 -m pip install psycopg2-binary

Installing Psycopg2 Using Anaconda

Anaconda is a distribution of Python and R for scientific computing and data science. You can install Anaconda by going to the installation instructions. Once you have installed Anaconda, you can create a virtual environment and install psycopg2.

To create a conda environment you can use the following command:

conda create -n psycopg2 python=3.8

You can specify a different Python 3 version if you like. Ideally, choose the latest version of Python. Next, you will activate the project container. You will see “psycopg2” in parentheses next to the command line prompt.

source activate psycopg2

Now you’re ready to install psycopg2 using conda.

Once you have activated your conda environment, you can install psycopg2 using the following command:

conda install -c anaconda psycopg2

Check Psycopg2 Version

Once you have successfully installed psycopg2, you can check its version. If you used pip to install psycopg2, you can use pip show from your terminal.

python3 -m pip show psycopg2-binary
Name: psycopg2-binary
Version: 2.9.3
Summary: psycopg2 - Python-PostgreSQL Database Adapter

Second, within your python program, you can import psycopg2 and then reference the __version__ attribute:

import psycopg2
print(psycopg2.__version__)
2.9.3

If you used conda to install psycopg2, you could check the version using the following command:

conda list -f psycopg2
# Name                    Version                   Build  Channel
psycopg2                  2.8.5            py38hddc9c9b_0    anaconda

Summary

Congratulations on reading to the end of this tutorial. The modulenotfounderror occurs if you misspell the module name, incorrectly point to the module path or do not have the module installed in your Python environment. If you do not have the module installed in your Python environment, you can use pip to install the package. However, you must ensure you have pip installed on your system. You can also install Anaconda on your system and use the conda install command to install psycopg2.

Go to the online courses page on Python to learn more about Python for data science and machine learning.

For further reading on missing modules in Python, go to the article:

  • How to Solve Python ModuleNotFoundError: no module named ‘urllib2’.
  • How to Solve ModuleNotFoundError: no module named ‘plotly’.
  • How to Solve Python ModuleNotFoundError: no module named ‘boto3’.

Have fun and happy researching!

Introduction

This is an article with a specific topic about how to solve error message as it exists in the title of the article. It occurs when a django application runs. It happens after adding the following content in the settings.py script inside the project folder :

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'xxxx',
        'USER': 'postgres',
        'PASSWORD' : '',
        'HOST' : 'xxx.xxx.xxx.xxx',
        'PORT' : 'xxxx',
    }
}

The above line configurations is a specific script for connecting the django application to PostgreSQL database server. Suddenly, after saving the file, the runserver application execution is generating error. The error message is ModuleNotFoundError: No module named ‘psycopg2’. The following is the complete output error message :

user@hostname:~/python/my-django/users$ python3 manage.py makemigrations
Traceback (most recent call last):e
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 20, in 
    import psycopg2 as Database
ModuleNotFoundError: No module named 'psycopg2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 21, in 
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 994, in _gcd_import
  File "", line 971, in _find_and_load
  File "", line 955, in _find_and_load_unlocked
  File "", line 665, in _load_unlocked
  File "", line 678, in exec_module
  File "", line 219, in _call_with_frames_removed
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/contrib/auth/models.py", line 2, in 
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/contrib/auth/base_user.py", line 47, in 
    class AbstractBaseUser(models.Model):
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/models/base.py", line 117, in __new__
    new_class.add_to_class('_meta', Options(meta, app_label))
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/models/base.py", line 321, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/models/options.py", line 204, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "v/python/django-env/lib/python3.6/site-packages/django/db/__init__.py", line 28, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/utils.py", line 201, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/utils.py", line 110, in load_backend
    return import_module('%s.base' % backend_name)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/user/python/django-env/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 24, in 
    raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2'

Solving the Problem

To solve the above problem, there are several solutions and those solutions have a connection with one another. But the most important thing is the focus of the solution is in the existance of the ‘psycopg2’. How can the module with the name of ‘psycopg2’ can be available for further usage in the django application.  There are several steps or solution to solve the problem. The following is the list of the alternative steps or solution :

1. Installing the psycopg2 package.

2. Install the psycopg2 package to the virtual environment.

In the following section, there will be a further detail explanation about the above steps for solving the problem.

Installing psycopg2 package to the operating system using apt tool

Since it is the Linux Ubuntu distribution, execute the following command to install the ‘psycopg2’ module :

(django-env) (base) user@hostname:~/python/my-django/users$ sudo apt-get -y install python3-psycopg2
[sudo] password for user: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  python-psycopg2-doc
The following NEW packages will be installed:
  python3-psycopg2
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 152 kB of archives.
After this operation, 838 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-psycopg2 amd64 2.7.4-1 [152 kB]
Fetched 152 kB in 2s (85.6 kB/s)                    
Selecting previously unselected package python3-psycopg2.
(Reading database ... 328999 files and directories currently installed.)
Preparing to unpack .../python3-psycopg2_2.7.4-1_amd64.deb ...
Unpacking python3-psycopg2 (2.7.4-1) ...
Setting up python3-psycopg2 (2.7.4-1) ...
(django-env) (base) user@hostname:~/python/my-django/users$ 

The above command execution is relying on the apt tool program. Just make sure to choose the correct package name since the chosen package for the installation above is using python3. So, the package name for installation is ‘python3-psycopg. Make sure to search the compatible package using ‘apt search’ command.

Apparently, in this case, after executing the above command for installing psycopg2 to the operating system, the error message still occurs. So, if the error message is still occurs, just do the following step as another action to complete the solution.

Installing psycopg2 module in the python virtual environment using pip

The following is the command for executing psycopg2 to the running python virtual environment. The python virtual environment is ‘django-env’. The command for installing psycopg2 module is ‘pip install psycopg2’.

(django-env) (base) user@hostname:~/python/my-django/users$ pip install psycopg2
Collecting psycopg2
Downloading https://files.pythonhosted.org/packages/5c/1c/6997288da181277a0c29bc39a5f9143ff20b8c99f2a7d059cfb55163e165/psycopg2-2.8.3.tar.gz (377kB)
100% |████████████████████████████████| 378kB 1.4MB/s
Building wheels for collected packages: psycopg2
Running setup.py bdist_wheel for psycopg2 ... error
Complete output from command /home/user/python/django-env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-nf3652og/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('rn', 'n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpb8vrk172pip-wheel- --python-tag cp36:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help

error: invalid command 'bdist_wheel'

----------------------------------------
Failed building wheel for psycopg2
Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
Running setup.py install for psycopg2 ... error
Complete output from command /home/user/python/django-env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-nf3652og/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('rn', 'n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-zxhvlzhf-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/python/django-env/include/site/python3.6/psycopg2:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.6/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/psycopg
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.3 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=100009 -DHAVE_LO64=1 -I/home/user/python/django-env/include -I/usr/include/python3.6m -I. -I/usr/include/postgresql -I/usr/include/postgresql/10/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.6/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:27:0:
./psycopg/psycopg.h:34:10: fatal error: Python.h: No such file or directory
#include
^~~~~~~~~~
compilation terminated.

It appears you are missing some prerequisite to build the package from source.

You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.

For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Command "/home/user/python/django-env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-nf3652og/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('rn', 'n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-zxhvlzhf-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/python/django-env/include/site/python3.6/psycopg2" failed with error code 1 in /tmp/pip-build-nf3652og/psycopg2/
(django-env) (base) user@hostname:~/python/my-django/users$


Installation of psycopg2-binary module in the python virtual environment using pip

Apparently, the solution in the previous step failed. There is an information about how to proceed the installation. It gives a solution to install a binary package with the name of ‘psycopg2-binary’. The following is the command execution :

(django-env) (base) user@hostname:~/python/my-django/users$ pip install psycopg2-binary==2.7.4
Collecting psycopg2-binary==2.7.4
Downloading https://files.pythonhosted.org/packages/5f/0b/aa7078d3f6d27d951c38b6a1f4b99b71b2caecebb2921b2d808b5bf0e2e0/psycopg2_binary-2.7.4-cp36-cp36m-manylinux1_x86_64.whl (2.7MB)
100% |████████████████████████████████| 2.7MB 194kB/s
Installing collected packages: psycopg2-binary
Successfully installed psycopg2-binary-2.7.4
(django-env) (base) user@hostname:~/python/my-django/users$

In the output above, the python virtual environment is ‘django-env’. According to the output message of the running django web server, after the execution of the above step ends in a success, the error message disappear and the problem solved.

Понравилась статья? Поделить с друзьями:

Читайте также:

  • From keras datasets import mnist error
  • From django db import models ошибка
  • From django conf urls import url ошибка
  • Frmtread на магнитоле пионер error 11
  • Frmt read ошибка пионер

  • 0 0 голоса
    Рейтинг статьи
    Подписаться
    Уведомить о
    guest

    0 комментариев
    Старые
    Новые Популярные
    Межтекстовые Отзывы
    Посмотреть все комментарии