Fatal python error cannot recover from stack overflow python runtime state initialized

Created on 2020-11-30 06:01 by xxm, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg382111 — (view)	Author: Xinmeng Xia (xxm)	Date: 2020-11-30 06:01
The following program 1 can crash in Python 3. We have reproduce it in the Python version 3.5, 3.6, 3.7, 3.8, 3.9, 3.10. This bug seems to be similar to issue 36272, however, the tracking system shows issue 36272 has been fixed and the program 2, which triggers issue 36272, will not cause crash (“core dump”) in Python 3.8, 3.9, 3.10. We have attached the stack trace in the end of this report. Program 1: ======================== import logging def rec(): try: logging.error("foo") except: pass rec() rec() ======================== Program 2: ======================== import logging def rec(): logging.error("foo") rec() rec() ====================== The error message is like following: “ ERROR:root:foo ERROR:root:foo ERROR:root:foo ERROR:root:foo … ERROR:root:foo ERROR:root:foo ERROR:root:foo Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow. Python runtime state: initialized Current thread 0x00007f0fa440b700 (most recent call first): File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 420 in usesTime File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 643 in usesTime File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 675 in format File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 938 in format File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 1094 in emit File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 963 in handle File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 1673 in callHandlers File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 1611 in handle File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 1601 in _log File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 1483 in error File "/usr/local/python310/lib/python3.10/logging/__init__.py", line 2080 in error File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 8 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec …. File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec File "/home/xxm/Desktop/methodfuzzer/error/loggingtest.py", line 12 in rec ... Aborted (core dumped)”
msg382112 — (view)	Author: Dennis Sweeney (Dennis Sweeney) *	Date: 2020-11-30 06:51
This might be the expected behavior. See https://bugs.python.org/issue25222 If you already caught a RecursionError and you keep recursing anyway, once you go 50 levels beyond sys.getrecursionlimit(), the interpreter crashes regardless of what is `except`ed. In /Python/ceval.c, there's this: if (tstate->overflowed) { if (tstate->recursion_depth > recursion_limit + 50) { /* Overflowing while handling an overflow. Give up. */ Py_FatalError("Cannot recover from stack overflow."); } return 0; } In your Program 2, when the interpreter raises a `RecursionError`, it is raised normally and everything is fine. In your Program 1, when the interpreter raises a `RecursionError`, it is `except`ed, so the interpreter thinks it's okay to keep going, and when it does, it raises more `RecursionError`s, which it keeps `except`ing, until it finally can't go any farther ( > 50 + sys.getrecursionlimit()), and has no option but to crash. "Cannot recover from stack overflow." seems to make sense to me: when the interpreter tries to recover, the code won't let it.
msg382114 — (view)	Author: Ronald Oussoren (ronaldoussoren) *	Date: 2020-11-30 08:10
See also #42500
msg382117 — (view)	Author: Xinmeng Xia (xxm)	Date: 2020-11-30 08:19
But program like following program 3 will not cause any core dump, RecursionError is also being caught in this Recursion. program 3 def rec(): try: rec() except: pass rec() Beside,I use sys.setrecursionlimit(80), and the program 1 still cause core dump.I print sys.getrecursionlimit(),the value is 1000. 80 is << 50 +1000, it shouldn't cause core dump.
msg382128 — (view)	Author: Dennis Sweeney (Dennis Sweeney) *	Date: 2020-11-30 09:04
sys.getrecursionlimit() returns whatever was passed to the most recent call of sys.setrecursionlimit(...), with some system default (here 1000). Catching a RecursionError might be fine sometimes, but the issue is that Program 1 catches a RecursionError *and then keeps recursing more* rather than stopping. I think it might have to be the responsibility of the Python user to make sure that if a RecursionError is to be caught, that the program can recover without making things much worse. It's my understanding that the extra buffer of +50 is to make sure that the programmer has room to stop the overflow and do any necessary cleanup [1]. If no attempt is made to clean up, then it seems reasonable to me that Python should crash, unless there's some idea of what could happen that I'm missing. The interpreter could allow arbitrary recursion during the cleanup until the C stack overflows, but that sort of defeats the point of the recursion checker. It could raise some new ExtraSuperRecurionError, but that doesn't fix anything: what if *that* error is caught ;) ? [1] to get back to a recursion depth lower than some lower threshold: https://github.com/python/cpython/blob/master/Include/internal/pycore_ceval.h#L98
msg382151 — (view)	Author: Mark Shannon (Mark.Shannon) *	Date: 2020-11-30 14:24
Duplicate of 42500

Issue

I imatates QT’s official documentation to write a calcuator example.

I try to rewrite the source code into python and make some changes. My code is as blow:

import sys

from PySide6.QtWidgets import (QApplication, QSizePolicy, QWidget, QToolButton, QGridLayout)
from PySide6 import QtCore


class Advanced_Calculator(QWidget):
    def __init__(self):
        super(Advanced_Calculator, self).__init__()

        self.mainwindow = QGridLayout(self)

        # Digit buttons and operator buttons

        self.widget_button = QWidget()

        self.button_digit = []

        for i in range(0, 10):
            self.button_digit.append(Button(str(i)))

        self.button_factor = Button("!")
        self.button_lbracket = Button("(")
        self.button_rbracket = Button(")")
        self.button_backspace = Button("<-")
        self.button_division = Button("/")
        self.button_log = Button("log")
        self.button_multiply = Button("X")
        self.button_sqrt = Button("√")
        self.button_minus = Button("-")
        self.button_power = Button("^")
        self.button_plus = Button("+")
        self.button_abs = Button("|x|")
        self.button_const = Button("Const")
        self.button_dot = Button(".")
        self.button_equal = Button("=")

        # Buttons layout with 0 spacing

        self.layout_button = QGridLayout()
        self.layout_button.setSpacing(0)

        self.layout_button.addWidget(self.button_factor, 0, 0, 1, 1)
        self.layout_button.addWidget(self.button_lbracket, 0, 1, 1, 1)
        self.layout_button.addWidget(self.button_rbracket, 0, 2, 1, 1)
        self.layout_button.addWidget(self.button_backspace, 0, 3, 1, 1)
        self.layout_button.addWidget(self.button_division, 0, 4, 1, 1)
        self.layout_button.addWidget(self.button_log, 1, 0, 1, 1)

        for i in range(1, 10):
            self.layout_button.addWidget(self.button_digit[i], 3 - ((i - 1) // 3), (i - 1) % 3 + 1, 1, 1)

        self.layout_button.addWidget(self.button_multiply, 1, 4, 1, 1)
        self.layout_button.addWidget(self.button_sqrt, 2, 0, 1, 1)
        self.layout_button.addWidget(self.button_minus, 2, 4, 1, 1)
        self.layout_button.addWidget(self.button_power, 3, 0, 1, 1)
        self.layout_button.addWidget(self.button_plus, 3, 4, 1, 1)
        self.layout_button.addWidget(self.button_power, 3, 0, 1, 1)
        self.layout_button.addWidget(self.button_power, 3, 0, 1, 1)
        self.layout_button.addWidget(self.button_power, 3, 0, 1, 1)
        self.layout_button.addWidget(self.button_power, 3, 0, 1, 1)
        self.layout_button.addWidget(self.button_abs, 4, 0, 1, 1)
        self.layout_button.addWidget(self.button_const, 4, 1, 1, 1)
        self.layout_button.addWidget(self.button_digit[0], 4, 2, 1, 1)
        self.layout_button.addWidget(self.button_dot, 4, 3, 1, 1)
        self.layout_button.addWidget(self.button_equal, 4, 4, 1, 1)

        self.widget_button.setLayout(self.layout_button)

        # button layout set to mainwindow

        self.mainwindow.addWidget(self.widget_button)


class Button(QToolButton):
    def __init__(self, text, parent=None):
        super(Button, self).__init__(parent)

        self.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
        self.setText(text)

    def sizeHint(self) -> QtCore.QSize:
        size = self.sizeHint()
        size.setHeight(size.height + 20)
        size.setWidth(max(size.width(), size.height()))
        # size.rheight() += 20
        # size.rwidth() = max(size.width(), size.height())

        return size


if __name__ == "__main__":
    app = QApplication([])

    Calculator = Advanced_Calculator()
    Calculator.show()

    sys.exit(app.exec())

When I debug these code, the «Fatal Python error: Cannot recover from stack overflow» occurs. And now it is known to me that the error is caused by the sizeHint() function. Where is the problem?

By the way, the function was rewrite by me with my comprehending of the sizeHint() function. And the official documentation uses the two lines code blow which I made them comments, and it doesn’t work, too. Well, I also can’t figure out where is the problem.

Solution

Consider the code…

def sizeHint(self) -> QtCore.QSize:
    size = self.sizeHint()
    size.setHeight(size.height + 20)
    size.setWidth(max(size.width(), size.height()))
    # size.rheight() += 20
    # size.rwidth() = max(size.width(), size.height())

    return size

The line…

size = self.sizeHint()

causes Button::sizeHint to call itself leading to infinite recursion and, hence, a stack overflow. Perhaps you meant to call the base class implementation instead…

size = super(Button, self).sizeHint()

Answered By — G.M.

I’ve come to a dead end, and after excessive (and unsuccessful) Googling, I need help.

I’m building a simple PyQt4 Widget where it lies out a grid of 60×80 squares, each initialized to None. If the user clicks on that box it changes color based on how many times left-clicked, defined by this list:

self.COLORS=[
        (0, 0, 255),        #WATER
        (255, 210, 128),    #SAND
        (0, 128, 0),       #GREEN
        (255, 255, 0),    #YELLOW
        (255, 165, 0),    #ORANGE
        (255, 0, 0)          #RED

]

If the user right clicks, it flood fills an area, using the common recursive flood fill algo. This works perfectly for small spaces, however if the space is large enough the program fails with the error Fatal Python error: Cannot recover from . I have no idea how to fix this, perhaps a flood fill that isn’t recursive?

All squares and subsequent color codes are stored in self.cells so by setting self.cells[(y,x)]=1 would set cell (y,x) to the Sand color.

Here is the program in whole.

import sys
from PyQt4 import QtGui, QtCore

class Example(QtGui.QWidget):

    def __init__(self, cell_size=10, swidth=800, sheight=600):
        QtGui.QWidget.__init__(self)
        self.resize(swidth,sheight)

        self.cell_size = cell_size
        self.height = sheight
        self.width = swidth
        self.columns = self.width // self.cell_size
        self.rows = self.height // self.cell_size

        self.COLORS=[
                (0, 0, 255),        #WATER
                (255, 210, 128),    #SAND
                (0, 128, 0),       #GREEN
                (255, 255, 0),    #YELLOW
                (255, 165, 0),    #ORANGE
                (255, 0, 0)          #RED

        ]

        self.cells = {(x,y):None for x in range(1,self.columns+1) for y in range(1,self.rows+1)}        

    def translate(self,pixel_x, pixel_y):
        "Translate pixel coordinates (pixel_x,pixel_y), into grid coordinates"
        x = pixel_x * self.columns // self.width + 1
        y = pixel_y * self.rows // self.height  + 1
        return x,y

    def check_cell(self,x,y):
        if self.cells[(x,y)] <= 0:
            self.cells[(x,y)]=0
        elif self.cells[(x,y)] >= len(self.COLORS)-1:
            self.cells[(x,y)]=len(self.COLORS)-1
        else:
            pass

    def draw_cell(self, qp, col, row):
        x1,y1 = (col-1) * self.cell_size, (row-1) * self.cell_size
        x2,y2 = (col-1) * self.cell_size + self.cell_size, (row-1) * self.cell_size + self.cell_size 
        qp.drawRect(x1, y1, x2-x1, y2-y1)

    def color_cell(self, qp, col, row):
        qp.setBrush(QtGui.QColor(*self.COLORS[self.cells[(col,row)]]))
        self.draw_cell(qp, col, row)

    def draw_grid(self, qp):
        qp.setPen(QtGui.QColor(128,128,128)) # gray
        # Horizontal lines
        for i in range(self.rows):
            qp.drawLine(0, i * self.cell_size, self.width, i * self.cell_size)
        # Vertical lines
        for j in range(self.columns):
            qp.drawLine(j * self.cell_size, 0, j * self.cell_size, self.height)

    def set_all(self, type):
        self.cells = {(x,y):type for x in range(1,self.columns+1) for y in range(1,self.rows+1)}  
        self.repaint()

    def fill(self, x, y, type):
        print(x,y)
        if x < 1 or x >= self.columns+1 or y < 1 or y >= self.rows+1:
            return
        if self.cells[(x,y)] != None:
            return
        self.cells[(x,y)] = type
        self.repaint()
        self.fill(x+1, y, type)
        self.fill(x-1, y, type)
        self.fill(x, y+1, type)
        self.fill(x, y-1, type)


    def paintEvent(self, e):
        qp = QtGui.QPainter()
        qp.begin(self)
        self.draw_grid(qp)
        for row in range(1, self.rows+1):
            for col in range(1, self.columns+1):
                if self.cells[(col,row)] != None:
                    self.color_cell(qp, col, row)
        qp.end()

    def drawPoints(self, qp):
        size = self.size()

        for i in range(1000):
            x = random.randint(1, size.width()-1)
            y = random.randint(1, size.height()-1)
            qp.drawPoint(x, y)  

    def mousePressEvent(self, e):
        x,y = self.translate(e.pos().x(),e.pos().y())

        if e.button() == QtCore.Qt.LeftButton:
            if self.cells[(x,y)] == None:
                self.cells[(x,y)]=0
            else:
                self.cells[(x,y)]+=1
                self.check_cell(x,y)

        elif e.button() == QtCore.Qt.RightButton:
            self.fill(x,y,0)
            '''
            if self.cells[(x,y)] == None:
                self.cells[(x,y)]=0
            else:  
                self.cells[(x,y)]-=1
                self.check_cell(x,y)
            '''            
        else: pass

        self.repaint()

    def save(self):
        return self.cells

    def open(self, new_cells):
        self.cells=new_cells
        self.repaint()


def main():
    app = QtGui.QApplication(sys.argv)
    ex = Example()
    ex.show()
    sys.exit(app.exec_())


if __name__ == '__main__':
    main()

Can anyone help diagnose the problem or perhaps point in a direction to fix it?