Developing especially complex applications can lead to crazy technical situations such as segmentation faults. This situation can happen due to different factors, and tracing the issue with your code is important.
Some of the biggest causes of segmentation faults are illegal memory location (using your code to access memory that you don’t have access to), fetching huge datasets, infinite recursion, etc. In this article, we will show you how to manage segmentation faults in Python.
Use settrace to Manage Segmentation Fault in Python
When faced with a segmentation fault error, it is important to know that you will have to rewrite your code.
Knowing the part to rewrite is a good starting point. That’s where sys.trace comes in.
The sys module allows us to check some variables and interact with the interpreter, and the settrace function allows us to trace the program execution and serves a Python source code debugger.
So with a case of segmentation fault, we can easily trace what calls are made and when everything happens.
The trace functions are important to the whole process, taking three arguments: frame, event, and arg. The frame takes the current stack frame, the event takes a string, and the arg takes shape depending on the event we parse.
So, within your code, you can import the sys module, create your trace function, and pass the trace function through the settrace method, which makes for a higher-order function. Then, you place your code that creates a segmentation fault.
Let’s create a trace for a test() function that prints two lines of code.
import sys
def trace(frame, event, arg):
print("%s, %s:%d" % (event, frame.f_code.co_filename, frame.f_lineno))
return trace
def test():
print("Line 8")
print("Line 9")
sys.settrace(trace)
test()
Output:
call, c:UsersakinlDocumentsPythonsegment.py:7
line, c:UsersakinlDocumentsPythonsegment.py:8
call, C:Python310libencodingscp1252.py:18
line, C:Python310libencodingscp1252.py:19
return, C:Python310libencodingscp1252.py:19
Line 8call, C:Python310libencodingscp1252.py:18
line, C:Python310libencodingscp1252.py:19
return, C:Python310libencodingscp1252.py:19
line, c:UsersakinlDocumentsPythonsegment.py:9
call, C:Python310libencodingscp1252.py:18
line, C:Python310libencodingscp1252.py:19
return, C:Python310libencodingscp1252.py:19
Line 9call, C:Python310libencodingscp1252.py:18
line, C:Python310libencodingscp1252.py:19
return, C:Python310libencodingscp1252.py:19
return, c:UsersakinlDocumentsPythonsegment.py:9
You can see the event occurring with each line, call, line, and return. With these, you can track every action the Python interpreter is doing and the output it gives.
You can see the code outputs — Line 8call, C:Python310libencodingscp1252.py:18 and Line 9call, C:Python310libencodingscp1252.py:18. Therefore, with a segmentation fault, we can trace where the problems start and work our way from there.
Sometimes you’ll get a segmentation fault in Python and your process will crash, this is due to a C module attempting to access memory beyond reach.
Output in this case will be very limited:
Segmentation fault
To get a full traceback we need to enable the Python faulthandler module.
First add the following to the top of your module.
import faulthandler; faulthandler.enable()
Then re-run your program with the faulthandler startup flag.
Passed as an argument.
# pass as an argument python -Xfaulthandler my_program.py # Or as an environment variable. PYTHONFAULTHANDLER=1 python my_program.py
Now you will get a detailed traceback.
Fatal Python error: Segmentation fault Current thread 0x00007f0a49caa700 (most recent call first): File "", line 219 in _call_with_frames_removed File "", line 922 in create_module File "", line 571 in module_from_spec File "", line 658 in _load_unlocked File "", line 684 in _load File "/usr/local/lib/python3.6/imp.py", line 343 in load_dynamic File "/usr/local/lib/python3.6/imp.py", line 243 in load_module File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24 in swig_import_helper File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 1023 in _handle_fromlist File "/usr/local/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 49 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/usr/local/lib/python3.6/site-packages/tensorflow/__init__.py", line 24 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/usr/local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 5 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/usr/local/lib/python3.6/site-packages/keras/backend/__init__.py", line 83 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 1023 in _handle_fromlist File "/usr/local/lib/python3.6/site-packages/keras/utils/conv_utils.py", line 9 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 1023 in _handle_fromlist File "/usr/local/lib/python3.6/site-packages/keras/utils/__init__.py", line 6 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 1023 in _handle_fromlist File "/usr/local/lib/python3.6/site-packages/keras/__init__.py", line 3 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 941 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/opt/app/ai_platform/forecasting.py", line 7 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/opt/app/ai_platform/customer_operations.py", line 12 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "/opt/app/app.py", line 13 in File "", line 219 in _call_with_frames_removed File "", line 678 in exec_module File "", line 665 in _load_unlocked File "", line 955 in _find_and_load_unlocked File "", line 971 in _find_and_load File "", line 994 in _gcd_import File "/usr/local/lib/python3.6/importlib/__init__.py", line 126 in import_module File "/usr/local/lib/python3.6/site-packages/celery/utils/imports.py", line 101 in import_from_cwd File "/usr/local/lib/python3.6/site-packages/kombu/utils/imports.py", line 56 in symbol_by_name File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 506 in symbol_by_name File "/usr/local/lib/python3.6/site-packages/celery/app/utils.py", line 355 in find_app File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 503 in find_app File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 481 in setup_app_from_commandline File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 279 in execute_from_commandline ... Segmentation fault
Happy debugging!
Уведомления
- Начало
- » Python для новичков
- » ошибка сегментирования на 64-разр.
#1 Ноя. 4, 2011 09:23:05
ошибка сегментирования на 64-разр.
Почему режеться указатель при передачи в сишный модуль? Это баг Питона или неправильное кодирование? Как можно устранить?
На более маленьких массивах все ok. Т.ж. на 32-битных системах работает без проблем.
$ cat test.py
#!/usr/bin/python
import ctypes
import numpy
from sys import path
path.append('./../')
from module.matrix import *
A = myones(200,200)
print A$ cat ../module/libmatrix.c
void ones_matrix(
const int n,
const int m,
double* mtx)
{
for(int i=0; i<n; ++i)
for(int j=0; j<m; ++j)
*(mtx++)=(i+j)/2.;
}$ gcc -std=gnu99 -fpic -shared -g -pg libmatrix.c -o libmatrix.so
$ cat ../module/matrix.py
#!/usr/bin/python
from numpy import ctypeslib, ndarray, zeros
from ctypes import c_double, byref, cdll
from os import path
from sys import platform
path = path.dirname(path.realpath(__file__))
if 'linux' in platform:
mod_matrix_maker=cdll.LoadLibrary("%s/libmatrix.so" % path)def myones(n,m):
A=ndarray((n,m),dtype='d')
mod_matrix_maker.ones_matrix(n,m,A.reshape(-1,1).ctypes.data)
return A
$ gdb -arg python ./test.py
GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /usr/bin/python...(no debugging symbols found)...done.
(gdb) start
Temporary breakpoint 1 at 0x41aa40
Starting program: /usr/bin/python ./test.py
[Thread debugging using libthread_db enabled]Temporary breakpoint 1, 0x000000000041aa40 in main ()
(gdb) next
Single stepping until exit from function main,
which has no line number information.
0x00000000004ee1e0 in Py_Main ()
(gdb) next
Single stepping until exit from function Py_Main,
which has no line number information.Program received signal SIGSEGV, Segmentation fault.
0x00007ffff34bd5e2 in ones_matrix (n=200, m=200, mtx=0xf346e010) at libmatrix.c:11
11 *(mtx++)=(i+j)/2.;
$ pmap -d 7419
7419: /usr/bin/python ./test.py
Address Kbytes Mode Offset Device Mapping
0000000000400000 2252 r-x-- 0000000000000000 0fd:00000 python2.7
0000000000832000 4 r---- 0000000000232000 0fd:00000 python2.7
0000000000833000 420 rw--- 0000000000233000 0fd:00000 python2.7
000000000089c000 5184 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff346e000 316 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff34bd000 4 r-x-- 0000000000000000 0fd:00002 libmatrix.so
00007ffff34be000 2044 ----- 0000000000001000 0fd:00002 libmatrix.so
00007ffff36bd000 4 r---- 0000000000000000 0fd:00002 libmatrix.so
00007ffff36be000 4 rw--- 0000000000001000 0fd:00002 libmatrix.so
00007ffff36bf000 292 r-x-- 0000000000000000 0fd:00000 mtrand.so
00007ffff3708000 2048 ----- 0000000000049000 0fd:00000 mtrand.so
00007ffff3908000 4 r---- 0000000000049000 0fd:00000 mtrand.so
00007ffff3909000 112 rw--- 000000000004a000 0fd:00000 mtrand.so
00007ffff3925000 4 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff3926000 36 r-x-- 0000000000000000 0fd:00000 fftpack_lite.so
00007ffff392f000 2044 ----- 0000000000009000 0fd:00000 fftpack_lite.so
00007ffff3b2e000 4 r---- 0000000000008000 0fd:00000 fftpack_lite.so
00007ffff3b2f000 4 rw--- 0000000000009000 0fd:00000 fftpack_lite.so
00007ffff3b30000 212 r-x-- 0000000000000000 0fd:00000 libquadmath.so.0.0.0
00007ffff3b65000 2044 ----- 0000000000035000 0fd:00000 libquadmath.so.0.0.0
00007ffff3d64000 4 r---- 0000000000034000 0fd:00000 libquadmath.so.0.0.0
00007ffff3d65000 4 rw--- 0000000000035000 0fd:00000 libquadmath.so.0.0.0
00007ffff3d66000 84 r-x-- 0000000000000000 0fd:00000 libgcc_s.so.1
00007ffff3d7b000 2044 ----- 0000000000015000 0fd:00000 libgcc_s.so.1
00007ffff3f7a000 4 r---- 0000000000014000 0fd:00000 libgcc_s.so.1
00007ffff3f7b000 4 rw--- 0000000000015000 0fd:00000 libgcc_s.so.1
00007ffff3f7c000 1104 r-x-- 0000000000000000 0fd:00000 libgfortran.so.3.0.0
00007ffff4090000 2044 ----- 0000000000114000 0fd:00000 libgfortran.so.3.0.0
00007ffff428f000 4 r---- 0000000000113000 0fd:00000 libgfortran.so.3.0.0
00007ffff4290000 8 rw--- 0000000000114000 0fd:00000 libgfortran.so.3.0.0
00007ffff4292000 9108 r-x-- 0000000000000000 0fd:00000 liblapack.so.3gf.0
00007ffff4b77000 2044 ----- 00000000008e5000 0fd:00000 liblapack.so.3gf.0
00007ffff4d76000 4 r---- 00000000008e4000 0fd:00000 liblapack.so.3gf.0
00007ffff4d77000 16 rw--- 00000000008e5000 0fd:00000 liblapack.so.3gf.0
00007ffff4d7b000 1076 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff4e88000 20 r-x-- 0000000000000000 0fd:00000 lapack_lite.so
00007ffff4e8d000 2048 ----- 0000000000005000 0fd:00000 lapack_lite.so
00007ffff508d000 4 r---- 0000000000005000 0fd:00000 lapack_lite.so
00007ffff508e000 4 rw--- 0000000000006000 0fd:00000 lapack_lite.so
00007ffff508f000 16 r-x-- 0000000000000000 0fd:00000 _compiled_base.so
00007ffff5093000 2044 ----- 0000000000004000 0fd:00000 _compiled_base.so
00007ffff5292000 4 r---- 0000000000003000 0fd:00000 _compiled_base.so
00007ffff5293000 4 rw--- 0000000000004000 0fd:00000 _compiled_base.so
00007ffff5294000 772 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff5355000 12 r-x-- 0000000000000000 0fd:00000 _heapq.so
00007ffff5358000 2044 ----- 0000000000003000 0fd:00000 _heapq.so
00007ffff5557000 4 r---- 0000000000002000 0fd:00000 _heapq.so
00007ffff5558000 8 rw--- 0000000000003000 0fd:00000 _heapq.so
00007ffff555a000 260 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff559b000 152 r-x-- 0000000000000000 0fd:00000 scalarmath.so
00007ffff55c1000 2044 ----- 0000000000026000 0fd:00000 scalarmath.so
00007ffff57c0000 4 r---- 0000000000025000 0fd:00000 scalarmath.so
00007ffff57c1000 8 rw--- 0000000000026000 0fd:00000 scalarmath.so
00007ffff57c3000 612 r-x-- 0000000000000000 0fd:00000 libblas.so.3gf.0
00007ffff585c000 2044 ----- 0000000000099000 0fd:00000 libblas.so.3gf.0
00007ffff5a5b000 4 r---- 0000000000098000 0fd:00000 libblas.so.3gf.0
00007ffff5a5c000 4 rw--- 0000000000099000 0fd:00000 libblas.so.3gf.0
00007ffff5a5d000 20 r-x-- 0000000000000000 0fd:00000 _dotblas.so
00007ffff5a62000 2044 ----- 0000000000005000 0fd:00000 _dotblas.so
00007ffff5c61000 4 r---- 0000000000004000 0fd:00000 _dotblas.so
00007ffff5c62000 4 rw--- 0000000000005000 0fd:00000 _dotblas.so
00007ffff5c63000 84 r-x-- 0000000000000000 0fd:00000 _sort.so
00007ffff5c78000 2044 ----- 0000000000015000 0fd:00000 _sort.so
00007ffff5e77000 4 r---- 0000000000014000 0fd:00000 _sort.so
00007ffff5e78000 4 rw--- 0000000000015000 0fd:00000 _sort.so
00007ffff5e79000 304 r-x-- 0000000000000000 0fd:00000 umath.so
00007ffff5ec5000 2044 ----- 000000000004c000 0fd:00000 umath.so
00007ffff60c4000 4 r---- 000000000004b000 0fd:00000 umath.so
00007ffff60c5000 16 rw--- 000000000004c000 0fd:00000 umath.so
00007ffff60c9000 8 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff60cb000 536 r-x-- 0000000000000000 0fd:00000 multiarray.so
00007ffff6151000 2044 ----- 0000000000086000 0fd:00000 multiarray.so
00007ffff6350000 4 r---- 0000000000085000 0fd:00000 multiarray.so
00007ffff6351000 44 rw--- 0000000000086000 0fd:00000 multiarray.so
00007ffff635c000 8 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff635e000 124 r-x-- 0000000000000000 0fd:00000 _ctypes.so
00007ffff637d000 2048 ----- 000000000001f000 0fd:00000 _ctypes.so
00007ffff657d000 4 r---- 000000000001f000 0fd:00000 _ctypes.so
00007ffff657e000 16 rw--- 0000000000020000 0fd:00000 _ctypes.so
00007ffff6582000 4 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff6583000 4072 r---- 0000000000000000 0fd:00000 locale-archive
00007ffff697d000 1620 r-x-- 0000000000000000 0fd:00000 libc-2.13.so
00007ffff6b12000 2044 ----- 0000000000195000 0fd:00000 libc-2.13.so
00007ffff6d11000 16 r---- 0000000000194000 0fd:00000 libc-2.13.so
00007ffff6d15000 4 rw--- 0000000000198000 0fd:00000 libc-2.13.so
00007ffff6d16000 24 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff6d1c000 524 r-x-- 0000000000000000 0fd:00000 libm-2.13.so
00007ffff6d9f000 2044 ----- 0000000000083000 0fd:00000 libm-2.13.so
00007ffff6f9e000 4 r---- 0000000000082000 0fd:00000 libm-2.13.so
00007ffff6f9f000 4 rw--- 0000000000083000 0fd:00000 libm-2.13.so
00007ffff6fa0000 92 r-x-- 0000000000000000 0fd:00000 libz.so.1.2.3.4
00007ffff6fb7000 2044 ----- 0000000000017000 0fd:00000 libz.so.1.2.3.4
00007ffff71b6000 4 r---- 0000000000016000 0fd:00000 libz.so.1.2.3.4
00007ffff71b7000 4 rw--- 0000000000017000 0fd:00000 libz.so.1.2.3.4
00007ffff71b8000 1568 r-x-- 0000000000000000 0fd:00000 libcrypto.so.1.0.0
00007ffff7340000 2048 ----- 0000000000188000 0fd:00000 libcrypto.so.1.0.0
00007ffff7540000 100 r---- 0000000000188000 0fd:00000 libcrypto.so.1.0.0
00007ffff7559000 40 rw--- 00000000001a1000 0fd:00000 libcrypto.so.1.0.0
00007ffff7563000 16 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7567000 296 r-x-- 0000000000000000 0fd:00000 libssl.so.1.0.0
00007ffff75b1000 2044 ----- 000000000004a000 0fd:00000 libssl.so.1.0.0
00007ffff77b0000 12 r---- 0000000000049000 0fd:00000 libssl.so.1.0.0
00007ffff77b3000 20 rw--- 000000000004c000 0fd:00000 libssl.so.1.0.0
00007ffff77b8000 8 r-x-- 0000000000000000 0fd:00000 libutil-2.13.so
00007ffff77ba000 2044 ----- 0000000000002000 0fd:00000 libutil-2.13.so
00007ffff79b9000 4 r---- 0000000000001000 0fd:00000 libutil-2.13.so
00007ffff79ba000 4 rw--- 0000000000002000 0fd:00000 libutil-2.13.so
00007ffff79bb000 8 r-x-- 0000000000000000 0fd:00000 libdl-2.13.so
00007ffff79bd000 2048 ----- 0000000000002000 0fd:00000 libdl-2.13.so
00007ffff7bbd000 4 r---- 0000000000002000 0fd:00000 libdl-2.13.so
00007ffff7bbe000 4 rw--- 0000000000003000 0fd:00000 libdl-2.13.so
00007ffff7bbf000 96 r-x-- 0000000000000000 0fd:00000 libpthread-2.13.so
00007ffff7bd7000 2044 ----- 0000000000018000 0fd:00000 libpthread-2.13.so
00007ffff7dd6000 4 r---- 0000000000017000 0fd:00000 libpthread-2.13.so
00007ffff7dd7000 4 rw--- 0000000000018000 0fd:00000 libpthread-2.13.so
00007ffff7dd8000 16 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7ddc000 132 r-x-- 0000000000000000 0fd:00000 ld-2.13.so
00007ffff7e1c000 260 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7e5d000 4 rwx-- 0000000000000000 000:00000 [ anon ]
00007ffff7e5e000 780 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7f53000 540 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7ff9000 8 rw--- 0000000000000000 000:00000 [ anon ]
00007ffff7ffb000 4 r-x-- 0000000000000000 000:00000 [ anon ]
00007ffff7ffc000 4 r---- 0000000000020000 0fd:00000 ld-2.13.so
00007ffff7ffd000 8 rw--- 0000000000021000 0fd:00000 ld-2.13.so
00007ffffffcc000 204 rw--- 0000000000000000 000:00000 [ stack ]
ffffffffff600000 4 r-x-- 0000000000000000 000:00000 [ anon ]
mapped: 85000K writeable/private: 10260K shared: 0K
Офлайн
- Пожаловаться
#2 Ноя. 4, 2011 12:37:32
ошибка сегментирования на 64-разр.
потому что нужно учитывать strides — массив не обязан быть непрерывным в памяти. А лучше смотреть на .ctypes и не трогать .reshape.
Офлайн
- Пожаловаться
#3 Ноя. 4, 2011 15:27:14
ошибка сегментирования на 64-разр.
Андрей Светлов
потому что нужно учитывать strides — массив не обязан быть непрерывным в памяти. А лучше смотреть на .ctypes и не трогать .reshape.
Можно поподробнее ?
удаление reshape не изменяет ситуации:
def myones(n,m):
A=ndarray((n,m),dtype='d')
mod_matrix_maker.ones_matrix(n,m,A.ctypes.data)
return A
Офлайн
- Пожаловаться
#4 Ноя. 4, 2011 16:03:11
ошибка сегментирования на 64-разр.
Stride: The distance (in bytes) between the two consecutive elements along an axis. Numpy expresses strides in bytes because the distance between elements is not necessarily a multiple of the element size.
Или нужно еще подробней?
Офлайн
- Пожаловаться
#5 Ноя. 4, 2011 21:45:09
ошибка сегментирования на 64-разр.
Андрей Светлов
Stride: The distance (in bytes) between the two consecutive elements along an axis. Numpy expresses strides in bytes because the distance between elements is not necessarily a multiple of the element size.
Или нужно еще подробней?
ошибка сегментирования даже для вот такого кода возникает (при записи в самую первую позицию не нужно же учитывать stride):
void ones_matrix(.
const int n,.
const int m,.
double* mtx).
{
*mtx = 1.0;
}
Офлайн
- Пожаловаться
#7 Ноя. 5, 2011 21:44:46
ошибка сегментирования на 64-разр.
к сожалению со strides не прошло, при этом на 32-битных и 64-битных системах strides одинаковые значения имеет, ну и соответственно опять при трассировке указатель на первый элемент mtx обрезан на 64-битной.
$ cat libmatrix.c
#include<math.h>
#include<stdio.h>void ones_matrix( const int n0, const int n1, const int s0, const int s1, double* mtx )
{
int byte_offset;
for(int i=0; i<n0; ++i)
for(int j=0; j<n1; ++j)
{
printf("i=%i j=%i n0=%i n1=%i s0=%i s1=%in",i,j,n0,n1,s0,s1);
byte_offset = s0*i + s1*j;
mtx[byte_offset] = (i+j)/2.;
}
}$ cat matrix.py
#!/usr/bin/python
from numpy import ctypeslib, ndarray, zeros
from ctypes import c_double, byref, cdll
from os import path
from sys import platform
path = path.dirname(path.realpath(__file__))
if 'linux' in platform:
mod_matrix_maker=cdll.LoadLibrary("%s/libmatrix.so" % path)def myones(n,m):
A=ndarray((n,m),dtype='d')
s0=A.strides[0]
s1=A.strides[1]
print "A.shape", A.shape
print "A.strides", A.strides
mod_matrix_maker.ones_matrix(n,m,s0,s1,A.ctypes.data)
return A$ ../test/test.py
A.shape (400, 400)
A.strides (3200, 8)
i=0 j=0 n0=400 n1=400 s0=3200 s1=8
Ошибка сегментирования
Офлайн
- Пожаловаться
#8 Ноя. 5, 2011 22:00:48
ошибка сегментирования на 64-разр.
Андрей Светлов
http://docs.python.org/py3k/library/ctypes.html?highlight=argtypes#ctypes._FuncPtr.argtypes
По умолчанию ctypes аргументы считает как int, насколько помню
Для amd64
sizeof(int) == 4
sizeof(long) == 8
sizeoof(void*) == 8
дошло !
mod_matrix_maker.ones_matrix(n,m,s0,s1, A.ctypes.data_as(POINTER(c_long)))
спасибо!
P.S. strides все таки нужно учитывать ?
здесь вопрос в чем, если писать так
$ cat libmatrix.c
#include<math.h>
#include<stdio.h>void ones_matrix( const int n0, const int n1, const int s0, const int s1, double* mtx )
{
int byte_offset;
int size_double = sizeof(double);
for(int i=0; i<n0; ++i)
for(int j=0; j<n1; ++j)
{
printf("i=%i j=%i n0=%i n1=%i s0=%i s1=%in",i,j,n0,n1,s0,s1);
byte_offset = s0*i + s1*j;
mtx[byte_offset/size_double] = (i+j)/2.;
}
}
то какая разница в таком смещении: здесь тоже смещение на sizeof(double):
или Питон может свое смещение устанавливать, например не 8 байт, а как-то неплотно, например 10 байт для каждого элемента?
или имеется ввиду что для массива n0xn1 мы может получить strides не с двумя измерениями , а с многими , как бы учитывающие хранение его не в непрерывной памяти. а в разрывной ?
или всетаки массив строго в непрерывной памяти хранится ?
Отредактировано (Ноя. 5, 2011 22:21:52)
Офлайн
- Пожаловаться
#9 Ноя. 7, 2011 14:40:29
ошибка сегментирования на 64-разр.
Нет гарантии, что без strides будет всегда и везде работать. Для оптимального выравнивания бывает полезно вставить пустые байты в конце строки, например. И т.д.
Офлайн
- Пожаловаться
- Начало
- » Python для новичков
- » ошибка сегментирования на 64-разр.
- Versions
$ python -V
Python 3.6.7
$ pip install -U pip setuptools
Requirement already up-to-date: pip in ./.virtualenvs/opnsl/lib/python3.6/site-packages (19.1)
Requirement already up-to-date: setuptools in ./.virtualenvs/opnsl/lib/python3.6/site-packages (41.0.1)
$ pip freeze | egrep 'cryptography|cffi'
cffi==1.12.3
cryptography==2.6.1
- cryptography installed in virtualenv using the debug version of python
- Clear steps for reproducing your bug
If run in the terminal, the error is being thrown as I close the terminal. When run in a script, the error is thrown immediately.
source
from cryptography.hazmat.bindings.openssl import binding
output
* ob
object : <refcnt 0 at 0x7fd4f75f77c8>
type : tuple
refcount: 0
address : 0x7fd4f75f77c8
* op->_ob_prev->_ob_next
object : <refcnt 0 at 0x7fd4f75f77c7>Segmentation fault (core dumped)
Admittedly, this by itself probably isn’t really a problem. However, I suspect this is at the root of some other segfaults I’m seeing. I am unable to discern if it’s actually a cffi problem though, so I’d love some feedback.
Background
I’m running django channels with uvloop. DB access occurs in separate threads via the concurrent.futures.ThreadPoolExecutor. When each task is done, the thread pool executor calls del on the worker task. The segfault typically occurs when the current thread is in that call, however, when that’s not the case, another thread is in that call.
Here’s what I’ve been able to gather from a core dump with the debug version of python in my tests:
In this case, the current thread was garbage collecting:
(gdb) py-bt
Traceback (most recent call first):
Garbage-collecting
File "/opt/project/my_virtualenv/lib/python3.6/site-packages/django/utils/functional.py", line 111, in __promise__
def __wrapper__(self, *args, **kw):
...
The core dump shows it was specifically at this section of code:
https://github.com/python/cpython/blob/3.6/Modules/gcmodule.c#L365
Meanwhile, there existed another thread that was previously in the delete call
(gdb) py-bt
Traceback (most recent call first):
Waiting for the GIL
<built-in method SSL_free of CompiledLib object at remote 0x20e0918>
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 71 in _worker
del work_item
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
I think the fact that it occurred in two threads is irrelevant. The problem is that the deallocator calls back to python for some reason and the garbage collector kicks in before the deallocation process is finished, leading to the error mentioned in the comment in the cpython source.
If it’s any help, these are the last few calls that occurred in the thread calling back to python land. The core has some sensitive data, so I can’t upload it, however I’m willing to paste whatever might be helpful.
#0 0x00007f3bb61abf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f3a2dff9670, expected=0, futex_word=0xa7c8ac <gil_cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1 __pthread_cond_wait_common (abstime=0x7f3a2dff9670, mutex=0xa7c840 <gil_mutex>, cond=0xa7c880 <gil_cond>) at pthread_cond_wait.c:539
#2 __pthread_cond_timedwait (cond=cond@entry=0xa7c880 <gil_cond>, mutex=mutex@entry=0xa7c840 <gil_mutex>, abstime=abstime@entry=0x7f3a2dff9670) at pthread_cond_wait.c:667
#3 0x000000000052d0b1 in PyCOND_TIMEDWAIT (us=5000, mut=0xa7c840 <gil_mutex>, cond=0xa7c880 <gil_cond>) at ../Python/condvar.h:103
#4 take_gil (tstate=tstate@entry=0xfd387a0) at ../Python/ceval_gil.h:224
#5 0x000000000052d39e in PyEval_RestoreThread (tstate=tstate@entry=0xfd387a0) at ../Python/ceval.c:369
#6 0x00007f3bb2ad0465 in _cffi_f_SSL_free (self=<optimized out>, arg0=<optimized out>) at build/temp.linux-x86_64-3.6-pydebug/_openssl.c:51462
#7 0x000000000049b3bf in _PyCFunction_FastCallDict (func_obj=func_obj@entry=<built-in method SSL_free of CompiledLib object at remote 0x20e0918>, args=args@entry=0x7f3a2dff97d0, nargs=nargs@entry=1,
kwargs=kwargs@entry=0x0) at ../Objects/methodobject.c:209
#8 0x000000000044a6b4 in _PyObject_FastCallDict (func=func@entry=<built-in method SSL_free of CompiledLib object at remote 0x20e0918>, args=args@entry=0x7f3a2dff97d0, nargs=1, kwargs=kwargs@entry=0x0)
at ../Objects/abstract.c:2313
#9 0x000000000044b7b8 in PyObject_CallFunctionObjArgs (callable=callable@entry=<built-in method SSL_free of CompiledLib object at remote 0x20e0918>) at ../Objects/abstract.c:2827
#10 0x00007f3bb2d96505 in gcp_finalize (destructor=destructor@entry=<built-in method SSL_free of CompiledLib object at remote 0x20e0918>, origobj=origobj@entry=<_cffi_backend.CData at remote 0x7f3a8405db90>)
at c/_cffi_backend.c:1960
#11 0x00007f3bb2d967aa in cdatagcp_dealloc (cd=cd@entry=0x7f3a8405dd88) at c/_cffi_backend.c:1998
Как я могу отладить ошибку сегментации Python?
Мы пытаемся запустить наш код Python на SuSE 12.3. Мы получаем воспроизводимые ошибки сегментации. Код Python работал на других платформах без ошибок сегментации в течение многих лет.
Мы пишем только Python, без расширения C….
Каков наилучший способ отладить это? Я знаю немного ansi c, но это было десять лет назад….
Python 2.7.5
Обновить
Ошибка сегментации возникает при выключении интерпретатора.
Я могу запустить скрипт несколько раз:
python -m pdb myscript.py arg1 arg1
continue
run
continue
run
Но ошибки сегментации случаются, если я выхожу из pdb с помощью Ctrl-D.
Обновление 2
Теперь я пытаюсь отладить его с помощью gdb:
gdb
> file python
> run myscript.py arg1 arg2
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffefbe2700 (LWP 15483)]
0x00007ffff7aef93c in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
(gdb) bt
#0 0x00007ffff7aef93c in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
#1 0x00007ffff7af5303 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.7.so.1.0
#2 0x00007ffff7adc858 in ?? () from /usr/lib64/libpython2.7.so.1.0
#3 0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0
#4 0x00007ffff7af1082 in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
#5 0x00007ffff7af233d in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
#6 0x00007ffff7af233d in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
#7 0x00007ffff7af5303 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.7.so.1.0
#8 0x00007ffff7adc5b6 in ?? () from /usr/lib64/libpython2.7.so.1.0
#9 0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0
#10 0x00007ffff7ad9171 in ?? () from /usr/lib64/libpython2.7.so.1.0
#11 0x00007ffff7ad840d in PyObject_Call () from /usr/lib64/libpython2.7.so.1.0
#12 0x00007ffff7aeeb62 in PyEval_CallObjectWithKeywords () from /usr/lib64/libpython2.7.so.1.0
#13 0x00007ffff7acc757 in ?? () from /usr/lib64/libpython2.7.so.1.0
#14 0x00007ffff7828e0f in start_thread () from /lib64/libpthread.so.0
#15 0x00007ffff755c7dd in clone () from /lib64/libc.so.6
Обновление 3
Я установил gdbinit из http://hg.python.org/cpython/file/default/Misc/gdbinit
и символы отладки из http://download.opensuse.org/debug/distribution/12.3/repo/oss/suse/x86_64/
(gdb) pystack
No symbol "_PyUnicode_AsString" in current context.
Что теперь?
Обновление 4
Мы установили новый RPM (python-2.7.5-3.1.x86_64). У нас меньше сегфоутов, но они все же случаются. Вот ссылка на репозиторий:
http://download.opensuse.org/repositories/devel:/languages:/python:/Factory/openSUSE_12.3/x86_64/
Обновление 5
Решил мою первоначальную проблему:
Это было http://bugs.python.org/issue1856 (выключение (выход) может зависнуть или segfault при запущенных потоках демона)
Связанный: Обнаружение закрытия интерпретатора в потоке демона