Struct error unpack requires a buffer of 16 bytes

I try to compile a python file into a .exe file, but when i execute the process, an error raises: unpack requires a buffer of 16 bytes:
how can i solve this problem?

Running auto-py-to-exe v2.19.0
Building directory: C:UsersUsuarioAppDataLocalTemptmpy37hk3r4
Provided command: pyinstaller --noconfirm --onefile --console --icon "C:/Users/Usuario/Desktop/Programacion/depositphotos_2076034-stock-photo-question-mark-and-thinker.ico"  "C:/Users/Usuario/Desktop/Programacion/como_sera_tu_vida.py"
Recursion Limit is set to 5000
Executing: pyinstaller --noconfirm --onefile --console --icon C:/Users/Usuario/Desktop/Programacion/depositphotos_2076034-stock-photo-question-mark-and-thinker.ico C:/Users/Usuario/Desktop/Programacion/como_sera_tu_vida.py --distpath C:UsersUsuarioAppDataLocalTemptmpy37hk3r4application --workpath C:UsersUsuarioAppDataLocalTemptmpy37hk3r4build --specpath C:UsersUsuarioAppDataLocalTemptmpy37hk3r4

55099 INFO: PyInstaller: 5.0.1
55099 INFO: Python: 3.10.1
55111 INFO: Platform: Windows-10-10.0.19044-SP0
55129 INFO: wrote C:UsersUsuarioAppDataLocalTemptmpy37hk3r4como_sera_tu_vida.spec
55144 INFO: UPX is not available.
55160 INFO: Extending PYTHONPATH with paths
['C:\Users\Usuario\Desktop\Programacion']
55607 INFO: checking Analysis
55608 INFO: Building Analysis because Analysis-01.toc is non existent
55609 INFO: Reusing cached module dependency graph...
55661 INFO: Caching module graph hooks...
55741 INFO: running Analysis Analysis-01.toc
55743 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by C:UsersUsuarioAppDataLocalProgramsPythonPython310python.exe
55792 INFO: Analyzing C:UsersUsuarioDesktopProgramacioncomo_sera_tu_vida.py
55797 INFO: Processing module hooks...
55798 INFO: Loading module hook 'hook-difflib.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55799 INFO: Loading module hook 'hook-distutils.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55811 INFO: Loading module hook 'hook-distutils.util.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55827 INFO: Loading module hook 'hook-encodings.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55911 INFO: Loading module hook 'hook-heapq.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55923 INFO: Loading module hook 'hook-lib2to3.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55975 INFO: Loading module hook 'hook-multiprocessing.util.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
55987 INFO: Loading module hook 'hook-pickle.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
56003 INFO: Loading module hook 'hook-sysconfig.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
56018 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
56033 INFO: Loading module hook 'hook-xml.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
56093 INFO: Loading module hook 'hook-_tkinter.py' from 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks'...
56242 INFO: checking Tree
56243 INFO: Building Tree because Tree-03.toc is non existent
56255 INFO: Building Tree Tree-03.toc
56346 INFO: checking Tree
56348 INFO: Building Tree because Tree-04.toc is non existent
56365 INFO: Building Tree Tree-04.toc
56469 INFO: checking Tree
56476 INFO: Building Tree because Tree-05.toc is non existent
56492 INFO: Building Tree Tree-05.toc
56522 INFO: Looking for ctypes DLLs
56547 INFO: Analyzing run-time hooks ...
56555 INFO: Including run-time hook 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_subprocess.py'
56571 INFO: Including run-time hook 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_inspect.py'
56574 INFO: Including run-time hook 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_pkgutil.py'
56587 INFO: Including run-time hook 'C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_multiprocessing.py'
56605 INFO: Looking for dynamic libraries
57191 INFO: Looking for eggs
57206 INFO: Using Python library C:UsersUsuarioAppDataLocalProgramsPythonPython310python310.dll
57207 INFO: Found binding redirects: 
[]
57228 INFO: Warnings written to C:UsersUsuarioAppDataLocalTemptmpy37hk3r4buildcomo_sera_tu_vidawarn-como_sera_tu_vida.txt
57263 INFO: Graph cross-reference written to C:UsersUsuarioAppDataLocalTemptmpy37hk3r4buildcomo_sera_tu_vidaxref-como_sera_tu_vida.html
57289 INFO: checking PYZ
57301 INFO: Building PYZ because PYZ-01.toc is non existent
57316 INFO: Building PYZ (ZlibArchive) C:UsersUsuarioAppDataLocalTemptmpy37hk3r4buildcomo_sera_tu_vidaPYZ-01.pyz
57639 INFO: Building PYZ (ZlibArchive) C:UsersUsuarioAppDataLocalTemptmpy37hk3r4buildcomo_sera_tu_vidaPYZ-01.pyz completed successfully.
57673 INFO: checking PKG
57678 INFO: Building PKG because PKG-01.toc is non existent
57694 INFO: Building PKG (CArchive) como_sera_tu_vida.pkg
58875 INFO: Building PKG (CArchive) como_sera_tu_vida.pkg completed successfully.
58882 INFO: Bootloader C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbootloaderWindows-64bitrun.exe
58898 INFO: checking EXE
58913 INFO: Building EXE because EXE-01.toc is non existent
58929 INFO: Building EXE from EXE-01.toc
58945 INFO: Copying bootloader EXE to C:UsersUsuarioAppDataLocalTemptmpy37hk3r4applicationcomo_sera_tu_vida.exe.notanexecutable
59112 INFO: Copying icon to EXE
59121 INFO: Copying icons from ['C:\Users\Usuario\Desktop\Programacion\depositphotos_2076034-stock-photo-question-mark-and-thinker.ico']
An error occurred while packaging
Traceback (most recent call last):
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesauto_py_to_exepackaging.py", line 131, in package
    run_pyinstaller()
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstaller__main__.py", line 178, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstaller__main__.py", line 59, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbuildingbuild_main.py", line 845, in main
    build(specfile, distpath, workpath, clean_build)
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbuildingbuild_main.py", line 767, in build
    exec(code, spec_namespace)
  File "C:UsersUsuarioAppDataLocalTemptmpy37hk3r4como_sera_tu_vida.spec", line 24, in <module>
    exe = EXE(
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbuildingapi.py", line 524, in __init__
    self.__postinit__()
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbuildingdatastruct.py", line 173, in __postinit__
    self.assemble()
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerbuildingapi.py", line 629, in assemble
    icon.CopyIcons(build_name, self.icon)
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerutilswin32icon.py", line 215, in CopyIcons
    return CopyIcons_FromIco(dstpath, [srcpath])
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerutilswin32icon.py", line 150, in CopyIcons_FromIco
    for i, f in enumerate(icons):
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerutilswin32icon.py", line 114, in __init__
    entry.fromfile(file)
  File "C:UsersUsuarioAppDataLocalProgramsPythonPython310libsite-packagesPyInstallerutilswin32icon.py", line 71, in fromfile
    self._fields_ = list(struct.unpack(self._format_, data))
struct.error: unpack requires a buffer of 16 bytes

Project output will not be moved to output folder
Complete.

Кто сказал, что нельзя делать низкоуровневые вещи на Python? Конечно, можно. Давайте научимся упаковывать данные из Python в байты и распаковывать их обратно.

Встроенный модуль struct как раз создан для этих целей. В низкоуровневом деле важны детали, а именно размер каждого элемента данных, их порядок в структуре, а также порядок байт для многобайтовых типов данных. Для определения этих деталей модуль struct вводит форматные строки (не путать с str.format, там другой формат).

Начнем с простого примера:

>>> import struct
>>> struct.pack("hhl", 1, 2, 3)
b'x01x00x02x00x00x00x00x00x03x00x00x00x00x00x00x00'

Здесь происходит вот что. Мы берем три числа: 1, 2, 3 и пакуем их в байты, таким образом, что первое и второе числа трактуются как тип short int (4 байта на моей машине), а последнее, как long int (8 байт на моей машине). Это типы не из Python, а из языка Си. Ознакомьтесь с типами языка Си, если хотите понимать, что они из себя представляют и какой размер в байтах имеют.

Обратная распаковка байт в кортеж значений по заданному формату:

>>> struct.unpack("hhl", b'x01x00x02x00x00x00x00x00x03x00x00x00x00x00x00x00')
(1, 2, 3)

Форматы запаковки и распаковки должны совпадать, иначе данные будут неправильно интерпретированы или испорчены, или же вообще возникнет ошибка из-за того, что размер данных не подходит под ожидаемый формат (struct.error):

>>> struct.unpack("hhl", b'x01x02x03')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: unpack requires a buffer of 16 bytes

Обратите внимание, что я выше писал, что размер элемента «h» – 4 байта именно на моей машине. Может статься так, что на машине с другим процессором, архитектурой или просто с другой версией ОС размер типа будет другой. Для 32 битных систем, это обычно будет 2 байта.

Но, что если данных передаются по сети или через носители информации между системами с разной или неизвестной заранее архитектурой? Конечно, у struct есть средства на такие случаи. Первый символ форматной строки обозначит порядок байт. Обратите внимание на таблицу:

Нативный – значит родной для конкретно вашей машины и системы. По умолчанию порядок байт и размер типов данных как раз нативный (символ @).

Стандартный размер – размер, который фиксирован стандартом и не зависит от текущей платформы. Например, char всегда 1 байт, а int – 4 байта. Если мы планируем распространять запакованные байты, мы должны гарантировать, что размер типов будет всегда стандартный. Для этого подходит любой из символов «=«, «<«, «>«, «!» в начале форматной строки.

Little-endian и big-endian

Little-endian и big-endian – это два основных порядка байт. Представим, что у нас есть короткое целое (short int), и оно занимает два (2) байта. Какой из байтов должен идти сначала, а какой в конце?

В big-endian порядок от старшего байта к младшему. В little-endian порядок от младшего байта к старшему. Как узнать на Python какой порядок байт в системе:

>>> import sys
>>> sys.byteorder
'little'

Давайте наглядно посмотрим как пакуются байты при разных порядках. Для числа 258 в форме short младший байт будет = 2, а старший = 1:

258 = 2*2⁰ + 1*2⁸

>>> struct.pack("<h", 258)  # little-endian
b'x02x01'
>>> struct.pack(">h", 258)  # big-endian
b'x01x02'

Как видите порядок байт противоположный для разных случаев.

В сетевых протоколах принято использовать big-endian (символ «!» – псевдоним к «>«), а на большинстве современных настольных систем используется little-endian.

Таблица типов данных

Теперь ознакомимся с таблицей типов данных, которая дает соответствие символу форматной строки (код преобразования) с Си-типом данных, Python-типом данных и стандартный размером. Еще раз: стандартный размер будет только, если задан первый символ как «<«, «>«, «!» или «=«. Для «@» или по умолчанию – размер данных определяется текущей системой (платформо-зависимо).

Коды «e«, «f«, «d» используют бинарный формат IEEE-754.

Код «x» это просто байт набивки. Он не попадает в распакованные данные, а нужен, чтобы выравнивать данные. «x» при запаковке забиваются пустыми байтами. Пример: «пусто-число-пусто-пусто-число-пусто»:

>>> struct.pack(">xBxxBx", 255, 128)
b'x00xffx00x00x80x00'

>>> struct.unpack('>xBxxBx', b'x00xffx00x00x80x00')
(255, 128)

О форматной строке

Если в форматной строке перед символом кода – число, то значит этот символ повторяется столько раз, сколько указывает число. Два кусочка кода аналогичны:

>>> struct.pack(">3h", 1, 2, 3)
b'x00x01x00x02x00x03'

>>> struct.pack(">hhh", 1, 2, 3)
b'x00x01x00x02x00x03'

Для строк (коды «s» и «p«) надо указывать число байт – длину строки, иначе будет считаться 1 байт:

>>> struct.pack("ss", b"abc", b"XYZW")  # не указал длину - потерял байты
b'aX'

>>> struct.pack("3s4s", b"abc", b"XYZW")
b'abcXYZW'

10s – одна 10-символьная строка, а 10c – 10 отдельных символов:

>>> struct.unpack('10c', b'abracadabr')
(b'a', b'b', b'r', b'a', b'c', b'a', b'd', b'a', b'b', b'r')

>>> struct.unpack('10s', b'abracadabr')
(b'abracadabr',)

Можно вставлять пробелы между отдельными элементами форматной строки (но нельзя отделать число от символа). Пробелы игнорируются при чтении строки и нужны для удобства чтения кода программистом:

>>> struct.pack('>6sh?', b'python', 65, True)
b'pythonx00Ax01'

>>> struct.pack('> 6s h ?', b'python', 65, True)  # тоже, но с пробелами
b'pythonx00Ax01'

>>> struct.unpack('> 6s h ?', b'pythonx00Ax01')
(b'python', 65, True)

Полезности

Можно вычислить размер данных из форматной строки без фактической запаковки или распаковки данных:

>>> struct.calcsize('> 6s h ?')
9

Удобно распаковывать байты прямо в именованные кортежи:

>>> from collections import namedtuple
>>> Student = namedtuple('Student', 'name serialnum school gradelevel')
>>> record = b'raymond   x32x12x08x01x08'
>>> Student._make(struct.unpack('<10sHHb', record))
Student(name=b'raymond   ', serialnum=4658, school=264, gradelevel=8)

Запаковка в буффер со смещением struct.pack_into(format, buffer, offset, v1, v2, ...):

>>> buffer = bytearray(40)
>>> struct.pack_into('h l', buffer, 10, 3432, 340840)
>>> buffer
bytearray(b'x00x00x00x00x00x00x00x00x00x00hrx00x00x00x00x00x00h3x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00')
>>>

Распаковка из буффера со смещением:

>>> x, y = struct.unpack_from('h l', buffer, 10)
>>> x, y
(3432, 340840)

Распаковка нескольких однотипных структур:

>>> chunks = struct.pack('hh', 10, 20) * 5  
>>> chunks  # 5 одинаковых штук
b'nx00x14x00nx00x14x00nx00x14x00nx00x14x00nx00x14x00'

>>> [(x, y) for x, y in struct.iter_unpack('hh', chunks)]
[(10, 20), (10, 20), (10, 20), (10, 20), (10, 20)]

Специально для канала @pyway. Подписывайтесь на мой канал в Телеграм @pyway 👈 

14 120

Introduction

A while ago I wrote an article on common issues when using auto-py-to-exe. In this post, I aim to replace that post and explain more in greater depth.

Please do not feel offended if I have referred you to this page, it’s just the case that many other people have asked the same question you are asking and you haven’t been able to find the answers I or others have provided.

A Small Disclaimer

I cannot guarantee this post or any advice I give will fix all your issues with auto-py-to-exe and PyInstaller. This tool allows you to select options and give data to be fed into PyInstaller, that means that a lot of the errors that occur will be raised by PyInstaller, meaning I can’t go and change how it works. Typically though, errors brought up in PyInstaller are due to incorrect configuration, thus this post aims to help you fix anything set up incorrectly.

If you have an issue that isn’t mentioned in this post, chuck it in the comments below and I will look into adding it if we can find a solution.

What is auto-py-to-exe?

auto-py-to-exe is a project I made in my free time to make packaging Python scripts to an executable/bundle easier. The interface uses Chromes app mode and a small Python server in the backend.

When the user presses «Convert .py to .exe», all the data in the interface is sent to PyInstaller and then the output from PyInstaller is sent back to the interface. auto-py-to-exe then cleans things up in the background when done (files/folders generated by PyInstaller) and allows you to find your project in the output folder in your current working directory.

Installation

To install auto-py-to-exe, make sure you have Python 3.4 or above installed. Ideally, you will also want Chrome installed but this is not a must; it will open in your default browser if Chrome is not found and the only thing different will be the interface size. Execute the following in the command prompt / terminal to install auto-py-to-exe:

python -m pip install auto-py-to-exe

Wait for this to finish and make sure that the last line contains «Successfully installed auto-py-to-exe» dash some version. You might get a message about your version of pip but you can ignore it. Now to run the project, execute the following in the terminal:

This will open auto-py-to-exe and the output will be saved in output/ in where your current working directory is.

Possible Gevent Issue

There could be a situation when installing auto-py-to-exe that you are told you require Microsoft Visual C++ Build Tools. This occurs because the file that was downloaded from PyPI to be installed needs to be built and these tools will build it.

Generally, the way to fix this would be to install the tools and run it again, but instead, you can get a pre-built version. To do this, go to gevents PyPI page and click «Download files» on the right to bring you to the files that are on PyPI for this project. You will want to find the file that matches your setup and is a .whl file, for example, I am using Python 3.7 on Windows so I would look for cp37 in the «Python version» column, Wheel in the «File type» column and then in the files that match these two criteria, match my OS — Windows. This means the file I will want is gevent-1.3.7-cp37-cp37m-win_amd64.whl (currently gevent is at 1.3.7).

The Layout of the Interface and What Things Do

At the top of the interface, you are asked for the script location. This is the script you call to run your project, so if your project is one script, put that here, if your project is more than one script, put the script that starts it here. The inputs outline will become blue if the file exists, otherwise it will be red.

Next, you need to choose between one-file and one-directory. These are relatively similar but when using one-file with extra files like images or data files you may need to modify your script to account for path changes. The difference between these methods and people ignoring the extra step required generally result in the bulk of complaints I get.

Under that is a selection of whether you want a console window to appear or not. Simply if you are developing a GUI application or something that doesn’t need the console to appear, use «Window Based». If you do want the console to appear, keep «Console Based Selected» but keep in mind that the script needs to block somewhere otherwise execution will end and the terminal will disappear immediately; a workaround for this is to put input() at the end of the script to hold execution until input is provided.

After this, you can select an icon for your executable. Make sure that the files are .ico and don’t just rename a file to be a .ico — that is not how file types work.

Next, there is a section to add files to your executable that are not Python files. Since PyInstaller doesn’t find extra files like images you need to add them manually. When adding an entry, the file path/folder path needs to the in the box on the left and the destination on the right; the destination is the folder in the executable.

• If you put a folder on the left, all files will be placed in the folder (and sub-directories keeping structure) specified on the right.
• If you put a file on the left, this file will be put in the folder specified with the filename the same as the original.

If you’re using one-file mode, you’ll be provided a link to stackoverflow which tells you how to implement a small wrapper to find the files you added. I explain why you have to do this towards the bottom of this post including an example; if you do not follow this step errors will occur.

Under this is the advanced tab which holds all PyInstallers extra flags as well as where to save the project and setting the maximum recursion depth. Here are a few flags/options that can help with things:

• —name: The name of the output folder/executable
• —hidden-import: If the executable says a module is missing, make sure you have it installed and add it here; you can separate multiple modules by a comma.
• —debug: Set this to all to help make debugging a lot easier

The last tab is for auto-py-to-exe specific settings:

• Output Directory: The directory that the output of the application is put into.
• Increase Recursion Limit: If a RecursionError occurs, make sure this is enabled to set the depth to 5000.
• Manual Argument Input: A simple input to manually add to the current command.

The «Current Command» section tells you what would be called if you wanted to execute this in a terminal and the convert button is under this. After the conversion, you can clear the output or open the output folder where the executable/package was saved.

General Steps

The process of packaging a Python project to an executable is simple in most cases.

1. Start auto-py-to-exe by executing auto-py-to-exe in the terminal
2. Search for your script
3. Decide on whether you want the output to be a single executable or a directory (a directory has many benefits discussed later)
4. Decide if you want a console window or not
5. Add an icon if you desire
6. Add any extra files your project requires that aren’t Python files (e.g. images, csv, databases, …)
7. Click the big blue button at the bottom to package the project
8. Open the location of the executable/package and run it

If you have any issues with running your script, it may be due to incorrect configuration. This means you will now have to go through a debugging process to find what is occurring.

Debugging

To debug, make sure you have set up the fields in auto-py-to-exe like you had previously — all your extra files and other settings. Now go to the «Advanced» section and under the title «How to generate» put all in the box beside —debug. This will print out messages to the console to help you debug.

When debugging, make sure you select the «Console Based» button and using «One Directory» will help remove basic problems; you can switch back to «One File» when there are no more bugs and then fix the ones associated with «One File» mode. When all bugs are removed, you can go back to «Window Based» if you wish.

Re-package your project and open up the command prompt / terminal and cd to the directory where the executable / package was output to. If you are using one-directory cd into that directory. Now execute ./my_project.exe (substitute the names). This will run the executable file and any errors you were missing before will be output to the console and preserved.

Look at what these errors are saying and fix them. Now repackage like you just did and keep fixing bugs and repackaging until there are no more left and your project is working properly. After this, you can then move to one-file if you were using one-directory and want one-file, and repeat the process by still staring it with cmd and debugging on.

When everything is done and there are no more bugs in the modes you want, you can enable «Window Based» if you want and remove all from —debug.

Not every project can be converted to one-file. This is due to files being used by the application being lost every time the application starts/stops. To understand this more go down to «Demonstration of Files References After Packaging».

How to Fix Specific Issues

These are some of the issues I have come across or others have asked me about and ways to fix them.

Fatal Error: failed to execute

This means something has gone wrong as it’s giving you a visual warning about it; this is not an error, it’s a warning; the real error has been printed to stdout/stderr. If you open the executable using the terminal or something else that will preserve the console output, you will most likely see a Python error telling you what went wrong. Fixing this and repackaging is the solution to this issue.

PermissionError: [Errno 13] Permission denied: …

This occurs because you are trying to modify files in a directory you do not have access to. A way to fix this is to run the script with admin privileges by opening cmd as admin and then running auto-py-to-exe one you have cd’ed to the directory you want the output to be in.

One reason this could occur is that you have opened cmd and am in System32. Make sure you do not accidentally modify files in this directory so make sure you are in a directory where you want to write files to when running auto-py-to-exe.

FileNotFoundError: [Errno 2] No such file or directory

This error is saying that a file you referenced does not exist. This can mean one of two things:

• You have not included the file (you may have not realised you needed to as it is part of a library)
• You have included the file but are not referencing it correctly

For situations where you have not included the file, you need to use the «Additional Files» section in the UI to add the required file or folder. You can identify the required file by looking at the error and then also work out the destination based on where it was being looked for.

When adding a file or folder in auto-py-to-exe, there are two boxes to fill out: the source location and the destination.

• The source location is where the file currently is on your disk. Using «Add Files» or «Add Folder» will auto-populate this field for you.
• The destination is where the file/folder will be put inside the final executable/folder. For example, if you have a file called icon.png and your code expects it to be in the folder «assets/images/» relative to your script, then you should put in «assets/images/».

For files that have been included but this error is still showing, package to one-directory instead of one-file to make sure things are working. Check out «The one-file Resource Wrapper» for a function that will help identify the root your application is currently running in.

RecursionError

This error is now prevented by the application by default by setting the recursion limit to 5000. If you want to disable this, there is an option in the Settings tab to.

AttributeError: module ‘enum’ has no attribute ‘IntFlag’

Try executing python -m pip uninstall enum34 to stop enum conflicts. This is also a fix for the similar issue «Fatal Python error: Py_Initialize: unable to load the file system codec».

ModuleNotFoundError: No module named x / ImportError: No module named x

This means a particular module (‘x’ in this case) was not added to the package. I have seen this occur with packages in the pandas library and win32api; as long as you can identify the package (e.g. ‘x’), then it is very easy to fix.

To fix this in the UI, open the advanced tab and find the --hidden-import input. Simply paste the module name into this input and then repackage. If the original error is still appearing, you have done this incorrectly.

For example, if you are missing pandas._libs.tslib, add ‘pandas._libs.tslib’ into the input by —hidden-import. Additionally, you can add more than one module, for example, pandas._libs.tslib, win32api. (See the question mark by the input for more information).

Alternatively, you may have installed auto-py-to-exe in one Python environment (a single installation or venv) and installed your dependent package in a different Python environment. Take a look at «How to Manage Multiple Python Distributions» for help on how to identify if you’ve done this.

ERR_CONNECTION_REFUSED

You need to be connected to the internet. This error appears when you cannot reach the servers.

Command «python setup.py egg_info» failed with error code 1

Update setuptools using pip install --upgrade setuptools.

FileNotFoundError

Make sure the file you are referencing actually exists, this error says that the file you are referencing doesn’t exist. If you are using one-file mode, make sure you have followed the necessary steps.

struct.error: unpack requires a buffer of 16 bytes

This typically occurs when you have supplied --icon / -i with a file that is not a .ico file. Please note that simply changing the extension of a file to .ico does not make it a .ico file; you will need to use a converter of some sort to convert your file (e.g. png, jpg) into a .ico.

General Questions Constantly Asked

How do I Convert All My Python Files?

Add the entry point in the script location. PyInstaller searches for imports to get the rest of the Python files required to run, so as long as you use the import keyword, your files will be added.

For example, if you have a layout like:

📂root
 ┣ 📂my_package
 ┃ ┣ 📜__init__.py
 ┃ ┣ 📜__main__.py
 ┃ ┗ 📜utils.py
 ┗ 📜start.py

Then package start.py — PyInstaller will find the rest as long as you are using the import keyword.

If you don’t have something like start.py and use your application like a module (python -m [module_name]) then create a file like start.py which imports the package and runs it as you would expect. For example, this is what I do with auto-py-to-exe:

from my_package import __main__

__main__.__name__ = '__main__' # Make it look like `my_package.__main__` is the main script
__main__.run()  # Call your main function if required

The Terminal Just Opens and Closes But There Are No Errors

If you double click to run your Python script, what happens? Does it open and close also? That means this tool has done its job correctly and the script is finishing just like it should.

You most likely think the output should stay visible because you are always using IDLE or an IDE and that’s what those tools do. Add a statement like input() at the end of your script to block execution and wait for you to press enter before closing.

Alternatively, there may be an error occurring which means you need to follow the debugging steps above.

‘python’/’pip’/’auto-py-to-exe’ is not recognised as an internal or external command, operable program or batch file.

This occurs because the path that these executables are located in is not on your path; thus cmd doesn’t know where to look for them. You need to add these paths to the PATH environment variable so cmd knows where to look for the executable you are trying to execute.

I created a script that can help set up these paths automatically, which can be found in my «Fix: ‘python’ is not recognized as an internal or external command» post.

If you want a more manual approach to understand what is occurring, watch my video on how to setup Pythons PIP. This will show you how to add the Scripts folder, but I also recommend you add the root Python directory (same path without ‘Scripts’) so the python command will work. The paths you will need to add can be found using the commands below:

import os, sys

# If 'python' is not recognised
print (os.path.dirname(sys.executable))

# If 'pip'/'auto-py-to-exe'/... is not recognised
print (os.path.dirname(sys.executable) + "\Scripts")

If you are using something older than Windows 10, setting the PATH environment variable will be a bit different than what is demonstrated in the video. If it is just one line, add a «;» to the end and then paste the path after the «;«.

The exe Doesn’t Work on Another Computer

This may be an architecture issue. PyInstaller will build an executable using the architecture of the machine it was built with. This means if you are using a 32bit machine, it will create a 64bit executable. As with any other programs, you cannot run 64bit on 32bit but you can run 32bit on 64bit. Thus I recommend using 32bit python or compiling on a 32bit machine so it will work on both architectures.

Can I Create an Executable on One Operating System and Run it on Another?

PyInstaller’s documentation states:

PyInstaller is tested against Windows, MacOS X, and Linux. However, it is not a cross-compiler; to make a Windows app you run PyInstaller on Windows, and to make a Linux app you run it on Linux, etc.

PyInstaller is cross-platform, meaning the tool itself works on many operating systems, however it is not a cross-compiler, meaning the output from the tool cannot run across multiple operating systems.

There are ways to use things like «wine» on Linux to create an executable on Linux that can run on Windows, but this is out of the scope of PyInstaller.

What is the __pycache__ folder?

This is Python bytecode generated by Python. You can delete this if you wish.

My Antivirus Detected the exe as a Virus

This is your anti-virus vendors fault. Check out this. There is nothing I or PyInstaller can do to stop this. After seeing this pop up a lot more recently I stated in brentvollebregt/auto-py-to-exe#122 that you have a few options:

• Tell your antivirus that these files are fine
• Switch to a different and more accurate anti-virus
• Notify your antivirus vendor about the false positive

Typically files are quarantined during the build process which is done within a temporary directory by default.
This directory can be overridden using the --build-directory-override [FOLDER_PATH] argument.
From here, if you whitelist your custom build directory and the output directory, you could potentially stop your antivirus from quarantining the files.

Lots of Warnings Appear in the Output

These warnings can be ignored in most cases. I have not currently found a situation where these are an issue, after all, they are only warnings. These warnings typically match the format WARNING: lib not found: api-ms-win-crt-<specific dll> dependency of <file>.

A YouTube user comment on the video related to this project that if you add C:WindowsSystem32downlevel to your PATH variable, these DLLS that were previously not being found can now be found. This is because the files missing are commonly found in this folder. Adding this folder to your path to now successfully locate these files can also speed up packaging times.

The Output Executable is Huge

Sometimes pyinstaller tries to be smart and will add packages it sees in your environment even if you are not using them in the project being packaged. This can lead to output executables being tens to hundreds of megabytes in size.

To get around this, the easiest way would be to:

1. Create a new/clean virtual environment
2. Install auto-py-to-exe into it
3. Install the required modules for your project
4. Use the auto-py-to-exe in this virtual environment to package your script

Doing this will mean pyinstaller doesn’t see the packages you don’t need bundled which can lead to smaller packages.

«VCRUNTIME140.dll» is either not designed to run on Windows or it contains an error

Try selecting the --noupx button in the advanced tab.

LoadLibraryEx / The file cannot be accessed by the system

I have seen quite a few people get tracebacks containing the following error messages:

• win32ctypes.pywin32.pywintypes.error: (1920, 'LoadLibraryEx', 'The file cannot be accessed by the system'
• OSError: [WinError 1920] The file cannot be accessed by the system

Looking at the traceback for these situations, it is very common that I see a path like C:Users<USER>AppDataLocalPackagesPythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0LocalCache... which tells me they’re using Python from the Windows Store. As stated in pyinstaller/pyinstaller#4941, we have been told by a maintainer that:

We don’t support using python from the windows store when not using a virtual environment. Please use a virtual environment; that should fix this.

So to fix this issue, download a distribution of Python from python.org or use a virtual environment. This solution is also noted in brentvollebregt/auto-py-to-exe#141.

Supporting Stack Overflow questions here, here and here.

If you install Python from python.org to fix this issue, you will need to install auto-py-to-exe in this new distribution. You will then also need to make sure you execute auto-py-to-exe from the newly installed distribution and not the old python from the windows store distribution. Executing where auto-py-to-exe can tell you where it will be execute from (the top result).

The exe Runs But Does Some Unexpected Things

A lot of people say their python script runs fine in IDLE, PyCharm, Conda or something similar but after packaging, the executable functions differently.

Examples of executables functioning differently include but are not limited to:

• A console window appearing
• Console windows appearing and closing

This difference is most likely because the script is now running in a different environment or started in a different way.

If you double-click your exe to run it, you can test how your script will run before packaging by doing the following:

1. Set the default application for .py files on your machine to the python.exe binary you are using to run your .py file
2. Double-click on your .py script to run it
3. Witness how your application runs

If you are packaging to a single .exe file (onefile), set pythonw.exe as the default application for .py files rather than python.exe.

If your script doesn’t run as expected when following this, it shows there is something wrong with the implementation of your script.

Additional Information and Explanations

Some things that are clear to people that use Python a lot are not always clear to new people. Here are some discussions about why and how things occur.

Using concurrent.futures

It was brought to my attention by William Lake in this GitHub issue that when using concurrent.futures for any multi-threading/processing, it is recommended to add the following import:

from multiprocessing import freeze_support

and this line to your main:

William had stated that before adding these lines, he kept getting errors regarding concurrent.futures even though it worked fine before packaging.

A related issue on bugs.python.org recommending the fix

The Difference Between One Directory and One File

One directory puts all your files in one folder. You can easily add and remove files like you normally would in a folder. When your script modifies a file in its folder, the file will still be modified when you run the script again.

One file mode is a bit different, instead of putting all the files in a folder, it puts them in something like a zip file which is contained in the end executable. When you run the executable, the files contained internally are unpacked to a new temporary directory. Due to one file unpacking on startup, it is a lot slower to start.

Also due to the files being unpacked to a new temporary directory on execution, the files that you modified added that were in the same directory as the executable will not be there on the next run because they are now in a different unknown folder. This means when creating files, you will want to use an absolute path that is not where the project files are (could use something like APPDATA on Windows).

Demonstration of Files References After Packaging

I made a small Python file that finds where it is located and then tries to print the contents of a file if it exists otherwise will create a new one.

import sys, os
if getattr(sys, 'frozen', False): # we are running in a bundle
    bundle_dir = sys._MEIPASS # This is where the files are unpacked to
else: # normal Python environment
    bundle_dir = os.path.dirname(os.path.abspath(__file__))

print ('Location : ' + bundle_dir) # Where the base file exists

file = bundle_dir + '\test.txt'
print ('File is at: ' + os.path.abspath(file)) # Absolute path of target file
if os.path.isfile(file):
    with open(file, "r") as f:
        print ('Contents:n' + f.read()) # Print contents of file if it exists
else:
    print ('Created a new file') # Create a file if it doesn't exist

with open(file, "a") as f:
    f.write('New Linen') # Add a new line to see if is there next time

input() # Block to keep terminal alive

Running this un-packaged (.py)

The first time it will create a file beside the script. Every time it is run, the new line will be added and this will be shown in the output. This means When you modify a file, it will stay modified.

Running this when packaged using one directory

The first time it will create a file in the directory (which will be beside the .exe). Every time it is run, the new line will be added and this will be shown in the output. This means When you modify a file, it will stay modified.

Running this when packaged using one file

Every time this is run, it will create a new file in the temporary directory. This occurs because the .exe unpacks to a new directory every time it is run, so instead of finding old files (which can disappear any time because they are in the temp folder) you may as well use an absolute reference to somewhere else.

The one-file Resource Wrapper

As described in the section above, non-python files that are bundled into an executable when using one-file mode need care when being referenced due to how they are un-packaged.

In the interface, when one-file mode is selected and the additional files section is expanded, a note can be found that references a StackOverflow question regarding «Bundling data files with PyInstaller (—onefile)». The first answer provides a nice snippet of code:

import sys, os
def resource_path(relative_path):
    """ Get the absolute path to the resource, works for dev and for PyInstaller """
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")

    return os.path.join(base_path, relative_path)

When referencing files using relative references, instead of using open('folder/my-file.jpg'), you will want to use open(resource_path('folder/my-file.jpg')). This appends the relative path you provided to the current / extracted location to make an absolute file reference which is safe to use.

The reason you need to use this extra bit of code is because a one-file exe will unpack all of it’s contents to a new folder in the operating systems temporary directory. This means the current working directory initially set in the application will not be where the files have been unpacked to unlike one-directory. This is why using relative references will work in one-directory but not in one-file — you need to adjust for the fact that the root of the project is now somewhere different.