Error unable to load dependency hdf5 make sure hdf5 is installed properly

h5py installation issue #2321

The log ends in:

Whole log attached for good measure.
firelog.zip

The text was updated successfully, but these errors were encountered:

I see you built with —honour-petsc-dir . Our autodetection of the HDF5 library is not great. At present our solution to compiled libraries that are not «basic» is «PETSc compiled them for us, so we can find them in $PETSC_DIR/$PETSC_ARCH/. «. My guess here is that your PETSc was not built with —download-hdf5 (you might have used —with-hdf5 but I think our sniffing is not good enough).

If you have an HDF5 lying around and want to use it, run HDF5_DIR=/path/to/hdf5 firedrake-install (such that /path/to/hdf5/lib contains libhdf5 and /path/to/hdf5/include` contains the headers).

@JDBetteridge our errors could be much better here!

Longer term we want a more robust solution building on package managers, but a combination of funding and lack of anything being perfect has meant that to date we have this somewhat bespoke solution.

To get a successful install I think you need to set PETSC_CONFIGURE_OPTIONS=»—with-hdf5-dir=$HDF5_DIR» , even if you have built your own PETSc.

However, I agree this is confusing and the error message needs to be improved (amongst other things that need to be cleaned up in the install script!).

To get a successful install I think you need to set PETSC_CONFIGURE_OPTIONS=»—with-hdf5-dir=$HDF5_DIR» , even if you have built your own PETSc.

That’s not how I read the code. That function get_petsc_options is never called with —honour-petsc-dir provided.

But HDF5 is found to link h5py against here:

hdf5_dir = os . environ . get ( «HDF5_DIR» , «%s/%s» % ( petsc_dir , petsc_arch ))

You’re quite correct, the block of code has been moved since the last time I was running the install script against a prebuilt HDF5. Setting HDF5_DIR makes more sense anyway.

We should document this on the install page.

My guess here is that your PETSc was not built with —download-hdf5

Correct, and easily enough fixed.

If I may continue in this ticket:
fire.zip

On MacOS, we probably assume that homebrew is providing gfortran (and so it’s probably gfortran 10 or 11) which barfs building when building old fortran libraries (like the mpif90 module bindings) that have argument mismatches. So we send in -fallow-argument-mismatch in the libsupermesh build. But your fortran compiler doesn’t like that.

The relevant part of the CMakeLists.txt in libsupermesh does:

So it’s meant to only add the flag for gfortran versions that have it. But I have to say that cmake is a closed book to me.

What does gfortran —version say along with

/Users/eijkhout/Installation/petsc/petsc-3.16.1/macx-clang-hdf5-debug/bin/mpif90 -show
/Users/eijkhout/Installation/petsc/petsc-3.16.1/macx-clang-hdf5-debug/bin/mpif90 —version

I’ll spare you having to unpack the log file:

Is my gfortran older or newer than the one from homebrew?

Pretty sure 8.5.0 is less than 10. So what I don’t understand is why cmake is determining the opposite.

My cmake-fu is pretty weak, so I actually don’t even know how to printf debug this.

And I get one line further every day.

Actually, I’m now on a Linux box.

That’s because PETSc only creates an mpicc in that location if it uses its own downloaded mpich. If you use an external MPI it doesn’t. But I see that you have the —mpicc and such flags for that.

Where are you creating this build directory? How can I make sure that nothing gets built outside my account? So I’d prefer it if you didnt’ write in /tmp even.

Where are you creating this build directory? How can I make sure that nothing gets built outside my account? So I’d prefer it if you didnt’ write in /tmp even.

This appears to be a recent change to how pip/setuptools installs packages (previously it would copy them into a temporary directory), now it builds «in-place». But of course you don’t have permissions to build in-place in the PETSc source tree (because that’s a central package). This is #2314 which I think @JDBetteridge was working on.

To additionally avoid never writing to /tmp, I think you’re going to have to set TMPDIR to a scratch directory in $HOME (make sure it exists, otherwise python falls back to /tmp).

Thanks for your patience, we’re in a phase where a bunch of package updates that we have been relying on providing particular behaviour have changed, and we haven’t had time to adapt yet.

Thanks. I think you need a fix from us/setuptools. Previously, if you said pip install /local/directory , setuptools would copy to a temporary location and build there. They changed that behaviour recently to no longer copy.

This naturally assumes that the directory you are trying to install from is writeable by you, which is not the case here.

What we need to do is do that copy step manually to then be able to install. This needs someone to write some code.

An alternative is to switch up the pip “build backend” that petsc4py uses to one that supports the copying but I don’t know how feasible that is. Python packaging is in rather a state of flux at the moment and I don’t think best practices have been pinned down again yet.

A workaround for now might be firedrake-install —pip-install ‘pip which puts a version of pip in the venv that is old enough. But this is an untested idea.

Hm. I definitely do not use pip explicitly.

Anyway, is that pip literal sytax? (Minus the smart quotes in your suggestion.) The installer doesn’t take it.

sounds like the whole —pip-install option is unknown.

Oh crap. Your suggested option had not only smart quotes but also some unicode type of dash. Might be a good idea to turn that stuff off.

Oh crap. Your suggested option had not only smart quotes but also some unicode type of dash. Might be a good idea to turn that stuff off.

Sorry, phone. Thought I had, but clearly not.

Anyway, that syntax is apparently accepted, but it doesn’t solve the problem.

Ok, I built a petsc in my own account that includes hdf5 and now the installation actually works. But I shouldn’t have to do that. I’m on a cluster where I should use as much as possible the installed system software.

But even with the activate script I still get

which is not the hdf5 that’s in petsc:

I think this is the hdf5 that’s built into my python.

But I shouldn’t have to do that. I’m on a cluster where I should use as much as possible the installed system software.

FWIW, we consider PETSc to be a moving target dependency (rather than something you should rely on the cluster to provide). Not least because we’re often not compatible with a release version. However, I agree that it should be easier to build PETSc using system-installed versions of dependent packages.

But even with the activate script I still get

which is not the hdf5 that’s in petsc:

I think this is the hdf5 that’s built into my python.

What is h5py.*.so linked against (it will live somewhere like /path/to/firedrake/lib/site-packages/python-3.7/h5py/ ) ? Did you still have HDF5_DIR defined in your environment when running firedrake-install ?

I don’t know why your python is built with hdf5 support (I thought python itself didn’t care about it).

Источник

Can’t install H5PY — HDF5.dll dependency not found

I’m trying to install a library which relies on h5py. However, when I try to install it I get this error:

Collecting h5py Using cached h5py-3.1.0.tar.gz (371 kB) Installing build dependencies . done Getting requirements to build wheel . done Installing backend dependencies . done Preparing wheel metadata . done Collecting markdown>=2.6.8 Using cached Markdown-3.3.3-py3-none-any.whl (96 kB) Collecting werkzeug>=0.11.10 Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB) Requirement already satisfied, skipping upgrade: setuptools in c:usersuserdesktopprojectspythonmlagentsvenvlibsite-packages (from protobuf>=3.6.1- tensorflow==1.12.0) (50.3.2) Using legacy ‘setup.py install’ for termcolor, since package ‘wheel’ is not installed. Building wheels for collected packages: h5py Building wheel for h5py (PEP 517) . error ERROR: Command errored out with exit status 1: command: ‘c:usersuserdesktopprojectspythonmlagentsvenvscriptspython.exe’ ‘c:usersuserdesktopprojectspythonmlagentsvenvlibsite-packages pip_vendorpep517_in_process.py’ build_wheel ‘C:UsersUSERAppDataLocalTemptmpf62jt8ch’ cwd: C:UsersUSERAppDataLocalTemppip-install-7diegpjlh5py Complete output (70 lines): running bdist_wheel running build running build_py creating build creating buildlib.win32-3.8 creating buildlib.win32-3.8h5py copying h5pyh5py_warnings.py -> buildlib.win32-3.8h5py copying h5pyipy_completer.py -> buildlib.win32-3.8h5py copying h5pyversion.py -> buildlib.win32-3.8h5py copying h5py_init_.py -> buildlib.win32-3.8h5py creating buildlib.win32-3.8h5py_hl copying h5py_hlattrs.py -> buildlib.win32-3.8h5py_hl copying h5py_hlbase.py -> buildlib.win32-3.8h5py_hl copying h5py_hlcompat.py -> buildlib.win32-3.8h5py_hl copying h5py_hldataset.py -> buildlib.win32-3.8h5py_hl copying h5py_hldatatype.py -> buildlib.win32-3.8h5py_hl copying h5py_hldims.py -> buildlib.win32-3.8h5py_hl copying h5py_hlfiles.py -> buildlib.win32-3.8h5py_hl copying h5py_hlfilters.py -> buildlib.win32-3.8h5py_hl copying h5py_hlgroup.py -> buildlib.win32-3.8h5py_hl copying h5py_hlselections.py -> buildlib.win32-3.8h5py_hl copying h5py_hlselections2.py -> buildlib.win32-3.8h5py_hl copying h5py_hlvds.py -> buildlib.win32-3.8h5py_hl copying h5py_hl_init_.py -> buildlib.win32-3.8h5py_hl creating buildlib.win32-3.8h5pytests copying h5pytestscommon.py -> buildlib.win32-3.8h5pytests copying h5pytestsconftest.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_attribute_create.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_attrs.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_attrs_data.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_base.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_big_endian_file.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_completions.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dataset.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dataset_getitem.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dataset_swmr.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_datatype.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dimension_scales.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dims_dimensionproxy.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_dtype.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_errors.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_file.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_file2.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_file_image.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_filters.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_group.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5d_direct_chunk.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5f.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5p.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5pl.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_h5t.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_objects.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_selections.py -> buildlib.win32-3.8h5pytests copying h5pyteststest_slicing.py -> buildlib.win32-3.8h5pytests copying h5pytests_init_.py -> buildlib.win32-3.8h5pytests creating buildlib.win32-3.8h5pytestsdata_files copying h5pytestsdata_files_init_.py -> buildlib.win32-3.8h5pytestsdata_files creating buildlib.win32-3.8h5pyteststest_vds copying h5pyteststest_vdstest_highlevel_vds.py -> buildlib.win32-3.8h5pyteststest_vds copying h5pyteststest_vdstest_lowlevel_vds.py -> buildlib.win32-3.8h5pyteststest_vds copying h5pyteststest_vdstest_virtual_source.py -> buildlib.win32-3.8h5pyteststest_vds copying h5pyteststest_vds_init_.py -> buildlib.win32-3.8h5pyteststest_vds copying h5pytestsdata_filesvlen_string_dset.h5 -> buildlib.win32-3.8h5pytestsdata_files copying h5pytestsdata_filesvlen_string_dset_utc.h5 -> buildlib.win32-3.8h5pytestsdata_files copying h5pytestsdata_filesvlen_string_s390x.h5 -> buildlib.win32-3.8h5pytestsdata_files running build_ext Loading library to get version: hdf5.dll error: Unable to load dependency HDF5, make sure HDF5 is installed properly error: Could not find module ‘hdf5.dll’. Try using the full path with constructor syntax.

ERROR: Failed building wheel for h5py Failed to build h5py ERROR: Could not build wheels for h5py which use PEP 517 and cannot be installed directly

I realize from that that i»m missing HDF5 however I could not find a proper way to install it. And even if I do, I have no idea where h5py wants its dll to be in order to be read.

Edit I am using Python 3.8.0 And if someone wonders what library I tried to install is tensorflow.

is not working for me so I used the direct wheel file for version 1.14.0

Источник

[Question] version validate should be more strict? #1588

To assist reproducing bugs, please include the following:

when we run python setup.py configure —hdf5-version=X.Y.Z , we should be a little more strict with the version string, thus to say, the version number should be non-negtive:

Lines 42 to 49 in 2caa3d8

I think maybe we can change it like this:

@takluyver what do you think?

The text was updated successfully, but these errors were encountered:

I guess so. Is it a problem you’ve run into?

I’m actually not sure if it’s ever useful to pass in the HDF5 version manually. The build process tries to get the version number from the library using ctypes:

Lines 289 to 296 in 2caa3d8

If it can do that, I don’t think there’s any good reason to supply the version manually. And if it can’t find the library, the chances are that the build will fail anyway.

Thank you for your reminding. I didn’t realize that there is auto-detect verison function.
but then, why not just remove that validate_version funtion away if this kind of configuration is not set manually.

That’s kind of what I’m thinking, but I’m not 100% sure — maybe there’s some scenario I haven’t thought of where it is useful to manually override the auto-detected HDF5 version, or where it can find HDF5 to build but can’t get the version number for some reason.

I think Windows (for whatever reason) has tended to have issues with version detection?

That could be. I guess that most of us maintaining it are using Linux/Mac, so anything Windows specific may not get much attention.

I also wonder: is it possible to get the HDF5 version number from the header files while compiling? I’m not well up on C stuff, but maybe it’s possible to drop the ctypes version detection thing that way.

Источник

Hdf5.dll

Ошибки Hdf5.dll

Обнаруженные проблемы hdf5.dll с UE_4.20 включают:

• «Отсутствует файл Hdf5.dll.»
• «Отсутствует hdf5.dll. «
• «Hdf5.dll нарушение прав доступа.»
• «Не удается зарегистрировать hdf5.dll. «
• «Файл C:WindowsSystem32\hdf5.dll не найден.»
• «Не удалось запустить UE_4.20. Отсутствует необходимый компонент: hdf5.dll. Пожалуйста, установите UE_4.20 заново.»
• «Не удалось запустить UE_4.20, так как hdf5.dll не найден. Повторная установка UE_4.20 может исправить это. «

Проблемы UE_4.20 hdf5.dll возникают при установке, во время работы программного обеспечения, связанного с hdf5.dll, во время завершения работы или запуска или реже во время обновления операционной системы. При появлении ошибки hdf5.dll запишите вхождения для устранения неполадок UE_4.20 и помогите Epic Games, Inc. найти причину.

Создатели Hdf5.dll Трудности

Чаще всего поврежденный (или отсутствующий) hdf5.dll вызывает проблему. Обычно проблемы UE_4.20 возникают из-за того, что hdf5.dll является файлом из внешнего источника.

Повреждение hdf5.dll происходит во время неожиданного завершения работы, вирусов или других проблем, связанных с UE_4.20s. Когда файл hdf5.dll поврежден, он не может быть загружен должным образом и представит сообщение об ошибке.

В других случаях проблемы реестра с hdf5.dll могут быть источником проблемы UE_4.20. Сломанные ссылки на DLL-файлы могут помешать правильной регистрации файла DLL, давая вам ошибку hdf5.dll Оставшиеся разделы реестра UE_4.20 или hdf5.dll, перемещенные или отсутствующие hdf5.dll, плохие установки или удаления, могут нарушить ссылки на пути к файлам hdf5.dll.

В частности, проблемы hdf5.dll, созданные:

• Поврежденная или недопустимая запись реестра hdf5.dll.
• Файл hdf5.dll поврежден от заражения вредоносными программами.
• Аппаратная неисправность Epic Games, Inc. (например, принтер) вызвала повреждение hdf5.dll.
• Другая установка приложения перезаписала правильную версию hdf5.dll.
• hdf5.dll злонамеренно или ошибочно удален другой программой (кроме UE_4.20).
• hdf5.dll злонамеренно (или ошибочно) удален другой мошенникой или действительной программой.

(venv) olivierskonieczny@MacBook-Pro-olivier AI % python3 -m pip install tensorflow-macos

Collecting tensorflow-macos
  Using cached tensorflow_macos-2.7.0-cp38-cp38-macosx_11_0_arm64.whl (179.0 MB)
....
....
....
Building wheels for collected packages: h5py
  Building wheel for h5py (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for h5py (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      running bdist_wheel
      running build
      running build_py
      running build_ext
      Loading library to get build settings and version: libhdf5.dylib
      error: Unable to load dependency HDF5, make sure HDF5 is installed properly
      error: dlopen(libhdf5.dylib, 0x0006): tried: 'libhdf5.dylib' (no such file), '/usr/lib/libhdf5.dylib' (no such file), '/private/var/folders/rh/4d7_g4zj06l16r4kl40s7rcm0000gn/T/pip-install-bukptyvu/h5py_993a67c380d14b6788f3e17bb8badb62/libhdf5.dylib' (no such file), '/usr/lib/libhdf5.dylib' (no such file)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for h5py
Failed to build h5py
ERROR: Could not build wheels for h5py, which is required to install pyproject.toml-based projects

% pip --version
pip 22.0.2 from /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8)

2 ответа

Лучший ответ

xcode-select --install

chmod +x Miniforge3-MacOSX-arm64.sh

./Miniforge3-MacOSX-arm64.sh

conda config --set auto_activate_base false

conda create --name conda_tf python=3.8

conda activate conda_tf

conda install -c apple tensorflow-deps

 pip install tensorflow-macos

conda deactivate

conda activate conda_tf

# Remove the root prefix
rm -rf $(conda info --base)

# Configuration file, common between conda installations
rm ~/.condarc
# Environment locations and system information
rm -rf ~/.conda