Error torch has an invalid wheel dist info directory not found

I try’d to install the new hummingbird 0.2.0 release in a new venv and the installation failed because the hummingbird wheel specified a pytorch version that seems to have a broken wheel or the hummingbird wheel is missing the full pytorch installation command (which is: pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html). It’s also not possible to get this exact pytorch install command from the official pytorch website. So its not possible to install the requirered pytorch version by just looking at the pytorch website where you only can get the 1.7.1 or the 1.6.0 version of pytorch.

Right now you have to find the pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html command somewhere in the internet and install this specific version before you can install hummingbird-ml

Collecting hummingbird-ml
  Downloading hummingbird_ml-0.2.0-py2.py3-none-any.whl (151 kB)
Collecting numpy<=1.19.4,>=1.15
  Using cached numpy-1.19.4-cp37-cp37m-win_amd64.whl (12.9 MB)
Collecting onnxconverter-common<=1.7.0,>=1.6.0
  Using cached onnxconverter_common-1.7.0-py2.py3-none-any.whl (64 kB)
Collecting scikit-learn<=0.23.2,>=0.21.3
  Using cached scikit_learn-0.23.2-cp37-cp37m-win_amd64.whl (6.8 MB)
Collecting joblib>=0.11
  Downloading joblib-1.0.0-py3-none-any.whl (302 kB)
Collecting scipy>=0.19.1
  Downloading scipy-1.6.0-cp37-cp37m-win_amd64.whl (32.5 MB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-2.1.0-py3-none-any.whl (12 kB)
Collecting torch<=1.7.0,>=1.4.*
  Downloading torch-1.7.0-cp37-cp37m-win_amd64.whl (184.0 MB)

ERROR: torch has an invalid wheel, .dist-info directory not found

Interesting. Thanks for reporting this. Apparently something happened to torch over the break because on the 31st the pipeline was working. We will investigate on Monday.

I tried pip-installing our 0.2.0 release just now in a new Linux container (with torch-1.7.0-cp37-cp37m-manylinux1_x86_64.whl) and had no issues, so maybe this is specific to Windows. Can you please confirm @speedfreakw …was this on windows (I see «win_amd64» so I’m assuming yes)?

I noticed there is a stackoverflow post complaining about this exact same torch version you mentioned above (torch-1.7.0-cp37-cp37m-win_amd64.whl)….something about a version error related to CUDA. Does any of that help resolve the issue?

The pipeline passed again just now on Windows with Python 3.7 with:

Run pip install torch==1.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.7.0+cpu
  Downloading https://download.pytorch.org/whl/cpu/torch-1.7.0%2Bcpu-cp37-cp37m-win_amd64.whl (184.2 MB)

but this is the +cpu version of torch, as the pipeline does not have GPU’s unfortunately!!

What version of CUDA do you have?

Yes Iam using Windows 10 64-bit an
I have to use Cuda 11.0+ as my RTX 3080 dosen’t support CUDA 10. But the installation of Pytorch with Cuda 10 should pass on my PC anyway.

I tryed some different install commands to see whats working and whats not:
pip install torch===1.7.0 -f https://download.pytorch.org/whl/torch_stable.html — works
pip install torch===1.7.0+cu110 -f https://download.pytorch.org/whl/torch_stable.html -works
pip install torch==1.7.0 -f https://download.pytorch.org/whl/torch_stable.html -works
pip install torch==1.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html -works
pip install torch==1.7.0+cu110 -f https://download.pytorch.org/whl/torch_stable.html -works
pip install torch==1.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html -works
pip install torch==1.7.0+cu102 -f https://download.pytorch.org/whl/torch_stable.html -fails
pip install humingbird-ml -fails

So it seems to be an issue with the CUDA Version 10.2 which funnily enough isnt even supported by my GPU and I dont know why hummingbird is picking that exact package

I think is more a problem with pypi then with Hummingbird. By default for windows machines probably torch picks the cu102 version which does not work in your case. The correct way of installing Hummingbird is to first get the desired pytorch version, and then install Hummingbird. If you go with Hummingbird first, you are basically doing pip install torch which may not work in some cases (although my hope was that the default version is CPU, not cu102, but what can you do).

Thanks for the info. We didn’t change anything related to the torch installer, so I am not sure if it is a change on their (pytorch) end. In the ticket you mention that this is a new problem with pip install of hb 0.2.0…do older versions of hb (ex 0.1.0) still work on your machine without issue? I think your problem is related to the pypi/torch installer.

A workaround might be to just install torch first (like we specify in the readme for other users who are having version issues), and then to install hummingbird, which hopefully avoids this issue. (But of course we want to make installing and using hummingbird as easy as possible, so this information you are providing is very useful!)

Installing Version 0.1.0 of Hummingbird is also not working.
Yes it seems that the pipy Version 1.7.0 of Pytorch is broken on Windows which is set as fixed reqirement in Hummingbird. The pipy Version 1.7.1 or 1.6.0 of Pytorch works without a Problem.

First trying, I had preinstalled the Version 1.7.1 of pytorch and then tryed to install hummingbird, but the hummingbird pip package automatically uninstalls Versions of Pytorch newer then Version 1.7.0

This is because we haven’t update Hummingbird to 1.7.1 yet, and v0.2 at most supports 1.7. We will add support for 1.7.1 soon, but it will take a bit until we roll a new version of hummingbird with 1.7.1 support to pypi. @ksaur, perhaps we can have a v0.2.1 with added support for pytorch 1.7.1.

Confirmed that pip install pulls torch-1.7.1 with the hummingbird-ml==0.2.1, so hopefully this avoids the downgrade on hb install!

It’s not ideal that we have to keep tracking new torch versions, but when we allowed torch<1.4.0 we got all kinds of unexpected breaking changes/perf regressions, so the incremental approach seems less disruptive at least for now.

Isnt it also possible to specify only a minimium version in the requirements instead of an specific version? Newer Versions should not break old conventions if properly coded. Both variants have theire own advantages and disadvantages. Manualy updating is a lot of work but ensures that it works all the time. A minimum version is much less work and works a long time until it breaks, but it will break eventually.

Thank you, for now its working this way

pkg1>=1.0  # min
pkg2==2.0  # specific
pkg3>=1.0,<=2.0  # range

Thanks for the suggestion @speedfreakw! What you are suggesting is the solution we were having before, and we found that pretty much any new pytorch release break something. This was annoying because then the version on pypi remained broken until the next release. While our current solution is suboptimal, at least it is not breaking every time a new pytorch release arrive and we smoothly add support for newer version of pytorch as we create new releases.

Oh alright so pytorch has a habit of changing a lot of things they maybe shouldn’t or at least should not change in the minor versions Otherwise you could at least use torch==1.7.* to skip updating the minor version manualy.

Yes, using something like torch==1.7.* was how we got in trouble last time. Thanks for the suggestion, but I think we’ll stay as-is.

And yes, haha, I am indeed very familiar with how to specify min versions!

Windows Install «ERROR: torch has an invalid wheel, .dist-info directory not found» Without Anaconda

Windows Install «ERROR: torch has an invalid wheel, .dist-info directory not found» Without Anaconda

Never mind then, I missed that PyTorch is completely unsupported on Windows with Python 3.6.

Windows Install «ERROR: torch has an invalid wheel, .dist-info directory not found» Without Anaconda