Error checking compiler version for cl - Исправление ошибок и поиск оптимальных решений проблем

📚 Installation

the dependencies torch-scatter, torch-sparse and torch cluster are not installed correctly
I ran the command for pytorch scatter and get the output (same for sparse and cluster):

running install
running bdist_egg
running egg_info
writing torch_scatter.egg-infoPKG-INFO
writing dependency_links to torch_scatter.egg-infodependency_links.txt
writing top-level names to torch_scatter.egg-infotop_level.txt
reading manifest file ‘torch_scatter.egg-infoSOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘torch_scatter.egg-infoSOURCES.txt’
installing library code to buildbdist.win-amd64egg
running install_lib
running build_py
running build_ext
C:UsersJonasAnaconda3envsschnetenvlibsite-packagestorchutilscpp_extension.py:184: UserWarning: Error checking compiler version for cl: ‘utf-8’ codec can’t decode byte 0x81 in position 62: invalid start byte
warnings.warn(‘Error checking compiler version for {}: {}’.format(compiler, error))
creating buildbdist.win-amd64egg
creating buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testtest_backward.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testtest_forward.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testtest_max_min.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testtest_multi_gpu.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testtest_std.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7testutils.py -> buildbdist.win-amd64eggtest
copying buildlib.win-amd64-3.7test_init_.py -> buildbdist.win-amd64eggtest
creating buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scatteradd.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scatterdiv.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scattermax.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scattermean.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scattermin.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scattermul.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scatterscatter_cpu.cp37-win_amd64.pyd -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scatterscatter_cuda.cp37-win_amd64.pyd -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scatterstd.py -> buildbdist.win-amd64eggtorch_scatter
copying buildlib.win-amd64-3.7torch_scattersub.py -> buildbdist.win-amd64eggtorch_scatter
creating buildbdist.win-amd64eggtorch_scatterutils
copying buildlib.win-amd64-3.7torch_scatterutilsext.py -> buildbdist.win-amd64eggtorch_scatterutils
copying buildlib.win-amd64-3.7torch_scatterutilsgen.py -> buildbdist.win-amd64eggtorch_scatterutils
copying buildlib.win-amd64-3.7torch_scatterutils_init_.py -> buildbdist.win-amd64eggtorch_scatterutils
copying buildlib.win-amd64-3.7torch_scatter_init_.py -> buildbdist.win-amd64eggtorch_scatter
byte-compiling buildbdist.win-amd64eggtesttest_backward.py to test_backward.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtesttest_forward.py to test_forward.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtesttest_max_min.py to test_max_min.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtesttest_multi_gpu.py to test_multi_gpu.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtesttest_std.py to test_std.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtestutils.py to utils.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtest_init_.py to init.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatteradd.py to add.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterdiv.py to div.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scattermax.py to max.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scattermean.py to mean.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scattermin.py to min.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scattermul.py to mul.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterstd.py to std.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scattersub.py to sub.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterutilsext.py to ext.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterutilsgen.py to gen.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterutils_init_.py to init.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatter_init_.py to init.cpython-37.pyc
creating stub loader for torch_scatterscatter_cpu.cp37-win_amd64.pyd
creating stub loader for torch_scatterscatter_cuda.cp37-win_amd64.pyd
byte-compiling buildbdist.win-amd64eggtorch_scatterscatter_cpu.py to scatter_cpu.cpython-37.pyc
byte-compiling buildbdist.win-amd64eggtorch_scatterscatter_cuda.py to scatter_cuda.cpython-37.pyc
creating buildbdist.win-amd64eggEGG-INFO
copying torch_scatter.egg-infoPKG-INFO -> buildbdist.win-amd64eggEGG-INFO
copying torch_scatter.egg-infoSOURCES.txt -> buildbdist.win-amd64eggEGG-INFO
copying torch_scatter.egg-infodependency_links.txt -> buildbdist.win-amd64eggEGG-INFO
copying torch_scatter.egg-infotop_level.txt -> buildbdist.win-amd64eggEGG-INFO
writing buildbdist.win-amd64eggEGG-INFOnative_libs.txt
zip_safe flag not set; analyzing archive contents…
torch_scatter.pycache.scatter_cpu.cpython-37: module references file
torch_scatter.pycache.scatter_cuda.cpython-37: module references file
creating ‘disttorch_scatter-1.2.0-py3.7-win-amd64.egg’ and adding ‘buildbdist.win-amd64egg’ to it
removing ‘buildbdist.win-amd64egg’ (and everything under it)
Processing torch_scatter-1.2.0-py3.7-win-amd64.egg
creating c:usersjonasanaconda3envsschnetenvlibsite-packagestorch_scatter-1.2.0-py3.7-win-amd64.egg
Extracting torch_scatter-1.2.0-py3.7-win-amd64.egg to c:usersjonasanaconda3envsschnetenvlibsite-packages
Adding torch-scatter 1.2.0 to easy-install.pth file

Installed c:usersjonasanaconda3envsschnetenvlibsite-packagestorch_scatter-1.2.0-py3.7-win-amd64.egg
Processing dependencies for torch-scatter==1.2.0
Finished processing dependencies for torch-scatter==1.2.0

Environment

OS: Windows 10
Python version: 3.7
PyTorch version: 1.1.0
CUDA/cuDNN version: 10.1, V10.1.168
GCC version: gcc (MinGW.org GCC-8.2.0-3) 8.2.0
How you tried to install PyTorch Geometric and its extensions (pip, source): source using
Any other relevant information: using visual studio 2019

Checklist

[x ] I followed the installation guide.
[x ] I cannot find my error message in the FAQ.
[ x] I set up CUDA correctly and can compile CUDA code via nvcc.
[x ] I have cloned the repository and tried a manual installation from source.
I do have multiple CUDA versions on my machine.
[x ] I checked if the official extension example runs on my machine.
The offical extension example runs on my machine.

##additional info:

running the first example works

import torch
from torch_geometric.data import Data

edge_index = torch.tensor([[0, 1, 1, 2],
[1, 0, 2, 1]], dtype=torch.long)
x = torch.tensor([[-1], [0], [1]], dtype=torch.float)

data = Data(x=x, edge_index=edge_index)

but transfering to the GPU

device = torch.device(‘cuda’)
data = data.to(device)

results in the error:
Traceback (most recent call last):
File «», line 1, in
File «C:UsersJonasAnaconda3envsschnetenvlibsite-packagestorch_geometricdatadata.py», line 247, in to
return self.apply(lambda x: x.to(device), keys)
File «C:UsersJonasAnaconda3envsschnetenvlibsite-packagestorch_geometricdatadata.py», line 233, in apply
self[key] = func(item)
File «C:UsersJonasAnaconda3envsschnetenvlibsite-packagestorch_geometricdatadata.py», line 247, in
return self.apply(lambda x: x.to(device), keys)
File «C:UsersJonasAnaconda3envsschnetenvlibsite-packagestorchcuda_init_.py», line 163, in _lazy_init
torch._C._cuda_init()
RuntimeError: CUDA error: unknown error

Using cuda with torch but without importing torch_geometric, works. It seems that importing torch_geometric is corrupting the use of cuda. I also tried with cuda 10.0 but it did not make a difference.

Thank you in advance
Jonas

Источник

I’m using Anaconda and I have installed PyTorch using the following command: pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

Now I’m getting the following error in torch/utils/cpp_extension.py:

UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified

I’m using Windows 10 and I have installed Visual Studio Community 2022 and Visual Studio Build Tools 2022, please see screenshots below.

Does somebody what is wrong or missing?

Edit: I’m using Cuda 11.6. I have now also installed Visual Studio 2019 including the build tools. Now the above error is gone but I have a new error:

Traceback (most recent call last):
  File "C:UsersmyUserAnaconda3envsparlailibsite-packagestorchutilscpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "C:UsersmyUserAnaconda3envsparlailibsubprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:UsersmyUserParlAIserverserver.py", line 3, in <module>
    from parlai.utils.safety import OffensiveStringMatcher, OffensiveLanguageClassifier
  File "c:usersmyUserparlaiparlaiutilssafety.py", line 10, in <module>
    from parlai.agents.transformer.transformer import TransformerClassifierAgent
  File "c:usersmyUserparlaiparlaiagentstransformertransformer.py", line 15, in <module>
    from parlai.core.torch_generator_agent import TorchGeneratorAgent
  File "c:usersmyUserparlaiparlaicoretorch_generator_agent.py", line 48, in <module>
    from parlai.ops.ngram_repeat_block import NGramRepeatBlock
  File "c:usersmyUserparlaiparlaiopsngram_repeat_block.py", line 23, in <module>
    ngram_repeat_block_cuda = load(
  File "C:UsersmyUserAnaconda3envsparlailibsite-packagestorchutilscpp_extension.py", line 1202, in load
    return _jit_compile(
  File "C:UsersmyUserAnaconda3envsparlailibsite-packagestorchutilscpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "C:UsersmyUserAnaconda3envsparlailibsite-packagestorchutilscpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "C:UsersmyUserAnaconda3envsparlailibsite-packagestorchutilscpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'ngram_repeat_block_cuda': [1/2] C:Program FilesNVIDIA GPU Computing ToolkitCUDAv11.6binnvcc --generate-dependencies-with-compile --dependency-output ngram_repeat_block_cuda_kernel.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=ngram_repeat_block_cuda -DTORCH_API_INCLUDE_EXTENSION_H -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchinclude -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludetorchcsrcapiinclude -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludeTH -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludeTHC "-IC:Program FilesNVIDIA GPU Computing ToolkitCUDAv11.6include" -IC:UsersmyUserAnaconda3envsparlaiInclude -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c c:usersmyUserparlaiparlaiclibcudangram_repeat_block_cuda_kernel.cu -o ngram_repeat_block_cuda_kernel.cuda.o
FAILED: ngram_repeat_block_cuda_kernel.cuda.o
C:Program FilesNVIDIA GPU Computing ToolkitCUDAv11.6binnvcc --generate-dependencies-with-compile --dependency-output ngram_repeat_block_cuda_kernel.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=ngram_repeat_block_cuda -DTORCH_API_INCLUDE_EXTENSION_H -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchinclude -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludetorchcsrcapiinclude -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludeTH -IC:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludeTHC "-IC:Program FilesNVIDIA GPU Computing ToolkitCUDAv11.6include" -IC:UsersmyUserAnaconda3envsparlaiInclude -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c c:usersmyUserparlaiparlaiclibcudangram_repeat_block_cuda_kernel.cu -o ngram_repeat_block_cuda_kernel.cuda.o
C:/Users/myUser/Anaconda3/envs/parlai/lib/site-packages/torch/includec10/macros/Macros.h(143): warning C4067: unexpected tokens following preprocessor directive - expected a newline
C:/Users/myUser/Anaconda3/envs/parlai/lib/site-packages/torch/includec10/macros/Macros.h(143): warning C4067: unexpected tokens following preprocessor directive - expected a newline
C:/Users/myUser/Anaconda3/envs/parlai/lib/site-packages/torch/includec10/core/SymInt.h(84): warning #68-D: integer conversion resulted in a change of sign

C:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludepybind11cast.h(1429): error: too few arguments for template template parameter "Tuple"
          detected during instantiation of class "pybind11::detail::tuple_caster<Tuple, Ts...> [with Tuple=std::pair, Ts=<T1, T2>]"
(1507): here

C:UsersmyUserAnaconda3envsparlailibsite-packagestorchincludepybind11cast.h(1503): error: too few arguments for template template parameter "Tuple"
          detected during instantiation of class "pybind11::detail::tuple_caster<Tuple, Ts...> [with Tuple=std::pair, Ts=<T1, T2>]"
(1507): here

2 errors detected in the compilation of "c:/users/myUser/parlai/parlai/clib/cuda/ngram_repeat_block_cuda_kernel.cu".
ngram_repeat_block_cuda_kernel.cu
ninja: build stopped: subcommand failed.

Источник

Old
5th September 2021, 10:52

#41

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Also did anyone manage to install this in a portable Vapoursynth environment on Windows?
Calling:

Code:

python -m pip install --upgrade vsbasicvsrpp

first failed with

Code:

ERROR: Could not find a version that satisfies the requirement vapoursynth==54 (from versions: 39, 40, 41, 42, 43, 44, 45, 46, 47, 47.1, 47.2, 48, 49, 50, 51)
ERROR: No matching distribution found for vapoursynth==54

after renaming the dummy ‘VapourSynth-53.dist-info’, I created to install VSGAN, to ‘VapourSynth-54.dist-info’, calling:

Code:

python -m pip install --upgrade vsbasicvsrpp

failed with:

Code:

OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

calling:

Code:

set CUDA_HOME=I:/Hybrid/64bit/Vapoursynth/Lib/site-packages/torch/cuda

(not toally sure this is correct)
and then:

Code:

python -m pip install --upgrade vsbasicvsrpp

it fails with:

Code:

I:Hybrid64bitVapoursynthLibsite-packagestorchutilscpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
      warnings.warn(f'Error checking compiler version for {compiler}: {error}')

I get the same error when calling:

Code:

python -m pip install mmcv-full==1.3.12 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.htm

-> that’s the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know.

Cu Selur

__________________
Hybrid here in the forum, homepage

Last edited by Selur; 5th September 2021 at 11:05.

Old
5th September 2021, 14:37

#42

| Link

Registered User

Join Date: Sep 2007

Posts: 5,096

Quote:

Originally Posted by Selur

Is there a difference between BasicVSR and BasicVSR++ if model 0-2 are used, or is it the same resizing as BasicVSR (model 0-2) and additional models for cleaning?

I’ve only done a few tests so far , but some early observations/comments — basicvsrpp is marginally better with the same model interval size compared to basicvsr. Not a major difference. The default interval size is different, 30 for ++, vs 7*2+1=15

Models 3-5 are from the NTIRE 2021 Quality enhancement of heavily compressed videos Challenge , which take HEVC compressed videos using fixed qp and low bitrate encodings — so those pre-trained models should factor in some compression degredation (at least HEVC type, not necessarily MPEG2, or AVC). It’ s nice to see some other types of degradation training and models, but 3 and 5 tend to be very smooth (ie. no detail) . 4 has more detail but more artifacts. Models 3-5 don’t upscale

I haven’t done enough testing to see if using a much larger interval size helps or hinders in general. It appears a very small interval size is worse. Larger sizes take more memory and are slower

Old
5th September 2021, 14:51

#43

| Link

Registered User

Join Date: Sep 2007

Posts: 5,096

Quote:

Originally Posted by Selur

-> that’s the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know.

Quote:

I:Hybrid64bitVapoursynthLibsite-packagestorchutilscpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
warnings.warn(f’Error checking compiler version for {compiler}: {error}’)

Not sure, I used installed environment, but I had problems at first. My errors msg was slightly different — it needed MS Visual C++ compiler to build the «wheels» to install other components. I’m wondering how the compiler is accessed in a «portable» environment ?

Old
5th September 2021, 15:29

#44

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Quote:

I’m wondering how the compiler is accessed in a «portable» environment ?

No clue either, haven’t run into the problem before.

Cu Selur

__________________
Hybrid here in the forum, homepage

Old
5th September 2021, 16:25

#45

| Link

Registered User

Join Date: Sep 2007

Posts: 5,096

HolyWu added update a few hours ago and made install «easier» on Windows. Maybe try this new one
https://github.com/HolyWu/vs-basicvsrpp

Quote:

Installing mmcv-full on Windows is a bit complicated as it requires Visual Studio and other tools to compile CUDA ops. So I have uploaded the built file compiled with CUDA 11.1 for Windows users and you can install it by executing the following command.

Code:

pip install https://github.com/HolyWu/vs-basicvsrpp/releases/download/v1.0.0/mmcv_full-1.3.12-cp39-cp39-win_amd64.whl

Old
5th September 2021, 20:49

#46

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Thanks ! using that call it works for me too.

Cu Selur

__________________
Hybrid here in the forum, homepage

Old
5th November 2021, 20:42

#47

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Has anyone tried https://github.com/HolyWu/vs-swinir ? (didn’t want to create a new thread )
-> man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)

__________________
Hybrid here in the forum, homepage

Last edited by Selur; 5th November 2021 at 22:21.

Old
6th November 2021, 08:11

#48

| Link

Registered User

Join Date: Aug 2002

Location: Italy

Posts: 275

Quote:

Originally Posted by Selur

Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.

HolyWu has just ported — after my request — SwinIR to VS, can someone make some «real world» test with it ?

https://github.com/HolyWu/vs-swinir

Quote:

Originally Posted by Selur

man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)

For individual frames (= pics) you can test it here, but a video-oriented colab notebook like this would be great (I don’t own a discrete GPU at all) !

EDIT
Out of curiosity: do you think the new Apple chips (M1 Pro / Max) could speed up operations?

Last edited by PatchWorKs; 6th November 2021 at 08:50.

Old
6th November 2021, 13:34

#49

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Quote:

Out of curiosity: do you think the new Apple chips (M1 Pro / Max) could speed up operations?

Without:
a. pytorch support
b. rewriting of the exitistn plugins
-> no

Quote:

For individual frames (= pics)

I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.

__________________
Hybrid here in the forum, homepage

Old
7th November 2021, 19:54

#50

| Link

Registered User

Join Date: Oct 2001

Posts: 422

Quote:

Originally Posted by Selur

I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.

thanx for testing How much vram did it use in your case? Is it really that GPU demanding or might the slow-down caused by not enough VRAM ? If you have something a unskilled person like me could use and test, I could throw it into a 12GB VRAM card and see what happens…

Old
8th November 2021, 09:49

#51

| Link

Registered User

Join Date: Aug 2002

Location: Italy

Posts: 275

Quote:

Originally Posted by Selur

Without:
a. pytorch support
b. rewriting of the exitistn plugins
-> no

Well, pytorch support SEEMS on the go:
https://github.com/pytorch/pytorch/issues/47702

Quote:

Originally Posted by Selur

I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.

Of course (that’s why a colab can help), but can you please post some visual results ?

Thx !

Old
8th November 2021, 18:32

#52

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

here are a few examples:

used:

Code:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'E:clipsVTS_02_1-Sample-Beginning.demuxed.m2v'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading E:clipsVTS_02_1-Sample-Beginning.demuxed.m2v using D2VSource
clip = core.d2v.Source(input="E:/Temp/m2v_5d36292e1f7f53fd6e26be51d50bbf8c_853323747.d2v")
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# DEBUG: vsTIVTC changed scanorder to: progressive
# cropping the video to 704x480
clip = core.std.CropRel(clip=clip, left=8, right=8, top=0, bottom=0)
from vsswinir import SwinIR
# adjusting color space from YUV420P8 to RGBS for VsSwinIR
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# resizing using SwinIR
clip = SwinIR(clip=clip, task="real_sr_large", scale=4, tile_x=352, tile_y=240, tile_pad=16, device_type="cuda", device_index=0) # 2816x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1920, h=1474, kernel="lanczos", interlaced=False, interlacedd=False)
# adjusting output color from: RGB48 to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

some more using RealSR_large:

Cu Selur

__________________
Hybrid here in the forum, homepage

Last edited by Selur; 8th November 2021 at 18:50.

Old
9th November 2021, 08:21

#53

| Link

Registered User

Join Date: Aug 2002

Location: Italy

Posts: 275

Very nice results (especially on the faces), even if — of course — not yet optimal for everything…

…btw I hope to see a SwinIR version optimized for videos too.

Last edited by PatchWorKs; 9th November 2021 at 08:25.

Old
9th November 2021, 21:08

#54

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Quote:

not yet optimal for everything…

Espeically the last example shows some real issues

Quote:

…btw I hope to see a SwinIR version optimized for videos too.

first a way faster version would be needed

__________________
Hybrid here in the forum, homepage

Old
10th November 2021, 08:13

#55

| Link

Registered User

Join Date: Aug 2002

Location: Italy

Posts: 275

Quote:

Originally Posted by Selur

first a way faster version would be needed

Already asked, of course: https://github.com/JingyunLiang/SwinIR/issues/47

Note: I’ve also just «fed» @HolyWu with this awesome collection, let’s see if other interesting «VS-ports» will come out…

Last edited by PatchWorKs; 10th November 2021 at 10:34.

Old
11th November 2021, 11:05

#56

| Link

Registered User

Join Date: Oct 2001

Posts: 422

Quote:

Originally Posted by PatchWorKs

Very nice results (especially on the faces), even if — of course — not yet optimal for everything…

…btw I hope to see a SwinIR version optimized for videos too.

Is a video version planed?

for upscaling, algos like esrgan (single image) are not very suitable for real-life content. Too much flickering. So unless SwinIR doesn�t get some extensions for multi-frame usage / flow detection /whatever, one will always get flickering / stutters / inkonsistent movement…

Old
18th November 2021, 17:52

#57

| Link

Registered User

Join Date: Apr 2009

Posts: 478

Nvidia just open sourced Nvidia image scaling… Would this be a candidate for the next filter?

Old
19th November 2021, 20:09

#58

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

Has anyone tested https://github.com/HolyWu/vs-hinet ?

Here are a few screen shots: (not sure what to make of them and for what content this is really useful)

Mode: Deblur GoPro

Mode: Deblur REDS

Mode: denoise

Mode: derain

__________________
Hybrid here in the forum, homepage

Last edited by Selur; 19th November 2021 at 21:32.

Old
20th November 2021, 08:22

#59

| Link

Registered User

Join Date: Aug 2002

Location: Italy

Posts: 275

According to your tests on that frame, the highest fidelity seems to be achieved by derain model, btw here are some questions:

how fast is it ?
how does it performs (in terms of both speed and fidelity) compared to xClean ?
its description claims that it perform «restoring» (aka denoising ?) function, but since is BasicSR-based does it upscale too ?

Last but not least (even if OT): did you tried RIFE ?
https://github.com/HolyWu/vs-rife

Last edited by PatchWorKs; 20th November 2021 at 16:31.

Old
20th November 2021, 10:17

#60

| Link

Registered User

Join Date: Oct 2001

Location: Germany

Posts: 6,914

speed: ~2-3fps for sd content, so not that slow
xClean: no clue about xClean, haven’t played around with it too many options for my taste (+ would need to add znedi3 and nnedi3cl support to it)
upscale: at least the current interface offers no upscaling and the method does not upscale
rife: yes, I like it (with sceneChange added). Waiting for FrameRateConverter to properly support it

__________________
Hybrid here in the forum, homepage

Источник

📚 Installation

Environment

Checklist

running the first example works

data = Data(x=x, edge_index=edge_index)

but transfering to the GPU

device = torch.device(‘cuda’) data = data.to(device)

Читайте также:

device = torch.device(‘cuda’)
data = data.to(device)