Proxmox gpu passthrough nvidia error 43

To summarize:

1. Trying to passthrough the GPU so that I can have a Linux host to manage the computer while the users only see Windows.

2. Hardware is the following:

— AMD Athlon X4 845 CPU
— ASRock FM2A88M PRO3+ Motherboard
— 8GB of DDR3 Kingston RAM
— 120GB SSD
— Nvidia GT 710 GPU​

3. Config file for the VM on ProxMox is the following:

        bios: ovmf
        bootdisk: virtio0
        cores: 4
        cpu: host
        efidisk0: local-lvm:vm-100-disk-2,size=128K
        hostpci0: 02:00,pcie=1,x-vga=on,romfile=GK208_BIOS_FILE.bin
        machine: q35
        memory: 7168
        name: Test-Windows
        net0: virtio=96:24:CF:B4:AA:A2,bridge=vmbr0
        numa: 0
        ostype: win10
        scsihw: virtio-scsi-pci
        smbios1: uuid=f675c872-c390-4668-9c48-423f5b4ff239
        sockets: 1
        usb0: host=6-1.2 # Mouse & Keyboard
        usb1: host=2-4 # Other
        usb2: host=3-1.2.3.4 # Physical
        usb3: host=1-1.2.3.4 # USB Ports
        virtio0: local-lvm:vm-100-disk-1,cache=writeback,size=90G

4. The BIOS bin file is parsed correctly and UEFI capable. The rom parsing instructions comes from the ProxMox Wiki.

        root@pve-001:~/rom-parser# ./rom-parser /usr/share/kvm/GK208_BIOS_FILE.bin
            Valid ROM signature found @600h, PCIR offset 190h
                PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000
                PCIR: revision 0, vendor revision: 1
            Valid ROM signature found @fc00h, PCIR offset 1ch
                PCIR: type 3 (EFI), vendor: 10de, device: 128b, class: 030000
                PCIR: revision 3, vendor revision: 0
                    EFI: Signature Valid, Subsystem: Boot, Machine: X64
            Last image

The Motherboard has the issue where it shadows the BIOS of the GPU in the first PCI-e slot. As far as I’m aware, providing a rom file is required in this situation for GPU passthrough.

5. I’ve followed the following guides trying a mix and match of settings:

https://forum.proxmox.com/threads/gpu-passthrough-tutorial-reference.34303/

https://forum.proxmox.com/threads/nvidia-gpu-passthrough-assistance.27237/

https://pve.proxmox.com/wiki/Pci_passthrough

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

Any other reasons for Nvidia driver code 43? from VFIO

I realize this isn’t Fedora 26 or RyZen, but useful info regardless:

https://level1techs.com/article/ryzen-gpu-passthrough-setup-guide-fedora-26-windows-gaming-linux

I could’ve easily missed a step in one of them given all the information I’m trying to combine here.

6. I have the latest Nvidia Drivers installed (version 388).

7. IOMMU is working:

root@pve-001:~# dmesg | grep -e IOMMU
[    0.615425] AMD-Vi: IOMMU performance counters supported
[    0.617069] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[    0.618547] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

root@pve-001:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/7/devices/0000:00:13.2
/sys/kernel/iommu_groups/7/devices/0000:00:13.0
/sys/kernel/iommu_groups/5/devices/0000:00:11.0
/sys/kernel/iommu_groups/3/devices/0000:00:09.0
/sys/kernel/iommu_groups/11/devices/0000:00:15.2
/sys/kernel/iommu_groups/11/devices/0000:00:15.0
/sys/kernel/iommu_groups/11/devices/0000:05:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:03.0
/sys/kernel/iommu_groups/1/devices/0000:02:00.1
/sys/kernel/iommu_groups/1/devices/0000:00:03.1
/sys/kernel/iommu_groups/1/devices/0000:02:00.0
/sys/kernel/iommu_groups/8/devices/0000:00:14.2
/sys/kernel/iommu_groups/8/devices/0000:00:14.0
/sys/kernel/iommu_groups/8/devices/0000:00:14.3
/sys/kernel/iommu_groups/6/devices/0000:00:12.2
/sys/kernel/iommu_groups/6/devices/0000:00:12.0
/sys/kernel/iommu_groups/4/devices/0000:00:10.0
/sys/kernel/iommu_groups/12/devices/0000:00:18.4
/sys/kernel/iommu_groups/12/devices/0000:00:18.2
/sys/kernel/iommu_groups/12/devices/0000:00:18.0
/sys/kernel/iommu_groups/12/devices/0000:00:18.5
/sys/kernel/iommu_groups/12/devices/0000:00:18.3
/sys/kernel/iommu_groups/12/devices/0000:00:18.1
/sys/kernel/iommu_groups/2/devices/0000:00:08.0
/sys/kernel/iommu_groups/10/devices/0000:00:14.5
/sys/kernel/iommu_groups/0/devices/0000:00:02.2
/sys/kernel/iommu_groups/0/devices/0000:00:02.0
/sys/kernel/iommu_groups/0/devices/0000:01:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:14.4

8. The vfio-pci driver is being used correctly:

root@pve-001:~# lspci -k
02:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1)
Subsystem: eVga.com. Corp. GK208 [GeForce GT 710B]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau

Other relevant information:

root@pve-001:~# pveversion -v
    proxmox-ve: 5.1-25 (running kernel: 4.13.4-1-pve)
    pve-manager: 5.1-35 (running version: 5.1-35/722cc488)
    pve-kernel-4.13.4-1-pve: 4.13.4-25
    libpve-http-server-perl: 2.0-6
    lvm2: 2.02.168-pve6
    corosync: 2.4.2-pve3
    libqb0: 1.0.1-1
    pve-cluster: 5.0-15
    qemu-server: 5.0-17
    pve-firmware: 2.0-3
    libpve-common-perl: 5.0-20
    libpve-guest-common-perl: 2.0-13
    libpve-access-control: 5.0-7
    libpve-storage-perl: 5.0-16
    pve-libspice-server1: 0.12.8-3
    vncterm: 1.5-2
    pve-docs: 5.1-12
    pve-qemu-kvm: 2.9.1-2
    pve-container: 2.0-17
    pve-firewall: 3.0-3
    pve-ha-manager: 2.0-3
    ksm-control-daemon: 1.2-2
    glusterfs-client: 3.8.8-1
    lxc-pve: 2.1.0-2
    lxcfs: 2.0.7-pve4
    criu: 2.11.1-1~bpo90
    novnc-pve: 0.6-4
    smartmontools: 6.5+svn4324-1
    zfsutils-linux: 0.7.2-pve1~bpo90

But after all that, I still get Error Code 43. Note that the Microsoft Basic Display Adapter is not listed under Display Adapters.

Is it because I’m using VirtIO? All the guides seem to be using SCSI without VirtIO. I’d like as much performance as possible, but if it’s just not happening with VirtIO, I can use SCSI.

I am getting this error when I run the VM:

kvm: -device vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0,romfile=/usr/share/kvm/GK208_BIOS_FILE.bin: Failed to mmap 0000:02:00.0 BAR 3. Performance may be slow

I have disabled the Framebuffer in the kernel settings:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on video=vesafb:off,efifb:off"

But I am still seeing BIOS messages, up to a point, on the display.

GPU passthrough code 43 (unable to resolve)

jishnuj

New Member

I’m running an AMD laptop (ASUS ROG Strix G513QC with integrated AMD graphics and a dedicated NVIDIA GPU (RTX 3050). I had fixed the code 43 for a moment by disabling and re-enabling the driver but now that no longer works.

GRUB_CMDLINE_LINUX_DEFAULT=»quiet amd_iommu=on textonly video=efifb ff video=vesafb ff»

root@vpj:/etc/default# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with «#» are ignored.

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

root@vpj:/etc/modprobe.d# cat blacklist.conf
blacklist radeon
blacklist nouveau
blacklist nvidia

root@vpj:/etc/modprobe.d# cat iommu_unsafe_interrupts.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1

root@vpj:/etc/modprobe.d# cat pve-blacklist.conf
# This file contains a list of modules which are not supported by Proxmox VE

root@vpj:/etc/modprobe.d# cat vfio.conf
options vfio-pci ids=10de:25a5,10de:2291 disable_vga=1

root@vpj:/etc/pve/qemu-server# cat /etc/pve/qemu-server/100.conf
agent: 1
args: -cpu ‘host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off’
balloon: 0
bios: ovmf
boot: order=sata0
cores: 6
cpu: host,hidden=1,flags=+pcid,hv-vendor-id=proxmox
hostpci0: 01:00.0,pcie=1,romfile=Asus.RTX3050Mobile.4096.210519.rom
hostpci1: 01:00.1,pcie=1
ide2: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
machine: q35
memory: 10000
name: windows
net0: virtio=42:18:EF:5E:C1:4E,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
sata0: local-lvm:vm-100-disk-0,size=300G
scsihw: virtio-scsi-pci
smbios1: uuid=2e5a76dc-cba9-4761-bd5e-4c0702892ef5
sockets: 1
vga: none
vmgenid: e40e446e-28c9-4699-93e1-59a7ae425a62

01:00.0 VGA compatible controller: NVIDIA Corporation Device 25a5 (rev a1)
Subsystem: ASUSTeK Computer Inc. Device 10ec
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
01:00.1 Audio device: NVIDIA Corporation Device 2291 (rev a1)
Subsystem: ASUSTeK Computer Inc. Device 10ec
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel

Have Windows 10 Pro installed with the latest NVIDIA graphics drivers (there are no beta drivers available). I have virtio-win drivers installed as well. Any ideas on what might be causing a code 43? I was told that it’s because the VM knows it’s a VM and the host has to trick it or something like that.

Источник

[SOLVED] Quadro GPU Passthrough to Windows shows code 43

mega_neo

New Member

Unfortunately I’m starting one more post regarding GPU passthrough, like many before. I’ve been reading several different posts and trying some different options but to no avail. I’m even trying the new nvidia drivers that are said to allow GPU passthrough but that also didn’t help.

I thought my life would be easier since I’m working with a nvidia Quadro GPU but that doesn’t seem to be the truth.
I hope you can get me in the right direction so I can finally solve this stupid error 43.

So, I’m using a HP workstation:

Machine: HP Z230 SFF
CPU: i7 4790
Ram: 8GB (to be updated to 32 when i find the 4 ram sticks)
GPU: Nvidia Quadro K1200
SDD: 120 GB SSD for Proxmox (with the VM disk inside it-also to be updated).

Using Proxmox VE 6.4-5
Guest VM: Windows 10 Pro

Following the several manuals around, I started by adding the Grub options to /etc/default/grub:

I’m using the ACS option due to the fact that my GPU was in the same IOMMU group as the PCI bridge PCIe x16 controller. I also tried the multifunction option put didn’t se a major difference.
*EDIT*: I was told that the grouping with the bridge is not a problem since the OS is able to work it that, so I took the ACS option out. Just wanted to let this here to show what I’ve tried already.

I’ve also added the required modules to /etc/modules:

So, that’s it. I’ve been fighting with this for a couple of weeks without a solution in sight.
I see the GPU being recognized in Windows so I think the VM on itself might not be the problem. I’m even able to install the most up to date driver.
If the problem was a recognition i think the driver would not even install as it doesn’t see a nvidia GPU.

So I really don’t know where to look any more tips for this. I’m really out of resources.
If you guys could have a look at the configs and see if you spot anything wrong or something missing please let me know.

Thank you in advance

mega_neo

New Member

I think I got it.

In my case, I found a website talking about what changed in Proxmox VE 6.2 and how to fix the GPU passthrough.

From Proxmox 6.2 the module

is already included in the kernel, so that may change somethings.

I follow the instructions on they solved their issue, and adding:

in the grub file, also solved my issue.

Don’t have code 43 in the device manager and GPU-z shows not only the nvidia details (as in the past), but also the sensors are on while before only had to entries with no changes.

Maybe this can helps anyone in the future.

There was also another change I did. I took out the romfile fro my VM config since, as indicated by someone else on my reddit post, the romfile i extracted maybe was not having the correct template or even not correct at all due to:

If your host is initialising the card on startup (e.g. it’s set as the primary GPU) this dumps the vBIOS that already got trashed by initialisation, so it’s worthless. You get precisely the same result by not using the romfile argument at all.

On the other hand, if your host isn’t initialising the card on startup, you don’t need to supply a clean vBIOS file anyway, since it hasn’t been ruined by initialisation.

This method is only useful for systems where you can temporarily add another GPU as the primary, leaving your target GPU unmolested. Then you can dump the clean vBIOS, remove the temporary GPU, and you’re good to go.

LocalUser

New Member

I’m using an HP640 Workstation with,
CPU: Intel Xeon E5-2650 v4
RAM: 32GB
GPU: Quadro K2200

basically (or at least close to) your system @mega_neo. However, I can’t seem to get the past «Code 43».

update-grub
update-initramfs -u
reboot

Adding «pc» to args: -machine as stated in the tutorial yields the following error.
kvm:/usr/share/qemu-server/pve-q35-4.0.cfg:1: Bus ‘pcie.0’ not found
TASK ERROR: start failed: QEMU exited with code 1

removing pc makes it bootable, with «Code 43«.

Without commenting out vfio.conf the following error arises,
kvm: -device vfio-pci,host=0000:02:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-vga=on,multifunction=on: vfio 0000:02:00.0: failed getting region info for VGA region index 8: Invalid argument
device does not support requested feature x-vga

Since you have a consumer CPU you have the iGPU enabled for the host I guess? Is that where it falls apart for me.

LocalUser

New Member

Solved mine (Code 43) with a nudge from @avw by adjusting,

/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT=»quiet intel_iommu=on iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream video=efifb ff video=vesafb ff»

leesteken

Famous Member

/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT=»quiet intel_iommu=on iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream [U]video=efifb:off video=vesafb:off» [/U]

/etc/modprobe.d/vfio.conf video= parameter to two separate ones (because the video parameter does not support multiple values separate by commas).
Just curious: Do you really need pcie_acs_override ? I would advice against it because it break proper isolation. Do you also need the vfio-pci ids ? Does this not prevent any display of stating Proxmox and make troubleshooting much harder? I would expect a Quadro to properly reset without binding it to vfio-pci manually.

LocalUser

New Member

The main fix was indeed splitting the video parameter.

The pcie_acs_override parameter is not necessary. Removed it from grub config and rebooted, GPU in the VM still works.

vfio-pci ids is not necessary if you want to connect a screen to the GPU. However, now it is impossible to connect by Remote Desktop into the VM with GPU passed to it. Why it behaves like that now is unclear to me. Must be that the host is accessing the GPU now for some reason after trying with/without vfio-pci ids . Could that be it. Error in the VM is now Code 12. Does it make sense?

I am running Proxmox headless, and I would like to have remote access to some VMs with the ability to use the GPU.

Edit:
The VM is unable to start with the passed GPU as primary ( x-vga=1 )
Procedure Removed the passed GPU from the VM, rebooted, then RDP was working again.
Procedure Added the passed GPU to the VM again, rebooted, then RDP was working with Code 12 on the GPU (maybe drivers).
Procedure Rebooted the VM again, then RDP was not working. However, the GPU outputs the VM on a screen so that is working.

Источник

[SOLVED] GPU PASSTHROUGH ( Error 43 while there is output from the gpu’s hdmi )

teokoul

New Member

I am new on the proxmox.
I am trying to make gpu passthrough on a windows 10 vm.

I am thinking that I am in a good way because in device manager of windows10 vm I can see the graphic card.
Also, I can see the windows home screen in my monitor.

The problem is that i got error43 in device manager.

I install the nvidia drivers with geforce experience.
**If i disable the device and enable it again the error 43 gone. After 1 minute the error43 come back again.**

What should I do to fix this?

Stefan_R

Proxmox Retired Staff

Remove the -args line from your config, it is not necessary. Also, do you actually need the romfile= parameter? Usually it is not necessary, and might even break things.

Can you please also post the output of the following commands:

Best regards,
Stefan

Do you already have a Commercial Support Subscription? — If not, Buy now and read the documentation

teokoul

New Member

Remove the -args line from your config, it is not necessary. Also, do you actually need the romfile= parameter? Usually it is not necessary, and might even break things.

Can you please also post the output of the following commands:

Thank you for the reply. I do not saw your answer before. I made a fresh install and now I am not able to run the VM when I try the GPU passthrough.

The output of preversion -v is :

The output of cat /proc/cmdline is:

There is no output from the lsmod | grep nvidia

What do you think?

I will give a try again with the rom file.

Thank you for your time!

jjk.saji

New Member

# lsmod | grep nvidia
i2c_nvidia_gpu 16384 0
root@server-4:

teokoul

New Member

was going through thread, to understand GPU pass through .
It looks strange for me, why you lsmod not giving nvdia results back, when you did the re-install was the card fixed properly
I have been trying out proxmox in 4 machines, all of them has nvdia and lsmod shows the results correctly

Mine is default installation, me have not done any command line to enable NVIDIA drives
As said in all the 4 machines it shows NVIDIA entry for lsmod
thanks
Joseph John

Thank you for the reply.

I am not sure what is going on. The lsmod | grep nvidia has no output.

I don’t know if that helps but the lspci | grep NVIDIA has output:

Источник

[SOLVED] Nvidia GPU passthrough code 43

unf0rg0tt3n

Active Member

I have followed a guide (this one) on how to passthrough a GPU.
Everything okay, but in the win10 guest I have a code 43 on the gpu.

I have two Nvidia Quadro600’s running in the system.
My qemu config:

Anyone a idea on how to troubleshoot or fix this problem?

tinfever

Member

I haven’t worked with Quadro cards but I had to set the CPU flag for hv_vendor_id=whatever as mentioned in this thread https://forum.proxmox.com/threads/modify-cpu-parameters.38686/.

Here is the line I have used before in the config file. I include a lot of the default flags from ‘qm showcmd yourvm#’ since I believe I read that the latter -cpu args command will overwrite the default one that Proxmox uses.

args: -cpu ‘host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_tlbflush,hv_ipi,kvm=off,hv_vendor_id=whatever’

elurex

Active Member

I would suggest the following config for your qemu if you are using PVE 6.0

hostpci0: 04:00,pcie=1,x-vga=1
machine: pc-q35-3.1

unf0rg0tt3n

Active Member

I haven’t worked with Quadro cards but I had to set the CPU flag for hv_vendor_id=whatever as mentioned in this thread https://forum.proxmox.com/threads/modify-cpu-parameters.38686/.

Here is the line I have used before in the config file. I include a lot of the default flags from ‘qm showcmd yourvm#’ since I believe I read that the latter -cpu args command will overwrite the default one that Proxmox uses.

args: -cpu ‘host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_tlbflush,hv_ipi,kvm=off,hv_vendor_id=whatever’

unf0rg0tt3n

Active Member

Adding the commands to 100.conf and reinstalling Windows10 actually worked for a bit.
Device manager popped: «success»!

Ofc Windows needed to reboot and the following occurs:

And. bootloop.
My config after adding the sugested:

tinfever

Member

Check what «qm showcmd 100» gives you regarding cpu flags. Make sure you aren’t specifying any flags in your manually added args line that aren’t normally included by proxmox defaults. Different CPUs have different flags so if you call one that doesn’t work with yours, it wouldn’t surprise me if strange things happened.

Also, the ‘hv_vendor_id=whatever’ was literal. It doesn’t matter what is that field as long as it isn’t something Nvidia has hard coded their drivers to block.

I’m also not sure how the pc-q35-3.1 and x-vga=1 are going to change things since I haven’t worked with those.

Ideally I’d suggest just getting your «qm showcmd 100» and copying the -cpu settings, adding the key ‘kvm=off,hv_vendor_id=whatever’ bits to the end, and then putting that whole thing in your -args command.

Also, there is a chance that Quadro cards won’t work the same way. I’ve only worked with GTX cards in virtualization.

Источник

AMD APU

PCIe

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
update-initramfs -u

lcpci
26:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
 
lspci -n -s 26:00.0
26:00.0 0300: 10de:0a65 (rev a2)
 
echo "options vfio-pci ids=10de:0a65" > /etc/modprobe.d/vfio.conf
 
rmmod vfio-pci
modprobe vfio-pci
 
dmesg
[536167.737504] vfio-pci 0000:26:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[536170.155565] vfio-pci 0000:26:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[536170.173544] vfio_pci: add [10de:0a65[ffffffff:ffffffff]] class 0x000000/00000000

Error code 43

It is generic error code. Possible reasons:

KVM detected

BAR 3: can’t reserve

vfio-pci 0000:26:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref]
...
pcieport 0000:00:03.1: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:26:00.0
pcieport 0000:00:03.1: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:26:00.0
pcieport 0000:00:03.1: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:26:00.0

nomodeset textonly video=vesafb:off video=efifb:off

GRUB_CMDLINE_LINUX_DEFAULT="nomodeset textonly video=vesafb:off video=efifb:off"

root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=4 mitigations=off panic=30 nomodeset textonly video=vesafb:off video=efifb:off

update-grub
pve-efiboot-tool refresh

This device cannot find enough free resources that it can use. (Code 12)

Solution was to select “Primary GPU” on PCI Device (x-vga=1 option in Proxmox config file)