Task error cannot prepare pci pass through iommu not present

Don’t put GRUB_CMDLINE_LINUX_DEFAULT=“quiet intel_iommu=on” in /etc/kernel/cmdline because it is not part of GRUB; Just use quiet intel_iommu=on and nothing else. But this has only effect if you’re booting in UEFI mode on ZFS.

Don’t use GRUB_CMDLINE_LINUX and GRUB_CMDLINE_LINUX_DEFAULT with different options. Especially with options that do not apply to your system like video=astdrmfb. Just use this to see if it enables your IOMMU:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX=""

Run update-grub or proxmox-boot-tool refresh to apply changes to the kernel parameters. Use cat /proc/cmdline to check if your changes have been applied (after a reboot).

EDIT: Make sure to fully enable IOMMU or VT-d in your motherboard BIOS.

Either the message is incorrect, which you can double check with: for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done. If there are multiple groups it is actually working and the error message is wrong. If there is only one IOMMU group *, then it is not fully enabled in hardware or the BIOS.

Maybe IOMMU and/or VT-d is not fully enabled in the motherboard BIOS. You motherboard and CPU appear to be capable of VT-d according to Intel’s documentation, but either the BIOS does not enable it properly or there is a setting in the BIOS that you missed. Are you on the latest BIOS version for that motherboard?

EDIT: Did you load the required kernel modules to /etc/modules? And run update-initramfs -u before rebooting?
EDIT2: Sorry the f of the for in the command above got cut off.

@avw

Either the message is incorrect, which you can double check with: or d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf ‘IOMMU group %s ‘ «$n»; lspci -nns «${d##*/}»; done. If there are multiple groups it is actually working and the error message is wrong. If there is only one IOMMU group *, then it is not fully enabled in hardware or the BIOS.

i tried copying and pasting this command link and got a syntax error

-bash: syntax error near unexpected token `do’

Maybe IOMMU and/or VT-d is not fully enabled in the motherboard BIOS. You motherboard and CPU appear to be capable of VT-d according to Intel’s documentation, but either the BIOS does not enable it properly or there is a setting in the BIOS that you missed. Are you on the latest BIOS version for that motherboard?

I’m checking as i type on the bios, issue is that intel doesnt support this board so theres no updated bios, or any on their website anymore

im running the 0650 bios from 2014

EDIT: Did you load the required kernel modules to /etc/modules? And run update-initramfs -u before rebooting?

i ran this:
root@nexus:~# nano /etc/modules

this was the result (which was in the wiki)
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

then i updated with this
root@nexus:~# update-initramfs -u -k all

and the result was:
update-initramfs: Generating /boot/initrd.img-5.13.19-2-pve
Running hook script ‘zz-proxmox-boot’..
Re-executing ‘/etc/kernel/postinst.d/zz-proxmox-boot’ in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
root@nexus:~#

Sorry the first f of the command got cut off. It should be for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done.

EDIT: You can check if the modules are actually loaded using lsmod | grep vfio.

@avw

this is the result of for:
d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

IOMMU group * 00:00.0 Host bridge [0600]: Intel Corporation Xeon E5/Core i7 DMI2 [8086:3c00] (rev 06)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1a [8086:3c02] (rev 06)
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1b [8086:3c03] (rev 06)
00:02.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 2a [8086:3c04] (rev 06)
00:03.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode [8086:3c08] (rev 06)
00:03.2 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3c [8086:3c0a] (rev 06)
00:05.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Address Map, VTd_Misc, System Management [8086:3c28] (rev 06)
00:05.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Control Status and Global Errors [8086:3c2a] (rev 06)
00:05.4 PIC [0800]: Intel Corporation Xeon E5/Core i7 I/O APIC [8086:3c2c] (rev 06)
00:11.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Virtual Root Port [8086:1d3e] (rev 05)
00:16.0 Communication controller [0780]: Intel Corporation C600/X79 series chipset MEI Controller #1 [8086:1d3a] (rev 05)
00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection (Lewisville) [8086:1502] (rev 05)
00:1a.0 USB controller [0c03]: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 [8086:1d2d] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation C600/X79 series chipset High Definition Audio Controller [8086:1d20] (rev 05)
00:1c.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 [8086:1d10] (rev b5)
00:1c.2 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 3 [8086:1d14] (rev b5)
00:1c.3 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 4 [8086:1d16] (rev b5)
00:1c.5 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 6 [8086:1d1a] (rev b5)
00:1d.0 USB controller [0c03]: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 [8086:1d26] (rev 05)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev a5)
00:1f.0 ISA bridge [0601]: Intel Corporation C600/X79 series chipset LPC Controller [8086:1d41] (rev 05)
00:1f.2 SATA controller [0106]: Intel Corporation C600/X79 series chipset 6-Port SATA AHCI Controller [8086:1d02] (rev 05)
00:1f.3 SMBus [0c05]: Intel Corporation C600/X79 series chipset SMBus Host Controller [8086:1d22] (rev 05)
02:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host Controller [1033:0194] (rev 04)
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU116 [GeForce GTX 1660 SUPER] [10de:21c4] (rev a1)
03:00.1 Audio device [0403]: NVIDIA Corporation TU116 High Definition Audio Controller [10de:1aeb] (rev a1)
03:00.2 USB controller [0c03]: NVIDIA Corporation TU116 USB 3.1 Host Controller [10de:1aec] (rev a1)
03:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU116 USB Type-C UCSI Controller [10de:1aed] (rev a1)
04:00.0 SATA controller [0106]: Samsung Electronics Co Ltd XP941 PCIe SSD [144d:a800] (rev 01)
08:00.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6315 Series Firewire Controller [1106:3403] (rev 01)
09:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host Controller [1033:0194] (rev 04)
0a:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
3f:08.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link 0 [8086:3c80] (rev 06)
3f:09.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link 1 [8086:3c90] (rev 06)
3f:0a.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 0 [8086:3cc0] (rev 06)
3f:0a.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 1 [8086:3cc1] (rev 06)
3f:0a.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 2 [8086:3cc2] (rev 06)
3f:0a.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 3 [8086:3cd0] (rev 06)
3f:0b.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Interrupt Control Registers [8086:3ce0] (rev 06)
3f:0b.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Semaphore and Scratchpad Configuration Registers [8086:3ce3] (rev 06)
3f:0c.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0c.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0c.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0c.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 0 [8086:3cf4] (rev 06)
3f:0c.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 System Address Decoder [8086:3cf6] (rev 06)
3f:0d.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0d.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0d.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 06)
3f:0d.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 1 [8086:3cf5] (rev 06)
3f:0e.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Processor Home Agent [8086:3ca0] (rev 06)
3f:0e.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Processor Home Agent Performance Monitoring [8086:3c46] (rev 06)
3f:0f.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Registers [8086:3ca8] (rev 06)
3f:0f.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller RAS Registers [8086:3c71] (rev 06)
3f:0f.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 0 [8086:3caa] (rev 06)
3f:0f.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 1 [8086:3cab] (rev 06)
3f:0f.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 2 [8086:3cac] (rev 06)
3f:0f.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 3 [8086:3cad] (rev 06)
3f:0f.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 4 [8086:3cae] (rev 06)
3f:10.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 0 [8086:3cb0] (rev 06)
3f:10.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 1 [8086:3cb1] (rev 06)
3f:10.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 0 [8086:3cb2] (rev 06)
3f:10.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 1 [8086:3cb3] (rev 06)
3f:10.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 2 [8086:3cb4] (rev 06)
3f:10.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 3 [8086:3cb5] (rev 06)
3f:10.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 2 [8086:3cb6] (rev 06)
3f:10.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 3 [8086:3cb7] (rev 06)
3f:11.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 DDRIO [8086:3cb8] (rev 06)
3f:13.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 R2PCIe [8086:3ce4] (rev 06)
3f:13.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Ring to PCI Express Performance Monitor [8086:3c43] (rev 06)
3f:13.4 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 QuickPath Interconnect Agent Ring Registers [8086:3ce6] (rev 06)
3f:13.5 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 0 Performance Monitor [8086:3c44] (rev 06)
3f:13.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 1 Performance Monitor [8086:3c45] (rev 06)

lsmod | grep vfio:

vfio_pci 57344 0
vfio_virqfd 16384 1 vfio_pci
irqbypass 16384 8 vfio_pci,kvm
vfio_iommu_type1 40960 0
vfio 36864 2 vfio_iommu_type1,vfio_pci

-bash: syntax error near unexpected token `do’

The first character «f» is just misssing. Please compare my command execution and note the usage of «code»-tags for showing things like this:

root@pvec:~# for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU group * 00:00.0 Host bridge [0600]: Intel Corporation 8th/9th Gen Core Processor Host Bridge/DRAM Registers [Coffee Lake] [8086:3e33] (rev 07)
00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
...
...

(Just showing the syntax, not a problem or a solution.)

Best regards

It’s just a single group, so the message is right and IOMMU is not turned on. That means VT-d is not (fully) enabled in your motherboard BIOS.
Do you have a link to the manual of your motherboard?

Hi leesteken,

my Hardware is a Dell T5810, i swiched every option for VT-D on what i have found
CPU is a E5-2697v3

Virtualization Support

• Virtualization
• VT for Direct I/O
• Trusted Execution

Here you can find all Options

https://www.dell.com/support/manual…842e5c-f56b-48e9-811c-9994145825fd&lang=en-en

Still could not start my VM: TASK ERROR: cannot prepare PCI pass-through, IOMMU not present

vfio_pci 16384 0
vfio_pci_core 73728 1 vfio_pci
vfio_virqfd 16384 1 vfio_pci_core
irqbypass 16384 3 vfio_pci_core,kvm
vfio_iommu_type1 40960 0
vfio 45056 2 vfio_pci_core,vfio_iommu_type1

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUS_DEFAULT=»quiet intel_iommu=on»
GRUB_CMDLINE_LINUX=»»

VM Conf
args: -cpu ‘host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off’
bios: ovmf
boot: order=ide2;scsi0
cores: 4
cpu: host,hidden=1,flags=+pcid
cpuunits: 1024
efidisk0: DATASSD:vm-109-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:03:00,pcie=1
ide2: nas-lvm:iso/Windows10.iso,media=cdrom,size=4824896K
machine: pc-q35-6.2 (tried also q35 only)
memory: 4096
meta: creation-qemu=7.1.0,ctime=1675969002
name: VD04GPU
net0: virtio=52:A1:67:52:3D:CB,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: DATASSD:vm-109-disk-0,cache=writeback,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=fffe13df-e9b9-4ce0-bb5c-61c1ca72c1de
sockets: 1
vcpus: 4

I am thankful for every help!!

Sounds like an Intel processor.

You have not added intel_iommu=on as explained in the Proxmox manual, that’s why IOMMU is not working for you.

Thanks alot!
now it looks like it works

finaly editing the kernal commandline

nano /etc/kernel/cmdline ADD intel_iommu=on

than updating

proxmox-boot-tool refresh

dmesg | grep -e DMAR -e IOMMU
[ 0.011354] ACPI: DMAR 0x00000000A97EDAB0 0000DC (v01 DELL CBX3 00000001 INTL 20091013)
[ 0.011385] ACPI: Reserving DMAR table memory at [mem 0xa97edab0-0xa97edb8b]
[ 0.098610] DMAR: IOMMU enabled

VM also starting

it seems it’s not the complete output, either you did not capture the output completely
or it already rotated out.
it any case, i think i’d would be better to use ‘journalctl’ and pipe it into a file:

journalctl -b > /tmp/journal.log

and then upload the file here