配置网卡、硬盘直通可以增加硬盘写入速度、网卡减小延迟、增加访问速度,避免因为虚拟化之后的资源浪费。如果没有配置好pve环境与BIOS,添加设备时会出现No IOMMU detected, please activate it.See Documentation for further information.的报错,启动虚拟机失败:
提示如下:
Task viewer: VM/CT 108 – 控制台输出状态停止VM 108 not running
TASK ERROR: Failed to run vncproxy.
Task viewer: VM 108 – 启动输出状态停止TASK ERROR: IOMMU not present
下文记录如何进行pve主机与虚拟机配置:
一、首先配置主板的BIOS
在北桥配置里面配置 iommu mode 为enable
或者找到inter-vt 选择enable
二、配置pve系统:
1、将/etc/default/grub配置文件中
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
改为
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
如下所示:
root@gotohome:~# more /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX=""
# Disable os-prober, it might add menu entries for each guest
GRUB_DISABLE_OS_PROBER=true
# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console
# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
2、刷新文件与重启系统
root@gotohome:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.15.18-12-pve
Found initrd image: /boot/initrd.img-4.15.18-12-pve
Found memtest86 image: /boot/memtest86 .bin
Found memtest86 multiboot image: /boot/memtest86 _multiboot.bin
done
root@gotohome:~# reboot
3、使用dmesg | grep -e DMAR -e IOMMU命令检查是否成功,有多行反馈说明没有问题。
root@gotohome:~# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x000000007C1ED7E0 0000BC (v01 A M I OEMDMAR 00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.004000] DMAR: Host address width 46
[ 0.004000] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.004000] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0462 ecap f020fe
[ 0.004000] DMAR: RMRR base: 0x0000007c673000 end: 0x0000007c681fff
[ 0.004000] DMAR: ATSR flags: 0x0
[ 0.004000] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[ 0.004000] DMAR-IR: IOAPIC id 0 under DRHD base 0xfbffc000 IOMMU 0
[ 0.004000] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0
[ 0.004000] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.004000] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.004000] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 1.394789] DMAR: dmar0: Using Queued invalidation
[ 1.394807] DMAR: Setting RMRR:
[ 1.394957] DMAR: Setting identity map for device 0000:00:1a.0 [0x7c673000 - 0x7c681fff]
[ 1.395128] DMAR: Setting identity map for device 0000:00:1d.0 [0x7c673000 - 0x7c681fff]
[ 1.395154] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 1.395296] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 1.395315] DMAR: Intel(R) Virtualization Technology for Directed I/O
root@gotohome:~#
如果是AMD的CPU:详细可参考官方文档: https://pve.proxmox.com/wiki/Pci_passthrough#Enable_the_IOMMU
For AMD CPUs add “amd_iommu=on”, for example:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
Safe the changes and update grub:
update-grub
三、配置硬盘直通:
1、查看硬盘状态,记录序列号:
root@gotohome:~# ls /dev/disk/by-id
ata-HGST_HTS721010A9E630_JR1004D32B42HM
ata-KINGSTON_SA400S37240G_50026B76822ED9B7
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part1
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part2
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part3
dm-name-pve-root
dm-name-pve-swap
dm-name-pve-vm--100--disk--0
dm-name-pve-vm--101--disk--0
dm-name-pve-vm--101--state--LNMP
dm-name-pve-vm--102--disk--0
dm-name-pve-vm--103--disk--0
dm-name-pve-vm--104--disk--0
dm-name-pve-vm--104--state--base
dm-name-pve-vm--104--state--h5ai
dm-name-pve-vm--104--state--h5aizabbix
dm-name-pve-vm--104--state--zabbix
dm-name-pve-vm--104--state--zabbixh5ai
dm-name-pve-vm--104--state--zabbixnginx
dm-name-pve-vm--105--disk--0
dm-name-pve-vm--105--state--chushi
dm-name-pve-vm--106--disk--0
dm-name-pve-vm--107--disk--0
dm-name-pve-vm--108--disk--0
dm-name-ydcdata-hcdn
dm-name-ydcdata-pq_pcdn
dm-name-ydcdata-pq_pcdn.1
dm-uuid-LVM-fF8kU0DUvShjs7tjMqKIDs6G1CrZPZpEACVfkc9WihsBxVOdkSWDuUyOkT906XJL
dm-uuid-LVM-fF8kU0DUvShjs7tjMqKIDs6G1CrZPZpEDIgYxvfu2Y4fBK429jn2HycL8uqGb0ku
dm-uuid-LVM-fF8kU0DUvShjs7tjMqKIDs6G1CrZPZpEPnh0qmVZ0VfbwXlq7yyefamgpXRczsda
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW0EdoeXM5j31YC2l1fGtiQdnKFwGEoFBW
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW2QRntRG923snAxETGxCd5rNmjKBdcs8C
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW3eVHvuuXjhlN9rFwD7K7sBd1S3KIVcRO
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW57qXdqlWQlXcDsWti3Tq35GcByBvuRLq
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW8WG6sXdjOnTDGOMinroDSYFRWem5CqEZ
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW8ZyYMZcepidmYvfHAdGOoRf5Aa4bgP9u
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzW9rJQGzpWpGNeu5oUPxz7nDIp75TtNDFF
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWDlCw7G96BdI7rXzIWGukLgunQTNJ7Vy0
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWEmWf2FUpg6esAv2JwqhRbJL0CvzWZSsr
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWexQNMhVax0CP9DviAfa0mtfqXftBLvRs
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWjStkELRv2p6Rl4RBsnNXc5dKN8sjkOQM
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWLwgzkoaOtyhfxA8VmlPSF6eAUlCQ39Kk
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWqTwTnrdfd6BViHJAu2lES3kHgLjsjofP
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWtwDXrkxO5Eh6dJQDgUcloSSWdZHEpM2L
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWvsiEzEBzMnEW7Rqoi7n8mtfBNNVebV8l
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWWGvsCxqpAli7CY3AI6wSJgYciuTdDfYb
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWYIR0kwKTaHpjO0krvV60Fez0ObpUSKpX
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWYuwWXUXdFJQjnU3Ja0Q3bBvrmnpzHOsA
dm-uuid-LVM-OdLNWwcW08dmyc63EZIDhWBQBrHuzdzWZ6c6c0sLffp7onkOW9kvUPnXEpEDs4hF
lvm-pv-uuid-Dk2HcH-9kM4-20D8-TqWy-qHEX-q2kv-PR1Cob
lvm-pv-uuid-M2JekT-FH16-gVn2-PG8R-KVLl-h1ND-H4AcIb
wwn-0x5000cca8c8e13e68
wwn-0x50026b76822ed9b7
wwn-0x50026b76822ed9b7-part1
wwn-0x50026b76822ed9b7-part2
wwn-0x50026b76822ed9b7-part3
root@gotohome:~#
可以看到下面是存在的硬盘,显示硬盘品牌下面我把 ata-HGST_HTS721010A9E630_JR1004D32B42HM 与虚拟机直通
ata-HGST_HTS721010A9E630_JR1004D32B42HM
ata-KINGSTON_SA400S37240G_50026B76822ED9B7
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part1
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part2
ata-KINGSTON_SA400S37240G_50026B76822ED9B7-part3
2、确定虚拟机的id,如下图所示,100-108即为虚拟机的id
3、配置硬盘与虚拟机直通:
qm set 108 -sata1 /dev/disk/by-id/ata-HGST_HTS721010A9E630_JR1004D32B42HM
上面命令中,108为虚拟机id;ata-HGST_HTS721010A9E630_JR1004D32B42HM为上面第一步中找出要直通的硬盘
4、确认直通成功,硬盘sata1即为直通的硬盘,完成配置。
四、配置网卡直通:
1、在虚拟机硬件选项中,选择添加设备,PCI设备。
2、选择对应的网卡和端口:
3、检查是否添加成功:
最后确认是否需要重启虚拟机,建议在关闭虚拟机的情况下进行添加设备。