Which virtualization implementation? QEMU/KVM?
this post was submitted on 14 Nov 2025
9 points (90.9% liked)
Linux
67891 readers
693 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 7 years ago
MODERATORS
libvirt qemu/kvm.
I'm runnign into virtio bottlenecks I cant get out of now too. I cant do PCI passthru on this unless I give up the GPU on this board because I didnt consider that when I bought the hardware -_-
You can passthrough more than one device at a time. This is probably the source of your slow speeds.
Add a virtual nic to the opnsense VM and bind it to the host bridge, that way you can use both the VM network and the physical LAN without a lot of faf.
This board will not let me do pci passthru. Only the GPU slot and one of the m2 slots are in seperate IOMMU groups. Everything else is just dumped into the same IOMMU group.
Then you will have to do it with bridging.
Create a second bridge and bind only the wan interface and the physical interface of opnsense to it. Then create a 2nd interface for opnsense, but bind it to the brIdge libvirt uses to connect its guests.
Looking deeper into it, this is kinda what its been falling back to since the passthru wasnt working even though it was offered as an option in cockpit and it threw no errors trying to do it. I put the lan side in a bridge and tied the hosts stuff in that way, then put the wan on a macvtap to the wan. that working but performance isnt great. I ran some tests today against bare metal and while direct access to the NIC certainly improves things, its still not keeping up.
direct macbook to cable modem: 916/40 opnsense virtualized (with vlans and rules): 699/41 opnsense bare metal (with vlans and rules): 816/39 opnsense bare metal (with vlans and rules and hardware offload fully enabled): 824/40
the only rules in place were the defaults, the rule to block vlans from talking to eachother, and the rule to pass traffic to WAN. when virtualized, I cannot get PCI passthru so I was using macvtap interfaces and virtuio drivers with 4 threads and 4 pinned CPU threads.
CPU is a ryzen 5800XT NIC is a dual port intel I226V when virtualized, it was running under fedora server with QEMU/KVM q35 and given 8gigs of ram with hugepage memory and tested in both 2 and 4 thread resource allocation (all confirmed to be on the same 1 or 2 physical cores as the threads) and eventually even giving 4 threads to the virtuio driver (it was only claiming 1 thread before).
Bare metal IS definitely helping, so it looks like I need to swap out for a motherboard that can do proper PCI passthru of the NIC (now that I understand the limitations of IOMMU groups they specs of the board dont tell you about I hate them all the more.) but it still cant hit line rates. Theres no IDS or suricata or any of the fanciness turned on yet though, so I just dont understand why its this slow even bare metal.
also, I have isolated and devoted 2 threads, 1 core, of a ryzen 5800XT CPU to this VM and my WAN download traffic wont exceed 625mbps! I dont even have IDS or anything enabled yet. How can that be?
added another 2 threads/1 core to the pile. no change. I cant hit gigabit down on this thing.