Re: nvmm users - experience

[Reinoud Zandijk <reinoud%NetBSD.org@localhost>]

Hi,

On Sun, May 21, 2023 at 02:01:22PM +0000, Mathew, Cherry G.* wrote:
> I'm wondering if there are any nvmm(4) users out there - I'd like to
> understand what your user experience is - expecially for multiple VMs
> running simultaneously.
> 
> Specifically, I'd like to understand if nvmm based qemu VMs have
> interactive "jitter" or any such scheduling related effects.
> 
> I tried 10.0_BETA with nvmm and it was unusable for 3guests that I
> migrated from XEN, so I had to fall back.
> 
> Just looking for experiences from any users of nvmm(4) on NetBSD (any
> version, including -current is fine).

I'm a regular user of nvmm(4) but normally only use one nvmm/qemu instance at
a time. I think your assessment is correct in that its scheduling related.
This explains both the jitter and the irregular behaviour.

AFAIK in nvmm(4) the code runs until it reaches a page fault, a read/write of
a memory mapped device or some other event and then returns to Qemu. There is
AFAIK no timer involved and as long as the scheduled CPU doesn't get an
interrupt or hits a page fault it just continues on. Code that spins will thus
neglect virtual irqs generated until some random event or when the NetBSD
scheduler decides its time to yield this CPU. This can be considered a bug :)

I don't see references to timers being used in sys/dev/nvmm/ nor are CPU
specific timer/preemption support bits set.

With regards,
Reinoud