I would like to propose that we stop using the xen_system_time
timecounter in NetBSD dom0, i.e. in XEN3_DOM0.
As I wrote recently in the ongoing "timekeeping regression?" thread,
xen_system_time is really just a wrapper on using the CPU TSC as a
timecounter.
However in dom0 the RDTSC instruction is _not_ emulated[1], and, as I've
observed, having multiple vCPUs assigned to dom0, but not pinned, on a
machine with unstable TSCs, makes xen_system_time unstable and this of
course leads to timekeeping problems.
So, forcing the use of raw/un-emulated RDTSC on systems where it would
not normally be used if the system were running on bare metal seems
wrong.
Workarounds such as pinning dom0's vCPUs, or assigning only one vCPU,
are just that, poor workarounds (and possibly not even 100% reliable).
If we stop registering xen_system_time in dom0s (and do register TSC)
then the system will choose whatever timecounter is best for the
available hardware (just as if it were running on bare hardware).
I think, if I'm not missing something else important, that this _could_
also allow for significant simplification of the code in xen_clock.c.
On the other hand I say this as someone with no experience running
NetBSD (and Xen) on CPUs with TSC_INVARIANT support. Perhaps on such
newer CPUs it would make sense to allow domUs to use "native"
(un-emulated) RDTSC for a TSC timecounter (which could then have a
frequency of >1GHz). If domUs with tsc_mode=native, and/or with a TSC
timecounter available are desirable then xen_clock.c cannot be
simplified as the TSC will then not be fixed at the emulated 1GHz
frequency. See also [2].
Would any of this change for a PVH dom0? I don't know, but I think I
can say that a PVH kernel running in a dom0 should not use/register
xen_sytem_time either (see also [1]).
Note this all (probably) does not have any relation to the timekeeping
problems some of us have observed in domUs (where RDTSC is emulated).
[1] In the Xen code it doesn't look impossible to give a PV dom0 support
for tsc_mode=emulate, however currently there is code at the top of
time.c:tsc_set_info() that blatantly clears the TSC emulation flag
for the PV (and PVH) hardware domain (which is usually dom0) so for
now it is baked in that dom0 will never get RDTSC emulation.
[2] Note there's a comment in the Xen code about TSC emulation possibly
changing if a domU is migrated to another host, but of course a dom0
cannot be migrated.
--
Greg A. Woods <gwoods%acm.org@localhost>
Kelowna, BC +1 250 762-7675 RoboHack <woods%robohack.ca@localhost>
Planix, Inc. <woods%planix.com@localhost> Avoncote Farms <woods%avoncote.ca@localhost>
Attachment:
pgplbRRAPr9Lh.pgp
Description: OpenPGP Digital Signature