Re: Xen time issues (Re: HEADS UP: timecounters (branch simonb-timecounters) merged into -current)

[Frank Kardel <kardel%netbsd.org@localhost>]

Jed Davis wrote:

> Frank Kardel <kardel%netbsd.org@localhost> writes:
>
>
> []
>
> > [Keep in mind a specific environment: AMD 64 X4, gcc4, -march=athlon.
> > Intel seems to do fine given the reports in teck-kern@.]
>
> Somewhat fine; I've had problems with TSC stability with
> hyperthreading enabled -- independent of Xen, but a domU has nothing
> else to use for timing (and I believe Xen itself makes use of TSCs).
Hyperthreading shouldn't be an issue for AMD. But I have seen
reports about AMD TSC acting funny in some chips and environments

> > That confuses me. xen/xen/clock.c:xen_timer_handler does assume that
> > process_system_time is always less then shadow_system_time. If that is
> > violated delta becomes negative.
>
> It's (shadow_system_time + get_tsc_offset_ns()), which is -- assuming
> a reliable TSC -- the system_time as of right now, which is expected
> to come after processed_system_time.  More to the point, it's expected
> to be monotonically increasing, and thus greater now than when
> processed_system_time was last set to something <= it.
I am fine with the assumptions and they are in the code. They are a valid
model for time keeping.

Remark: A conceptual problem here is that elapsed time
(frequency derived) is mixed with the UTC time abstraction that 
sometimes (@start)
needs to be set to a sane value. The good news is that timecounters 
start out
with the lesser abstraction (fixed frequency counter) but they require the
monotonicity property.

> Clearly that's not always the case, though I'm not sure how.
This is what causes the trouble. I have the impression that
there is something lurking that leads to a violation of the
monotonicity assumption. I think we need to examine that
part very closely as it would give also a clue how to get
the timecounter abstraction right (see previous remark).

[...]

> > furthermore the negative delta will be assigned to ci->ci_cc.cc_denom.
> > Is this wanted?
>
> I'm not sure what else can be done there -- the underlying time source
> has gone backwards.
we should prevent that if possible. Not having looked into xen
itself: what manages the (shadow)system_time the hypervisor
or dom0 or a combination of those two?
Did anyone look how fbsd manages that - I haven't yet.

>  What does the timecounter framework do when a
> timecounter steps backwards or otherwise gives obviously wrong output?
It gets confused to put it mildly. 'negative numbers' will look like
jumps into the future, smaller counts will move the time backwards.
clockinterrupts will pick the current possibly wrong reading as the next
base for interpolation. so it is confusion at best, but certainly not a 
clock.

I think we need to find out why apparently the xen time scale
(shadow_system_time + get_tsc_offset_ns()) sometimes goes backwards (in

some environments at least).

Frank