Re: Qemu/nvmm - time in NetBSD guest system lags behind (with estd on host)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Wed, 31 Aug 2022 11:29:06 +0200
    From:        Matthias Petermann <mp%petermann-it.de@localhost>
    Message-ID:  <f7819d97-ea8f-96fe-8c7d-47db993cef88%petermann-it.de@localhost>

  | The guests' time increasingly lags behind with continued operation. Also 
  | the ntpd seems to have no compensating effect in the guests here.

This is a well known issue, ntp in the guest cannot help, the clock
drift is too much for its algorithms to handle.

  | What could be the reason for this? Can estd be a source of interference?

On estd, no.   The problem is that the qemu guest is running using time at
100 Hz, it expects a clock interrupt every 10ms.   To make that happen,
qemu (effectively) sleeps for 10ms, then signals a clock interrupt to the
guest.   But the host running qemu is also running with a clock at 100Hz.
When a process asks to sleep for 10ms, it needs 2 clock ticks to occur,
one which might happen an instant after the sleep request (be just 1us
or something since the request) and so would not be long enough, and another
10ms later - then we know that 10ms has passed.

If it happens (and always happened) just that way, all would be close enough
(and NTP would cope in the host with any minor time shift that occurs) but
what actually happens is that qemu wakes up from its 10ms sleep (or gets
a 10 ms SIGALRM - that difference doesn't matter) and as well as signalling
its guest, immediately requests a new 10ms sleep, for next time.

Here, rather than being 1us before the next clock intr happens, it is
more likely to be 1us after the previous one happened, ie: 9.999 ms
until the next one happens - we wait that long, then 10ms more, and the
sleep finishes - just about 20ms elapses to do one 10ms sleep.   The
guest is getting one clock interrupt every 20ms, but believing that 10ms
has elapsed, as that's what it requested.

Or that is how I understand it from what has been explained to me - the
details might not be exactly right (and I've never looked inside qemu)
but that's more or less the effect and its underlying cause.

  | All I could find so far is [1]. It is recommended to add the rtc switch 
  | to the qemu command. Is there any recommendation here in the meantime 
  | which setting works best with NetBSD?

About the rtc, no no idea.   But to deal with the problem, aside from
major NetBSD code rewrites (the so called tickless kernel) the one
solution that should work is to run the host with HZ set a lot higher,
and leave the guest(s) at 100Hz.

For any modern host (anything you'd really want to use to run a qemu
guest in production) running with HZ=1000 will be fine (you'll never
notice the tiny extra overhead).   Some of the NetBSD ports already
run at that kind of rate - alpha has been at 1024Hz forever (and these
days, alphas are slow processors - though they weren't compared to
others when that change was made).  With this, the 10ms interrupts might
actually occur about 10.1 ms apart, but that much drift NTP should be
able to handle.   If not, run the host with an even higher HZ rate,
even 10000 should work with a modern amd64 CPU (though I have never
tested that, nor heard of anyone who has - but 2000 should not be an issue).

If for some reason you cannot change the clock rate of the host (that is,
compile a new kernel with "options HZ=1000" in the config file) then make the
guests run with a much slower clock rate - nothing faster than 50Hz.

That should be acceptable (pdp-11's used to run at 50 or 60hz, and worked
OK) but needs to be even slower for clock drift issues.   The problem
is that if the OS clock rate is too slow, it will start to impact upon
(perceived) performance, and some application capabilities.

kre