Subject: Re: interrupts (was Clockticks lost) (long)
To: Erik E. Fair <fair@clock.org>
From: Bill Studenmund <wrstuden@loki.stanford.edu>
List: port-mac68k
Date: 01/25/1997 18:17:41
> 
> Bill,
> 	I readily admit that Apple made mistakes with the Macintosh's basic
> design. You should have heard the screams of frustration from the poor guy
> I know inside Apple who tried to implement full NTP for MacOS, only to
> discover the limitations of the completely suckful clock hardware Apple
> uses. Sub-second accuracy for that RTC chip is essentially impossible.
> 
> However, independent of the bad hardware question, there are things that we
> do in software that make things worse, i.e. disabling interrupts for "long"
> periods of time. This is an issue not just of good timekeeping, but of
> general system responsiveness - the less we do it, the snappier the system
> will respond to all interrupts, across the board.

I most heartily agree! I've actually been wondering if there are things
we can do to make all the I/O operations which would block interrupts
keep other services alive. Like how the modem port's kept alive w/ Localtalk
under MacOS.

> I recognize the need to serialize operations on certain kernel data
> structures to prevent inconsistent modifications, which would, in the long
> run, lead to corrupted data structures and to crashes. So, when we know
> that two sections of code modify the same data structure, one at an
> interrupt level, and one not, the one not at interrupt level is wrapped in
> an interrupt disabler. However, it is precisely in these sections of code
> (both at interrupt level and not) that we should think *very* carefully,
> and strive to absolutely minimize the instructions executed in such
> critical sections.

Hmm. I thin the current interrupt model has worked around this problem
by using "silo"s. For instance, the serial driver has an at-interrupt-
time part, and a not-at-interrupt-time part. They communicate via a
ring buffer and two pointers into the ring (one for each side). The
interupt-time pointer is marked volatile. Since they will each write
to different places, and the non-interrupt-time code doesn't cache the
interrupt-time pointer, it works ok.

I don't know how the other drivers work, so I can't comment on how they
deal with such concerns.

There are two other uses for blocking interrupts (I'm getting detailed
as I'm very interested in this subject, and hope we can figure something
out):

Twiddling the hardware (which we can't get around), and keeping other
parts of the code from messing with certain structures by preventing
a context switch out of the executing code.

For example, the SCC serial chip uses a two-access scheme to get at the
chips registers (first = address, second = data). Thus you can't have
the interrupt servicer come along during an access.

But I wish we could do something about the latter type of blockage.
spltty is one example. It's used so that only one piece of code is dorking
around with the tty read & write queues. But I think you really only need
to lock the one queue you are processing. Switching to semaphores would
be great here. Same with splsoftnet.

N.B. I'll admit that the last two sentences represent a LOT of work which
I'm not in a position to perform. :-)

> The amusing thing is that we're going to have to face this issue from a
> completely different angle - multiprocessor system support. Disabling
> interrupts won't help there - we will need a non-interrupt-based semaphore
> system to serialize access to kernel data structures for multiple
> processors if we have any hope of doing SMP. I think SMP a worthy goal -
> cheap MP systems are becoming available, e.g. PowerMac 9500/180MP, DayStar
> Genesis MP, Sun SparcStation 10's & 20's (getting cheap on the used
> market), to name a few; hell, if you wanna get really bizarre, I wonder if
> it's possible to run both CPUs in an "accelerated" Mac (e.g. an SE/30 with
> a faster CPU in the PDS). I'd like to be able to use their full power with
> an SMP NetBSD.

I AGREE!!!! When I get my PowerMac (a CHRP or PPCP or whatever) w/ ISA,
I'm going to look at options for adding a second CPU, in the hopes
of getting NetBSD up on a multi-head. :-)

There's a lot of work between here and there. Maybe one step would be
to agree on some sort of semaphore system, or some general framework
for SMP, even if it's a ways off.

I'm not sure about the SMP SE/30, though, as I'm not sure if the accelerators
have the hardware to let both CPU's run at once, and can coordinate boot
up. I'm not sure how exactly an SMP machine boots.

> Also, some things simply require guaranteed real-time response from the
> system (e.g. writing a CD-ROM with a CD-R drive, or schlepping bits to an
> audio device for playback) to work really right. We're going to have to
> clean up not just the device drivers (frankly, it won't be enough, long
> term, to simply say, "Well, then, don't use the tape drive when you're
> writing a CD-ROM!"), but also modify the scheduler so that, aside from the
> entirely too simple "nice" facility, the system can guarantee an
> arbitrary-sized CPU time slice at particular intervals to one or more
> processes. There have been quite a few papers given at USENIX over the
> years on this topic.

Have you been reading current-users? :-) There's been a thread about
getting libpthreads into the kernel, and also one about CD-R support
(which might mean some sort of real-time process a la Linux), in the
last week. Also, there are a number of folks who have multi-headed
boxes (i386's and I think a Sparc) who have posted before. There's
definitely interest out there.

Unfortunately I don't think this list (port-mac68k) is the best one for
much of this discussion. Not because folks aren't interested in SMP stuff,
but because many changes will have to happen in MI parts of code that all
NetBSD folks need to be involved. :-)

Take care,

Bill