Subject: HEADS UP: merging the newlock2 branch
To: None <current-users@netbsd.org>
From: Andrew Doran <ad@netbsd.org>
List: current-users
Date: 02/08/2007 15:20:15
Hi all,

This is a heads up to say that I'm going to merge the newlock2 branch into
-current tomorrow (Friday) at 20:00 UTC, and will send an announcement to
this list once complete. I have gotten a few questions about what it
provides and what the status is so here is a brief outline:

o It provides a new set of synchronization primitives in the kernel designed
  to make programming for multiprocessor systems easier and more efficient:
  mutexes, reader / writer locks, condition variables, sleep queues and MI
  memory barrier operations. A number of underlying kernel facilties have
  been made 'multiprocessor safe' including the scheduler, ktrace and the
  general purpose method of kernel synchronisation: sleep & wakeup.

o Some application facilities have been made MP safe and can now run without
  the "big lock" on multiprocessor systems, including signalling, SysV
  messaging, and system calls that inspect process state, for example:
  wait(). This increases the number of system calls that will run without
  the big lock from 1 up to 56, with more in the pipeline. For workloads
  that are fork intensive and make heavy use of signals this will show a
  small yet quantifiable benefit on multi-way systems.

o The branch introduces a new 1:1 threading model that allows multithreaded
  applications to take advantage of all available CPUs in a multi-way
  system. The scheduler activations implementation used from NetBSD 2.0
  through NetBSD 4.0 provides execellent performance on single CPU systems,
  but restricts any instance of a threaded application to a single CPU in
  the system. Given that multicore and multi-CPU systems are increasingly
  commonplace and that single threaded CPUs are rapidly disappearing from
  the market, we made the decision to move to a new threading model, on the
  basis that providing increased concurrency is now the most important
  factor in ensuring good performance for threaded workloads.

There are number of side effects that merging the branch will have, which I
will try to note:

o A number of ports will be broken by the merge: those that do not implement
  the generic soft interrupt facility, and those that not yet been brought
  into line with the machine independent kernel changes. Assuming that they
  provide generic soft interrupts, I belive that ports based on the
  following CPUs have all of the necessary changes and should work well:

	i386, amd64, sparc64, mips, powerpc, sparc, alpha

o Uniprocessor alpha systems will work well. There is currently a problem
  that prevents multiprocessor alpha systems from booting, which I am hoping
  will be resolved soon.

o Changes for systems with the following CPUs are in the pipeline, but not
  yet complete:

	arm, m68k, sh3, hppa

o After the branch is merged, up to date libc and libpthread libraries will
  be required for threaded applications to work. More simply put, this means
  that you will need to update your userland _and_ kernel in step if you
  want threaded applications to work as normal.

o While every effort has been made to test the changes introduced by the
  branch, they are significant in number and I fully expect that bugs will
  be uncovered. If you do run into problems then please make sure to log
  them via send-pr. If you're not able or willing to run a bleeding edge
  system, then it may be worthwhile holding off until any initial problems
  have been resolved.

Thanks,
Andrew