Subject: How u. was lost
To: None <port-vax@netbsd.org>
From: Toru Nishimura <nisimura@itc.aist-nara.ac.jp>
List: port-vax
Date: 06/06/2000 08:08:06
Here is the short memorandum once I referred to.

//// How the u. was lost ////

USPACE, used to be commonly referred as "u.",  is a twilight zone where
two different lights shine on sky belonging to distinct universes
in the same time.  It's a part of user process address space as well as
a part of kernel address space.  It holds processes' kernel stack, a
set of CPU register copy and flags.  USPACE, in whole, can be
considered as kernel runtime image or snapshot.

Traditional UNIX design used to have a special address constant UADDR.
UADDR was the address where 'struct user' variable was populated.  The
structure member could be handsomely referred by "u.<member>"
notations.  'struct user' was a large store which contains various
information and data necessary during processes' life time.  They all
were local to individual processes.

'struct proc' is another important store which describes individual
processes' natures and features.  Why are the two, 'struct user' and
'struct proc', separate structures exist for every single process? It's
because of consideration about memory consumption; 'struct proc' is
memory resident while USPACE which holds 'struct user' and kernel stack
can be swapped out to backing store just like as the contents of
processes' user address space can be moved away.   To keep 'struct
proc' small as possible, 'struct user' held everything local to process
and unnecessary for book keeping purpose while the process is swapped
out.

"u.<member>" was the major reference point of many kernel resources.
Frequently used 'struct proc' pointer for current process is referred
as u.u_proc.  It was necessary to understand how 'struct user' was
designed and used.  'struct proc' was the next one to be investigated.
In some senses, UNIX kernel was structured around "u." structure.

'struct user' was populated the lower end address of USPACE while
process kernel stack was placed at the opposite end of USPACE.  For
UNIX implementation whose entire address space was divided into 2
portions, one was in lower for user address space and another
in higher for kernel address space, USPACE was commonly layouted at
above user process stack and below kernel text starting address.

u.u_arg[] is for the temporal copy of system call arguments.  Some
UNIX implementation may have all of system call arguments in calling
user process' kernel stack as a part of trapframe, or other may have
all or part of system call arguments left on call process' user stack
before kernel starts system call processing.  In either case, system
call handler must prepare u.u_args[] array to have system call argument
values appropriately, by copying trapframe contents , or by reading
system call arguments left on user stack.  All of system call bodies
refer to u.u_arg[] as the start address of system arguments.

u.u_rval1 and u.u_rval2 are for system call return value.  The 2nd is
only used for limited number of system calls; lseek(2), pipe(2) and
fork(2).

u.u_error is the indication of error during system call processing.
When kernel is about to return to the calling address of user process,
u.u_error is consulted and several registers on trapframe are
overwritten appropriately to return the condition of system call
processing.   Carry flag in a special register was commonly used for
the system call error indication.  On successful return, two of
registers in trapframe contain the values left in u.u_rval1 and
u.u_rval2.  On error return, carry flag is set and error value left in
u.u_error is copied on the primary register for system call return
values.

"u.<member>" has several stores for non local jump in kernel address
space.  They are kinds of jmp_buf[] stores to hold selected number
of registers.   They are used for multiple purposes.  Process
context switch uses most frequently.  Another occasion is somehow
obscure because the store is used to hold the check point location
of system call entry whose address is longjmp destination when system
call processing waiting for event is interrupted by any reason.

Non local jump inside kernel address space was one of most complicated
nature of UNIX internal.  Combined with the way processes are arranged
by UNIX memory management, it's hard to track and understand the
control flow; it was a sort of magic.

4.3BSD-Reno release made a large restructuring on the interwinded
callee/caller relationship.  "u.<member>" referendum was reduced
greatly.  System call input arguments and return values were changed,
for simplicity, stored in arrays on kernel stack and passed to system
call processing routines.  Nothing peculiar because it just follows a
standard calling convention.  Non local jumps has been completely
eliminated.  In principle, only two locations make context switches;
tsleep() and the place where process returns to user mode.  The former
is the occasion of current process yields CPU ownership to other
processes waiting for an event to happen in future.  This is a
voluntary context switch.  The latter, lately named preempt(), is the
place for current process to check to see whether it is requested to
yield the CPU ownership.  Reasons vary; the process ran out processing
tick assigned for it and the scheduler marked it to abandon CPU, or
urgent condition was detected so far and the proper action were
prompted as soon as possible.  This is a preemption to yield control
flow.  Both cases are designed to call mi_switch() which is the sole
routine save and restore jump_buf[] store placed inside 'struct user',
named 'struct pcb', for an encapsulation for non local jump.

Given trends toward larger and luxuriant memory capacity, "u.<member>"
are relocated to 'struct proc' gradually.  'struct user' is now small
enough only to hold jump_buf[] store for context switch, and runtime
statistic book keeping record.  Newer kernel is considered structured
around curproc global variable as most of kernel resources can be
located via dereferencing pointers from curproc.

It does not make sense any more to keep the once bold design decision
behind to map 'struct user' at fixed location of UADDR.   Processes
have USPACE, 'struct user' and kernel stack, at distinct locations in
kernel address space, thus all of USPACEs coexist simultaneously at the
time.

Eventually "u.<member>" notation, the prominent characteristics of
UNIX kernel implementation, has passed away with UADDR constant.

--