> Date: Sat, 30 Dec 1995 16:10:28 -0500
> To: tech-kern@NetBSD.ORG, Jason Thorpe <thorpej@nas.nasa.gov>,
>         Chuck Cranor <chuck@maria.wustl.edu>
> From: chopps@water.emich.edu (Christian E. Hopps)
> Subject: Re: found anoncvs problem . . .
> 
> Note: I have cross-posted this to tech-kern as some of it might be of interest to non-m68k ports...
> 
> ...
> Regarding USRSTACK :)
> 
> Notice that the amiga port uses a much smaller USRSTACK than the hp300.  This was originally done by Markus Wild so as to be more compatible with SunOS.  I was going to change it (not seeing the large benifit in this compatibility) and then I discovered how usefull it was to leave it lower.  It free's up a bunch of phsycial memory for the system to use.  I'll demonstrate, first the amiga and hp300 pte.h constants (I list the amiga defines so you can better see the relationships they are the same for the hp300):
> 
> #define USRSTACK        0x0E000000
> #define VM_MAXUSER_ADDRESS      ((vm_offset_t)(USRSTACK))
> 
> #define AMIGA_UPTSIZE           roundup(VM_MAXUSER_ADDRESS / NPTEPG, NBPG)
> #define AMIGA_UPTBASE           0x10000000
> #define AMIGA_UPTMAXSIZE \
>     roundup((AMIGA_MAX_COREUPT * AMIGA_UPTSIZE), NBPG)
> #define AMIGA_MAX_KPTSIZE \
> 
> #define HP_MAX_PTSIZE   0x400000        /* max size of UPT */
> #define HP_MAX_KPTSIZE  0x100000        /* max memory to allocate to KPT */
> #define HP_PTBASE       0x10000000      /* UPT map base address */
> #define HP_PTMAXSIZE    0x70000000      /* UPT map maximum size */
> 
> #define USRSTACK        (-HIGHPAGES*NBPG)       /* Start of user stack */
> #define VM_MAX_KERNEL_ADDRESS   ((vm_offset_t)0xFFFFF000)
> 
> (NPTEG is 0x400 on the hp300 and 0x800 on the amiga becuase they use 4k and 8k pages)
> 
> Say the user requests maxusers of 16 thats 16 * 16 + 20 or a maxproc of 276.  Thus the number of KPT pages is
> amiga: howmany(((maxproc + 16) * AMIGA_UPTSIZE / NPTEPG), NBPG);
> amiga: howmany((292 * AMIGA_UPTSIZE / NPTEPG), NBPG);
> amiga: howmany((292 * USRSTACK / NPTEPG / NPTEPG), NBPG);
> amiga: howmany((292 * 56), NBPG);  /* thats 56 bytes per process */
> amiga: howmany(16352, NBPG);
> amiga: 2 pages or 16k.
> 
> hp300: howmany((292 * 0x400000 / NPTEPG), NBPG);
> hp300: howmany((292 * 4k), NBPG);  /* thats 4k bytes per process */
> hp300: howmany(1196032, NBPG);
> hp300: 146 pages or 584k.
> 
> So the amiga gains 568k of that valuable commodity, pageable ram.
> The other large benifit (and the one that ties this to the point of discussion) is that I can support a great many more processes.
> 
> With such a large USRSTACK on the hp300 the largest "maxproc" is 448.  Currently the amiga has a limit of 1024 procs (see MAX_CORE_UPT) and if I chose to use the same range as the hp300 for my page table map I would raise that to 16k processes (a ridiculous number for a single 040 or less system).
> 
> I not sure how pratical 448 processes on an hp300 is.  Is it possible to reach that number and still have a slightly usable system?
> 
> I think that there was some reason for the stack being so large on the hp300 (hpux compat?).  If not I recommend lowering it.  This may also be an issue for other non-m68k ports if they use similar algorithms.
> 
Yes, I put the stack there on the hp300 because that is where HP-UX had it.
I don't really recall any compatibility-related reasons for doing it that
way, it just seemed like the logical place to put it since it gave a
process access to the entire address space.  Of course, allowing for a
single-stack at one end of the address space is a rather single-threaded
mentality so I will retract the ``logical place'' argument.  However,
I still don't like the idea of arbitrarily restricting access to the
address space by lowering VM_MAXUSER_ADDRESS as you propose and, as I
am sure you are well aware, this is fixing a symptom and not the problem.

The problem is that kernel page table (KPT) pages are statically allocated.
When I originally did the hp300 pmap, KPT pages were dynamically allocated
just like user page table pages.  As you might suspect, this led to deadlocks
(can't remember a specific scenerio right now though) and I took the easy
way out and just statically allocated a pool of KPT pages.  To somewhat
mitigate this restriction, I implemented a cheezy pmap_collect routine on
the hp300 to reclaim ``empty'' KPT pages which came about whenever a
process was completely swapped out.  The truth of the matter was that I
considered one page per possible process ``in the noise'' for the 8 and
16MB machines I worked on (this was just a prototype remember).

It would seem that a combination of dynamically-allocated KPT pages and
a small statically-allocated ``reserve'' to use along the pageout path
(and garbage collection of unused pages of course) might solve the problem
but I will admit I have not thought about this problem in a long time.

A final observation: if you are still using the hp300-based Jolitz pmap
module for the x86, you cannot reduce the maximum size of a user page table
as you suggest.  The pmap module relies on the fact that there will be
exactly one KPT page describing every user page table (this KPT page
doubles as the segment table in a tricky bit of recursion).