Re: PR 9725 and the m68k pmap

[Izumi Tsutsui <tsutsui%ceres.dti.ne.jp@localhost>]

Hauke Fath wrote:

> now that netbsd-5 is safely on its way, I would like to come back to PR
> 9725 <http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=9725 >. As
> mentioned a while back, the issue stands between the 68k family and
> successful pkgsrc bulk builds.
> 
> During a discussion last August, Izumi Tsutsui mentioned that pmap would
> have to be re-written, while Michael Hitch hinted at some work he'd done on
> amiga to allow for larger user address space. Can you elaborate?

I have not discussed with Michael, but I think we have to know
how current m68k pmap (both for 020/030 and 040/060) has been
designed and implemented.

I wrote the following message on port-atari when I was working
on atari pmap, and I think they might be useful on this topic:
http://mail-index.NetBSD.org/port-atari/2009/01/03/msg000200.html

>> The traditional hp300 derived pmap (designed for 68851 and HP MMU)
>> uses two level table lookups for address translation.
>> The first one is the segment table and the second is the page table.
>> (there is a simple doc in arch/hp300/DOC/HPMMU.notes)

Note 68851/68030 support upto four level table lookups,
but hp300 pmap uses simple two level table lookups for old HPMMU.

>> On 4kbytes/page systems (hp300 etc.), both have 1024 entries.
>> On 8kbytes/page systems (i.e. on amiga and atari),
>> the segment table has 256 entries and the page table has
>> 2048 (== NPTEPG == PAGE_SIZE / sizeof(pt_entry_t)) entries.
>> (I don't know why these two traditional ports use different PAGE_SIZE)

(Note is@ said because we could do it to reduce TLB shortage)

>> On the other hand, 040/060 MMU only supports three level table lookups.
>> Both the first and second level tables have 128 entries, and
>> the third one (page table) has 1024 or 2048 entries per its page size
>> as well as 68851/68030.

(Note this is incorrect. 32 or 64 entries in each L3 table)

>> In -current pmap implementation, it uses an allocated page for the
>> segment table for 030 to put both the first and second lookup tables
>> (maybe to simplify implementation to support both machines).

As you can see pmap_bootstrap.c, L1 table and L2 table are allocated
in one 4KB page which is allocated as the STE page for 68851/68030 MMU.

On 68851/68030 with 4KB/page, one 4KB STE page has 1024 PTE page
entries and each 4KB PTE page has 1024 entries, so whole 4GB
address space (4KB * 1024 * 1024 = 4GB) can be mapped with
only one 4KB STE page.

On 68040/68060 with 4KB/page, both L1 table and L2 tables
have 128 entries, and each L3 table has 64 (SG4_LEV3SIZE) entries.
To map whole 4GB address space, we need upto 128 L2 tables.
(4KB * 128 * 128 * 64 = 4GB)

But on -current 040 pmap, L1 table is allocated at top of
the 4KB STE page for 68851/68030, and L2 tables are allocated after
the L1 table in the same 4KB page. Both L1 table and L2 tables
are 512 bytes (== SG4_LEV[12]SIZE * sizeof(st_entry_t)),
so we can have only upto 7 (== (4096 - 512) / 512) L2 tables.

In this case, we can map only 4KB * 128 * 7 * 64 (== ~224MB)
virtual address space on 040/060. This is what pmap_bootstrap.c
comments (on all hp300 derived pmap ports) say:

>>       * XXX cramming two levels of mapping into the single "segment"
>>       * table on the 68040 is intended as a temporary hack to get things
>>       * working.  The 224mb of address space that this allows will most
>>       * likely be insufficient in the future (at least for the kernel).

Furthermore, after yamt-km branch merge, one more L2 table is
required to map Sysptmap (which is allocated at the end of KVA),
so now we can have only six L2 tables.
(on some port more L2 tables will be used for other special mappings)


To solve this, I guess we have to add some code to allocate
a new page for additional L2 tables after all first L2 tables
in the first 4KB page have been occupie in
m68k/pmap_motorola.c:pmap_enter_ptpage().
(around "panic("enter: out of address space")" line)
I guess this is what Michael was working on.

Michael also said that his change required modifications
to all the m68k port's initial kernel MMU table setup.
I don't know what exact changes are needed, but I guess
L2 map allocation for Sysptmap (and also other special L2 mappings)
needs to be changed, to map those L2 tables at right after
other L2 tables (as amiga and atari do), not at the end of
the 4KB STE page, to expand L2 entries in the L1 table.
More other fixes might be required, of course.

(sorry for my insufficient English to mention such technical details)
---
Izumi Tsutsui