Re: 16KB page ibm4xx performance

[Masao Uebayashi <uebayasi%tombi.co.jp@localhost>]

On Mon, Aug 16, 2010 at 05:23:56PM +0900, Masao Uebayashi wrote:
> I'm testing XIP on OpenBlockS266 (405GPr).  It works, but it seems very
> slow.  If I disable neighbor fault for MADV_NORMAL, /bin/ksh as XIP
> starts up in doubled speed, and time -l shows 1/2 page reclaims.
> 
> I guess this platform needs serious TLB / cache tuning...

There are a mixture of problems:

- Our PowerPC ELF has RWX .data/.got/.plt.  If programs' .data sections
  are not aligned to 16KB, those are mapped as "overlay"; pages are always
  copied.

- Mapping executable pages by pmap_enter() is very expensive because of the
  __syncicache() operation.

- Neighbor fault tends to cause TLB shortage.  This is bad especially if exec
  mappings became victimes.

I can get useful speed on 405GPr now by doing the followings:

- Use static binaries.

- Disable neighbor faults.

*

I'd recommend all 405GPr users to use only static userland, so you
get 2x speed...

Masao

-- 
Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635

Index: sys/uvm/uvm_fault.c
===================================================================
RCS file: /cvsroot/src/sys/uvm/uvm_fault.c,v
retrieving revision 1.166.2.21
diff -u -r1.166.2.21 uvm_fault.c
--- sys/uvm/uvm_fault.c 17 Aug 2010 06:48:14 -0000      1.166.2.21
+++ sys/uvm/uvm_fault.c 18 Aug 2010 14:52:52 -0000
@@ -167,7 +167,11 @@
  */
 
 static const struct uvm_advice uvmadvice[] = {
+#if PAGE_SIZE < 16384
        { MADV_NORMAL, 3, 4 },
+#else
+       { MADV_NORMAL, 0, 0 },
+#endif
        { MADV_RANDOM, 0, 0 },
        { MADV_SEQUENTIAL, 8, 7},
 };