Well, I tried the new kernel with nsphy enabled, and be(4), ukphy
removed.
The performance dropped(!) from 2.6 MB/s to 2.27 MB/s. ;(

Here's "systat vm" output halfway through a big transfer:

---
    1 user     Load  0.11  0.18  0.20                  Mon May 20
22:17:59

          memory totals (in KB)             PAGING   SWAPPING     
Interrupts
         real   virtual    free             in  out   in  out      2712
total
Active   6204      6204   69044     ops                             273
lev1
All     24488     24488  404960     pages                             5
lev4
                                                                       
lev6
Proc:r  d  s  w       Csw  Trp  Sys  Int  Sof  Flt        forks    2234
lev7
     1     2           10        25 2715         1        fkppw     100
clock
                                                          fksvm        
lev12
   0.2% Sy   0.2% Us   0.0% Ni  77.5% In  22.1% Id        pwait     100
prof
|    |    |    |    |    |    |    |    |    |    |       relck
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%                   rlkok
                                                          noram
Namei         Sys-cache     Proc-cache                    ndcpy
    Calls     hits    %     hits     %                    fltcp
                                                          zfod
                                                          cow
Discs  fd0  sd0                                        64 fmin
seeks                                                  85 ftarg
xfers         1                                           itarg
Kbyte         6                                        43 wired
%busy       1.0                                           pdfre
---

Also, after removing the be cards from the machine, NOT the drivers
from the kernel, and also after removing the drivers from the kernel,
I've started to see these:

May 20 22:16:10 sik-gw1 /netbsd: stray interrupt ipl 0x7 pc=0xf0007604
npc=0xf0007608 psr=401.40c7<S,PS>

Fascinating, isn't it? ;)

Am I hitting the maximum achievable with this setup, as the interrupts
seem to be pretty heavily loaded?

If you have any patches I could try, I'm all ears. Are those "zero-copy"
patches by Jason R Thorpe appliable to 1.5.3 source tree?

Regards,

Tomi

PS: Please reply to the mailing list, as my e-mail address is obfuscated
to repel spammers.