Subject: Re: multilink
To: Ignatios Souvatzis <ignatios@cs.uni-bonn.de>
From: Brian Somers <brian@Awfulhak.org>
List: tech-net
Date: 06/06/2000 02:58:15
> On Thu, Jun 01, 2000 at 10:18:37AM +0100, Brian Somers wrote:
> > > On Wed, May 31, 2000 at 01:07:22AM +0100, Brian Somers wrote:
> > > 
> > > > User-ppp will do MP without any noticeable loss in performance. ...
> > > 
> > > ...for fast CPUs and slow links.
> > 
> > To an extent, yes.  I can get 1.5 Mb (bytes, not bits) through a 
> > synchronous back-to-back user-ppp connection on my laptop (PII366) 
> I notice this number...
> > with compression disabled.
> ... and this restriction.
> 
> at what %CPU usage, and what context switch rate?

100% CPU usage.  I can't measure the context switch rate at the 
moment, but the 1.5Mb is flat-out hammering the machine.

By back-to-back I mean that one ppp invocation is execing the other 
and they're talking via a socketpair() - CPU is intended to be the 
limiting factor as I'm trying to measure maximum throughput.

> > Also, I'd argue that on an SMP box, user-land MP ppp will beat the 
> > socks off any kernel implementation - at least for the forseeable 
> > (disclaimer: I know nothing about NetBSD SMP mutexes !!).  Especially 
> > when you introduce compression - usually better done in userland.
> 
> Not necessarily. We're already doing it at a low priority software interupt,
> and I guess we could move it to a kernel thread now. Besides: if you're doing
> it in userland, you'll get an increasing context switch rate if you have (due
> to your increasing CPU capabilities) an increasing data rate (packet sizes are
> limited). So you'll be limited by context switching abilities of your hardware
> (CPU, caches, translation tables), not by CPU power per se.

On the above i386 with FreeBSD I'm still hitting CPU limitations and 
I guess NetBSD is the same, but I take your point.

I'm not sure how easy it would be to move the if_ppp stuff into kernel 
threads... if you take a look at the code, it's not exactly layered 
as you might expect :-(

> You're talking about machines with 10^3 to 10^4 times the CPU power of
> the low-end NetBSD machines, and about 50 times the memory. You'll always
> have a class of links for a given class of machines where userland PPP is
> ok... and you'll always have another class where it doesn't really work.

Yes, I can't deny that the bouncing of packets in & out of the 
kernel murders things when resources are at a premium.  I guess 
I'm just arguing that user-ppp isn't limited to high CPU low 
bandwidth solutions as the general case is a single-invocation 
single or dual link connection.  Even PPPoE (over ethernet) doesn't 
hit resource problems on modern hardware....

> So you should mention the details of your setup, if you're publishing claims
> like:
> 
> > > > User-ppp will do MP without any noticeable loss in performance. ...

This was badly phrased. I *meant* it can do MP with an almost linear 
bandwidth increase - given that the CPU requirements are available.

> which is certainly true for fast modem to ISDN/double ISDN speeds on most of
> the supported machines... but not for every supported (by NetBSD) combination.

I was thinking of modem/isdn connections :-)

Having said that, I do virtually all my testing with either 
back-to-back connections on the same machine (``set device "!ppp 
-direct in"'') or PPPoverUDP... so in practice, my tests always 
hammer the machine and aren't really representative of anything 
except theoretical throughput.

My web site (http://www.Awfulhak.org/ppp.html) has links to various 
configuration possibilities for the curious.

> Of course, if you want to do multilink PPP, you have no choice currently.

And if you want to do PPP over Ethernet, sync cards, arbitrary programs, 
tcp or udp.  Even ppp over ISDN (i4b) is far easier with user-ppp.
User-ppp even deals with the ``first-connection'' problem :-)

There are trade-offs both ways...

> Regards,
> 	-is

Cheers.

-- 
Brian <brian@Awfulhak.org>                        <brian@[uk.]FreeBSD.org>
      <http://www.Awfulhak.org>                   <brian@[uk.]OpenBSD.org>
Don't _EVER_ lose your sense of humour !