Re: pty(4) 1024 bytes buffer limit

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

On Thu, 8 Sep 2011 17:45:38 -0400
Thor Lancelot Simon <tls%panix.com@localhost> wrote:

> On Thu, Sep 08, 2011 at 11:26:29AM -0400, Matthew Mondor wrote:
> > 
> > It would be nice to for instance be able to use an MTU of 3000 so that
> > there are less context switches, but unfortunately tracing the
> > processes show that 1024 bytes are read from the pty devices at most.
> 
> Are you sure using an MTU of 3000 would do much of anything?  Since
> almost all peers are connected by Ethernet somewhere along the line,
> you are unlikely to ever see packets larger than 1500 minus Ethernet
> framing size.

Indeed I could even avoid IP fragmentation with a low enough MTU, which
is what I tried in the initial setup (and am still using, because of
the 1024 bytes limit).

> How did you determine that the bottleneck for your application was
> context switches?  That the 1024-byte read size you're seeing is
> actually internal to the tty layer or ppp rather than application
> imposed in userspace?

I'm not sure that the bottleneck really are user context switches but I
highly suspect it, as the forwarding daemon is mostly idle while it
can't seem to send faster than about 178KB/sec (when using an MTU small
enough to avoid the 1024 bytes limit, without which performance drops
even more).  If I could test with higher MTU to move more work down
into the kernel and network, I could confirm or disprove :)

I wrote the application as a test, so am controlling the buffer size,
and am invoking pppd with the wanted mru setting.  While it's not
impossible that pppd imposes the limit, I've found some threads when
searching with people complaining about the same pty limit on NetBSD
and OpenBSD with it being absent on Linux.  But pppd also uses the
in-kernel ppp support I think, which is probably different than Linux's.

Also, although I didn't inspect carefully the whole if_ppp code, I
didn't see anything suggesting 1024 would be a limit, yet in the pty
code I do see TTYHOG:

/usr/include/sys/tty.h:#define  TTYHOG  1024

To definitely test if it's really a pty/tty limitation I could write a
small program and see, though; probably the best thing to confirm.
-- 
Matt