Re: Long round trip times on local net

["Steven M. Bellovin" <smb%cs.columbia.edu@localhost>]

On Thu, 12 Feb 2009 01:46:41 +0000
raymond.meyer%rambler.ru@localhost wrote:

> On Wed, 11 Feb 2009 20:04:01 -0500
> "Steven M. Bellovin" <smb%cs.columbia.edu@localhost> wrote:
> 
> > On Thu, 12 Feb 2009 00:50:50 +0000
> > raymond.meyer%rambler.ru@localhost wrote:
> > 
> > > I have two machines running NetBSD-5 beta, i386 and sparc64. Both
> > > machines are on a local network. When I ping i386 machine from
> > > sparc64 I notice the following output:
> > > 
> > > 64 bytes from 192.168.0.40: icmp_seq=49 ttl=255 time=0.261 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=50 ttl=255 time=0.254 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=51 ttl=255 time=1000.276 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=52 ttl=255 time=0.416 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=53 ttl=255 time=0.258 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=54 ttl=255 time=1000.250 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=55 ttl=255 time=0.391 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=56 ttl=255 time=0.256 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=57 ttl=255 time=0.264 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=58 ttl=255 time=0.275 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=59 ttl=255 time=0.257 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=60 ttl=255 time=1000.249 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=61 ttl=255 time=0.386 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=62 ttl=255 time=0.264 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=63 ttl=255 time=1000.248 ms
> > > 64 bytes from 192.168.0.40: icmp_seq=64 ttl=255 time=0.388 ms
> > > 
> > > Quite a few packets show round trip time of just over a second.
> > > If I ping sparc64 machine from i386, the round trip time for all
> > > packets is about 0.200 ms, i.e. there are absolutely no over 1
> > > second delays.
> > > 
> > > Does anyhone know why so many packets have such long delays? The
> > > i386 has a PCI network card that uses rtk driver, sparc64 uses
> > > hme driver. I've tried different CAT cables on both machines,
> > > which didn't make any difference. Can this be a hardware problem
> > > or a driver bug?
> > > 
> > Are you doing 'ping' or 'ping -n'?  I wonder if it's a DNS lookup
> > issue -- those figures are very suggestive of a 1-second timeout.
> > 
> > Hmm -- I wonder if an interrupt is getting lost.  That 1 second is
> > the interval to the next ping, after all.  Try it with '-i 5' and
> > see if you get 5-second delays.  Try it with another transmission
> > (say, via ttcp) running in the background and see if that gets rid
> > of the delays.
> > 
> > 
> >             --Steve Bellovin, http://www.cs.columbia.edu/~smb
> 
> OK here is what I have so far:
> 
> My sparc64 machine is Ultra 10, it has 1 onboard ethernet port and a
> Sun quad PCI ethernet card with 4 ethernet ports.
> 
> If I do my networking via quad PCI ethernet card, I have problems with
> long round trip times as detailed above. I also posted about having
> problems with slow NFS
> 
> http://mail-index.netbsd.org/netbsd-users/2009/01/27/msg002768.html
> 
> Which I now think is related to this issue.
> 
> If I do my networking via the onboard ethernet port, everything works
> fine, including NFS!
> 
This sure sounds like a lost interrupt problem or (possibly) something
incorrect in the FIFO setting.  It could be hardware, but my bet would
be on a driver bug.  (The "bug" could be as "simple" as not having the
proper work-around for buggy hardware.)  It would be good if you ran
those tests I suggest.  Another good test would be to boot some other
open source OS on the Sparc and see what happens.

Past that, I can't really offer any advice.


                --Steve Bellovin, http://www.cs.columbia.edu/~smb