Ignatios Souvatzis <ignatios%cs.uni-bonn.de@localhost> writes:
> On Sun, Feb 22, 2009 at 06:02:18PM -0500, Greg Troxel wrote:
>>
>> I have systems at BBN and at MIT set up for IPv6. The provider at BBN
>> is sixxs.net, and at MIT OCCAID. I did a cvs update -r netbsd-5 of a
>> netbsd-4 tree, and it took forever, as in went nearly 24h and didn't
>> finish.
>
> What - ftp.netbsd.org or cvs.netbsd.org?
Both. I had 'cvs update' be really slow, and then I did more controlled
tests with ftp.netbsd.org. The tcpdump2xplot I sent was from MIT to
ftp.netbsd.org.
> I can ftp from ftp.netbsd.org at a rate of about 150 to 180 kB/s, depending
> on the client machine/location, with v6, v4 is actually 15% slower
> 125 kBytes/s).
>
> I didn't check cvs.netbsd.org, but use it regularly, and afaik mostly
> over IPv6.
Thanks - so that confirms that some paths to cvs.netbsd.org work ok.
> I'd guess broken path mtu detection.
I suspected that as well. But the tcpdump from MIT shows an initial
small segment (http reply headers I think), and then a gap and a 1208
byte segment. The gap is 2416, so it seems like that's made up of 2 x
1208 rather that a single 2416.
08:19:52.237824 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: S
242525615:242525615(0) win 32768 <mss 1440,nop,wscale
0,sackOK,nop,nop,nop,nop,timestamp 0 0> [flowlabel 0x6e6b2]
08:19:52.316087 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: S
1528261171:1528261171(0) ack 242525616 win 4096 <mss 1440,nop,wscale
6,nop,nop,timestamp 1 0,sackOK,nop,nop>
08:19:52.316206 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 1 win
33120 <nop,nop,timestamp 0 1> [flowlabel 0x6e6b2]
08:19:52.316403 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: P
1:144(143) ack 1 win 33120 <nop,nop,timestamp 0 1> [flowlabel 0x6e6b2]
08:19:52.585859 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: . ack 144
win 65 <nop,nop,timestamp 1 0> [flowlabel 0x9f14c]
08:19:52.997808 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: P
1:234(233) ack 144 win 67 <nop,nop,timestamp 2 0> [flowlabel 0x9f14c]
08:19:53.189964 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 234
win 33120 <nop,nop,timestamp 2 2> [flowlabel 0x6e6b2]
08:19:55.351518 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
2650:3858(1208) ack 144 win 67 <nop,nop,timestamp 7 2> [flowlabel 0x9f14c]
08:19:55.351658 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 234
win 33120 <nop,nop,timestamp 6 2,nop,nop,sack sack 1 {2650:3858} > [flowlabel
0x6e6b2]
08:19:58.270901 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
234:1442(1208) ack 144 win 67 <nop,nop,timestamp 13 6> [flowlabel 0x9f14c]
08:19:58.271107 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 1442
win 31912 <nop,nop,timestamp 12 13,nop,nop,sack sack 1 {2650:3858} > [flowlabel
0x6e6b2]
08:19:59.851562 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
1442:2650(1208) ack 144 win 67 <nop,nop,timestamp 16 12> [flowlabel 0x9f14c]
08:19:59.851707 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 3858
win 30704 <nop,nop,timestamp 15 16> [flowlabel 0x6e6b2]
08:19:59.851812 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 3858
win 33120 <nop,nop,timestamp 15 16> [flowlabel 0x6e6b2]
08:20:01.432344 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
3858:5066(1208) ack 144 win 67 <nop,nop,timestamp 19 15> [flowlabel 0x9f14c]
08:20:01.630382 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 5066
win 33120 <nop,nop,timestamp 19 19> [flowlabel 0x6e6b2]
08:20:03.703481 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
5066:6274(1208) ack 144 win 67 <nop,nop,timestamp 24 19> [flowlabel 0x9f14c]
08:20:03.900473 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 6274
win 33120 <nop,nop,timestamp 23 24> [flowlabel 0x6e6b2]
08:20:05.474196 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
6274:7482(1208) ack 144 win 67 <nop,nop,timestamp 27 23> [flowlabel 0x9f14c]
08:20:05.670563 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 7482
win 33120 <nop,nop,timestamp 27 27> [flowlabel 0x6e6b2]
08:20:07.745308 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
7482:8690(1208) ack 144 win 67 <nop,nop,timestamp 32 27> [flowlabel 0x9f14c]
08:20:07.940675 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 8690
win 33120 <nop,nop,timestamp 31 32> [flowlabel 0x6e6b2]
08:20:09.516371 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
8690:9898(1208) ack 144 win 67 <nop,nop,timestamp 35 31> [flowlabel 0x9f14c]
08:20:09.710760 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 9898
win 33120 <nop,nop,timestamp 35 35> [flowlabel 0x6e6b2]
08:20:11.787314 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
9898:11106(1208) ack 144 win 67 <nop,nop,timestamp 40 35> [flowlabel 0x9f14c]
08:20:11.980870 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: . ack 11106
win 33120 <nop,nop,timestamp 39 40> [flowlabel 0x6e6b2]
08:20:12.603766 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: F
144:144(0) ack 11106 win 33120 <nop,nop,timestamp 41 40> [flowlabel 0x6e6b2]
08:20:12.681748 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: . ack 145
win 67 <nop,nop,timestamp 42 41> [flowlabel 0x9f14c]
08:20:13.558138 ftp.NetBSD.org.http > 2001:4830:2446:3e::6a9.62119: .
11106:12314(1208) ack 145 win 67 <nop,nop,timestamp 43 41> [flowlabel 0x9f14c]
08:20:13.558304 2001:4830:2446:3e::6a9.62119 > ftp.NetBSD.org.http: R
242525760:242525760(0) win 0 [flowlabel 0x4d1b9]
traceroute6 to ftp.netbsd.org (2001:4f8:4:7:230:48ff:fe31:43f2) from
2001:4830:2446:3e::6a9, 64 hops max, 12 byte packets
1 limekiller.ipv6.mit.edu 0.862 ms 0.754 ms 0.668 ms
2 bbr01-tu100.bstn01.occaid.net 1.831 ms 1.863 ms 1.443 ms
3 bbr01-g1-0.chcg01.occaid.net 28.35 ms 28.283 ms 28.323 ms
4 bbr01-p2-1.kscy01.occaid.net 41.744 ms 41.56 ms 41.351 ms
5 bbr01-p1-0.snfc02.occaid.net 77.397 ms 77.215 ms 77.407 ms
6 2001:4f8:4:b::1 77.569 ms 77.716 ms 77.968 ms
7 sfo2-sql1.r1.sql1.isc.org 78.787 ms 78.007 ms 78.14 ms
8 * * *
9 *^C
and traceroute from ftp.netbsd.org back to my machine:
traceroute6 to 2001:4830:2446:3e::6a9 (2001:4830:2446:3e::6a9) from
2001:4f8:4:7:230:48ff:fe31:43f2, 64 hops max, 12 byte packets
1 isc-router.NetBSD.org 260.727 ms 252.026 ms 3.278 ms
2 sfo2-sql1.r2.sfo2.isc.org 1.772 ms 1.419 ms 2.001 ms
3 2001:4f8:4:b:2d0:b7ff:fee1:4d0f 1.396 ms 1.536 ms 1.46 ms
4 bbr01-p3-0.kscy01.occaid.net 37.117 ms 37.173 ms 36.914 ms
5 bbr01-g2-0.chcg01.occaid.net 49.973 ms 50.724 ms 50.524 ms
6 bbr01-t2-0.bstn01.occaid.net 76.982 ms 77.254 ms 76.912 ms
7 mit-ic-1137-bos.customer.occaid.net 77.596 ms 78.013 ms 77.478 ms
8 2001:4830:2446:3e::6a9 78.275 ms 78.973 ms 78.148 ms
which looks pretty much like the same path.
I tried wget on ftp.netbsd.org fetching from MIT, and v6 speed was 2.6
KB/s vs 370 KB/s IPv4.
Looking at it from my end, it seems like there is a queue someplace with
a 1-packet limit. My machine sent 1208 bytes of TCP data in two packets
immediately following the reply headers, and then another segment when
it got the header ack. The strange thing is that after recovering from
that loss the next burst of three results in a SACK for the third
packet.
08:28:29.938497 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: S
1607206229:1607206229(0) win 32768 <mss 1440,nop,wscale
6,sackOK,nop,nop,nop,nop,timestamp 1 0> [flowlabel 0x51d4f]
08:28:29.938549 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: S
1696404153:1696404153(0) ack 1607206230 win 65535 <mss 1440,nop,wscale
2,nop,nop,timestamp 0 1,sackOK,nop,nop>
08:28:30.017269 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 1 win
517 <nop,nop,timestamp 1 0> [flowlabel 0x51d4f]
08:28:30.017329 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: . ack 1 win
33120 <nop,nop,timestamp 0 0> [flowlabel 0xb71e2]
08:28:30.017418 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: P
1:120(119) ack 1 win 517 <nop,nop,timestamp 1 0> [flowlabel 0x51d4f]
08:28:30.017959 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: P
1:282(281) ack 120 win 33120 <nop,nop,timestamp 0 1> [flowlabel 0xb71e2]
08:28:30.018007 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
282:1490(1208) ack 120 win 33120 <nop,nop,timestamp 0 1> [flowlabel 0xb71e2]
08:28:30.018018 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
1490:2698(1208) ack 120 win 33120 <nop,nop,timestamp 0 1> [flowlabel 0xb71e2]
08:28:30.294648 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 282
win 517 <nop,nop,timestamp 1 0> [flowlabel 0x51d4f]
08:28:30.294679 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
2698:3906(1208) ack 120 win 33120 <nop,nop,timestamp 1 1> [flowlabel 0xb71e2]
08:28:30.375438 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 282
win 517 <nop,nop,timestamp 1 0,nop,nop,sack sack 1 {2698:3906} > [flowlabel
0x51d4f]
08:28:31.792686 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
282:1490(1208) ack 120 win 33120 <nop,nop,timestamp 4 1> [flowlabel 0xb71e2]
08:28:31.873800 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 1490
win 498 <nop,nop,timestamp 4 4,nop,nop,sack sack 1 {2698:3906} > [flowlabel
0x51d4f]
08:28:31.873835 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
1490:2698(1208) ack 120 win 33120 <nop,nop,timestamp 4 1> [flowlabel 0xb71e2]
08:28:31.955199 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 3906
win 479 <nop,nop,timestamp 5 4> [flowlabel 0x51d4f]
08:28:31.955227 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
3906:5114(1208) ack 120 win 33120 <nop,nop,timestamp 4 1> [flowlabel 0xb71e2]
08:28:31.955238 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
5114:6322(1208) ack 120 win 33120 <nop,nop,timestamp 4 1> [flowlabel 0xb71e2]
08:28:31.955250 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
6322:7530(1208) ack 120 win 33120 <nop,nop,timestamp 4 1> [flowlabel 0xb71e2]
08:28:32.036761 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 3906
win 517 <nop,nop,timestamp 5 4,nop,nop,sack sack 1 {6322:7530} > [flowlabel
0x51d4f]
08:28:33.452776 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
3906:5114(1208) ack 120 win 33120 <nop,nop,timestamp 7 1> [flowlabel 0xb71e2]
08:28:33.534299 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 5114
win 498 <nop,nop,timestamp 8 7,nop,nop,sack sack 1 {6322:7530} > [flowlabel
0x51d4f]
08:28:33.534330 2001:4830:2446:3e::10e.http > ftp.NetBSD.org.57000: .
5114:6322(1208) ack 120 win 33120 <nop,nop,timestamp 7 1> [flowlabel 0xb71e2]
08:28:33.615930 ftp.NetBSD.org.57000 > 2001:4830:2446:3e::10e.http: . ack 7530
win 479 <nop,nop,timestamp 8 7> [flowlabel 0x51d4f]
Given the replies I've received, I now suspect this has something to
with OCCAID or MIT rather than ftp.netbsd.org, and will dig in further
locally.
Attachment:
pgpBJ1Ouyd81x.pgp
Description: PGP signature