Yes, you are encountering a fragmentation problem, but some intermediate
NAT/routers fix up this problem by dynamically modifiying the MSS option in
initial TCP connections from client PCs. The PC clients are unaware that
there is a PPPoE MTU between themselves and their intended service. What
these NAT routers do is modify the TCP option for MSS during a SYN sequence
to take into account the MTU size of the PPPoE interface, so as to avoid
fragmentation, as well as PMTU blackhole problems. It works pretty well,
this maybe an option that might be considered for a future release in the
NetBSD code. (NOTE: I said "option")

Randy

-----Original Message-----
From: Martin Husemann
To: tech-net@NetBSD.ORG
Sent: 4/14/01 1:57 PM
Subject: bad tcp sums

I have a problem connecting to a pop3 server when connected over PPPoE
(MTU 1492), while everything works using an ISDN connection (MTU 1500).

I get bad tcp cksums for fragmented packets.

Forcing the MSS of the connection to 1412 fixes the problem, but a
router
probably should not mess with MSSs negotiated by the peers, or do I
misunderstand this? (The rp-pppoe package has an otpion to clamp MSS.)

Does this ring a bell for someone? I'm suspecting a broken stack on the
remote router (probably a Linux system). The NetBSD client is running
NetBSD-current.

What options do I have to fix this?


Martin

tcpdump: listening on tlp0
22:42:33.319725 {netbsd-client}.62699 > {linux-server}.pop3: F [tcp sum
ok] 1205370870:1205370870(0) ack 1090582912 win 17520 (DF) (ttl 64, id
50377)
22:42:33.928624 {linux-server}.pop3 > {netbsd-client}.62699: . [tcp sum
ok] 1461:1461(0) ack 1 win 32735 (DF) (ttl 52, id 28229)
22:42:37.616032 {netbsd-client}.62688 > {linux-server}.pop3: S [tcp sum
ok] 207863030:207863030(0) win 16384 <mss 1460> (DF) (ttl 64, id 50410)
22:42:37.769037 {linux-server}.pop3 > {netbsd-client}.62688: S [tcp sum
ok] 222344672:222344672(0) ack 207863031 win 32736 <mss 1460> (ttl 52,
id 65123)
22:42:37.769157 {netbsd-client}.62688 > {linux-server}.pop3: . [tcp sum
ok] 1:1(0) ack 1 win 17520 (DF) (ttl 64, id 50413)
22:42:37.851589 {linux-server}.pop3 > {netbsd-client}.62688: P [tcp sum
ok] 1:44(43) ack 1 win 32736 (DF) (ttl 52, id 65165)
22:42:37.852485 {netbsd-client}.62688 > {linux-server}.pop3: P [tcp sum
ok] 1:21(20) ack 44 win 17520 (DF) (ttl 64, id 50417)
22:42:38.164842 {linux-server}.pop3 > {netbsd-client}.62688: P [tcp sum
ok] 44:57(13) ack 21 win 32736 (DF) (ttl 52, id 65287)
22:42:38.169717 {netbsd-client}.62688 > {linux-server}.pop3: P [tcp sum
ok] 21:36(15) ack 57 win 17520 (DF) (ttl 64, id 50420)
22:42:38.375061 {linux-server}.pop3 > {netbsd-client}.62688: P [tcp sum
ok] 57:62(5) ack 36 win 32736 (DF) (ttl 52, id 65344)
22:42:38.428995 {netbsd-client}.62688 > {linux-server}.pop3: . [tcp sum
ok] 36:36(0) ack 62 win 17520 (DF) (ttl 64, id 50427)
22:42:41.389218 {netbsd-client}.62688 > {linux-server}.pop3: P [tcp sum
ok] 36:42(6) ack 62 win 17520 (DF) (ttl 64, id 50449)
22:42:41.468079 {linux-server}.pop3 > {netbsd-client}.62688: P [tcp sum
ok] 62:71(9) ack 42 win 32736 (DF) (ttl 52, id 504)
22:42:41.482161 {netbsd-client}.62688 > {linux-server}.pop3: P [tcp sum
ok] 42:48(6) ack 71 win 17520 (DF) (ttl 64, id 50450)
22:42:41.563073 {linux-server}.pop3 > {netbsd-client}.62688: P [tcp sum
ok] 71:76(5) ack 48 win 32736 (DF) (ttl 52, id 519)
22:42:41.564226 {linux-server}.pop3 > {netbsd-client}.62688: F [tcp sum
ok] 76:76(0) ack 48 win 32736 (ttl 52, id 520)
22:42:41.564298 {netbsd-client}.62688 > {linux-server}.pop3: . [tcp sum
ok] 48:48(0) ack 77 win 17520 (DF) (ttl 64, id 50451)
22:42:41.564624 {netbsd-client}.62688 > {linux-server}.pop3: F [tcp sum
ok] 48:48(0) ack 77 win 17520 (DF) (ttl 64, id 50452)
22:42:41.571255 {netbsd-client}.62686 > {linux-server}.pop3: S [tcp sum
ok] 393771036:393771036(0) win 16384 <mss 1460> (DF) (ttl 64, id 50457)
22:42:41.648015 {linux-server}.pop3 > {netbsd-client}.62686: S [tcp sum
ok] 1966123666:1966123666(0) ack 393771037 win 32736 <mss 1460> (ttl 53,
id 54493)
22:42:41.648098 {netbsd-client}.62686 > {linux-server}.pop3: . [tcp sum
ok] 1:1(0) ack 1 win 17520 (DF) (ttl 64, id 50458)
22:42:41.650525 {linux-server}.pop3 > {netbsd-client}.62688: . [tcp sum
ok] 77:77(0) ack 49 win 32735 (DF) (ttl 52, id 538)
22:42:41.740156 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 1:44(43) ack 1 win 32736 (DF) (ttl 53, id 54512)
22:42:41.750485 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 1:21(20) ack 44 win 17520 (DF) (ttl 64, id 50461)
22:42:41.825081 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 44:57(13) ack 21 win 32736 (DF) (ttl 53, id 54529)
22:42:41.826658 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 21:36(15) ack 57 win 17520 (DF) (ttl 64, id 50462)
22:42:41.917726 {linux-server}.pop3 > {netbsd-client}.62686: . [tcp sum
ok] 57:57(0) ack 36 win 32736 (DF) (ttl 53, id 54544)
22:42:42.814094 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 57:63(6) ack 36 win 32736 (DF) (ttl 53, id 54690)
22:42:42.829122 {netbsd-client}.62686 > {linux-server}.pop3: . [tcp sum
ok] 36:36(0) ack 63 win 17520 (DF) (ttl 64, id 50463)
22:42:45.821614 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 36:42(6) ack 63 win 17520 (DF) (ttl 64, id 50464)
22:42:45.895607 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 63:77(14) ack 42 win 32736 (DF) (ttl 53, id 55210)
22:42:45.897490 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 42:48(6) ack 77 win 17520 (DF) (ttl 64, id 50465)
22:42:45.970313 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 77:84(7) ack 48 win 32736 (DF) (ttl 53, id 55220)
22:42:45.980738 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 48:54(6) ack 84 win 17520 (DF) (ttl 64, id 50466)
22:42:46.054656 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 84:90(6) ack 54 win 32736 (DF) (ttl 53, id 55235)
22:42:46.229177 {netbsd-client}.62686 > {linux-server}.pop3: . [tcp sum
ok] 54:54(0) ack 90 win 17520 (DF) (ttl 64, id 50470)
22:42:46.247278 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 84:90(6) ack 54 win 32736 (DF) (ttl 53, id 55252)
22:42:46.247343 {netbsd-client}.62686 > {linux-server}.pop3: . [tcp sum
ok] 54:54(0) ack 90 win 17520 (DF) (ttl 64, id 50471)
22:42:46.303728 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 90:211(121) ack 54 win 32736 (DF) (ttl 53, id 55260)
22:42:46.310168 {netbsd-client}.62686 > {linux-server}.pop3: P [tcp sum
ok] 54:70(16) ack 211 win 17520 (DF) (ttl 64, id 50472)
22:42:46.384381 {linux-server}.pop3 > {netbsd-client}.62686: P [tcp sum
ok] 211:228(17) ack 70 win 32736 (DF) (ttl 53, id 55270)
22:42:46.429166 {netbsd-client}.62686 > {linux-server}.pop3: . [tcp sum
ok] 70:70(0) ack 228 win 17520 (DF) (ttl 64, id 50473)
22:42:46.665190 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 55317:1472@0+) (ttl
52)
22:42:47.144732 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 55374:1472@0+) (ttl
52)
22:42:47.720557 {linux-server}.pop3 > {netbsd-client}.62699: P [bad tcp
cksum 4b3a!] 1:1453(1452) ack 1 win 32736 (frag 31523:1472@0+) (ttl 51)
22:42:48.104988 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 55529:1472@0+) (ttl
52)
22:42:50.025098 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 55858:1472@0+) (ttl
52)
22:42:53.865343 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 56521:1472@0+) (ttl
52)
22:43:01.546983 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 57866:1472@0+) (ttl
52)
22:43:16.905231 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 60170:1472@0+) (ttl
52)
22:43:47.625367 {linux-server}.pop3 > {netbsd-client}.62686: P [bad tcp
cksum 560e!] 228:1680(1452) ack 70 win 32736 (frag 942:1472@0+) (ttl 52)
22:43:50.648441 {netbsd-client}.62686 > {linux-server}.pop3: F [tcp sum
ok] 70:70(0) ack 228 win 17520 (DF) (ttl 64, id 50628)
22:43:50.721515 {linux-server}.pop3 > {netbsd-client}.62686: . [tcp sum
ok] 1688:1688(0) ack 71 win 32735 (DF) (ttl 53, id 1866)

250 packets received by filter
0 packets dropped by kernel