>Number:         10994
>Category:       kern
>Synopsis:       Weird ipnat/tcp bug where nated packets are garbled
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Sep 11 18:22:00 PDT 2000
>Closed-Date:
>Last-Modified:
>Originator:     Bill Studenmund
>Release:        Both 1.4.3_ALPHA and 1.5_ALPHA
>Organization:
Zembu Labs
>Environment:
Powermac on internal net, with System: NetBSD tanis 1.5D NetBSD 1.5D

IP-NAT box:
System: NetBSD vespasia 1.5_ALPHA2 NetBSD 1.5_ALPHA2 (SKULLPORT) #2:System: NetBSD vespasia 1.5_ALPHA2 NetBSD 1.5_ALPHA2 (SKULLPORT) #2:

and also the 1.4.3_ALPHA kernel which was put up in July.

Destenation:	ftp.netbsd.org

>Description:

I have a powermac running NetBSD on my internal net, and I have the
internal net NATd to the outside world. I usually ssh to ftp.netbsd.org to
log into icb. Often, these connections will just die with a read reset by
peer message.

So I ran tcpdump on both my internal and external interfaces. Under both
1.4.3 and 1.5_alpha, I have seen the same behavior: The last packets sent
from my machine to nbftp, which look like repeated pushes, are covering a
range other than the range they were covering when sent to the NAT box.
Of course this causes the other side to reset the connection.

Here are three tcp dump sets. In each set is the output of both tcpdump -i
rtk0 (external interface) and tcpdump -i rtk1 (internal interface). The
only manipulation I've done to the se logs is to trim the hostnames down
to just tanis (the powermac, on the internal net), dsl (my dsl address),
and ftp (ftp.netbsd.org).

Since it seems fairly easy to impliment, I can test different things if
desired.

From NAT box to nbftp

16:03:18.790404 dsl.1006 > nbftp.ssh: . ack 1729 win 17520 <nop,nop,timestamp 56165 858580>
16:03:18.796373 nbftp.ssh > dsl.1006: . 1729:2229(500) ack 220 win 17520 <nop,nop,timestamp 858581 56165> [tos 0x10]
16:03:18.797286 dsl.1006 > nbftp.ssh: . ack 2361 win 16888 <nop,nop,timestamp 56165 858581>
16:03:54.787277 dsl.1006 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56237 858581>
16:03:55.616433 dsl.1006 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56238 858581>
16:03:57.615973 dsl.1006 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56242 858581>
16:04:01.615140 dsl.1007 > nbftp.ssh: P 747722286:747722306(20) ack 2709672657 win 17520 <nop,nop,timestamp 56250 858581>
16:04:09.613247 dsl.1008 > nbftp.ssh: P 747722286:747722306(20) ack 2709672657 win 17520 <nop,nop,timestamp 56266 858581>
16:04:09.730244 nbftp.ssh > dsl.1008: R 2709672657:2709672657(0) win 0


From tanis (powermac) to NAT box

16:03:18.790279 tanis.65509 > nbftp.ssh: . ack 1729 win 17520 <nop,nop,timestamp 56165 858580>
16:03:18.796513 nbftp.ssh > tanis.65509: . 1729:2229(500) ack 220 win 17520 <nop,nop,timestamp 858581 56165> [tos 0x10]
16:03:18.797164 tanis.65509 > nbftp.ssh: . ack 2361 win 16888 <nop,nop,timestamp 56165 858581>
16:03:54.787144 tanis.65509 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56237 858581>
16:03:55.616299 tanis.65509 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56238 858581>
16:03:57.615826 tanis.65509 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56242 858581>
16:04:01.614935 tanis.65509 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56250 858581>
16:04:09.613049 tanis.65509 > nbftp.ssh: P 220:240(20) ack 2361 win 17520 <nop,nop,timestamp 56266 858581>
16:04:09.730362 nbftp.ssh > tanis.65509: R 2709672657:2709672657(0) win 0

Second one:
From NAT box to nbftp:

16:58:04.576570 ftp.ssh > dsl.1009: . ack 6684 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.702182 dsl.1009 > ftp.ssh: P 6724:6744(20) ack 10360 win 17520 <nop,nop,timestamp 62738 865152>
16:58:04.783038 ftp.ssh > dsl.1009: . ack 6684 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.784194 ftp.ssh > dsl.1009: . ack 6684 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.784691 dsl.1009 > ftp.ssh: P 6684:6744(60) ack 10360 win 17520 <nop,nop,timestamp 62738 865152>
16:58:05.351759 dsl.1009 > ftp.ssh: P 6684:6744(60) ack 10360 win 17520 <nop,nop,timestamp 62739 865152>
16:58:06.072092 dsl.1009 > ftp.ssh: P 6744:6764(20) ack 10360 win 17520 <nop,nop,timestamp 62741 865152>
16:58:07.351278 dsl.1009 > ftp.ssh: P 6684:6764(80) ack 10360 win 17520 <nop,nop,timestamp 62743 865152>
16:58:11.350407 dsl.1010 > ftp.ssh: P 2263583694:2263583774(80) ack 1283869543 win 17520 <nop,nop,timestamp 62751 865152>
16:58:19.348588 dsl.1011 > ftp.ssh: P 2263583694:2263583774(80) ack 1283869543 win 17520 <nop,nop,timestamp 62767 865152>
16:58:19.478348 ftp.ssh > dsl.1011: R 1283869543:1283869543(0) win 0

From power mac to NAT box:

16:58:04.576687 ftp.ssh > tanis.65505: . ack 6080 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.702056 tanis.65505 > ftp.ssh: P 6120:6140(20) ack 9409 win 17520 <nop,nop,timestamp 62738 865152>
16:58:04.783151 ftp.ssh > tanis.65505: . ack 6080 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.784295 ftp.ssh > tanis.65505: . ack 6080 win 17520 <nop,nop,timestamp 865153 62737> [tos 0x10]
16:58:04.784577 tanis.65505 > ftp.ssh: P 6080:6140(60) ack 9409 win 17520 <nop,nop,timestamp 62738 865152>
16:58:05.351615 tanis.65505 > ftp.ssh: P 6080:6140(60) ack 9409 win 17520 <nop,nop,timestamp 62739 865152>
16:58:06.071935 tanis.65505 > ftp.ssh: P 6140:6160(20) ack 9409 win 17520 <nop,nop,timestamp 62741 865152>
16:58:07.351128 tanis.65505 > ftp.ssh: P 6080:6160(80) ack 9409 win 17520 <nop,nop,timestamp 62743 865152>
16:58:11.350180 tanis.65505 > ftp.ssh: P 6080:6160(80) ack 9409 win 17520 <nop,nop,timestamp 62751 865152>
16:58:19.348371 tanis.65505 > ftp.ssh: P 6080:6160(80) ack 9409 win 17520 <nop,nop,timestamp 62767 865152>
16:58:19.478480 ftp.ssh > tanis.65505: R 1283869543:1283869543(0) win 0


Third one:
From NAT box to nbftp:

17:03:12.949990 ftp.ssh > dsl.1015: P 5544:5584(40) ack 3224 win 17520 <nop,nop,timestamp 865769 63350> [tos 0x10]
17:03:12.950537 dsl.1015 > ftp.ssh: . ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:12.988916 dsl.1015 > ftp.ssh: P 3724:3744(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.088799 dsl.1015 > ftp.ssh: P 3744:3764(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.168774 dsl.1015 > ftp.ssh: P 3764:3784(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.308861 dsl.1015 > ftp.ssh: P 3784:3804(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.388813 dsl.1015 > ftp.ssh: P 3804:3824(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.568812 dsl.1015 > ftp.ssh: P 3824:3844(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.719563 dsl.1015 > ftp.ssh: P 3844:3864(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.818674 dsl.1015 > ftp.ssh: P 3864:3884(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:13.938578 dsl.1015 > ftp.ssh: P 3884:3904(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.018582 dsl.1015 > ftp.ssh: P 3904:3924(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.198521 dsl.1015 > ftp.ssh: P 3924:3944(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.278544 dsl.1015 > ftp.ssh: P 3944:3964(20) ack 5584 win 17520 <nop,nop,timestamp 63358 865765>
17:03:14.778692 dsl.1015 > ftp.ssh: P 3224:3964(740) ack 5584 win 17520 <nop,nop,timestamp 63358 865765>
17:03:16.954296 ftp.ssh > dsl.1015: P 5544:5584(40) ack 3224 win 17520 <nop,nop,timestamp 865777 63350> [tos 0x10]
17:03:16.954856 dsl.1015 > ftp.ssh: . ack 5584 win 17520 <nop,nop,timestamp 63363 865765>
17:03:20.777351 dsl.1016 > ftp.ssh: P 2994441819:2994442559(740) ack 2895994808 win 17520 <nop,nop,timestamp 63370 865765>
17:03:20.904228 ftp.ssh > dsl.1016: R 2895994808:2895994808(0) win 0
17:03:24.945327 ftp.ssh > dsl.1015: P 5544:5584(40) ack 3224 win 17520 <nop,nop,timestamp 865793 63350> [tos 0x10]
17:03:24.945513 dsl.1015 > ftp.ssh: R 2994441819:2994441819(0) win 0

From powermac to NAT box:

17:03:12.950128 ftp.ssh > tanis.65503: P 5544:5584(40) ack 3224 win 17520 <nop,nop,timestamp 865769 63350> [tos 0x10]
17:03:12.950416 tanis.65503 > ftp.ssh: . ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:12.988794 tanis.65503 > ftp.ssh: P 3724:3744(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.088669 tanis.65503 > ftp.ssh: P 3744:3764(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.168628 tanis.65503 > ftp.ssh: P 3764:3784(20) ack 5584 win 17520 <nop,nop,timestamp 63355 865765>
17:03:13.308708 tanis.65503 > ftp.ssh: P 3784:3804(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.388687 tanis.65503 > ftp.ssh: P 3804:3824(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.568659 tanis.65503 > ftp.ssh: P 3824:3844(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.719434 tanis.65503 > ftp.ssh: P 3844:3864(20) ack 5584 win 17520 <nop,nop,timestamp 63356 865765>
17:03:13.818550 tanis.65503 > ftp.ssh: P 3864:3884(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:13.938448 tanis.65503 > ftp.ssh: P 3884:3904(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.018446 tanis.65503 > ftp.ssh: P 3904:3924(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.198372 tanis.65503 > ftp.ssh: P 3924:3944(20) ack 5584 win 17520 <nop,nop,timestamp 63357 865765>
17:03:14.278411 tanis.65503 > ftp.ssh: P 3944:3964(20) ack 5584 win 17520 <nop,nop,timestamp 63358 865765>
17:03:14.778524 tanis.65503 > ftp.ssh: P 3224:3964(740) ack 5584 win 17520 <nop,nop,timestamp 63358 865765>
17:03:16.954438 ftp.ssh > tanis.65503: P 5544:5584(40) ack 3224 win 17520 <nop,nop,timestamp 865777 63350> [tos 0x10]
17:03:16.954739 tanis.65503 > ftp.ssh: . ack 5584 win 17520 <nop,nop,timestamp 63363 865765>
17:03:20.777118 tanis.65503 > ftp.ssh: P 3224:3964(740) ack 5584 win 17520 <nop,nop,timestamp 63370 865765>
17:03:20.904357 ftp.ssh > tanis.65503: R 2895994808:2895994808(0) win 0

>How-To-Repeat:

ssh from my powermac to ftp.netbsd.org, and type a lot of text.

>Fix:

Unknown.

>Release-Note:
>Audit-Trail:
>Unformatted: