Subject: ip_len byte order in fr_fastroute
To: None <tech-net@netbsd.org>
From: Ken Raeburn <raeburn@raeburn.org>
List: tech-net
Date: 11/04/2007 09:19:43
I've been having a problem with my gif tunnel in NetBSD/i386 3.1 and  
4.0RC2: It's basically become unusable.  A large portion of the  
packets going out have the length field of the encapsulated packet in  
host byte order rather than network byte order.  (I only ran 3.1  
briefly before updating to 4.0RC2; before that, 2.0 was working  
fairly well for me.)

My tunnel configuration uses ipfilter to force traffic originating on  
my local subnet to go out through the tunnel, because of ISP  
filters.  When a packet from a server here arrives on the NetBSD  
gateway box, the incoming filter sees that the source address is one  
of my public ones, and the destination is non-local, so it copies it  
out to the outgoing gif queue.  (It may be that only the packets  
originating locally but not on the NetBSD box itself are corrupted.   
The mail server on the NetBSD box itself, accessed with one of the  
addresses delegated to me via the tunnel, seems to be okay.)

I tweaked the code a little bit to set a hardware watchpoint on the  
length field in gif_output, and delete it in in_gif_output where I'd  
already determined the host-order value has been put in place.  The  
hardware watchpoint was tripped in fr_fastroute, with the following  
stack trace according to ddb:

fr_fastroute+0x18c
fr_check+0x605
fr_check_wrapper+0x97
pfil_run_hooks+0x91
ip_input+0x5dd
ipintr+0x24

fr_fastroute+0x18c is at line 1252:

		if (i) {
/* --> */		ip->ip_len = ntohs(ip->ip_len);
			ip->ip_off = ntohs(ip->ip_off);
		}

called from fr_check, fil.c:2730:

		} else if ((fdp->fd_ifp != NULL) &&
			   (fdp->fd_ifp != (struct ifnet *)-1)) {
			/* this is for to rules: */
			(void) fr_fastroute(fin->fin_m, mp, fin, fdp);
			m = *mp = NULL;
		}

So, if I've got it right, fr_fastroute gets a handle on the packet  
with its length in host order, sets it to network order for the call  
to the output routine, and then sets it back to host order, just in  
case the caller of fr_fastroute wants to do something else with the  
packet (e.g., in the dup-to case, I assume).  Then the gif soft  
interrupt handler is invoked, and outputs on the real network  
interface an encapsulated packet with its length in the host's byte  
order.

Is the packet eventually supposed to get its length restored to  
network order once again before the soft interrupt for gif can be  
serviced?  Or should I look at making a copy of the packet in  
gif_output, or maybe patch up the length in gif_intr or  
in_gif_output?  (I haven't looked to see if IPv6 tunnels have a  
similar problem, but I'm just using the routing table to get IPv6  
traffic to the right output queue.)

Ken