Subject: Re: tcpflow vs. lo0 on 2.0_BETA
To: Greg Troxel <gdt@ir.bbn.com>
From: Guy Harris <guy@alum.mit.edu>
List: tech-net
Date: 08/20/2004 16:26:03
> On tcpdump.org, I found the following commit:
>
> http://cvs.tcpdump.org/cgi-bin/cvsweb/libpcap/gencode.c? 
> r1=1.138&r2=1.139

The checkin comment for that change was:

	Handle DLT_NULL correctly - the AF_ value is in host byte order, which
	means that we should "htonl()" it before using it in BPF expressions
	*but*, if we're reading a capture file from a machine with the opposite
	byte order from ours, we should byte-swap it before "htonl()"ing it.

	Handle OpenBSD DLT_LOOP as well - it's like DLT_NULL except that the  
AF_
	value is in *network* byte order.

	Don't support checking for inbound or outbound packets except on those
	data link types that supply an inbound/outbound qualifier (DLT_SLIP and
	DLT_PPP) - this came from OpenBSD's libpcap, delta 1.12 to "gencode.c".

and the corresponding gencode.c code in current CVS tcpdump.org libpcap  
(and change 1.139 is in libpcap 0.6) is:

	case DLT_NULL:
	case DLT_LOOP:
	case DLT_ENC:
		/*
		 * For DLT_NULL, the link-layer header is a 32-bit
		 * word containing an AF_ value in *host* byte order,
		 * and for DLT_ENC, the link-layer header begins
		 * with a 32-bit work containing an AF_ value in
		 * host byte order.
		 *
		 * In addition, if we're reading a saved capture file,
		 * the host byte order in the capture may not be the
		 * same as the host byte order on this machine.
		 *
		 * For DLT_LOOP, the link-layer header is a 32-bit
		 * word containing an AF_ value in *network* byte order.
		 *
		 * XXX - AF_ values may, unfortunately, be platform-
		 * dependent; for example, FreeBSD's AF_INET6 is 24
		 * whilst NetBSD's and OpenBSD's is 26.
		 *
		 * This means that, when reading a capture file, just
		 * checking for our AF_INET6 value won't work if the
		 * capture file came from another OS.
		 */
		switch (proto) {

		case ETHERTYPE_IP:
			proto = AF_INET;
			break;

#ifdef INET6
		case ETHERTYPE_IPV6:
			proto = AF_INET6;
			break;
#endif

		default:
			/*
			 * Not a type on which we support filtering.
			 * XXX - support those that have AF_ values
			 * #defined on this platform, at least?
			 */
			return gen_false();
		}

		if (linktype == DLT_NULL || linktype == DLT_ENC) {
			/*
			 * The AF_ value is in host byte order, but
			 * the BPF interpreter will convert it to
			 * network byte order.
			 *
			 * If this is a save file, and it's from a
			 * machine with the opposite byte order to
			 * ours, we byte-swap the AF_ value.
			 *
			 * Then we run it through "htonl()", and
			 * generate code to compare against the result.
			 */
			if (bpf_pcap->sf.rfile != NULL &&
			    bpf_pcap->sf.swapped)
				proto = SWAPLONG(proto);
			proto = htonl(proto);
		}
		return (gen_cmp(0, BPF_W, (bpf_int32)proto));

"The BPF interpreter will convert it to network byte order" means that  
the load done by the BPF program will load the word assuming it's  
big-endian in the packet header, so if it was written to the savefile  
as little-endian, it'll be byte-swapped.

> So, I'd say tcpflow is incorrect and should expect host order, and
> byteswapped if (p->sf.rfile != NULL && p->sf.swapped).

Right, although "pcap_t" is opaque, so that should be "byteswapped if  
pcap_file(p) != NULL && pcap_is_swapped(p)".  In addition, a pcap_t is  
zeroed out when allocated, and "p->sf.swapped" is set only if a  
byte-swapped savefile is seen, so "byteswapped if pcap_is_swapped(p)"  
should be sufficient - "pcap_is_swapped(p)" will always be false on a  
live capture.

(It should also handle DLT_ENC, if that's defined, the same way it  
handles DLT_LOOP, and should handle DLT_LOOP as host byte order, in  
order to make OpenBSD users happy.)