tech-net archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Two NetBSD networking stack questions

On Wed, Nov 9, 2011 at 11:38 AM, David Young <> wrote:
> On Tue, Nov 08, 2011 at 04:22:46PM -0700, Andrew Collins wrote:
>> I'm working on an embedded NetBSD product, and I've hit two issues
>> with the networking stack that I'm curious about.
>> The first question is related to ether_output's route lookups.  In
>> cases where a route has RTF_GATEWAY is set, and there is no rt_gwroute
>> or rt_gwroute isn't RTF_UP, a route lookup is done, and after
>> obtaining a usable route for the gateway the rt_ifp is checked against
>> the current ifp to make sure the route is on the same interface we're
>> trying to send from.  What I find odd is that this is *not* done for
>> cases where we find a usable rt_gwroute right off the bat, and in
>> practice, I see certain situations where a cloned route on another
>> interface is picked up for rt_gwroute, then used during the ARP
>> request for the gateway.  It ends up sort of working, because
>> arpresolve uses the current ifp to make the ARP request so it gets
>> sent out the right interface, but we end up in this weird situation
>> where we have a route on a different ifp holding information about a
>> link level address for a host off of our ifp.  Is there a reason that
>> rt_gwroute isn't checked to ensure rt_ifp == ifp even when we don't do
>> a lookup?
> How does one reproduce this situation where rt_gwroute->rt_ifp != ifp?

The way I ran into was it was with a default route which was an
interface route on an interface capable of arp, and hence was cloning.
 A packet happened to get
sent to an address that later on ends up being the gateway for a route
on a separate interface.  When this packet gets sent we create a
cloned route on the original
interface.  Shortly afterwards, the route on the 2nd interface gets
created with a gateway pointing to that address, and rt_gwroute picks
up the cloned route on other

I can repro in a similar fashion:

Destination        Gateway            Flags    Refs      Use    Mtu  Interface
default            link#10            UCS         1        0      -  vlan1
192.168.0/24       link#10            UC          0        0      -  vlan1
192.168.10/24       link#7             UC          2        0      -  vlan2

In this setup, ping out to, get:

Destination        Gateway            Flags    Refs      Use    Mtu  Interface
default            link#10            UCS         1        0      -  vlan1
192.168.0/24       link#10            UC          0        0      -  vlan1
192.168.10/24       link#7             UC          2        0      -  vlan2       link#10            UHLc        1       38      -  vlan1

Now bring up an interface at, and then set up a
gateway route for an address reachable through this interface:

Destination        Gateway            Flags    Refs      Use    Mtu  Interface
default            link#10            UCS         1        0      -  vlan1
192.168.0/24       link#10            UC          0        0      -  vlan1
192.168.10/24       link#7             UC          2        0      -  vlan2
192.168.20/24      link#12            UC          0        0      -  vlan3       link#10            UHLc        1       38      -  vlan1      UGHS        0        0      -  vlan3

After doing this, if you ping,  you'll see arps out on
vlan3 for, but after dropping into ddb and looking in

rtentry=0x8669b990 flags=0x807 refcnt=0 use=4 expire=0
 mask=[NULL] gw=[16,2,0,0,192,168,20,100,0,0,0,0,0,0,0,0]
 ifp=0xcaf8e020 (vlan3) ifa=0xc01bce00
 gwroute=0x8669baa0 llinfo=0x0

rtentry=0x8669baa0 flags=0x2405 refcnt=1 use=1 expire=1320944433
 mask=[NULL] gw=[19,18,10,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
 ifp=0xc01bb800 (vlan1) ifa=0xc01bca00
 gwroute=0x0 llinfo=0xc00253e0
  la_rt=0x8669baa0 la_hold=0x862b5300, la_asked=0x3

The gwroute of our route points to a route on vlan1, and we have the
cloned route on vlan1 holding llinfo and la_hold for information
meant for vlan3.

This is obviously a pretty weird setup, but since we check the other
cases for rt_ifp == ifp in ether_output, and it's *possible* to get
into this state,
I don't see why we don't check this for all cases.

>> The second question is related to PPP.  I'm running into a case where
>> packets getting sent to a local address are actually sent out on the
>> wire for PPP devices.  For example:
>> pppoe1: flags=8851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST> mtu 1492
>>         inet -> netmask 0xff000000
>> If I ping, I actually see packets getting emitted on the
>> wire where the src addr == dst addr.  Is this espected behavior?
> What does 'route -n get' say?

# route -n get
   route to:
destination: default
       mask: default
 local addr:
  interface: pppoe1
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0         0         0

# netstat -r -n
Routing tables

Destination        Gateway            Flags    Refs      Use    Mtu  Interface
default        UGS         1        1      -  pppoe1          UH          2      458  33176  lo0      UH          1        0      -  pppoe1

I tried this is as an interface route as well and got the same result
(packets on the wire that is).  Am I just screwing up the default

> Dave
> --
> David Young             OJC Technologies is now Pixo
>     Urbana, IL   (217) 344-0444 x24

Home | Main Index | Thread Index | Old Index