tech-net archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: IFF_SIMPLEX vs bridge(4)
>>> [...apparent IFF_SIMPLEX vs bridge(4) issue...]
>> I don't have time now to post the full details [...]
> Hm. If you're seeing it twice, then either you're bridging it twice
> or the interface isn't really SIMPLEX. Do you see it twice from an
> external listener, on either port?
I don't (yet) know.
Here's the situation. The machine is 4.0 i386 Xen. It has five
hardware network interfaces, ne0 rtk0 ex0 vr0 sk0, but only two of them
(ne0 and vr0) are actually involved. (ex0 is also in use but isn't, I
believe, relevant; rtk0 and sk0 are up but unused.) It's being used to
play with some new (to us) DSL hardware; there are three CPEs connected
to it, of which only one is relevant. The DSL stuff seemed to be
working, but we were having some trouble getting it to behave in real
use; while trying to track it down, I ran into this issue.
Hardware connections:
- ex0 is connected to the house LAN.
- ne0 is connected to the DSLAM uplink.
- vr0 is connected to the relevant CPE's LAN port.
- rtk0 is connected to another CPE's LAN port (not used here).
- The DSLAM is configured for the relevant CPE's traffic to come out
the uplink tagged vlan 112. (The other CPEs are tagged vlans 18 and
19, but I think those don't matter.)
There are two domUs, d1 and d2 (well, there's a third set up, but it
isn't running and thus shouldn't have any relevance) and, of course,
the managing dom0.
Conceptually, what I am trying to set up looks like this:
House LAN ---------+---------------+------------
| |
+--+---+ +--+---+
| d1 | | d2 |
+--+---+ +--+---+
| |
CPE DSLAM uplink
But, because of xen, it's not quite that simple. Here's the truth
(including a private network between the domUs which does not appear on
the above simplified diagram but which I'm including here - for
completeness, not because I think it really bears on the problem):
House LAN
|
| +------------------------- dom0 -------------------------+
| | |
| | +---------+ |
| | | bridge0 | |
| | +-+--+--+-+ |
| | | | | |
+---+--------------+ | | |
| |ex0 | | |
| |10.10.10.39/23 | | |
| | | | +------ domU d1 ------+ |
| | | | | |
| +---------+ | | | xennet0 | |
| | bridge1 | | +---------+ 00:16:3e:65:11:19 | |
| +--+---+--+ | | 10.10.10.59/23 | |
| | | | | | |
| | | | | xennet1 | |
CPE ----+-----+ +-------|------------+ 00:16:3e:0c:7f:99 | |
|vr0 | | 172.16.1.1/24 | |
| | | | |
| | | xennet2 | |
DSLAM ----+--------+ | +-----+ 00:16:3e:7f:f9:1e | |
|ne0 | | | | 10.0.0.1/24 | |
| | | | +---------------------+ |
| | | +---+-----+ |
| +----+----+ | | bridge4 | |
| | vlanif | | +---+-----+ |
| | | | | +------ domU d2 ------+ |
| | VLAN | | | | | |
| | | | | | xennet0 | |
| | vlan112 | +------|-----+ 00:16:3e:48:f5:90 | |
| +----+----+ | | 10.10.10.61/23 | |
| 172.16.1.2 | | | |
| | | | xennet1 | |
| | +-----------|-----+ 00:16:3e:16:3b:a4 | |
| | | | | 172.16.1.3/24 | |
| | | | | | |
| +--+---+--+ | | xennet2 | |
| | bridge2 | +-----+ 00:16:3e:16:c1:38 | |
| +---------+ | 10.0.0.2/24 | |
| +---------------------+ |
| |
+--------------------------------------------------------+
In text form, here's the config:
- dom0 ex0 is configured 10.10.10.39/23.
- ne0, vr0, and rtk0 are up but have no addresses of their own.
- There are five bridge interfaces, bridge0 through bridge4.
- Each domU's xennet0 is configured with an address in 10.10.10.0/23
and its corresponding xvif interface is a member of bridge0.
- Each domU's xennet2 is configured with an address in 10.0.0.0/24 and
its corresponding xvif interface is a member of bridge4.
- bridge1 bridges vr0 and the xvif corresponding to domU d1's xennet1.
- domU d1's xennet1 is configured with 172.16.1.1/24.
- bridge2 bridges vlan112 and the xvif corresponding to domU d2's
xennet1.
- domU d2's xennet1 is configured with 172.16.1.3/24.
- vlan112 is configured "vlan 112 vlanif ne0" and has address
172.16.1.2/24.
To reproduce the problem: on d2 I tcpdump xennet1 and on the dom0 I
tcpdump vr0 and vlan112. Then I have d1 "ping -n -c 1 172.16.1.3".
The xennet1 and vlan112 tcpdumps show
00:16:3e:0c:7f:99 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp
who-has 172.16.1.3 tell 172.16.1.1
00:16:3e:16:3b:a4 > 00:16:3e:0c:7f:99, ethertype ARP (0x0806), length 42: arp
reply 172.16.1.3 is-at 00:16:3e:16:3b:a4
but the tcpdump on vr0 shows the request twice
00:16:3e:0c:7f:99 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp
who-has 172.16.1.3 tell 172.16.1.1
00:16:3e:0c:7f:99 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp
who-has 172.16.1.3 tell 172.16.1.1
00:16:3e:16:3b:a4 > 00:16:3e:0c:7f:99, ethertype ARP (0x0806), length 64: arp
reply 172.16.1.3 is-at 00:16:3e:16:3b:a4
and when I "brconfig bridge1", it says
00:16:3e:16:3b:a4 vr0 1179 flags=0<>
00:16:3e:0c:7f:99 vr0 1179 flags=0<>
However, every interface in sight (vr0, ne0, xvif*.*, vlan112, and even
ex0) turns out to be marked SIMPLEX, and bridge2 has got it right:
00:16:3e:16:3b:a4 xvif10.1 1194 flags=0<>
00:16:3e:0c:7f:99 vlan112 1194 flags=0<>
So it's not just SIMPLEX interacting badly with bridge(4), but I'm not
sure what it is.
Having d1 ping 172.16.1.2 works just as badly.
It has something to do with broadcasts, though; if I manually set an
ARP entry for 172.16.1.2 or 172.16.1.3 with the correct MAC and *then*
ping the relevant address, bridge1 learns the affected MAC address
correctly, and traffic works fine - as long as the ARP entry is around.
And if - even with the ARP entries in place - I ping 172.16.1.255, the
broadcast echo request is duplicated according to vr0's tcpdump, and
bridge1 moves the affected MAC address to vr0. (Only 172.16.1.1
actually _responds_ to the broadcast ping, but that appears to be
irrelevant.) Further unicast traffic from d1 - such as a ping to
172.16.1.2 or .3 using the hardwired ARP table entries - moves it back
again.
I should check out the "external listener" question - interpose a hub
between vr0 and the CPE, snoop it, and see what is actually present on
the wire. I can probably do that sometime Thursday - I don't expect to
get back to this before then. (The lack of a duplicate on vlan112
argues that it isn't duplicated on the wire, but nothing beats actually
checking to see. :)
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse%rodents-montreal.org@localhost
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Home |
Main Index |
Thread Index |
Old Index