Subject: device disable/enable and packet
To: None <tech-net@netbsd.org>
From: enami tsugutomo <enami@sm.sony.co.jp>
List: tech-net
Date: 10/30/2002 19:43:19
Some of network driver like fxp or ex disable or enable device when an
interface is down or up respectively.

When an interface is up, kernel sends some packets (and they are
actually dma'ed to controller, but won't be sent to wire until link is
up).  On the other hands, kernel will initialize the driver (i.e.,
call fxp_init etc) for some reason.  If the initializing is initiated
before the packet is really sent to wire, the packet is lost.

If I modify fxp_start to just return when IFM_ACTIVE isn't set, it
looks like all packets are sent to wire.  But, of course you can do
this only when driver or device can report link status.

Or, ifconfig can have special command to enable/disable device?  But
IMHO, it is just hide the problem most case (i.e., problem won't
appear if device isn't disabled).

How do you think?  (this is ethernet and packet allways may lost? :-)

enami.

Here is an example:

Packet kernel thought sent:

(ifconfig fxp0 up; v6 stateless auto addrconf starts)

fxp_init: status = 0x0, active = 0x0
fxp_init: status = 0x1, active = 0x22
fxp_start: dequeue
  mbuf 0xc21eb000 type 2 len 54 next 0xc21eb200 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 ff a1 04 dd 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 00 00 00 00 00 00 00 00 00 00
  0020: 00 00 00 00 00 00 ff 02 00 00 00 00 00 00 00 00
  0030: 00 01 ff a1 04 dd
fxp_txintr: 1
fxp_init: status = 0x1, active = 0x22
fxp_start: dequeue
  mbuf 0xc21eb300 type 2 len 54 next 0xc21eb500 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 74 b7 9b 93 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 00 00 00 00 00 00 00 00 00 00
  0020: 00 00 00 00 00 00 ff 02 00 00 00 00 00 00 00 00
  0030: 00 02 74 b7 9b 93
fxp_txintr: 1
fxp_init: status = 0x1, active = 0x22
fxp_init: status = 0x3, active = 0x100026
fxp_start: dequeue
  mbuf 0xc21eb500 type 1 len 78 next 0x0 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 ff a1 04 dd 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 18 3a ff 00 00 00 00 00 00 00 00 00 00
  0020: 00 00 00 00 00 00 ff 02 00 00 00 00 00 00 00 00
  0030: 00 01 ff a1 04 dd 87 00 bd 28 00 00 00 00 fe 80
  0040: 00 00 00 00 00 00 02 02 b3 ff fe a1 04 dd
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21ebc00 type 2 len 54 next 0xc21eba00 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 ff a1 04 dd 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 fe 80 00 00 00 00 00 00 02 02
  0020: b3 ff fe a1 04 dd ff 02 00 00 00 00 00 00 00 00
  0030: 00 01 ff a1 04 dd
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21ebb00 type 2 len 54 next 0xc21eb900 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 74 b7 9b 93 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 fe 80 00 00 00 00 00 00 02 02
  0020: b3 ff fe a1 04 dd ff 02 00 00 00 00 00 00 00 00
  0030: 00 02 74 b7 9b 93
fxp_txintr: 1
fxp_init: status = 0x3, active = 0x100026
fxp_start: dequeue
  mbuf 0xc21ebe00 type 2 len 54 next 0xc21eb800 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 00 00 02 02 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 fe 80 00 00 00 00 00 00 02 02
  0020: b3 ff fe a1 04 dd ff 02 00 00 00 00 00 00 00 00
  0030: 00 00 00 00 02 02
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21daa00 type 2 len 54 next 0xc21dac00 flags 0x202<PKTHDR,MCAST>
  0000: 33 33 00 00 02 02 00 02 b3 a1 04 dd 86 dd 60 00
  0010: 00 00 00 20 00 01 fe 80 00 00 00 00 00 00 02 02
  0020: b3 ff fe a1 04 dd ff 02 00 00 00 00 00 00 00 00
  0030: 00 00 00 00 02 02
fxp_txintr: 1
fxp_init: status = 0x3, active = 0x100026
fxp_init: status = 0x3, active = 0x100026
fxp_init: status = 0x3, active = 0x100026
fxp_start: dequeue
  mbuf 0xc21e8f00 type 1 len 42 next 0x0 flags 0x102<PKTHDR,BCAST>
  0000: ff ff ff ff ff ff 00 02 b3 a1 04 dd 08 06 00 01
  0010: 08 00 06 04 00 01 00 02 b3 a1 04 dd c0 a8 01 01
  0020: 00 00 00 00 00 00 c0 a8 01 01
fxp_txintr: 1

(ifconfig fxp0 down; link down)

fxp_init: status = 0x1, active = 0x22
fxp_init: status = 0x1, active = 0x22

(ifconfig fxp0 192.168.1.1; v4 address is assigned)

fxp_start: dequeue
  mbuf 0xc21e9d00 type 1 len 66 next 0x0 flags 0x102<PKTHDR,BCAST>
  0000: ff ff ff ff ff ff 00 02 b3 a1 04 dd 08 00 45 00
  0010: 00 34 01 ca 00 00 01 11 33 9f c0 a8 01 01 c0 a8
  0020: 01 ff 02 08 02 08 00 20 76 3c 01 01 00 00 00 00
  0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0040: 00 10
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21e9c00 type 1 len 42 next 0x0 flags 0x202<PKTHDR,MCAST>
  0000: 01 00 5e 00 00 02 00 02 b3 a1 04 dd 08 00 45 00
  0010: 00 1c 01 cd 00 00 01 01 16 69 c0 a8 01 01 e0 00
  0020: 00 02 0a 00 f5 ff 00 00 00 00
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21e8800 type 1 len 42 next 0x0 flags 0x202<PKTHDR,MCAST>
  0000: 01 00 5e 00 00 02 00 02 b3 a1 04 dd 08 00 45 00
  0010: 00 1c 01 ce 00 00 01 01 16 68 c0 a8 01 01 e0 00
  0020: 00 02 0a 00 f5 ff 00 00 00 00
fxp_txintr: 1
fxp_start: dequeue
  mbuf 0xc21e8d00 type 1 len 42 next 0x0 flags 0x202<PKTHDR,MCAST>
  0000: 01 00 5e 00 00 02 00 02 b3 a1 04 dd 08 00 45 00
  0010: 00 1c 01 cf 00 00 01 01 16 67 c0 a8 01 01 e0 00
  0020: 00 02 0a 00 f5 ff 00 00 00 00
fxp_txintr: 1


The packet caught by peer tcpdump:

19:13:51.198431 :: > ff02::1:ffa1:4dd: icmp6: neighbor sol: who has fe80::202:b3ff:fea1:4dd
                         6000 0000 0018 3aff 0000 0000 0000 0000
                         0000 0000 0000 0000 ff02 0000 0000 0000
                         0000 0001 ffa1 04dd 8700 bd28 0000 0000
                         fe80 0000 0000 0000 0202 b3ff fea1 04dd
19:13:54.101564 fe80::202:b3ff:fea1:4dd > ff02::1:ffa1:4dd: HBH icmp6: multicast listener report max resp delay: 0 addr: ff02::1:ffa1:4dd [hlim 1]
                         6000 0000 0020 0001 fe80 0000 0000 0000
                         0202 b3ff fea1 04dd ff02 0000 0000 0000
                         0000 0001 ffa1 04dd 3a00 0100 0502 0000
                         8300 bda5 0000 0000 ff02 0000 0000 0000
                         0000 0001 ffa1 04dd
19:13:54.601512 fe80::202:b3ff:fea1:4dd > ff02::2:74b7:9b93: HBH icmp6: multicast listener report max resp delay: 0 addr: ff02::2:74b7:9b93 [hlim 1]
                         6000 0000 0020 0001 fe80 0000 0000 0000
                         0202 b3ff fea1 04dd ff02 0000 0000 0000
                         0000 0002 74b7 9b93 3a00 0100 0502 0000
                         8300 a60b 0000 0000 ff02 0000 0000 0000
                         0000 0002 74b7 9b93
19:13:56.919535 fe80::202:b3ff:fea1:4dd > ff02::202: HBH icmp6: multicast listener report max resp delay: 0 addr: ff02::202 [hlim 1]
                         6000 0000 0020 0001 fe80 0000 0000 0000
                         0202 b3ff fea1 04dd ff02 0000 0000 0000
                         0000 0000 0000 0202 3a00 0100 0502 0000
                         8300 c2a1 0000 0000 ff02 0000 0000 0000
                         0000 0000 0000 0202
19:14:04.831896 fe80::202:b3ff:fea1:4dd > ff02::202: HBH icmp6: multicast listener report max resp delay: 0 addr: ff02::202 [hlim 1]
                         6000 0000 0020 0001 fe80 0000 0000 0000
                         0202 b3ff fea1 04dd ff02 0000 0000 0000
                         0000 0000 0000 0202 3a00 0100 0502 0000
                         8300 c2a1 0000 0000 ff02 0000 0000 0000
                         0000 0000 0000 0202
19:14:47.259071 192.168.1.1 > 224.0.0.2: icmp: router solicitation [ttl 1]
                         4500 001c 01ce 0000 0101 1668 c0a8 0101
                         e000 0002 0a00 f5ff 0000 0000 0000 0000
                         0000 0000 0000 0000 0000 0000 0000
19:14:50.319117 192.168.1.1 > 224.0.0.2: icmp: router solicitation [ttl 1]
                         4500 001c 01cf 0000 0101 1667 c0a8 0101
                         e000 0002 0a00 f5ff 0000 0000 0000 0000
                         0000 0000 0000 0000 0000 0000 0000