On Tue, 15 May 2001, Jun-ichiro itojun Hagino wrote:

>
> >I have a gif tunnel configured between two NetBSD/i386 1.5.1_BETA
> >systems. Without IPSec transport mode configured for the underlying
> >real IP addresses, the gif tunnel with transport OSPF multicast
> >packets (ie. Zebra OSPF hello to 224.0.0.5) without problem.
> >
> >If I configure IPSec between the real IP addresses of the tunnel, the
> >OSPF multicast packets never make it to the other side of the tunnel.
> >Normal packets (TCP, UDP, ICMP) work fine.
>
> 	could you try running tcpdump on gif interface as well as ethernet?
> 	watch both inbound and outbound interface.  what kind of traffic
> 	do you see and what kind of traffic you don't see?
> 	# tcpdump -n -i gif0	(packet with inner header)
> 	# tcpdump -n -i tlp0	(packet with innter + outer header)
> 	the point is to know which layer is losing (or corrupting) packets.
>
> 	watch netstat -sn.  which number increases while you run ospfd?
> 	taking diff between netstat -sn output always help.
>

The packets appear to get lost on the destination machine during IPSec
processing or gif unpacking: The packet arrives on the real interface
but never comes out of the gif tunnel:

sending gif:
20:40:54.473567 10.10.1.10.143 > 10.11.11.12.2119: P 46:69(23) ack 31 win 10494 <nop,nop,timestamp 502102177 52710547> (DF)
20:40:54.554669 10.0.0.5 > 224.0.0.5:  OSPFv2-hello 44: rtrid 10.10.1.1 backbone [ttl 1]
20:41:04.573470 10.0.0.5 > 224.0.0.5:  OSPFv2-hello 44: rtrid 10.10.1.1 backbone [ttl 1]

sending tlp0:
20:40:54.473711 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb722)
20:40:54.554764 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb723)
20:41:04.573575 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb724)


receiving tlp0:
23:40:54.521550 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb722)
23:40:54.602720 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb723)
23:41:04.621764 x.x.x.x > y.y.y.y ESP(spi=2001,seq=0x1cb724)

receiving gif:
23:40:54.533225 10.11.11.12.2119 > 10.10.1.10.143: . ack 70 win 31856 <nop,nop,timestamp 52710558 502102177> (DF)
[OSPF hello packets missing here]
23:41:07.591626 10.11.1.10.912 > 10.10.1.21.669:  udp 28 (DF)

(The sending machine is on Pacific time, the receiving machine on
Eastern time).

The output of two consecutive  netstat -sn is more difficult to
interpret since the connection and the machines are currently in
production and carry a lot of traffic on other interfaces. The two
netstats for the diff below were taken on the destination machine at 2
seconds apart, with the lost packet arriving at at 00:11:57.


***************
*** 1,6 ****
! Tue May 15 00:11:56 EDT 2001
  ip:
!       104443918 total packets received
        0 bad header checksums
        0 with size smaller than minimum
        0 with data size < data length
--- 1,6 ----
! Tue May 15 00:11:58 EDT 2001
  ip:
!       104443934 total packets received
        0 bad header checksums
        0 with size smaller than minimum
        0 with data size < data length
***************
*** 13,24 ****
        0 malformed fragments dropped
        88 fragments dropped after timeout
        131 packets reassembled ok
!       26192115 packets for this host
        0 packets for unknown/unsupported protocol
!       78042781 packets forwarded (0 packets fast forwarded)
!       188755 packets not forwardable
        41896 redirects sent
!       33737509 packets sent from this host
        256395 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        1 output packet discarded due to no route
--- 13,24 ----
        0 malformed fragments dropped
        88 fragments dropped after timeout
        131 packets reassembled ok
!       26192125 packets for this host
        0 packets for unknown/unsupported protocol
!       78042786 packets forwarded (0 packets fast forwarded)
!       188756 packets not forwardable
        41896 redirects sent
!       33737510 packets sent from this host
        256395 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        1 output packet discarded due to no route
***************
*** 58,65 ****
        0 membership reports received for groups to which we belong
        0 membership reports sent
  tcp:
!       862663 packets sent
!               516485 data packets (341864677 bytes)
                5319 data packets (3883264 bytes) retransmitted
                223546 ack-only packets (260059 delayed)
                0 URG only packets
--- 58,65 ----
        0 membership reports received for groups to which we belong
        0 membership reports sent
  tcp:
!       862664 packets sent
!               516486 data packets (341864785 bytes)
                5319 data packets (3883264 bytes) retransmitted
                223546 ack-only packets (260059 delayed)
                0 URG only packets
***************
*** 87,93 ****
        92115 connections established (including accepts)
        92643 connections closed (including 1198 drops)
        468 embryonic connections dropped
!       512739 segments updated rtt (of 472301 attempts)
        6255 retransmit timeouts
                39 connections dropped by rexmit timeout
        0 persist timeouts (resulting in 0 dropped connections)
--- 87,93 ----
        92115 connections established (including accepts)
        92643 connections closed (including 1198 drops)
        468 embryonic connections dropped
!       512739 segments updated rtt (of 472302 attempts)
        6255 retransmit timeouts
                39 connections dropped by rexmit timeout
        0 persist timeouts (resulting in 0 dropped connections)
***************
*** 113,130 ****
        179 duplicate SYNs received for entries already in the cache
        14 SYNs dropped (no route or no space)
  udp:
!       5315093 datagrams received
        0 with incomplete header
        0 with bad data length field
        0 with bad checksum
        5228 dropped due to no socket
!       2930740 broadcast/multicast datagrams dropped due to no socket
        0 dropped due to full socket buffers
        2379125 delivered
!       5091637 PCB hash misses
        2402899 datagrams output
  ipsec:
!       18044699 inbound packets processed successfully
        0 inbound packets violated process security policy
        24112 inbound packets with no SA available
        2 invalid inbound packets
--- 113,130 ----
        179 duplicate SYNs received for entries already in the cache
        14 SYNs dropped (no route or no space)
  udp:
!       5315102 datagrams received
        0 with incomplete header
        0 with bad data length field
        0 with bad checksum
        5228 dropped due to no socket
!       2930749 broadcast/multicast datagrams dropped due to no socket
        0 dropped due to full socket buffers
        2379125 delivered
!       5091646 PCB hash misses
        2402899 datagrams output
  ipsec:
!       18044700 inbound packets processed successfully
        0 inbound packets violated process security policy
        24112 inbound packets with no SA available
        2 invalid inbound packets
***************
*** 135,141 ****
        0 inbound packets considered authentic
        0 inbound packets failed on authentication
        ESP input histogram:
!               des-cbc: 18044701
        28636858 outbound packets processed successfully
        0 outbound packets violated process security policy
        0 outbound packets with no SA available
--- 135,141 ----
        0 inbound packets considered authentic
        0 inbound packets failed on authentication
        ESP input histogram:
!               des-cbc: 18044702
        28636858 outbound packets processed successfully
        0 outbound packets violated process security policy
        0 outbound packets with no SA available
***************
*** 339,345 ****
          E Timers: [     0 ticks]
               0 timers set          0 timers expired        0 timers
cancelled

!         C Timers: [4927698 ticks]
               0 timers set          0 timers expired        0 timers
cancelled
               0 inactive timers cancelled

--- 339,345 ----
          E Timers: [     0 ticks]
               0 timers set          0 timers expired        0 timers
cancelled

!         C Timers: [4927706 ticks]
               0 timers set          0 timers expired        0 timers
cancelled
               0 inactive timers cancelled


> itojun
>

-- 
    - aew