NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

kern/41874: in-kernel NFS server hates network interface aliases



>Number:         41874
>Category:       kern
>Synopsis:       in-kernel NFS server hates network interface aliases
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Aug 12 01:35:00 +0000 2009
>Originator:     Erik E. Fair
>Release:        NetBSD 5.0_STABLE
>Organization:
        The NetBSD Project
>Environment:
System: NetBSD digital.clock.org 5.0_STABLE NetBSD 5.0_STABLE (FLAPJACK2) #1: 
Wed Aug  5 16:42:56 PDT 2009 
root%fast.clock.org@localhost:/var/obj/sys/arch/sparc64/compile/FLAPJACK2
Architecture: sparc64
Machine: sparc64

total memory = 2048 MB
mainbus0 (root): SUNW,UltraAX-i2 (Netra T1 200): hostid 0xdeadbeef
cpu0 at mainbus0: SUNW,UltraSPARC-IIe @ 500 MHz, UPA id 0
cpu0: 32K instruction (32 b/l), 16K data (32 b/l), 1024K external (64 b/l)

kernel config:

include "arch/sparc64/conf/std.sparc64"
options         INCLUDE_CONFIG_FILE     # embed config file in kernel binary
ident           "FLAPJACK2"
maxusers        64
options         SUN4U           # sun4u - UltraSPARC
config          netbsd  root on ? type ?
options         KTRACE
options         SYSVMSG         # System V message queues
options         SYSVSEM         # System V semaphores
options         SYSVSHM         # System V shared memory
options         P1003_1B_SEMAPHORE      # p1003.1b semaphore support 
options         USERCONF        # userconf(4) support
options         SYSCTL_INCLUDE_DESCR    # Include sysctl descriptions in kernel
options         NFS_BOOT_BOOTPARAM
options         NFS_BOOT_DHCP
options         DDB                     # kernel dynamic debugger
options         DDB_HISTORY_SIZE=100    # enable history editing in DDB
options         DDB_VERBOSE_HELP        # enable verbose online help
options         DIAGNOSTIC      # extra kernel sanity checking
options         SCSIVERBOSE
options         PCIVERBOSE
options         MIIVERBOSE      # verbose PHY autoconfig messages
options         COMPAT_43       # 4.3BSD system interfaces
options         COMPAT_09       # NetBSD 0.9 binary compatibility
options         COMPAT_10       # NetBSD 1.0 binary compatibility
options         COMPAT_11       # NetBSD 1.1 binary compatibility
options         COMPAT_12       # NetBSD 1.2 binary compatibility
options         COMPAT_13       # NetBSD 1.3 binary compatibility
options         COMPAT_14       # NetBSD 1.4 binary compatibility
options         COMPAT_15       # NetBSD 1.5 binary compatibility
options         COMPAT_16       # NetBSD 1.6 binary compatibility
options         COMPAT_20       # NetBSD 2.0 binary compatibility
options         COMPAT_30       # NetBSD 3.0 binary compatibility
options         COMPAT_40       # NetBSD 4.0 binary compatibility
options         COMPAT_SUNOS    # SunOS 4.x binary compatibility
options         COMPAT_SVR4     # SunOS 5.x binary compatibility
options         COMPAT_SVR4_32  # SunOS 5.x 32-bit binary compatibility -- 
64-bit only
options         COMPAT_NETBSD32 # NetBSD/sparc binary compatibility -- 64-bit 
only
options         EXEC_AOUT       # execve(2) support for a.out binaries
options         EXEC_ELF32      # Exec module for SunOS 5.x binaries.
options         COMPAT_BSDPTY   # /dev/[pt]ty?? ptys.
file-system     FFS             # Berkeley Fast Filesystem
file-system     NFS             # Sun NFS-compatible filesystem client
file-system     KERNFS          # kernel data-structure filesystem
file-system     NULLFS          # NULL layered filesystem
file-system     OVERLAY         # overlay file system
file-system     MFS             # memory-based filesystem
file-system     FDESC           # user file descriptor filesystem
file-system     PROCFS          # /proc
file-system     CD9660          # ISO 9660 + Rock Ridge file system
file-system     UNION           # union file system
file-system     MSDOSFS         # MS-DOS FAT filesystem(s).
file-system     PTYFS           # /dev/pts/N support
file-system     SMBFS           # experimental - CIFS; also needs nsmb (below)
file-system     TMPFS           # Efficient memory file-system
file-system     PUFFS           # Userspace file systems (e.g. ntfs-3g & sshfs)
options         NFSSERVER       # Sun NFS-compatible filesystem server
options         QUOTA           # FFS quotas
options         SOFTDEP         # FFS soft updates support.
options         INET            # IP (Internet Protocol) v4
options         IPSEC           # IP security
options         IPSEC_ESP       # IP security (encryption part; define w/IPSEC)
options         IPSEC_NAT_T     # IPsec NAT traversal (NAT-T)
options         GATEWAY         # packet forwarding ("router switch")
options         MROUTING        # packet forwarding of multicast packets
options         PIM             # Protocol Independent Multicast
options         NETATALK        # AppleTalk (over Ethernet) protocol
options         NTP             # Network Time Protocol in-kernel support
options         PPS_SYNC        # Add serial line synchronization for NTP
options         PFIL_HOOKS      # Add pfil(9) hooks, intended for custom LKMs.
options         IPFILTER_LOG    # Add ipmon(8) logging for ipfilter device
options         IPFILTER_LOOKUP # ippool(8) support
options         PPP_BSDCOMP     # Add BSD compression to ppp device
options         PPP_DEFLATE     # Add deflate (libz) compression to ppp device
options         PPP_FILTER      # Add active filters for ppp (via bpf)
mainbus0 at root
cpu0    at mainbus0
psycho* at mainbus0                             # PCI-based systems
pci0    at psycho0                              # FLAPJACK2 built-in
ppb0    at pci0 dev 1 function 1                # FLAPJACK2 built-in
ppb1    at pci0 dev 1 function 0                # FLAPJACK2 built-in
pci1    at ppb0                                 # FLAPJACK2 built-in
pci2    at ppb1                                 # FLAPJACK2 built-in
ppb*    at pci?                                 # `APB' support.
pci*    at ppb?
ebus*   at pci?                                 # ebus devices
pcons0  at mainbus0                             # PROM console
auxio*  at ebus?                                # auxio registers
lpt*    at ebus?                                # parallel port
clock*  at ebus?
rtc*    at ebus?
timer*  at mainbus0                             # sun4c
com*    at ebus?                                # `com' driver for `su'
isp*    at pci? dev ? function ?        # Qlogic ISP [12]0x0 SCSI/FibreChannel
esiop0  at pci2 dev 8 function 0        # FLAPJACK2 built-in
esiop1  at pci2 dev 8 function 1        # FLAPJACK2 built-in
scsibus0 at esiop0                      # FLAPJACK2 built-in
scsibus1 at esiop1                      # FLAPJACK2 built-in
esiop*  at pci?                         # 53C875 and newer ("glm" compatible)
scsibus* at scsi?
sd0     at scsibus0 target 0 lun 0              # FLAPJACK2 built-in
sd1     at scsibus0 target 1 lun 0              # FLAPJACK2 built-in
sd*     at scsibus? target ? lun ?              # SCSI disks
cd*     at scsibus? target ? lun ?              # SCSI CD-ROMs
ses*    at scsibus? target ? lun ?              # SCSI SES/SAF-TE devices
uk*     at scsibus? target ? lun ?              # unknown SCSI
pciide* at pci? dev ? function ? flags 0x0000   # GENERIC pciide driver
aceride* at pci? dev ? function ?       # Acer Lab IDE controllers
atabus* at ata?
atapibus* at atapi?
wd*     at atabus? drive ? flags 0x0000
cd*     at atapibus? drive ? flags 0x0000       # ATAPI CD-ROM drives
sd*     at atapibus? drive ? flags 0x0000       # ATAPI disk drives
uk*     at atapibus? drive ? flags 0x0000       # ATAPI unknown
pseudo-device   accf_data               # "dataready" accept filter
pseudo-device   accf_http               # "httpready" accept filter
pseudo-device   vnd     
pseudo-device   ccd     4
options         RAID_AUTOCONFIG         # auto-configuration of RAID components
hme*            at pci? dev ? function ?        # network "hme" compatible
gem*    at pci? dev ? function ?        # Apple GMAC and Sun ERI gigabit enet
tlp*    at pci? dev ? function ?        # DECchip 21x4x and clones
wm*     at pci? dev ? function ?        # Intel 8254x gigabit
bge*    at pci? dev ? function ?        # Broadcom BM570x gigabit Ethernet
acphy*  at mii? phy ?                   # Altima AC101 and AMD Am79c874 PHYs
bmtphy* at mii? phy ?                   # Broadcom BCM5201 and BCM5202 PHYs
brgphy* at mii? phy ?                   # Broadcom BCM5400-family PHYs
ciphy*  at mii? phy ?                   # Cicada CS8201 Gig-E PHYs
dmphy*  at mii? phy ?                   # Davicom DM9101 PHYs
exphy*  at mii? phy ?                   # 3Com internal PHYs
icsphy* at mii? phy ?                   # Integrated Circuit Systems ICS189x
ikphy*  at mii? phy ?                   # Intel 82563 PHYs
inphy*  at mii? phy ?                   # Intel 82555 PHYs
iophy*  at mii? phy ?                   # Intel 82553 PHYs
igphy*  at mii? phy ?                   # Intel IGP01E1000
lxtphy* at mii? phy ?                   # Level One LXT-970 PHYs
makphy* at mii? phy ?                   # Marvell Semiconductor 88E1000 PHYs
nsphy*  at mii? phy ?                   # NS83840 PHYs
nsphyter* at mii? phy ?                 # NS83843 PHYs
qsphy*  at mii? phy ?                   # Quality Semiconductor QS6612 PHYs
rgephy* at mii? phy ?                   # Realtek 8169S/8110S internal PHYs
rlphy*  at mii? phy ?                   # Realtek 8139/8201L PHYs
sqphy*  at mii? phy ?                   # Seeq 80220/80221/80223 PHYs
tlphy*  at mii? phy ?                   # ThunderLAN PHYs
tqphy*  at mii? phy ?                   # TDK Semiconductor PHYs
ukphy*  at mii? phy ?                   # generic unknown PHYs
ohci*   at pci? dev ? function ?        # Open Host Controller
usb*    at ohci?
uhub*   at usb?
uhub*   at uhub? port ?
uhidev* at uhub? port ? configuration ? interface ?
ucycom* at uhidev? reportid ?
uhid*   at uhidev? reportid ?
ulpt*   at uhub? port ? configuration ? interface ?
umodem* at uhub? port ? configuration ?
ucom*   at umodem?
umass*  at uhub? port ? configuration ? interface ?
wd*     at umass?
uaudio* at uhub? port ? configuration ?
umidi* at uhub? port ? configuration ?
atu*    at uhub? port ?         # Atmel AT76C50XX based adapters
ral*    at uhub? port ?         # Ralink Technology RT25x0 802.11a/b/g
upl*    at uhub? port ?
ubsa*   at uhub? port ?         # Belkin serial adapter
ucom*   at ubsa? portno ?
uftdi*  at uhub? port ?         # FTDI FT8U100AX serial adapter
ucom*   at uftdi? portno ?
umct*   at uhub? port ?         # MCT USB-RS232 serial adapter
ucom*   at umct? portno ?
uplcom* at uhub? port ?         # I/O DATA USB-RSAQ2 serial adapter
ucom*   at uplcom? portno ?
uvscom* at uhub? port ?         # SUNTAC Slipper U VS-10U serial adapter
ucom*   at uvscom? portno ?
uscanner* at uhub? port ?
usscanner* at uhub? port ?
udsbr*  at uhub? port ?
radio*  at udsbr?
ugen*   at uhub? port ?
pseudo-device   loop
pseudo-device   ppp             
pseudo-device   pppoe
pseudo-device   tun             
pseudo-device   tap                     # virtual Ethernet
pseudo-device   gre                     # generic L3 over IP tunnel
pseudo-device   bpfilter
pseudo-device   agr                     # IEEE 802.3ad link aggregation
pseudo-device   ipfilter
pseudo-device   gif                     # IPv[46] over IPv[46] tunnel (RFC1933)
audio*          at audiobus?
midi*           at midibus?
options         WSEMUL_SUN              # sun terminal emulation
options         WS_DEFAULT_FG=WSCOL_BLACK
options         WS_DEFAULT_BG=WSCOL_LIGHT_WHITE
options         WSDISPLAY_COMPAT_USL            # VT handling
options         WSDISPLAY_COMPAT_RAWKBD         # can get raw scancodes
options         WSDISPLAY_DEFAULTSCREENS=4
options         FONT_GALLANT12x22               # PROM font look-alike
alipm*          at pci?
iic*            at alipm?
spdmem*         at iic? addr 0x54
spdmem*         at iic? addr 0x55
spdmem*         at iic? addr 0x56
spdmem*         at iic? addr 0x57
admtemp*        at iic? addr 0x18
pseudo-device   crypto                  # /dev/crypto device
pseudo-device   swcrypto                # software crypto implementation
pseudo-device   pty                     # pseudo-ttys (for network, etc.)
pseudo-device   rnd
pseudo-device   clockctl                # user control of clock subsystem
pseudo-device   ksyms                   # /dev/ksyms
pseudo-device   fss             4       # file system snapshot device
pseudo-device   lockstat                # lock profiling
options         FILEASSOC               # fileassoc(9) - required for Veriexec 
pseudo-device   veriexec                1
options VERIFIED_EXEC_FP_RMD160
options VERIFIED_EXEC_FP_SHA256
options VERIFIED_EXEC_FP_SHA384
options VERIFIED_EXEC_FP_SHA512
options VERIFIED_EXEC_FP_SHA1
options VERIFIED_EXEC_FP_MD5
pseudo-device   nsmb            # experimental - SMB requester
pseudo-device   drvctl
pseudo-device   putter


>Description:
        My NFS server was pressed into service as the house router.

        Some time after adding the necessary network interface aliases, NetBSD
        panic'd, and continued to panic just before getty appeared on console
        at every subsequent reboot.  Starting in single user mode and
        attempting to diagnose which service/program was causing the problem
        yielded this:

112 # nfsd -r -t -u
113 # Aug 10 16:46:07 Reader / writer lock error: rw_vector_enter: locking 
against myself

lock address : 0x000000000ead0fb8
current cpu  :                  0
current lwp  : 0x000000000ead4860
owner/count  : 0x000000000ead4860 flags    : 0x0000000000000004

panic: lock error
Stopped in pid 186.2 (nfsd) at  netbsd:cpu_Debugger+0x4:        nop

        Turning off NFS service in /etc/rc.conf made the system stable, and
        I needed IP routing a bit more urgently than NFS service.

        I have now installed a replacement router, removed the network
        interface aliases from the NFS server, and turned NFS service back
        on. It appears as stable as it had been before.

>How-To-Repeat:
        1. set up an NFS server with multiple network interfaces and some 
clients.

        2. add network interface aliases for additional addresses
        (same respective subnets) on two interfaces with the intent
        that the NFS server also route IP packets between those
        two nets, in place of a router.

        3. observe "panic: lock error" on the NFS server console.

        4. curse loudly that you don't need this kind of software
        problem on top of all the other very urgent infrastructure
        failures you're dealing with.

>Fix:
        



Home | Main Index | Thread Index | Old Index