kern/56433: -current (9.99.90) panic on RPI3 on boot

To: kern-bug-people%netbsd.org@localhost,gnats-admin%netbsd.org@localhost,netbsd-bugs%netbsd.org@localhost
Subject: kern/56433: -current (9.99.90) panic on RPI3 on boot
From: brad%anduin.eldar.org@localhost
Date: Sat, 2 Oct 2021 10:10:00 +0000 (UTC)
>Number:         56433
>Category:       kern
>Synopsis:       -current (9.99.90) panic on RPI3 on boot
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 02 10:10:00 +0000 2021
>Originator:     brad%anduin.eldar.org@localhost
>Release:        NetBSD-current 9.99.90
>Organization:
	eldar.org
>Environment:
	RPI 3
Architecture: evbarm
Machine: earmv6hf or earmv7hf
>Description:

I did a build of -current (9.99.90) from source taken on 2021-09-30 and had it
panic on my RPI 3.  I don't know if this is a known problem at this point.

This is how it was built, with $P being the location of my build artifacts:

build.sh -D $P/DIST -O $P/OBJ -T $P/TOOLS -R $P/REL -X /usr/xsrc -x -j2 -m evbarm -a earmv6hf release

I have been using this for a long time on RPI 0, 2 and 3.  I then pulled
the rpi.img.gz out of the REL directory and tried to boot it on the RPI
3.  The panic is as follows:

[   1.0000000] [ Kernel symbol table missing! ]
[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[   1.0000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[   1.0000000]     2018, 2019, 2020, 2021 The NetBSD Foundation, Inc.  All rights reserved.
[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[   1.0000000]     The Regents of the University of California.  All rights reserved.

[   1.0000000] NetBSD 9.99.90 (RPI2) #0: Fri Oct  1 16:01:41 EDT 2021
[   1.0000000]  brad%samwise.nat.eldar.org@localhost:/lhome/CURRENT_20210930/earmv6hf//OBJ/sys/arch/evbarm/compile/RPI2
[   1.0000000] total memory = 948 MB
[   1.0000000] avail memory = 915 MB
[   1.0000000] armfdt0 (root)
[   1.0000000] simplebus0 at armfdt0: Raspberry Pi 3 Model B Rev 1.2
[   1.0000000] simplebus1 at simplebus0
[   1.0000000] simplebus2 at simplebus0
[   1.0000000] cpus0 at simplebus0
[   1.0000000] simplebus3 at simplebus0
[   1.0000000] cpu0 at cpus0: 600 MHz Cortex-A53 r0p4 (Cortex V8A core)
[   1.0000000] cpu0: DC enabled IC enabled WB enabled EABT branch prediction enabled
[   1.0000000] cpu0: L1 32KB/64B 2-way (256 set) VIPT Instruction cache
[   1.0000000] cpu0: L1 32KB/64B 4-way (128 set) write-back-locking-C PIPT Data cache
[   1.0000000] cpu0: L2 512KB/64B 16-way (512 set) write-through PIPT Unified cache
[   1.0000000] vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
[   1.0000000] cpu1 at cpus0
[   1.0000000] cpu2 at cpus0
[   1.0000000] cpu3 at cpus0
[   1.0000000] fclock0 at simplebus2: 19200000 Hz fixed clock (osc)
[   1.0000000] simplebus4 at simplebus1
[   1.0000000] fclock1 at simplebus2: 480000000 Hz fixed clock (otg)
[   1.0000000] bcmicu0 at simplebus1
[   1.0000000] bcmcprman0 at simplebus1: BCM283x Clock Controller
[   1.0000000] bcmicu1 at simplebus1: Multiprocessor
[   1.0000000] gtmr0 at simplebus0: Generic Timer
[   1.0000000] gtmr0: interrupting on local_intc irq 3
[   1.0000000] armgtmr0 at gtmr0: Generic Timer (19200 kHz, virtual)
[   1.0000030] bcmaux0 at simplebus1
[   1.0000030] /soc/thermal@7e212000 at simplebus1 not configured
[   1.0000030] /soc/dsi@7e209000 at simplebus1 not configured
[   1.0000030] bcmgpio0 at simplebus1: GPIO controller 2835
[   1.0000030] bcmgpio0: pins 0..31 interrupting on icu irq 49
[   1.0000030] bcmgpio0: pins 32..54 interrupting on icu irq 50
[   1.0000030] gpio0 at bcmgpio0: 54 pins
[   1.0000030] /soc/firmware/gpio at simplebus4 not configured
[   1.0000030] bcmdmac0 at simplebus1: DMA0 DMA2 DMA4 DMA5 DMA8 DMA9 DMA10 DMA11
[   1.0000030] /soc/power at simplebus1 not configured
[   1.0000030] /wifi-pwrseq at simplebus0 not configured
[   1.0000030] /phy at simplebus0 not configured
[   1.0000030] bsciic0 at simplebus1: Broadcom Serial Controller
[   1.0000030] bsciic0: interrupting on icu irq 53
[   1.0000030] iic0 at bsciic0: I2C bus
[   1.0000030] bcmpmwdog0 at simplebus1: Power management, Reset and Watchdog controller
[   1.0000030] bcmmbox0 at simplebus1: VC mailbox
[   1.0000030] bcmmbox0: interrupting on icu irq 65
[   1.0000030] vcmbox0 at bcmmbox0
[   1.0000030] /soc/timer@7e003000 at simplebus1 not configured
[   1.0000030] /soc/txp@7e004000 at simplebus1 not configured
[   1.0000030] plcom0 at simplebus1: ARM PL011 UART
[   1.0000030] plcom0: txfifo disabled
[   1.0000030] plcom0: interrupting on icu irq 57
[   1.0000030] bcmsdhost0 at simplebus1: SD HOST controller
[   1.0000030] bcmsdhost0: interrupting on icu irq 56
[   1.0000030] bsciic1 at simplebus1: Broadcom Serial Controller
[   1.0000030] bsciic1: interrupting on icu irq 53
[   1.0000030] iic1 at bsciic1: I2C bus
[   1.0000030] com0 at simplebus1: BCM AUX UART, 1-byte FIFO
[   1.0000030] com0: console
[   1.0000030] com0: interrupting on icu irq 29
[   1.0000030] /soc/pwm@7e20c000 at simplebus1 not configured
[   1.0000030] sdhc0 at simplebus1: SDHC controller
[   1.0000030] sdhc0: interrupting on icu irq 62
[   1.0000030] bsciic2 at simplebus1: Broadcom Serial Controller
[   1.0000030] bsciic2: interrupting on icu irq 53
[   1.0000030] iic2 at bsciic2: I2C bus
[   1.0000030] /soc/vec@7e806000 at simplebus1 not configured
[   1.0000030] dwctwo0 at simplebus1: USB controller
[   1.0000030] dwctwo0: interrupting on icu irq 9
[   1.0000030] /soc/hdmi@7e902000 at simplebus1 not configured
[   1.0000030] /soc/gpu at simplebus1 not configured
[   1.0000030] genfb0 at simplebus1
[   1.0000030] wsdisplay0 at genfb0 kbdmux 1
[   1.0000030] vchiq0 at simplebus1: BCM2835 VCHIQ
[   1.0000030] /arm-pmu at simplebus0 not configured
[   1.0000030] gpioleds0 at simplebus0:
[   1.0000030] bcmrng0 at simplebus1: RNG
[   1.0000030] entropy: ready
[   1.0000030] cpu1: 600 MHz Cortex-A53 r0p4 (Cortex V8A core)
[   1.0000030] cpu1: DC enabled IC enabled WB enabled EABT branch prediction enabled
[   1.2578383] cpu1: L1 32KB/64B 2-way (256 set) VIPT Instruction cache
[   1.2578383] cpu1: L1 32KB/64B 4-way (128 set) write-back-locking-C PIPT Data cache
[   1.2678215] cpu1: L2 512KB/64B 16-way (512 set) write-through PIPT Unified cache
[   1.2778233] vfp1 at cpu1: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
[   1.2878232] cpu3panic: : 600 MHz Cortex-A53 r0p4 (Cortex V8A core)
[   1.2878232] kernel diagnostic assertion "l->l_stat == LSONPROC" failed: file "/usr/src/sys/kern/kern_sleepq.c", line 227
[   1.2978253] cpu1: Begin traceback...
[   1.3078253] cpu3:0x809aedbc:  DC enabled IC enabled WB enabled EABT branch prediction enabled
[   1.3178285] netbsd:db_paniccpu3: L1 32KB/64B 2-way (256 set) VIPT Instruction cache
[   1.3178285] cpu3: L1 32KB/64B 4-way (128 set) write-back-locking-C PIPT Data cache
[   1.3278278] cpu3: L2 512KB/64B 16-way (512 set) write-through PIPT Unified cache
[   1.3378291] +0x14
[   1.3378291] 0x809aedd4: vfp3 at cpu3: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
[   1.3478291] netbsd:vpanic+0x16c
[   1.3478291] 0x809aedec: cpu2: 600 MHz Cortex-A53 r0p4 (Cortex V8A core)
[   1.3578337] netbsd:__aeabi_uldivmodcpu2: DC enabled IC enabled WB enabled EABT
[   1.3678311] 0x809aee2c:  branch prediction enabled
[   1.3678311] netbsd:sleepq_enqueuecpu2: L1 32KB/64B 2-way (256 set) VIPT Instruction cache
[   1.3778336] +0x1a0
[   1.3778336] 0x809aee5c: cpu2: L1 32KB/64B 4-way (128 set) write-back-locking-C PIPT Data cache
[   1.3878332] cpu2: L2 512KB/64B 16-way (512 set) write-through PIPT Unified cache
[   1.3978354] netbsd:cv_enter+0xfc
[   1.3978354] 0x809aee84: vfp2 at cpu2: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
[   1.4078352] netbsd:cv_wait+0x44
[   1.4078352] 0x809aeeac: netbsd:xc_wait+0xa0
[   1.4178637] 0x809aeee4: netbsd:pic_establish_intr+0x2a0
[   1.4178637] 0x809aef14: netbsd:intr_establish_xname+0xa4
[   1.4278385] 0x809aef2c: netbsd:intr_establish+0x20
[   1.4278385] 0x809aef54: netbsd:bcm2836mp_intr_init+0x4c
[   1.4378399] 0x809aef6c: netbsd:arm_fdt_cpu_hatch+0x34
[   1.4378399] 0x809aef94: netbsd:cpu_hatch+0x1bc
[   1.4478405] cpu1: End traceback...
Stopped in pid 0.16 (system) at netbsd:cpu_Debugger+0x4:        bx      r14
db{1}>

>How-To-Repeat:

Try booting the rpi.img.gz created from a release build on a RPI3.  I
also noted that the evbarmv7hf build with the armv7.img.gz image
panics in exactly the same way.  I am going to try a aarch64 image
shortly and will probably try a kernel without MP enabled after that.

>Fix:

Don't know myself.  The same board works with 9.x so I don't really
think that it is a hardware problem.  One of the CPUs appears to panic
as it is being spun up, the l->l_stat == LSONPROC message, so maybe
changes to kqueue??
Prev by Date: Re: bin/56422 (zgrep -l sometimes hangs)
Next by Date: PR/56429 CVS commit: [netbsd-9] src
Previous by Thread: Re: bin/56422 (zgrep -l sometimes hangs)
Next by Thread: Re: kern/56433: -current (9.99.90) panic on RPI3 on boot
Indexes:
Home | Main Index | Thread Index | Old Index