NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

kern/56517: dual G4 macppc locks up during atf test run



>Number:         56517
>Category:       kern
>Synopsis:       dual G4 macppc locks up during atf test run
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Nov 23 16:30:01 +0000 2021
>Originator:     Martin Husemann
>Release:        NetBSD 9.99.92
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD gethsemane.aprisoft.de 9.99.92 NetBSD 9.99.92 (GETHSEMANE) #143: Tue Nov 23 09:35:58 CET 2021 martin%seven-days-to-the-wolves.aprisoft.de@localhost:/work/src/sys/arch/macppc/compile/GETHSEMANE macppc
Architecture: powerpc
Machine: macppc
>Description:

In general macppc tests give pretty good results (it used to be the best
architectures for several weeks, only last week i386 caught up).

But sometimes the test run locks up the machine (see ddb output below), new
processes hang immediately (like the login process on console).
Unfortunately I have been unable to reproduce this with a LOCKDEBUG kernel
so far.

Stopped in pid 0.2 (system) at  netbsd:zstty_stint+0x1d8:       b       netbsd:z
stty_stint+0x144
db{0}> ps
PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
20444 20444 3   0         0           1e018080               cron tstile
15423 15423 3   0         0           3753f080               cron tstile
24198 24198 3   0         0           3750d3c0               cron tstile
29986 29986 3   0         0           2e505680               cron tstile
25241 25241 3   0         0           2e505080               cron tstile
1503  1503 3   0         0           373cf9c0               cron tstile
26462 26462 3   0         0           3858e700               cron tstile
21470 21470 3   0         0           2dc220c0               cron tstile
1890  1890 3   0         0           2e5c2100               cron tstile
5260  5260 3   0         0           374e8d00               cron tstile
24252 24252 3   0         0           3755b6c0               cron tstile
19306 19306 3   0         0           3753f380               cron tstile
17979 17979 3   0         0           37a6e6c0               cron tstile
24366 24366 3   0         0           2e583380               cron tstile
24933 24933 3   0         0           1c9aec40               cron tstile
16527 16527 3   0         0           3771e100               cron tstile
26070 26070 3   0         0           2e505980               cron tstile
21277 21277 3   0         0           3750d9c0               cron tstile
12738 12738 3   0         0           1f2f7d00               cron tstile
9318  9318 3   0         0           2e3106c0               cron tstile
20974 20974 3   0         0           3769c080               cron tstile
27646 27646 3   0         0           2e310cc0               cron tstile
1380  1380 3   0         0           380b9c40               cron tstile
23888 23888 3   0         0           2e602080               cron tstile
27645 27645 3   0         0           1cb8fa00               cron tstile
14582 14582 3   0         0           3741ec40               cron tstile
26478 26478 3   0         0           2e0a0380               cron tstile
22454 22454 3   0         0           2eb45c80               cron tstile
12464 12464 3   0   1000000           3769c680           ifconfig tstile
3834  9104 3   0       180           10f6e400        rump_server parked
3834  5502 3   0       180           375cd9c0        rump_server parked
3834  8007 3   0       180           37ba8380        rump_server parked
3834  5387 3   1       180           2e263080        rump_server parked
3834  9088 3   1       180           377a93c0        rump_server parked
3834  7555 3   1       180           2e485c80        rump_server parked
3834  8597 3   0       180           37448100        rump_server parked
3834  4866 3   0       180           376799c0        rump_server parked
3834  6824 3   0       180           37a3e340        rump_server parked
3834  3245 3   0       180           2e4e5c40        rump_server parked
3834  9876 3   0       180           37accc80        rump_server parked
3834  5797 3   0       180           1994b040        rump_server parked
3834  9346 3   1       180           376793c0        rump_server parked
3834  7839 3   1       180           1090fc80        rump_server parked
3834  3880 3   1       180           2e0c0cc0        rump_server parked
3834  8835 3   1       180           37c78100        rump_server parked
3834  5744 3   1       180           2e4e5340        rump_server parked
3834  3955 3   1       180           2e1c0a00        rump_server parked
3834  4707 3   1       180           3764e700        rump_server parked
3834  8256 3   1       180           375cd3c0        rump_server parked
3834  11184 3   1       180           1bf2fcc0        rump_server parked
3834  5750 3   1       180           2e004980        rump_server parked
3834  7164 3   1       180           373fc680        rump_server parked
3834  7084 3   1       180           3755b9c0        rump_server parked
3834  9025 3   1       180           3761ac80        rump_server parked
3834  5386 3   1       180           37a3e940        rump_server parked
3834  8425 3   1       180           1c95b6c0        rump_server parked
3834  7359 3   1       180           2db04c40        rump_server parked
3834  11446 3   0       180           2e2c6c40        rump_server parked
3834  8874 3   0       180           37acc980        rump_server parked
3834  2214 3   1       180           37679cc0        rump_server parked
3834  9877 3   1       180           1cf2c0c0        rump_server parked
3834  5743 3   1       180           3759d100        rump_server parked
3834  6247 3   1       180           375fd080        rump_server parked
3834  5728 3   0       180           376be340        rump_server parked
3834  5207 3   1       180           2e583680        rump_server parked
3834  4872 3   0       180           2e5c2d00        rump_server parked
3834  9811 3   1       180           3746a9c0        rump_server parked
3834  9084 3   0       180           19e196c0        rump_server parked
3834  10975 3   0       180           3858ea00        rump_server parked
3834  9812 3   1       180           1e018980        rump_server parked
3834  6220 3   1       180           3746a0c0        rump_server parked
3834  5211 3   0       180           2da62c40        rump_server parked
3834  5746 3   1       180           1c95b0c0        rump_server parked
3834 >3373 7   1       100           2da62040        rump_server
3834  8483 3   0       180           2e2a5d00        rump_server parked
3834  5397 3   0       180           2dc72d00        rump_server parked
3834  8330 3   1       180           1f2f7700        rump_server parked
3834  6700 3   0       180           2e1b13c0        rump_server parked
3834  7420 3   1       180           373fc980        rump_server parked
3834  8106 3   1       180           3858e100        rump_server parked
3834  3957 3   1       180           37a5b700        rump_server parked
3834  8665 3   0       180           374bbc40        rump_server parked
3834  8175 3   0       180           3746a3c0        rump_server parked
3834  6448 3   1       180           1bbdd100        rump_server parked
3834  7326 3   1       180           1cbcb380        rump_server parked
3834  21959 3   1       180           1069fc40        rump_server parked
3834  10137 3   1       180           1bc8a980        rump_server parked
3834  7637 3   0       180           3746acc0        rump_server parked
3834  9679 3   1       180           198cd080        rump_server parked
3834  5977 3   0       180           3762b340        rump_server parked
3834  5472 3   1       180           3460f980        rump_server parked
3834  6493 3   0       180           2e564c40        rump_server parked
3834  5483 3   1       180           2e543400        rump_server parked
3834  3899 3   1       180           374e8a00        rump_server parked
3834  5491 3   1       180           380b9640        rump_server parked
3834  5486 3   0       180           2e602380        rump_server parked
3834  8691 3   1       180           2e5e2040        rump_server parked
3834  10139 3   0       180           1844e380        rump_server parked
3834  9609 3   1       180           375cd0c0        rump_server parked
3834  5497 3   1       180           2e602980        rump_server parked
3834  7677 3   1       180           193f09c0        rump_server parked
3834  7619 3   1       180           2dc223c0        rump_server parked
3834  6568 3   1       180           375cd6c0        rump_server parked
3834  8606 3   0       180           373fcc80        rump_server parked
3834  8588 3   1       180           19e19cc0        rump_server parked
3834  5498 3   0       180           2e2e7680        rump_server parked
3834  9541 3   1       180           3762b940        rump_server parked
3834  5402 3   1       180           3769c380        rump_server parked
3834  5396 3   1       180           2e112940        rump_server parked
3834  5383 3   0       180           37b8ed00        rump_server parked
3834  6398 3   1       180           1cbcbc80        rump_server parked
3834  9943 3   0       180           2e1c0d00        rump_server parked
3834  6751 3   1       180           2dc72700        rump_server parked
3834  5405 3   0       180           2e2c6940        rump_server parked
3834  5395 3   1       180           2e2c6340        rump_server parked
3834  5391 3   1       180           375cdcc0        rump_server parked
3834  8681 3   0       180           2e4a50c0        rump_server parked
3834  8609 3   0       180           120e4a00        rump_server parked
3834  5067 3   0       180           3756e340        rump_server parked
3834  5658 3   1       180           3750d0c0        rump_server parked
3834  5493 3   0       180           1bf59c40        rump_server parked
3834  9608 3   0       180           2e2a5700        rump_server parked
3834  8862 3   0       180           2e524cc0        rump_server parked
3834  25700 3   0       180           377a9cc0        rump_server parked
3834  4696 3   0       180           2e5a3cc0        rump_server parked
3834  4864 3   0       180           2d1916c0        rump_server parked
3834  9358 3   0       180           2db04940        rump_server parked
3834  10220 3   0       180           2e3103c0        rump_server parked
3834  5968 3   0       180           2e052cc0        rump_server parked
3834  5659 3   0       180           2d191cc0        rump_server parked
3834  8171 3   0       180           2e112640        rump_server parked
3834  8407 3   0       180           374bb640        rump_server parked
3834  8092 3   0       180           2e1d7080        rump_server parked
3834  5553 3   0       180           193f0cc0        rump_server parked
3834  5580 3   0       180           120e4700        rump_server parked
3834  8400 3   0       180           2e112040        rump_server parked
3834  4628 3   1       180           37c78400        rump_server parked
3834  6990 3   0       180           2db30980        rump_server parked
3834  5730 3   0       180           2e263980        rump_server parked
3834  8224 3   0       180           2e263380        rump_server parked
3834  6315 3   0       180           37b3fcc0        rump_server parked
3834  9206 3   1       180           375fd680        rump_server parked
3834  7417 3   0       180           37209d00        rump_server parked
3834  7921 3   1       180           1c04a940        rump_server parked
3834  7477 3   0       180           2e191640        rump_server parked
3834  4353 3   1       180           1c9ae040        rump_server parked
3834  8411 3   0       180           377a99c0        rump_server parked
3834  9437 3   0       180           3771e700        rump_server parked
3834  9775 3   1       180           2db04040        rump_server parked
3834  9089 3   0       180           37ba8c80        rump_server parked
3834  8911 3   0       180           2e1d7680        rump_server parked
3834  8144 3   0       180           2e0a0c80        rump_server parked
3834  6856 3   0       180           2e364c40        rump_server parked
3834  6180 3   1       180           10324a00        rump_server parked
3834  7241 3   1       180           3748e080        rump_server parked
3834  7670 3   0       180           2e191040        rump_server parked
3834  3242 3   0       180           1c04ac40        rump_server parked
3834  4505 3   1       180           1bf59940        vmem_rehash parked
3834  8527 3   0       1c0           1bf2f3c0            ioflush parked
3834  2852 3   1       180           2d1910c0             vdrain parked
3834  29485 3   0       180           2dc226c0            pdaemon parked
3834  28198 2   1       140           1bc8a080          entbutler
3834  3739 3   1       180           1bbdda00            xcall/1 parked
3834  3941 3   1       180           2dc72a00             sipbnc parked
3834  1614 3   0       1c0           1bf59340           rumpclk1 parked
3834  2500 3   1       180           1bbddd00            xcall/0 parked
3834  2947 3   1       180           1c95b9c0             rsi1/0 parked
3834  5853 3   1       180           1e018380             rsi0/0 parked
3834  1963 3   1       180           1994b340             rsi1/3 parked
3834  3228 3   1       180           1c9ae940             rsi0/3 parked
3834  3139 3   1       180           2dc22cc0             rsi1/2 parked
3834  3872 3   1       180           2de32d00             rsi0/2 parked
3834   458 3   1       180           1c9ae640             sipbnc parked
3834  5717 3   0       180           1069f340           rumpclk0 nanoslp
3834  24226 3   1       180           1cb8f100             rsi1/1 parked
3834  28354 3   0       180           1e018c80             rsi0/1 parked
3834  3834 3   0       180           1cbcb680        rump_server psem
5294  5294 3   1       180           19e199c0               tail kqueue
4557  4557 3   1       180           1994bc40               tcsh pause
4303  4303 3   0       180           198cd380               rxvt select
4143  4143 3   1       180           14e6c340               tcsh pause
3638  3638 3   0       180           1994b940               sshd select
3029  3029 3   1       180           10b7a9c0               sshd poll
1234  1234 3   1       180           193f03c0                tee pipe_rd
1184  1184 3   0       180           193f00c0         atf-report pipe_rd
1117  1117 3   0       180           1844ec80                tee pipe_rd
1242  1242 3   0       1c0           1844e980            atf-run pipe_rd
1377  1377 3   1       180           1844e680                 sh wait
1212  1212 3   0       180           10218cc0                 sh wait
1072  1072 3   0         0           1069f940              login tstile
1253  1253 3   0       180           10b7a6c0              getty nanoslp
953    953 3   0       180           10f6e100              getty nanoslp
1080  1080 3   0       180           10b7a0c0              getty nanoslp
1085  1085 3   0       180           10324400              getty nanoslp
809    809 3   0       180           10f6e700               cron nanoslp
1119  1119 3   1       180           14e6c040              inetd kqueue
1104  1104 3   0       180           14e6c640               sshd select
947    947 3   1       180           1844e080             powerd kqueue
946    946 3   0       180           14e6c940               ntpd pause
448    448 3   0       180           1069f640            syslogd kqueue
347    347 3   0       180           1090f080             dhcpcd poll
345    345 3   1       180           1090f680             dhcpcd poll
343    343 3   1       180           1090f380             dhcpcd poll
344    344 2   1         0           1090f980             dhcpcd
197    197 3   0       180           1069f040            wdogctl nanoslp
1        1 3   0       180           102996c0               init wait
0     29589 3   0       200           3769c980       bridge_rtage bridge_rtage
0     19051 3   0       200           3460fc80       bridge_rtage bridge_rtage
0     19266 3   0       200           3460f680       bridge_rtage bridge_rtage
0     2593 3   0       200           37acc080       bridge_rtage bridge_rtage
0     29408 3   0       200           37a6e0c0       bridge_rtage bridge_rtage
0     22464 3   0       200           2e191c40       bridge_rtage bridge_rtage
0     26104 3   0       200           2e1d7c80       bridge_rtage bridge_rtage
0     13542 3   0       200           1997f400       bridge_rtage bridge_rtage
0      201 3   1       200           102189c0            physiod physiod
0      165 3   0       200           10324100            ioflush syncer
0      164 3   1       200           10299cc0          pooldrain pooldrain
0      163 3   0       200           102186c0           pgdaemon pgdaemon
0      126 3   0       200           102183c0          swwreboot swwreboot
0      125 3   1       200           1022d100          atapibus0 sccomp
0      122 3   0       200           102180c0               usb1 usbevt
0      121 3   0       200           1022d400               usb0 usbevt
0      120 3   0       200           102999c0             npfgc0 npfgcw
0      119 3   1       200           102993c0            rt_free rt_free
0      118 3   1       200           102990c0              unpgc unpgc
0      117 3   0       200           1027dc80    key_timehandler key_timehandler

0      116 3   1       200           1027d980    icmp6_wqinput/1 icmp6_wqinput
0      115 3   0       200           1027d680    icmp6_wqinput/0 icmp6_wqinput
0      114 3   0       200           1027d380          nd6_timer nd6_timer
0      113 3   1       200           1027d080    carp6_wqinput/1 carp6_wqinput
0      112 3   0       200           1024bc40    carp6_wqinput/0 carp6_wqinput
0      111 3   1       200           1024b940     carp_wqinput/1 carp_wqinput
0      110 3   0       200           1024b640     carp_wqinput/0 carp_wqinput
0      109 3   1       200           1024b340     icmp_wqinput/1 icmp_wqinput
0      108 3   0       200           1024b040     icmp_wqinput/0 icmp_wqinput
0      107 3   0       200           1022dd00           rt_timer rt_timer
0      106 3   0       200           1022da00        vmem_rehash vmem_rehash
0      105 3   0       200           1022d700          entbutler entropy
0       96 3   0       380           5f8b3c80           fw0probe ieee1394
0       30 3   0       200           5f8b3980          cardslot0 evexit
0       29 3   0       200           5f8b3680         usbtask-dr usbtsk
0       28 3   0       200           5f8b3380         usbtask-hc usbtsk
0       27 3   0       200           5f8b3080            atabus2 atath
0       26 3   1       240           5f8e9c40            atabus1 atath
0       25 3   1       240           5f8e9940            atabus0 atath
0       24 3   0       200           5f8e9640               iic1 iicintr
0       23 3   0       200           5f8e9340                pmu wait
0       22 3   0       200           5f8e9040               iic0 iicintr
0       21 3   1       200           5f8f8d00            xcall/1 xcall
0       20 1   1       200           5f8f8a00          softser/1
0       19 1   1       200           5f8f8700          softclk/1
0       18 1   1       200           5f8f8400          softbio/1
0       17 1   1       200           5f8f8100          softnet/1
0       16 1   1       201           5f90fcc0             idle/1
0       15 3   0       200           5f90f9c0             sysmon smtaskq
0       14 3   0       200           5f90f6c0         pmfsuspend pmfsuspend
0       13 3   0       200           5f90f3c0           pmfevent pmfevent
0       12 3   0       200           5f90f0c0         sopendfree sopendfr
0       11 3   1       200           5fb20c80            iflnkst iflnkst
0       10 3   0       200           5fb20980           nfssilly nfssilly
0        9 3   0       200           5fb20680             vdrain vdrain
0        8 3   0       200           5fb20380          modunload mod_unld
0        7 3   0       200           5fb20080            xcall/0 xcall
0        6 1   0       200           5fb2ac40          softser/0
0        5 1   0       200           5fb2a940          softclk/0
0        4 1   0       200           5fb2a640          softbio/0
0        3 1   0       200           5fb2a340          softnet/0
0    >   2 1   0       201           5fb2a040             idle/0
0        0 3   0       200             c01a80            swapper uvm
db{0}> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12, ncolors=1
  376297 VM pages: 52176 active, 8 inactive, 2943 wired, 293831 free
  pages  9992 anon, 40844 file, 4238 exec
  freemin=512, free-target=682, wired-max=125432
  resv-pg=1, resv-kernel=10
  bootpages=8127, poolpages=26764
  faults=54290796, traps=57579591, intrs=14030916, ctxswitch=31608426
   softint=2936276, syscalls=0
  fault counts:
    noram=0, noanon=0, pgwait=0, pgrele=0
    ok relocks(total)=274315(274355), anget(retrys)=12763740(0), amapcopy=847757
1
    neighbor anon/obj pg=10006765/115520577, gets(lock/unlock)=29165328/274550
    cases: anon=8378514, anoncow=4385226, obj=24260330, prcopy=4904763, przero=1
1383605
  daemon and swap counts:
    woke=0, revs=0, scans=0, obscans=0, anscans=0
    busy=0, freed=0, reactivate=0, deactivate=0
    pageouts=0, pending=0, nswget=0
    nswapdev=1, swpgavail=1152017
    swpages=1152017, swpginuse=0, swpgonly=0, paging=0
db{0}> show pool
POOLCACHE : size 2153971720, align 946012180, ioff 1214406745, roflags 0x8001002
4
        pool 0x1194c8, alloc 0x2c1e0001
        minitems 958398464, minpages 2212429876, maxpages 2210332728, npages 946
012180
        itemsperpage 2436825140, nitems 2436825148, nout 1214407821, hardlimit 2
155806720
        nget 1015021714, nfail 2141776760, nput 2143744888
        npagealloc 2208366612, npagefree 959044028, hiwat 2210463768, nidle 9482
38968

[ 24500.8240384] trap: kernel read DSI trap @ 0x7c7f1c98 by 0x796524 (DSISR 0x40000000, err=14), lr 0x7964f4
[ 24500.8240384] panic: trap
[ 24500.8240384] cpu0: Begin traceback...
[ 24500.8240384] 0x10003810: at vpanic+0x12c
[ 24500.8240384] 0x10003840: at panic+0x50
[ 24500.8240384] 0x10003880: at trap+0x458
[ 24500.8240384] 0x10003940: kernel DSI read trap @ 0x7c7f1c98 by pool_print1+0x278: srr1=0x32
[ 24500.8240384]             r1=0x10003a10 cr=0x20204432 xer=0x20000000 ctr=0x20 dsisr=0x40000000
[ 24500.8240384] 0x10003a10: at pool_print1+0x244
[ 24500.8240384] 0x10003a60: at db_command+0x138
[ 24500.8240384] 0x10003b00: at db_command_loop+0xd0
[ 24500.8240384] 0x10003bd0: at db_trap+0xdc
[ 24500.8240384] 0x10003c00: at kdb_trap+0x128
[ 24500.8240384] 0x10003c40: at trapstart+0x95c
[ 24500.8240384] 0x10003d10: at intr_deliver+0xc8
[ 24500.8240384] 0x10003d30: at zsc_intr_hard+0x74
[ 24500.8240384] 0x10003d50: at zshard+0x18
[ 24500.8240384] 0x10003d60: at intr_deliver+0xe0
[ 24500.8240384] 0x10003da0: at pic_handle_intr+0xf8
[ 24500.8240384] 0x10003df0: at trapstart+0x6b0
[ 24500.8240384] 0x10003ec0: at __func__.1+0x65444
[ 24500.8240384] 0x10003ee0: at idle_loop+0x328
[ 24500.8240384] 0x10003f20: at cpu_lwp_bootstrap+0xc
[ 24500.8240384] saved LR(0xfb3f6279) is invalid.cpu0: End traceback...
Faulted in DDB; continuing...



>How-To-Repeat:
s/a

>Fix:
n/a



Home | Main Index | Thread Index | Old Index