NetBSD-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
kern/56353: Build hung
>Number: 56353
>Category: kern
>Synopsis: Build hung
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Mon Aug 09 07:05:00 +0000 2021
>Originator: Andreas Gustafsson
>Release: NetBSD 9.0_STABLE
>Organization:
>Environment:
System: NetBSD babylon5.netbsd.org 9.0_STABLE NetBSD 9.0_STABLE (BABYLON5-EPYC) #0: Wed Sep 9 05:05:23 UTC 2020 spz%franklin.NetBSD.org@localhost:/home/netbsd/9/amd64/obj/sys/arch/amd64/compile/BABYLON5-EPYC amd64
Architecture: x86_64
Machine: amd64
>Description:
On Aug 5, a build of NetBSD-current/amd64 running on
babylon5.netbsd.org hung. Logging in on Aug 8 and running
"ps -glaxdwww" showed the following process tree for the build
(sorry about the long lines):
666 17610 1122 0 85 0 17584 1364 wait I pts/8 0:00.00 | | `-- time nice ./build.sh -j 24 -D /tmp/build/2021.08.05.06.54.16-amd64/destdir -R /bracket/amd64/test/2021.08.05.06.54.16/release/release -T /tmp/build/2021.08.05.06.54.16-amd64/tools -O /tmp/build/2021.08.05.06.54.16-amd64/obj -m amd64 -V TMPDIR=/tmp -U release iso-image
666 2283 17610 34427 72 10 20960 2164 wait IN pts/8 0:00.02 | | `-- sh ./build.sh -j 24 -D /tmp/build/2021.08.05.06.54.16-amd64/destdir -R /bracket/amd64/test/2021.08.05.06.54.16/release/release -T /tmp/build/2021.08.05.06.54.16-amd64/tools -O /tmp/build/2021.08.05.06.54.16-amd64/obj -m amd64 -V TMPDIR=/tmp -U release iso-image
666 27809 2283 0 80 10 16808 2532 select SN pts/8 0:14.22 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake -j 24 release
666 2897 27809 34427 72 10 20408 1740 wait IN pts/8 0:00.00 | | `-- sh
666 11692 2897 0 80 10 16800 2552 select SN pts/8 0:06.63 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_= distribution
666 6459 11692 34427 72 10 20032 1740 wait IN pts/8 0:00.00 | | `-- sh
666 5151 6459 0 80 10 17112 2564 select SN pts/8 0:06.55 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_= NOPOSTINSTALL=1 build
666 15346 5151 36114 72 10 20028 1744 wait IN pts/8 0:00.00 | | `-- sh
666 22088 15346 0 80 10 16800 2556 select SN pts/8 0:02.46 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_= do-build
666 28934 22088 36114 72 10 20240 1744 wait IN pts/8 0:00.00 | | `-- sh
666 7534 28934 0 80 10 16812 2568 select SN pts/8 0:02.60 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_= BUILD_tools=no BUILD_lib=no dependall
666 13913 7534 36114 72 10 21128 1748 wait IN pts/8 0:00.01 | | `-- sh
666 21381 13913 0 80 10 16768 2424 select SN pts/8 0:01.15 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/ dependall
666 19128 21381 36114 72 10 20036 1748 wait IN pts/8 0:00.01 | | `-- sh
666 11978 19128 0 80 10 16496 2168 select SN pts/8 0:00.89 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/arch/ dependall
666 23154 11978 36114 72 10 20496 1748 wait IN pts/8 0:00.01 | | `-- sh
666 10154 23154 0 80 10 16440 2208 select SN pts/8 0:00.89 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/arch/amd64/ dependall
666 20194 10154 36128 72 10 20056 1752 wait IN pts/8 0:00.01 | | `-- sh
666 3165 20194 0 80 10 18376 2228 select SN pts/8 0:00.87 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/arch/amd64/../i386/stand/ dependall
666 1624 3165 35407 72 10 20056 1752 wait IN pts/8 0:00.00 | | `-- sh
666 28740 1624 0 80 10 16516 2204 select SN pts/8 0:00.41 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/arch/amd64/../i386/stand/boot/ dependall
666 26209 28740 36128 72 10 20488 1744 wait IN pts/8 0:00.00 | | `-- sh
666 23367 26209 0 80 10 20136 2576 select SN pts/8 0:00.44 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake _THISDIR_=sys/arch/amd64/../i386/stand/boot/biosboot/ dependall
666 15692 23367 36198 72 10 23412 1744 wait IN pts/8 0:00.00 | | `-- sh
666 8370 15692 0 80 10 17276 3068 select SN pts/8 0:00.47 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/nbmake realall
666 486 8370 36243 72 10 20176 1752 pipe_rd IN pts/8 0:00.00 | | `-- sh
666 10452 486 36243 72 10 20184 1200 wait IN pts/8 0:00.00 | | `-- sh
666 20791 10452 0 0 10 20452 1628 - DNE pts/8 0:00.00 | | |-- (sh)
666 21534 10452 36243 72 10 28912 2004 wait IN pts/8 0:00.00 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/bin/x86_64--netbsd-gcc -o boot.sym --sysroot=/tmp/build/2021.08.05.06.54.16-amd64/destdir -nostdlib -Wl,-N -Wl,-e,boot_start -Wl,-m,elf_i386 -Wl,-z,relro -Wl,-Ttext,0 -Wl,-cref biosboot.o boot2.o conf.o devopen.o exec.o exec_multiboot1.o exec_multiboot2.o vers.o /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/libsa.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/z/libz.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/kern/libkern.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/li
bsa.a (x86_64--netbsd-g)
666 24620 21534 36243 72 10 20452 1628 wait IN pts/8 0:00.00 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/libexec/gcc/x86_64--netbsd/10.3.0/collect2 -plugin /tmp/build/2021.08.05.06.54.16-amd64/tools/libexec/gcc/x86_64--netbsd/10.3.0/liblto_plugin.so -plugin-opt=/tmp/build/2021.08.05.06.54.16-amd64/tools/libexec/gcc/x86_64--netbsd/10.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/cckfKTsc.res --sysroot=/tmp/build/2021.08.05.06.54.16-amd64/destdir --eh-frame-hdr -dc -dp -dynamic-linker /usr/libexec/ld.elf_so -o boot.sym -L/tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0 -L/tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/lib -L/tmp/build/2021.08.05.06.54.16-amd64/destdir/usr/lib -N -e boot_start -m elf_i386 -z relro -Ttext 0 -cref biosboot.o boot2.o conf.o devopen.o exec.o exec_multiboot1.o exec_multiboot2.o vers.o /tmp/build/2
021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/libsa.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/z/libz.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/kern/libkern.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/libsa.a
666 10255 24620 36243 72 10 27188 8984 pipe_wr IN pts/8 0:00.03 | | `-- /tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/bin/ld -plugin /tmp/build/2021.08.05.06.54.16-amd64/tools/libexec/gcc/x86_64--netbsd/10.3.0/liblto_plugin.so -plugin-opt=/tmp/build/2021.08.05.06.54.16-amd64/tools/libexec/gcc/x86_64--netbsd/10.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/cckfKTsc.res --sysroot=/tmp/build/2021.08.05.06.54.16-amd64/destdir --eh-frame-hdr -dc -dp -dynamic-linker /usr/libexec/ld.elf_so -o boot.sym -L/tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0 -L/tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/lib -L/tmp/build/2021.08.05.06.54.16-amd64/destdir/usr/lib -N -e boot_start -m elf_i386 -z relro -Ttext 0 -cref biosboot.o boot2.o conf.o devopen.o exec.o exec_multiboot1.o exec_multiboot
2.o vers.o /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/libsa.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/z/libz.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/kern/libkern.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/i386/libi386.a /tmp/build/2021.08.05.06.54.16-amd64/obj/sys/arch/i386/stand/boot/biosboot/lib/sa/libsa.a
I attached to the ld process from the last line of the ps output using gdb:
babylon5.netbsd.org$ gdb /tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/bin/ld
[boilerplate elided]
Reading symbols from /tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/bin/ld...
(No debugging symbols found in /tmp/build/2021.08.05.06.54.16-amd64/tools/lib/gcc/x86_64--netbsd/10.3.0/../../../../x86_64--netbsd/bin/ld)
(gdb) attach 10255
Attaching to program: /tmp/build/2021.08.05.06.54.16-amd64/tools/x86_64--netbsd/bin/ld, process 10255
Reading symbols from /usr/lib/libc.so.12...
(No debugging symbols found in /usr/lib/libc.so.12)
Reading symbols from /usr/libexec/ld.elf_so...
(No debugging symbols found in /usr/libexec/ld.elf_so)
[Switching to LWP 1 of process 10255]
0x00007f1c59e427aa in write () from /usr/lib/libc.so.12
(gdb) where
#0 0x00007f1c59e427aa in write () from /usr/lib/libc.so.12
#1 0x00007f1c59f67716 in __sflush () from /usr/lib/libc.so.12
#2 0x00007f1c59f135d2 in _fwalk () from /usr/lib/libc.so.12
#3 0x00007f1c59f677b7 in fflush () from /usr/lib/libc.so.12
#4 0x00007f1c59f43814 in exit () from /usr/lib/libc.so.12
#5 0x00000000004d78b7 in xexit ()
#6 0x00000000004170b8 in main ()
(gdb) quit
A debugging session is active.
Inferior 1 [process 10255] will be detached.
Quit anyway? (y or n) y
Detaching from program: /tmp/build/2021.08.05.06.54.16-amd64/tools/x86_64--netbsd/bin/ld, process 10255
[Inferior 1 (process 10255) detached]
After detaching gdb, the hung process continued as if nothing had
happened and the build completed successfully, three days after
it started.
>How-To-Repeat:
Probably with difficulty, as the machine in case has been running
builds continuously for more than 9 months now and hung only once
so far.
>Fix:
Home |
Main Index |
Thread Index |
Old Index