Subject: building 3.0 in the background can "freeze" due to unsafe use of ksh
To: None <netbsd-users@netbsd.org>
From: Greg A. Woods <woods@planix.com>
List: netbsd-users
Date: 05/03/2006 15:29:12
--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
Building of 3.0 in the background (with nohup) can "freeze" with a stuck
configure script due to what appears to be unsafe use of ksh (i.e. when
$ENV is not cleared and thus all manner of things can get buggered).
I start build.sh with a very simple little wrapper script (that sets a
number of options and parameters for build.sh) as follows:
$ nohup ./myupdate.sh -j 1 >> /home/woods/becoming/NetBSD-3.0-i386-ppro.log 2>&1 < /dev/null &
At several points while building the tools it stops running I have to
bring the process into the foreground, and then send it a newline before
it will continue to run:
[1] + Stopped (tty input) nohup ./myupdate.sh -j 1 >> /home/woods/becomin
14:18 [56] $
^Z
[1] + Stopped nohup ./myupdate.sh -j 1 >> /home/woods/becomin
14:18 [57] $ j
[1] + 25657 Stopped nohup ./myupdate.sh -j 1 >> /home/woods/becomin
14:19 [58] $ bg
[1] nohup ./myupdate.sh -j 1 >> /home/woods/becomin
14:19 [59] $
14:19 [59] $ j
[1] + 25657 Running nohup ./myupdate.sh -j 1 >> /home/woods/becomin
The log file now contains garbage that would be produced by a special
function in my ~/.kshrc (used to set the window title upon "cd"):
----------------------------------------
dependall ===> tools/compat
mkdir -p include/sys include/machine include/rpc
rm -f config.cache
CC=cc CFLAGS=-O LDFLAGS= sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache
\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming/~ | woods[1]:not\007/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat
\033]0;sh://becoming/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007/bin/ksh: exit code: 1
\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming/~ | woods[1]:not\007/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat
\033]0;sh://becoming/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007/bin/ksh: exit code: 1
configure: creating cache config.cache
checking for gcc... cc
----------------------------------------
At the point when things were hung the following was running on that
terminal, and as one can see there's one ksh process which appears to
have been run without use of :
UID PID PPID CPU PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
1000 502 20445 24 10 0 152 768 - T ttyp0 0:00.00 /bin/sh -c _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*)\tthis="${dir}/"; real="${dir}" ;; .)\tthis="tools/"; real="/building/work/woods/m-NetBSD-3.0/tools" ;; *)\tthis="tools/${dir}/"; real="/building/work/woods/m-NetBSD-3.0/tools/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+\t(with: $@)}"; cd "${real}" && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget compat dependall
1000 540 606 0 18 0 360 964 pause I ttyp0 0:00.08 -ksh
0 606 352 0 10 0 172 1644 wait Is ttyp0 0:00.01 login
1000 748 19601 6 10 0 152 768 - T ttyp0 0:00.01 /bin/sh -c _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*)\tthis="${dir}/"; real="${dir}" ;; .)\tthis=""; real="/building/work/woods/m-NetBSD-3.0" ;; *)\tthis="${dir}/"; real="/building/work/woods/m-NetBSD-3.0/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+\t(with: $@)}"; cd "${real}" && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget tools dependall
1000 4176 25657 0 10 0 244 916 - T ttyp0 0:00.01 sh ./build.sh -m i386 -u -U -D /build/woods/becoming/NetBSD-3.0-i386-ppro-destdir -R /build/woods/becoming/NetBSD-3.0-i386-ppro-release -T /build/woods/becoming/NetBSD-3.0-i386-i386-tools -M /build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj distribution
1000 6070 25170 6 10 0 152 768 - T ttyp0 0:00.00 /bin/sh -c _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*)\tthis="${dir}/"; real="${dir}" ;; .)\tthis=""; real="/building/work/woods/m-NetBSD-3.0" ;; *)\tthis="${dir}/"; real="/building/work/woods/m-NetBSD-3.0/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+\t(with: $@)}"; cd "${real}" && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget . do-tools
1000 7649 20063 0 10 0 152 768 - T ttyp0 0:00.01 /bin/sh -c _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*)\tthis="${dir}/"; real="${dir}" ;; .)\tthis=""; real="/building/work/woods/m-NetBSD-3.0" ;; *)\tthis="${dir}/"; real="/building/work/woods/m-NetBSD-3.0/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+\t(with: $@)}"; cd "${real}" && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget . build NOPOSTINSTALL=1
1000 11678 502 24 10 0 624 1268 - T ttyp0 0:00.03 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_ tools/compat/
1000 13228 20046 24 10 0 160 824 - T ttyp0 0:00.01 sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache
1000 17181 13228 24 52 0 352 964 - T ttyp0 0:00.01 /bin/ksh -c \n as_lineno_1=$LINENO\n as_lineno_2=$LINENO\n as_lineno_3=`(expr $as_lineno_1 + 1) 2>/dev/null`\n test "x$as_lineno_1" != "x$as_lineno_2" &&\n test "x$as_lineno_3" = "x$as_lineno_2"
1000 19601 6070 6 10 0 652 1240 - T ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_
1000 20046 11678 24 10 0 144 728 - T ttyp0 0:00.00 /bin/sh -c CC=cc CFLAGS=-O LDFLAGS= sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache
1000 20063 4176 0 10 0 652 1244 - T ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake distribution
1000 20445 748 24 10 0 1392 2032 - T ttyp0 0:00.04 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_ tools/
1000 25170 7649 6 10 0 652 1244 - T ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_ NOPOSTINSTALL
1000 25657 540 0 18 0 260 860 - T ttyp0 0:00.01 /bin/ksh ./myupdate.sh -j 1
1000 26148 540 0 2 0 108 528 kqread I+ ttyp0 0:00.23 tail -f /home/woods/becoming/NetBSD-3.0-i386-ppro.log
When I examine the tools/compat/configure script I find code that SHOULD
disable use of $ENV:
# Work around bugs in pre-3.0 UWIN ksh.
$as_unset ENV MAIL MAILPATH
PS1='$ '
PS2='> '
PS4='+ '
However for some reason this doesn't seem to have the necessary effect.
These configure scripts will also, unfortunately, find /usr/pkg/bin/bash
(if that's installed) and re-run themselves with bash, possibly causing
similar problems.
I'm not sure what the proper fix is, except to try adding the above to
my own wrapper script, implying they should be in build.sh proper I
suppose.....
--
Greg A. Woods
H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com> Secrets of the Weird <woods@weird.com>
--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: iGJbPm10SgAFb3W9W62KhH7//Ds9ZZGn
iQA/AwUBRFkEiGZ9cbd4v/R/EQKZnACfaGS9YZrNScuGFeORgpbue2uTR4IAn0LK
pj6CMyoD2PXf/QoLDcIiqr5i
=Ufe4
-----END PGP SIGNATURE-----
--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1--