Subject: building 3.0 in the background can "freeze" due to unsafe use of ksh
To: None <netbsd-users@netbsd.org>
From: Greg A. Woods <woods@planix.com>
List: netbsd-users
Date: 05/03/2006 15:29:12
--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Building of 3.0 in the background (with nohup) can "freeze" with a stuck
configure script due to what appears to be unsafe use of ksh (i.e. when
$ENV is not cleared and thus all manner of things can get buggered).

I start build.sh with a very simple little wrapper script (that sets a
number of options and parameters for build.sh) as follows:

  $ nohup ./myupdate.sh -j 1 >> /home/woods/becoming/NetBSD-3.0-i386-ppro.log 2>&1 < /dev/null &

At several points while building the tools it stops running I have to
bring the process into the foreground, and then send it a newline before
it will continue to run:

  [1] + Stopped (tty input)  nohup ./myupdate.sh -j 1 >> /home/woods/becomin
  14:18 [56] $ 

  ^Z
  [1] + Stopped              nohup ./myupdate.sh -j 1 >> /home/woods/becomin
  14:18 [57] $ j  
  [1] + 25657 Stopped              nohup ./myupdate.sh -j 1 >> /home/woods/becomin
  14:19 [58] $ bg
  [1] nohup ./myupdate.sh -j 1 >> /home/woods/becomin
  14:19 [59] $ 
  14:19 [59] $ j
  [1] + 25657 Running              nohup ./myupdate.sh -j 1 >> /home/woods/becomin


The log file now contains garbage that would be produced by a special
function in my ~/.kshrc (used to set the window title upon "cd"):

----------------------------------------
dependall ===> tools/compat
mkdir -p include/sys include/machine include/rpc
rm -f config.cache
CC=cc CFLAGS=-O LDFLAGS=  sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache
\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming/~ | woods[1]:not\007/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat
\033]0;sh://becoming/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007/bin/ksh: exit code: 1
\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming//build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007\033]0;sh://becoming/~ | woods[1]:not\007/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat
\033]0;sh://becoming/build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj/building/work/woods/m-NetBSD-3.0/tools/compat | woods[1]:not\007/bin/ksh: exit code: 1
configure: creating cache config.cache
checking for gcc... cc
----------------------------------------


At the point when things were hung the following was running on that
terminal, and as one can see there's one ksh process which appears to
have been run without use of :

 UID   PID  PPID CPU PRI NI  VSZ  RSS WCHAN  STAT TTY      TIME COMMAND
1000   502 20445  24  10  0  152  768 -      T    ttyp0 0:00.00 /bin/sh -c _makedirtarget() {  dir="$1"; shift;  target="$1"; shift;  case "${dir}" in  /*)\tthis="${dir}/";  real="${dir}" ;;  .)\tthis="tools/";  real="/building/work/woods/m-NetBSD-3.0/tools" ;;  *)\tthis="tools/${dir}/";  real="/building/work/woods/m-NetBSD-3.0/tools/${dir}" ;;  esac;  show=${this:-.};  echo "${target} ===> ${show%/}${1:+\t(with: $@)}";  cd "${real}"  && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target};  };  _makedirtarget compat dependall 
1000   540   606   0  18  0  360  964 pause  I    ttyp0 0:00.08 -ksh 
   0   606   352   0  10  0  172 1644 wait   Is   ttyp0 0:00.01 login 
1000   748 19601   6  10  0  152  768 -      T    ttyp0 0:00.01 /bin/sh -c _makedirtarget() {  dir="$1"; shift;  target="$1"; shift;  case "${dir}" in  /*)\tthis="${dir}/";  real="${dir}" ;;  .)\tthis="";  real="/building/work/woods/m-NetBSD-3.0" ;;  *)\tthis="${dir}/";  real="/building/work/woods/m-NetBSD-3.0/${dir}" ;;  esac;  show=${this:-.};  echo "${target} ===> ${show%/}${1:+\t(with: $@)}";  cd "${real}"  && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target};  };  _makedirtarget tools dependall 
1000  4176 25657   0  10  0  244  916 -      T    ttyp0 0:00.01 sh ./build.sh -m i386 -u -U -D /build/woods/becoming/NetBSD-3.0-i386-ppro-destdir -R /build/woods/becoming/NetBSD-3.0-i386-ppro-release -T /build/woods/becoming/NetBSD-3.0-i386-i386-tools -M /build/woods/becoming/NetBSD-3.0-i386-i386-ppro-obj distribution 
1000  6070 25170   6  10  0  152  768 -      T    ttyp0 0:00.00 /bin/sh -c _makedirtarget() {  dir="$1"; shift;  target="$1"; shift;  case "${dir}" in  /*)\tthis="${dir}/";  real="${dir}" ;;  .)\tthis="";  real="/building/work/woods/m-NetBSD-3.0" ;;  *)\tthis="${dir}/";  real="/building/work/woods/m-NetBSD-3.0/${dir}" ;;  esac;  show=${this:-.};  echo "${target} ===> ${show%/}${1:+\t(with: $@)}";  cd "${real}"  && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target};  };  _makedirtarget . do-tools 
1000  7649 20063   0  10  0  152  768 -      T    ttyp0 0:00.01 /bin/sh -c _makedirtarget() {  dir="$1"; shift;  target="$1"; shift;  case "${dir}" in  /*)\tthis="${dir}/";  real="${dir}" ;;  .)\tthis="";  real="/building/work/woods/m-NetBSD-3.0" ;;  *)\tthis="${dir}/";  real="/building/work/woods/m-NetBSD-3.0/${dir}" ;;  esac;  show=${this:-.};  echo "${target} ===> ${show%/}${1:+\t(with: $@)}";  cd "${real}"  && /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_="${this}" "$@" ${target};  };  _makedirtarget . build NOPOSTINSTALL=1 
1000 11678   502  24  10  0  624 1268 -      T    ttyp0 0:00.03 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_ tools/compat/ 
1000 13228 20046  24  10  0  160  824 -      T    ttyp0 0:00.01 sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache 
1000 17181 13228  24  52  0  352  964 -      T    ttyp0 0:00.01 /bin/ksh -c \n  as_lineno_1=$LINENO\n  as_lineno_2=$LINENO\n  as_lineno_3=`(expr $as_lineno_1 + 1) 2>/dev/null`\n  test "x$as_lineno_1" != "x$as_lineno_2" &&\n  test "x$as_lineno_3"  = "x$as_lineno_2"  
1000 19601  6070   6  10  0  652 1240 -      T    ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_  
1000 20046 11678  24  10  0  144  728 -      T    ttyp0 0:00.00 /bin/sh -c CC=cc CFLAGS=-O LDFLAGS=  sh /building/work/woods/m-NetBSD-3.0/tools/compat/configure --cache-file=config.cache 
1000 20063  4176   0  10  0  652 1244 -      T    ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake distribution 
1000 20445   748  24  10  0 1392 2032 -      T    ttyp0 0:00.04 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_ tools/ 
1000 25170  7649   6  10  0  652 1244 -      T    ttyp0 0:00.02 /build/woods/becoming/NetBSD-3.0-i386-i386-tools/bin/nbmake _THISDIR_  NOPOSTINSTALL 
1000 25657   540   0  18  0  260  860 -      T    ttyp0 0:00.01 /bin/ksh ./myupdate.sh -j 1 
1000 26148   540   0   2  0  108  528 kqread I+   ttyp0 0:00.23 tail -f /home/woods/becoming/NetBSD-3.0-i386-ppro.log 


When I examine the tools/compat/configure script I find code that SHOULD
disable use of $ENV:

  # Work around bugs in pre-3.0 UWIN ksh.
  $as_unset ENV MAIL MAILPATH
  PS1='$ '
  PS2='> '
  PS4='+ '

However for some reason this doesn't seem to have the necessary effect.

These configure scripts will also, unfortunately, find /usr/pkg/bin/bash
(if that's installed) and re-run themselves with bash, possibly causing
similar problems.

I'm not sure what the proper fix is, except to try adding the above to
my own wrapper script, implying they should be in build.sh proper I
suppose.....

-- 
						Greg A. Woods

H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>       Secrets of the Weird <woods@weird.com>

--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: iGJbPm10SgAFb3W9W62KhH7//Ds9ZZGn

iQA/AwUBRFkEiGZ9cbd4v/R/EQKZnACfaGS9YZrNScuGFeORgpbue2uTR4IAn0LK
pj6CMyoD2PXf/QoLDcIiqr5i
=Ufe4
-----END PGP SIGNATURE-----

--pgp-sign-Multipart_Wed_May__3_15:29:10_2006-1--