Subject: Re: Making pkgsrc more robust
To: None <tech-pkg@NetBSD.org, rillig@NetBSD.org>
From: Gilles Dauphin <Gilles.Dauphin@enst.fr>
List: tech-pkg
Date: 02/07/2006 14:18:29
> From: Roland Illig <rillig@NetBSD.org>
> 
> Roland Illig wrote:
> > Hi,
> > 
> > to make pkgsrc (in general; especially the bulk builds) more robust 

And please look at pre-build : 
 if you don't use pkgview you don't have /usr/pkg/vulnerable
 and the pre-build break at line:
 
 -----------------------------
PKG_DBDIR=`(cd "${PKGLINT_PKG_DIR}" && ${BMAKE} show-var VARNAME=PKG_DBDIR)`
DEPOTBASE=`(cd "${PKGLINT_PKG_DIR}" && ${BMAKE} show-var VARNAME=DEPOTBASE)`
PKG_TOOLS_BIN=`(cd "${PKGLINT_PKG_DIR}" && ${BMAKE} show-var VARNAME=PKG_TOOLS_B
IN)`

for dbdir in "${PKG_DBDIR}" "${DEPOTBASE}"; do
        echo "pre-build> Removing all installed packages in ${dbdir}"
        if cd "$dbdir" 1>/dev/null 2>&1; then  <--- THIS BREAK THE BULK
                for pkg in *
                do
---------------------------

Gilles

 
> > against unforeseen errors, I'd like to add the command "set -e -u" at 
> > the top of every shell program that is used in the pkgsrc infrastructure.
> > 
> > The -e option terminates the program as soon as a command fails, which 
> > is much more often appropriate than just continuing. The current shell 
> > programs contain lots of lines like:
> > 
> >     cd "$some_directory"
> >     do_something
> > 
> > So what should be done if the directory $some_directory is not accessible?
> > 
> > The -u option terminates the program as soon as an uninitialized 
> > variable is used. This prevents from spelling mistakes. If you want to 
> > use a possibly-undefined variable, you can provide a default value using 
> > ${possibly_undefined_variable-default_value} instead of 
> > $possibly_undefined_variable.
> > 
> > Together these two options will probably make pkgsrc break more often 
> > than before, but almost always at points where the error checking had 
> > not be done in the past. These points must be fixed anyway, so this is 
> > just a help to find them early.
> 
> Today I had the idea to add the following to mk/defaults/mk.conf:
> 
> # This variable determines which amount of self-checks should be done
> # in pkgsrc. The higher this value is, the more packages will probably
> # fail. Currently, the levels 0, 1 and 2 are supported.
> PKG_STRICT_LEVEL?=      0
> 
> And the following for mk/bsd.pkg.mk:
> 
> .if ${PKG_STRICT_LEVEL} >= 1
> _PKG_DEBUG+=            set -e;
> .endif
> .if ${PKG_STRICT_LEVEL} >= 2
> _PKG_DEBUG+=            set -u;
> .endif
> 
> SCRIPTS_ENV+=           PKG_STRICT_LEVEL=${PKG_STRICT_LEVEL:Q}
> # (and maybe some other environment variable lists)
> 
> This would catch many errors in the packages, as well as some of the 
> infrastructure code. Standalone shell programs should then start with 
> the following two lines.
> 
> test "${PKG_STRICT_LEVEL-0}" -lt 1 || set -e
> test "${PKG_STRICT_LEVEL-0}" -lt 2 || set -u
> 
> Roland