Subject: Issues with the wrappers
To: None <tech-pkg@netbsd.org>
From: Krister Walfridsson <cato@df.lth.se>
List: tech-pkg
Date: 07/30/2006 14:47:16
I have noticed some issues with the wrappers while doing work on a package
that compiles quite a lot of files (lang/ghc).

1. Too much is cached in the wrapper cache

    Ghc use absolute paths to the files during compilation, with the
    result that all those input/output-filenames are added to the cache,
    and the cache has 65879 lines at end of compilation.  This is bad
    because /bin/sh does not handle big case statements efficiently [a],
    and each lookup consumes 0.2 seconds on my AMD Athlon 3500+ machine.
    This adds up, and hours are spent in this case-statement during the
    build of the package.

    I would assume that most filenames provided to the compiler are only
    used once, and do not need to be cached.  My suggestion is that
    we should not add the -o options or filenames that are used as input
    (or expressed differently, we should only add arguments that start
    with an '-' and their arguments, unless they are the '-o' option). [b]

2. Too much is logged in the .wrapper.log

    The wrappers log quite a lot of detailed information by default,
    with the result that my .work.log file is 176236028 bytes.

    The wrapper functionality is mature, so this detailed information
    is seldom needed, and you can in general determine what happened
    by looking at the input/output-lines (i.e. those lines marked as "[*]"
    and "<.>" in the .work.log).  I would therefore like to change the
    default behavior to only log the input/output-lines.

Comments?

    /Krister


a: I'm planning to profile the shell to see if it can be improved.
This does however not help non-NetBSD platforms (or older NetBSD
versions).

b: It may be that it is more common with e.g. archive files of the
form /usr/pkg/lib/libfoo.a that need to be transformed, in which case
my suggestion should be changed to read "we should not add the -o options
or filenames that are used as input, unless they need to be transformed".