lintpkgsrc important changes RFC (Was: lintpkgsrc 'Out of memory!' error)

[David Brownlee <abs%netbsd.org@localhost>]

On 2 September 2012 21:53, David Holland 
<dholland-pkgtech%netbsd.org@localhost> wrote:
> On Sun, Sep 02, 2012 at 11:08:23AM +0200, John Marino wrote:
>  > I'm not having luck with lintpkgsrc today.
>  >
>  > This command: lintpkgsrc -or -M /mech/bulk
>  > terminated with "Out of memory!"
>  >
>  > It's an 8GB machine, but 4600M are truncated since the 32-bit
>  > platform was installed on it.  Still, with nearly 4GB of ram,
>  > lintpkgsrc runs out of memory just identifying obsolete distfiles?
>  >
>  > Is there a way to work around this?
>
> rewrite it not in perl? :(

The problem with lintpkgsrc is that it tries to speed up the Makefile
processing by implementing (just enough of) a Makefile parser in perl.
Unfortunately as the usage of Makefile constructs gets more
sophisticated in pkgsrc this is probably a doomed arms race.

At this point I would suggest:

- "Installed package options" (-i -u) should die (they are better
covered by pkg_chk)

- The Makefile parsing should be ripped out and lintpkgsrc should just
call 'make' to extract values. It will be slow, but it will at least
be right, and the speed of pkgsrc Makefile parsing is an issue which
really needs to be addressed separately.

I actually quite like the idea of lintpkgsrc focussing towards some
form of per pkgsrc package cache recording package PKGNAME and other
relevant information, making the first run expensive but helping later
runs (some simple invalidation rules possibly based on newer files in
pkgsrc tree or mk.conf)

Any objections to linktpkgsrc being gutted and becoming slower but right? :)