Re: make: dependfile preference list

["Simon J. Gerraty" <sjg%crufty.net@localhost>]

On Thu, 19 Apr 2012 07:12:12 +0000, David Holland writes:
> >      .MAKE.DEPENDFILE_PREFERENCE
> >                      The ordered list of makefile names (default
> >                      `${.MAKE.DEPENDFILE:T}') that make will look for to
> >                      read
> >                      dependencies from.  Only the first found will be
> >                      read.
>
>Is there some reason this has to be a builtin and not just a piece of
>logic in a make library that wants to do this sort of thing?

Yes, I think so.  Adoption of meta mode, is much easier if it can be
seamless.  That is, the 1000's of makefiles that make up the bulk of the
tree "just work" either way.

I've just finished such a conversion of a very large tree at work with
many active developers.  It was a pure cross-build environment.
It took over a year and except for the last few months most developers
were largely unimpacted (unaware).

We build the entire tree (much bigger than say NetBSD) for over a
dozen architectures - in parallel in a single pass.  For a full tree
build the 0th bmake reads several thousand makefiles (Makefile.depend*)
before starting sub-makes all over the place.  I'm thinking I need to
make the job scheduling better - ie. don't start a submake unless you
can give it several tokens.
The real benefit for most developers though, is that from anywhere
in a clean tree, they can run make and build only and exactly what is
needed for that directory.

Being able to set .MAKE.DEPENDFILE (default .depend) is a big part of
that.

But one setting does not really suit everyone or even all parts of a
single tree. 

Originally I looked at doing one Makefile.depend per directory.
The dependencies for a directly - to a very large extent are independent
of $MACHINE - there are obvious exceptions, most of which can be handled
by a bit of filtering when updating the file.

That approach fails though when the likelyhood of a single directory
being built at the same time for multiple $MACHINE's is high - you have
to start using mutexes etc, no thanks.  So we opted for
Makefile.depend.${MACHINE} as the default - allows everyting to run in
parallel.

To the greatest extent possible we want the Makefile.depend.${MACHINE}
auto-updated - so we know they are correct, and can skip reviewing them
etc.   There are places though where a manually maintained file makes
much more sense - especially in the high level "pseudo" target area.
We have perhaps 2-300 manually maintained files which is < 1% of the
total.  

It would actually have been very nice for our SCM/tools folk, to have
been able to give clearly different names to manually edited files, so
that we could easily enforce code reviews etc on them, while skipping
for the autogenerated ones.  Fortunately for us nearly all the manual
files are in one part of the tree - but naming flexibility would have
been a huge win.

That and thinking about how to make this more palatable to the broader
BSD community, prompted the introduction of .MAKE.DEPENDFILE_PREFERENCE.

Yes, it *could* be done with makefiles but given
.MAKE.MAKEFILE_PREFERENCE, 
.MAKE.DEPENDFILE_PREFERENCE seemed natural and will allow a more elegant
solution.  Which is also important for getting people to adopt something
new.

--sjg