Re: make: avoid breaking backwards compatability

[Jarmo Jaakkola <jarmo.jaakkola+netbsd-toolchain%roskakori.fi@localhost>]

On Thu, Sep 04, 2014 at 04:28:11AM +0000, David Holland wrote:
> No, it's a file with the same basename (in potentially any directory)
> and a different suffix.

So you're saying that dir1/foo.c is accectable to make dir2/foo.o here?

    .PATH: dir1
    executable: dir2/foo.o

Because this is what I meant.  You take what is in the target name
(dir2/foo.o), replace the suffix (dir2/foo.X) and try to find an
existing target or a file from the current directory, or .PATH.

> That's the point of .PATH. This is something that (IIRC) POSIX doesn't
> have at all, even as VPATH, so it's no surprise that the description
> of suffix rules in POSIX doesn't support the feature or include the
> hooks wanted to use it successfully.

Yes, POSIX does not have it at all, but you will end up with the exact
same situation with examples I have provided, without .PATH.

>  > So, why should subdir/foo.c be the file we're looking for?  I have never
>  > seen it mentioned anywhere that a listed dependency should affect
>  > the implied source.  I must admit that I have experience only from
>  > NetBSD's make, gmake and reading the POSIX standard.
> 
> Because it's looking for sources. If a matching source is already
> provided, why should it search for another one? Then it'll have two
> and it has to guess, or in this case assume, which one you meant to be
> the "real" one.

But it is not a matching one, is it?  In my books "dir/foo" is not
the same as "foo".  It can be, if "dir" is in .PATH but there's no need
for the explicit dir prefix.

> As to why it would be provided, one reason (as we found when the build
> broke repeatedly) is to indicate specifically which of several
> potential matching sources to use. Otherwise there's no way to do this.

Of course there is.  What's wrong with saying dir/foo.o if that's what
you want?  If you want foo.o made from foo.S instead of foo.c, use
    .SUFFIXES: .S .c
instead of 
    .SUFFIXES: .c .S

If you need to consistently override on a file-by-file basis, then I'd
say your file hierarchy or build process needs refactoring.

>  >     .SUFFIXES: .a .b .c
>  >     .a.c: ; ...
>  >     .b.c: ; ...
>  >     test.c: test.b
>  >     test.a:
>  >     test.b:
>  > 
>  > .a.c should be used, not .b.c.  This is completely within the realm
>  > of POSIX, and I think it would have been specified that the dependencies
>  > must affect the implied rule / source selection process, if it really
>  > should work like that.
> 
> Why? POSIX doesn't have .PATH and there's no notion of the source of a
> suffix rule being anywhere but the current directory. So of course it
> doesn't have the apparatus to support it.

But a search from .PATH is not needed to get us into a situation where
there might be multiple matching sources.  Going by your example
of foo.c and foo.S, those two files could be in the same, current
directory.  There might even be files foo1..fooN with the same suffixes
and about half of them should be made from .S and the rest from .c.  Yes,
the overriding behavior would be nice, but it has not still been
specified.

And having test.b force make to choose .b.c instead of .a.c as is
dictated by .SUFFIXES ordering, is against POSIX.

> Which, bluntly, is exactly what one wants --
> there is no non-contrived case where it's reasonable for the 'extra
> dependency' to mean that I wanted to compile foo.c and I know that
> foo.c includes foo.S. If I'd written an 'extra dependency' on a .h
> file, there wouldn't be a .h.o rule to try to compile the header file
> on its own, because that isn't a sensible thing to do.

Of course there wouldn't be .h.o rule, but there are files other than
C files in the universe.  There might well be situations where my
examples are not contrived at all.

>  > I have to agree here.  However, this whole thing actually had nothing
>  > to do with .PATH.
> 
> It has a lot to do with .PATH because the point at which this behavior
> becomes needed is when .PATH contains multiple candidate sources and
> you want to explicitly specify which one to use.

No it hasn't, look above.  There just needs to be two candidate files
with different suffixes in the same directory.

> As I said above, without .PATH it's unnecessary because there can be
> only one possible $(.IMPSRC) for any given suffix rule and any given
> target.

Yes, only one $(.IMPSRC) per suffix rule, but there can be multiple
rule candidates, thus multiple $(.IMPSRC) candidates.  And in POSIXly
correct behavior it is .SUFFIXES which drives the selection, not
the dependencies of the target.

>  > If this actually turns out to be the consensus, I can reimplement this
>  > feature on top of the new functionality.
> 
> That doesn't necessarily seem like a good plan.

Yeah, in another mail you said that some compatibility code could be in
order.  How do you want it done?

How is it activated?  Should it work outwardly exactly like it did
before, or should there be changes?  Internally make should probably
at least look at the dependencies first, before it even tries
files or other targets.

It did for example not support multi-stage transformations earlier.
"foo.z: foo.x" would not have caused make to select ".x -> .y -> .z"
over ".w -> .z".  If implemented, this might give surprises, as make
can get quite creative with a large set of suffix rules.

Should all the dependencies be looked at, or only the first one?
Previously they were.  But I think that the overriding source usually
is the first one (the rest 50 or so are probably all header files).
It would also protect people at least from some cases where
the dependency unexpectedly comes from some complex chain of included
makefiles and it would force the user to be up-front when he means
to override.

Do we still want make to accept non-matching files?  E.g. "dir/foo.c"
for "foo.o" when dir is _not_ in .PATH.  Because here you're not
telling make to prefer one choice over another, you're telling it
to completely ignore what it would normally do.

-- 
Jarmo Jaakkola