Re: build dependencies vs tools

["Johnny C. Lam" <jlam%NetBSD.org@localhost>]

On Wed, Sep 13, 2017 at 04:55:15PM +0100, Jonathan Perkin wrote:
> * On 2017-09-13 at 15:46 BST, J. Lewis Muir wrote:
> 
> > On 09/11, Jonathan Perkin wrote:
> > > * On 2017-09-11 at 15:47 BST, Jan Danielsson wrote:
> > > 
> > > > Looking at some dependencies on pkgsrc.se and a thought occurred:
> > > > What's the difference between "build dependencies" and "USE_TOOLS"?
> > > 
> > > The main difference is that a tool can come from the operating system.
> > > 
> > > If an operating system comes with bison and you request bison via
> > > USE_TOOLS then, as long as the OS version works fine, it will be used
> > > for the build.  If you add bison as a BUILD_DEPENDS, then regardless
> > > of whether the OS provides it, the pkgsrc bison package will be pulled
> > > in.
> > 
> > What if bison had built-in support (i.e., had a builtin.mk file) and
> > the OS had a bison version that satisfied the built-in requirement, and
> > PREFER_NATIVE was set to yes, then a BUILD_DEPENDS of bison would use
> > the bison from the OS, right?
> 
> No.  builtin.mk and PREFER_* are buildlink concepts for headers and
> libraries.  The tools infrastructure does something similar but in a
> different way for programs.  BUILD_DEPENDS and other *DEPENDS have no
> such logic, they just add dependencies on pkgsrc packages.

DEPENDS and BUILD_DEPENDS are the underlying method of expressing
dependencies on other packages.

The two main pkgsrc frameworks encountered by package maintainers
that use these dependency methods are the buildlink and tools
frameworks.  They are independent of each other, but they both
ultimately reduce to adding a DEPENDS or BUILD_DEPENDS on the
correct package if needed.

buildlink (buildlink3.mk and builtin.mk) is a way to express
dependencies on other packages' headers and libraries and other
files used at build-time (*.pc files, etc.) and run-time (shared
libraries, etc.).  In general, if a package provides headers and
libraries, it should provide at least a buildlink3.mk file.  The
builtin.mk files are provided to hint to the buildlink machinery
on how to use OS-provided versions of those headers and
libraries.  PREFER_PKGSRC and PREFER_NATIVE are user-settable
variables that affect how buildlink uses the builtin.mk hints.

USE_TOOLS is a way to express build-time and run-time
dependencies on other packages' binaries.  These are often
script interpreters, e.g., perl, bash, etc.  Sometimes, we need
GNU versions of common utilities, e.g., gmake instead of make,
bison instead of yacc, etc.  USE_TOOLS is a shorthand to hide a
lot of makefile code to either use the OS-provided binary or to
add a dependency on the correct pkgsrc package, and to also use
a consistent name to invoke the tool, regardless of how the
OS-provided binary is named.

In the example above regarding bison, these are the scenarios
that the package maintainer should consider:

(1) Package just needs a yacc, not necessarily bison.

    USE_TOOLS+= yacc

(2) Package needs bison (yacc won't be enough).

    USE_TOOLS+= bison

(3) Package needs bison at run-time as well as at build-time.
    Perhaps it provides scripts that wrap GNU bison to extend
    the language accepted.

    USE_TOOLS+= bison:run

I hope this explanation demystifies how dependencies are
expressed in pkgsrc.

Regards,
-- 
Johnny C. Lam
jlam%NetBSD.org@localhost