Re: Initial LLVM/clang patch

[Alistair Crooks <agc%pkgsrc.org@localhost>]

On Thu, Feb 03, 2011 at 09:50:58PM +0100, Joerg Sonnenberger wrote:
> On Thu, Feb 03, 2011 at 05:18:39PM +0100, Alistair Crooks wrote:
> > On Thu, Feb 03, 2011 at 06:04:28PM +0200, Antti Kantee wrote:
> > > Seems like the only required action is "make checkout" which runs two
> > > svn commands.
> > 
> > If you start going down this route, then version information needs to be
> > retained somewhere for the external toolchain. Some minimum versino to build
> > the source tree needs to be placed in the source tree.
> 
> The revision numbers are already contained for the purpose of versioning
> inside the binary. The checkout target obtains exactly that revision(s)
> from the svn repository. All comparing etc is left to the subversion
> checkout command, including merging local changes with upstream etc.

Thanks, yeah, I understand the approach.  I still maintain that this
is the wrong way to do it. It's not the fact that one can do this with
svn, it's the fact that svn is being proposed to do this. Wrong way.
 
> > A subversion client is required.
> 
> A subversion client is needed if you want to play or develop, yes. If
> you build on NetBSD, you can obtain it from pkgsrc. If you are on Linux,
> you can use whatever your distribution uses etc. I don't consider this
> a big road block. If you want to help improve LLVM/clang, it is a
> non-issue since you are going to deal with the upstream repository
> anyway.

I wonder if we're talking past each other.  I'm looking at this for
the long haul - I want a vehicle for external toolchains that can be
used easily, requires only enough intrusion into the repo as is
absolutely essential for ordinary users to build using that external
toolchain.  If this is a developer only thing, it should not be part
of the standard tree.
 
> > > > 2. Doing this in src means that no binary packages for the toolchain.
> > > 
> > > why?
> > 
> > pkgsrc makes one for you by doing "make package". I can take the result,
> > optionally sign it, and move it to any machine I want and use it there.
> 
> Nothing stops you from that with the clang package. The patch provides
> two different things that pkgsrc can't easily replicate:
> 
> (1) Building a native clang compiler for running on NetBSD. Even if the
> host system you have is not NetBSD.
> 
> (2) Building a cross compiler and integrating it into the NetBSD tree.
> 
> This is not about providing an external toolchain, but about getting
> clang to the point where it can be fully used as normal in-tree
> toolchain. At that point or when it is far enough, it is perfectively
> sensible to do a real import. I just want to avoid the trouble of
> keeping the version synchronised or people committing local patches
> without bringing them up upstream first etc. As in: I want to avoid a
> divergenting development line.

Aha, OK, so we're coming at this from different viewpoints.  My view
is that if this external toolchain is not far enough along to be
committed to the tree, then it doesn't belong there.  If it doesn't
belong in-tree, special custom knobs to download it and build it also
do not belong in the tree.  If we can use external packages (via a
clang-devel fast-moving pkgsrc package, for example), then let's use
that. Yes, we'd need to add a "svn co" download method, but pkgsrc
would benefit from that too.
 
> > Now I'm sure it's possible to just tar up the external toolchain you've
> > built, and carry it around with you, but the basic mode of behaviour is
> > the same - everything you are doing is reproducing, in a different way,
> > with less testing, and with less features, what pkgsrc gives you for free.
> > On a number of platforms.
> > 
> > I'm not into luddite or amish games - neither am i advocating nih and a
> > "damn external toolchains" attitude. What I am saying is that a "hey,
> > let's do this for now, it's much easier, it's easy, it's simple" approach
> > has some drawbacks.
> 
> The reason for wanting the build infrastructure integrated is the very
> same reason always given for having modular Xorg in xsrc. The need for
> fetching a second repository is the very same as for modular Xorg. The
> only difference is that the cvs client is shipped with NetBSD.
> 
> The clang build from src/tools looks like what the other parts are
> doing. It is going to be integrated into the cross-compiling environment
> just like GCC and PCC already are. I have not included this part in this
> patch as it is more intrusive and needs some refinement. That would just
> obscure the issue for now.

Yeah, the problem I have is going out to the network, and downloading
sources by another external program.  Needing network, possibly
proxies, external programs built.  No way of checking that downloaded
sources are what we need.  No way of checking that we got the sources
from the right place.  Network connectivity from within a
tightly-controlled prison. Not when this can be done perfectly well from
within pkgsrc, anyway.
 
> > You haven't addressed my point, though - why are you trying to
> > re-invent pkgsrc?
> 
> It is as much a re-invention of pkgsrc as modular Xorg support is.

Not at all - you are proposing a fundamental shift in the way that src
would be built.  Xorg is all in the repos.  Whilst I'm all for
supporting external toolchains, adding all of this network
access/excess to a build process is not the correct way to do it.

Regards,
Alistair