tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Bootstrapping pkgsrc with a compiler

On 2020-07-06 20:39, Brook Milligan wrote:
On Jul 4, 2020, at 9:47 AM, Greg Troxel <> wrote:

Martin Husemann <> writes:

On Sat, Jul 04, 2020 at 09:49:28AM -0400, Greg Troxel wrote:
Listing what actual ABI issues exist today is useful.  But it does not
refute the statement that there are no stability specifications.  And
this is pkgsrc, with many compilers and many platforms.
I am trying to say that it is not C++ and ABI that are the big issue here,
but specific switches like what happened to the internal libstdc++ in NetBSD,
or "restricted" issues with specific compilers (gcc) and their C++ runtime.
I don't follow.   There are certainly multiple ABI issues, and what I am
saying is that we have to be careful, unless we can prove that there are
not and will not be issues.   The C++ ABI is not guaranteed stable.
There is also the NetBSD libstdc++ change.   And there are other compilers.

As you say, this is pkgsrc, with many compilers and many platforms. Many
of them may not see the lossage, but that does tell nothing about the others.
Right, so therefore we must build *everything* with the same compiler.

I thought you were arguing that this was unnecessary.
I’m not sure where this thread has gone, entirely.  However, I want to refocus so we do not make the perfect the enemy of the good.

My summary would be the following:

- The pkgsrc status quo uses a mixture of compilers across packages determined mostly by GCC_REQD.

- Unless the user specifies GCC_REQD greater than or equal to anything in a package, this results in a mixture of compilers being used across packages.

- At least for C++ packages, a mixture of compilers can lead to "bad things”.

- I proposed a minimal solution that would use a consistent compiler across everything except the prerequisites for the chosen compiler.  On one of my reference systems, that would leave 12 non-bootstrap packages built with the native compiler.  Yes, there could still be problems, but I feel that would be an improvement and is easily doable.

- Other suggestions are for a double bootstrap process that would build the toolchain and secondarily build the packages.  This would clearly be the best solution, as all packages would be built with the same compiler.  If we can implement that, then it is definitely preferable to my incremental solution.

To clarify what a double bootstrap process would look like, I implemented it manually as follows:

- bootstrap normally into a toolchain prefix.

- build a compiler package, here gcc8 with always-libgcc, and install it in the toolchain prefix

- create a bootstrap Makefile fragment with GCCBASE pointing to the toolchain gcc8

- set PATH to include bin, sbin, and gcc8/bin in the toolchain prefix so that the next bootstrap will use the toolchain compiler

- bootstrap a second time into the main prefix

- add bin and sbin from the main prefix to PATH

- build packages to install in the main prefix

I believe that this is building everything that gets installed into the main prefix with the compiler built in the toolchain prefix.  Thus, I believe that this would meet all the requirements so far identified.

I hope we can build the wrapper around our current bootstrap that will accomplish this or some equivalent.

I also hope we can include capability for doing this all with clang as well, in case that is selected instead of some version of gcc.


A wrapper for the 1st proposal has existed for years and is well-tested on CentOS, Darwin, and NetBSD:

The double bootstrap idea has some merit in that it eliminates the risk of ABI incompatibility.  However, I have not experienced any such problems in several years using auto-pkgsrc-setup + GCC_REQD on CentOS, nor any in about a year of testing on NetBSD.

It would not be difficult to adapt auto-pkgsrc-setup to support the double bootstrap method.



Home | Main Index | Thread Index | Old Index