tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Bootstrapping pkgsrc with a compiler

On 2020-07-03 14:17, Brook Milligan wrote:
On Jul 3, 2020, at 12:01 PM, Greg Troxel <> wrote:s
You raise a bunch of relevant issues, but I think the following snippet captures how we are thinking of this differently.

I think you are ignoring the ABI stability issues and relying on the
fact that recently one can mostly get away with ignoring that.  With
earlier gcc versions one could not.

So here's a question:  If on netbsd-8, you:

  run a bootstrap
  build gcc8
  set GCC_REQD=8
  (which is what I think you want the new bootstrap to do)


  1) how many packages does you have (built with gcc5)?

  2) Are any of them built with c++ and linking to libstdc++?

  3) Is it guaranteeed to be sound if those libs are included in some
     other program?

  4) If one does pkg_rolling-replace, or make replace, will each of them
     then be rebuilt with gcc8?   Will that cause any issues?  Can you
     prove by specs/guaranteees of ABI stability that it will not?
In my experience, setting USE_PKGSRC_GCC=yes and GCC_REQD=8 leads to all packages (except of course prerequisites for the compiler and the compiler itself) to be built with gcc8.  If you also set the always-libgcc option when building the compiler, then everything will be linked to that compiler’s library as well.  I have not done this for a bulk build, but that certainly seems to be the case for a selection of 500+ packages that I have tried recently.  From the documentation, that is also certainly what those variables are intended to accomplish.

Thus, the answer to your question (1) is 100% built with gcc8 and 0% built with gcc5.  Of course, if a package itself sets GCC_REQD=9, then that would be built with a different compiler.  We could instead fail because of the mismatch.

I think we are both concerned about consistency and stability.  I am not seeing how that is threatened by installing a selected compiler during bootstrap and sticking with that single choice throughout the build of packages.  I think that is exactly what those options do, modulo package-specific requests for a higher version which we could handle if we want.

It is true that using these variables leaves the prerequisites being built with a different compiler.  Perhaps that is a critical point.  If so, then we need a make replace or something equivalent on that set so that everything in the main tree is built with a single compiler.  Or perhaps we need to do something more complex as part of the bootstrap.  To handle that, perhaps the bootstrap should install the compiler and all its prerequisites in one place and then use that exclusively to build things installed elsewhere.  Is that what you meant by /usr/pkg/COMPILER?  I think that still requires only a single bootstrap to do, one that is not particularly different than what I suggested.  If the prerequisites are the issue, then it seems this is the best way to guarantee that all packages installed in the main tree are compiled with the same compiler.

Perhaps I am confused about these variables, either in intent or effect.  If so, please correct me.  Otherwise, I think it comes down to (i) whether the prerequisites are ok or should be replaced once the right compiler is built and (ii) whether we should fail if a package requires an even more recent compiler than is selected.


I haven't had time to read this thread carefully, but has anyone raised the possibility of a separate pkgsrc tree with just compilers, to be utilized when bootstrapping other trees?  Similar in concept to what pbulk does.  Use pkgsrc to build the compilers under one prefix using just the OS-supplied compiler, and use the pkgsrc compilers there as a base compiler for other trees.  That would eliminate the lengthy list of exempted packages in the mk.conf generated by auto-pkgsrc-setup.

I've also bootstrapped pkgsrc from caveman GCC installations in the past, and it proved to be a headache dodging all the idiosychrasies of GCC that can trip up pkgsrc builds.  Using the community-tested compiler packages in pkgsrc, but keeping them outside the tree that uses them might minimize man-hours needed to get around this problem.


Home | Main Index | Thread Index | Old Index