Re: Building NetBSD 11 Release from Source Fails on Linux/MacOS

[Greg Troxel <gdt%lexort.com@localhost>]

Helge Pfeiffer <ropf%itu.dk@localhost> writes:

> I am new to NetBSD. I am trying to build NetBSD 11 for amd64 x86_64 from source.
> I tried it both on a Linux (Pop!_OS 22.04 LTS, which is an Ubuntu 22.04) host and on a MacOS (Sequoia). On both hosts, I
> see the same issue. Building a release fails, see my detailed description below. 

Welcome.  I'm not sure what's wrong, but surely we will get you over
this.  Problems building on various hosts creep in, are discovered, and
are fixed, but I would be surprised about netbsd-11.

> Specifically, I am doing the following:
>
> 1. I checkout the source code of the NetBSD 11 branch from the Git repository mirror on GitHub:
>    `git clone --depth 1 https://github.com/NetBSD/src.git -b netbsd-11 netbsd-11-src`

Good call to take the branch tip vs the tag.

> 2. I build the tools
>    `./build.sh -U -u -j12 -m amd64 -a x86_64 -O ./obj tools`
>    Building the tools works. No errors are reported.
> 3. I try to build a release with the following command:
>    `./build.sh -U -u -j12 -m amd64 -a x86_64 -O ./obj release`

Almost certainly not related to the problem, but
  - Tools get built automatically so you don't need that explicitly.
    It is nice to validate that they build ok, though.
  - The amd64 machine defaults to (has to be) x86_64, so you can skip
    that, but it is not wrong and should not hurt.

> Step 3 fails. When inspecting the logs, I can see the following error:

The build is very many nested levels and you often have to go back
hundreds of lines to find the problem.  This is made harder with -j as
other branches finish and produce output.

You have -u, so you can re-run with less compiling.

(Best to put the build commands in a script if you haven't already.)

-j12 is of course useful for building faster but it makes the build log
harder to read.  I suggest removing the -j12 and running again.  You may
well have found the right/first/causative error but without -j is less
confusing.

My build.sh command, in case reading another one helps you:

  ./build.sh -m amd64 -j4 -x -u -U -O /usr/obj/gdt-11/amd64 -T /usr/obj/gdt-11/tools -D /usr/obj/gdt-11/destdir/amd64 -R /usr/obj/gdt-11/releasedir -X /n0/gdt/NetBSD-11/xsrc release

In theory, I think relative paths are fine for -O.  I prefer to have an
entire tree where I put the objdir, the tools dir, the destdir, and the
release dir.  I have multiple trees for various versions.

You will see -x and -X in my build line.  I am not suggesting at all
that you add X11 to the mix now.  My build command is just a literal
cut/paste from builds that work - on NetBSD 11.

> When inspecting the file `external/lgpl3/gmp/dist/mpn/x86_64/sqr_diag_addlsh1.asm`, I see that on line 33, a file
> `../config.m4` is referenced (33 include(`../config.m4').
> However, there is no file `config.m4` under `external/lgpl3/gmp/dist/mpn/` (nor under
> `external/lgpl3/gmp/dist/mpn/x86_64/`). The only `config.m4` files that I can find are under the respective lib
> directory, e.g., `external/lgpl3/gmp/lib/libgmp/arch/x86_64/config.m4` (I found them via `find external/lgpl3/gmp -name
> config.m4 | sort`).

In my setup, the sources are not written to at all.  Probably that is
what is happening to you because it is the standard approach.  I find
that I have the config.m4 file (current not 11):

  $ find /usr/obj/gdt-current/ -name config.m4
  /usr/obj/gdt-current/amd64/external/lgpl3/gmp/lib/libgmp/config.m4
  /usr/obj/gdt-current/amd64/tools/gmp/build/config.m4

The second copy is from gmp being built as part of the tools and the
first is from the build for it to be in the target system.

make will look for things in the objdir as well as the sources.

I suggest looking in the obj directories.

I have two theories each with weak support:

  The "-O ./obj" is causing the process of finding config.m4 to go
  badly.

  Something about building the toolchain on Linux or mac is not ok, but
  it seems the tools build finished without errors.



You can cd into the sources and run (my paths)

  /usr/obj/gdt-11/tools/bin/nbmake-amd64

This is the cross make, which has baked in where the objdir/destdir are.
It lets you isolate and redo one build step.  In your case, tools are in
some default location, perhaps ./tools (but it's been 15 years since I
did a build with defaults).


I would also suggest:

  either leave out -O, as I think using objdir is default

or

  pass explicit absolute paths for all the directories