Subject: WIP: USE_NEW_TOOLCHAIN for i386 Build from an Alpha Running OpenBSD
To: None <>
From: Chris Jepeway <>
List: tech-toolchain
Date: 09/10/2001 19:38:33
Well...don't shoot me, but I've made decent progress with
USE_NEW_TOOLCHAIN for an i386 -current build hosted on an
Alpha running OpenBSD-2.9.

I figured using a different O/S and a different processor
would uncover all sorta problems with the new toolchain that
might otherwise stay hidden.  Too, I only have clunky
133MHz i386 boxes for native builds, so I thought compiling
on my clunky 233MHz Alpha might save me some time.

Well, I was right on the first count :)

I started out intending to write up a cookbook-type document
for me and some colleagues that'd be helpful to anyone
working with USE_NEW_TOOLCHAIN.  That was a mistake--I should
have just logged every change I made as I made it.  Nevertheless,
having begun in a "do this first, do that next" format, I tried
to stick to it.  Since I've iterated over a few "make dependall"
runs, correcting whatever crops up, though, it's less of a
cookbook and more of a list of problems and how I fixed them.

By "decent progress," I mean I've got most userland binaries compiled.
For example, the Alpha builds a /bin/cat executable that runs on a
386 running NetBSD-1.5.1.  The execs that don't build yet are
hpftodit, most of the DHCP stuff, and a few RPC servers.  The
other "dependall" targets compiles.

I've not tried any kernel builds.  I'm just now starting on
wrestling with "make release".

I'm not totally sure that diffs to everything I've changed
would be helpful, since I doubt the changes I've made are
stylish enough.  However, I'll put one up tomorrow at

Lessee, what else?  I synced to -current as of Wed/Thu of last week.

Oh, and I hope I understand the ${MACHINE} vs. ${MACHINE_ARCH}
distinction properly. ${MACHINE} is the hosting architecture,
and ${MACHINE_ARCH} is the target architecture, right?  If I've
got that screwed up, well, this doc is just wrong.

If you Emacs, the doc's in outline format.

Chris <>.

* cd to the src dir

  % cd /export/netbsd-1.5/src
  % set p = $PWD


  __TOP			= /export/netbsd-1.5
  __SRC			= $(__TOP)/src
  __OBJ			= $(__TOP)/obj
  __TOOLARCH		= obsd-alpha
  __TARGARCH		= i386
  MKTOOLS		= yes
  MKOBJDIRS		= yes
  MKINFO		= no

* Build bmake

  % ( cd usr.bin/make ; make -f Makefile.boot MACHINE=alpha MACHINE_ARCH=alpha )

* Put bmake somewhere it can be found via PATH

  % cp usr.bin/make/bmake ~/bin

  Ideally, bmake would go in $(TOOLDIR)/bin

* Create obj dirs

  Even though Open and Net are similar, you can't just start on a build,
  because, eg, you need an "install" that groks "-r".

  % bmake -k -m $p/share/mk MAKECONF=$p/mk.conf do-make-obj \

* Add some macros that the toolchain .c files want

  Add these to tools/compat/compat_netbsd.h

	#ifndef __RCSID   
	#define __RCSID(x)

	#ifndef __COPYRIGHT   
	#define __COPYRIGHT(x)

* Fix mkdep

  mkdep won't use the C compiler from the new toolchain.
  Instead, it'll use the native compiler, which, of course,
  wrecks everything.

  First, fix the mkdep source to define the macro DEFAULT_CC only
  if it hasn't been already defined.

  In the file usr.sbin/mkdep.c, protect the #define of DEFAULT_CC
  by wrapping it in a #ifndef.  Change

	#define DEFAULT_CC		"cc"


	#ifndef DEFAULT_CC
	#define DEFAULT_CC		"cc"

  Next, change the file tools/mkdep/Makefile so it sets DEFAULT_CC
  by adding the line


  before the .include line.

* Make obj dirs for tools

  % ( cd tools ; bmake -m $p/share/mk 'BSDOBJDIR=$(TOOLDIR)' \
				      MAKECONF=$p/mk.conf obj )

* Fix usr.sbin/config/Makefile

  Protect .include of

	.ifndef HOSTPROG
	.include <>

* Fix files for gdb

  The files that come with NetBSD don't know
  about OpenBSD.  Fixing them consists of adding lines that
  look like

	alpha*-*-openbsd*)      gdb_host=obsd ;;

  to the case statements that set the gdb_host variable in the
  files gnu/dist/{toolchain,}/gdb/

* Add config files for OpenBSD/Alpha to tools/toolchain/gdb/config

  You'll need all the files from OpenBSD that match the glob *obsd*
  found in the directories /usr/src/gnu/usr.bin/binutils/gdb/config{,/alpha}.
  They should be copied to gnu/dist/toolchain/gdb/config{,/alpha}.

* Change tools/Makefile so it doesn't build texinfo

  texinfo won't build, since its configure writes to a file not in obj/.
  So, change the line

                toolchain .WAIT texinfo

  in tools/Makefile to		

                toolchain #.WAIT texinfo

* Fix tools/Makefile so it creates includes for gcc

  This one is ugly.  tools/Makefile changes the Makefile for gcc
  so that some targets like libgcc.a aren't made.  However, when
  those targets don't get made, then (some of) their dependencies
  don't get made either.  In particular, some of the include files
  gcc wants don't get made, and so not all of gcc is built.  The
  fix is ugly, and it might be overkill.  Nevertheless...

  For the target toolchain.configure in the file tools/Makefile,
  change the line that looks like

		Makefile.orig >Makefile


		Makefile.orig | \
		sed 's/^STMP_FIXPROTO =$$/STMP_FIXPROTO = stmp-int-hdrs/' \

* Make tools

  % ( cd tools ; bmake -m $p/share/mk MAKECONF=$p/mk.conf \
				      BSDOBJDIR='$(TOOLDIR)' build )

* Fix

  As it turns out, the install part of the gcc build isn't smart enough
  to figure out that it's building a cross-compiling version of cpp.
  As a consequence, you'll need to change the definition of CPP in
  tools/ from



	CPP=		${TOOLDIR}/bin/cpp

* mtree wants the user games to exist

  % sudo user add games

* make the distrib-dirs

  % su
  # bmake -m $PWD/share/mk MAKECONF=$PWD/mk.conf beforeinstall

* Fix some MACHINE vs. MACHINE_ARCH make variable problems

  MACHINE_ARCH is the architecture of the target machine.
  MACHINE is the arch of the building (aka host) machine.
  Some *.mk files use MACHINE instead of MACHINE_ARCH.  Fix them.

  In share/mk/, change the commands that update
  machine-links: target from

	    -rm -f machine && \
		ln -s $S/arch/${MACHINE}/include machine
	    -rm -f ${MACHINE_ARCH} && \
		ln -s $S/arch/${MACHINE_ARCH}/include ${MACHINE_ARCH}


	    -rm -f machine && \
		ln -s $S/arch/${MACHINE_ARCH}/include machine
    #	-rm -f ${MACHINE_ARCH} && \
    #	    ln -s $S/arch/${MACHINE_ARCH}/include ${MACHINE_ARCH}

  In the file share/mk/, change


  In the file sys/arch/Makefile, change all occurrences of

* Fix gnu/lib/libstdc++/config/Makefile to use MACHINE_ARCH

  In gnu/lib/libstdc++/config/Makefile, change

	INCSDIR=	/usr/include/${MACHINE}


	INCSDIR=	/usr/include/${MACHINE_ARCH}

* Make includes

  The includes target doesn't pick up, so you
  must set INSTALL to get the includes into $DESTDIR/usr/include,
  where the rest of the build system expects them to be.

  # bmake -k -m $PWD/share/mk MAKECONF=$PWD/mk.conf includes \
			      INSTALL='$(TOOLDIR)/bin/binstall' \
  # exit			 

  You need MKCRYPTO=no here.  Without it, you'll try to build
  compile_et, a binary which is needed to generate Kerberos .h
  files.  compile_et isn't part of the new toolchain, yet, so it
  bombs trying to link in setprogname().

  Ignore any errors about usr.bin/dbsym and in usr.sbin/pvctxctl,
  gnu/usr.bin/gcc/, and src/regress/lib/libc/arch/i386.

* Build the C startup libs

  % ( cd lib/csu ; bmake -m $PWD/share/mk MAKECONF=$PWD/mk.conf dependall )

* Add MKCRYPT=no to mk.conf

* Set some make variables in the environment

  % setenv MAKECONF $p/mk.conf
  % setenv MACHINE_ARCH i386

  I should have begun with these set, but better late than never.
  I'm not sure MACHINE_ARCH is necessary, but I (think I)
  encountered some problem that was fixed by setting it.

* Build the system libs

  % ( cd lib ; bmake -e -m $p/share/mk -k dependall MKSHARE=no )

  You need the -k so the missing include file <machine/ecoff_machdep.h>
  and a missing won't cause the depending to fail.

* Install the csu & system libs

  % su
  # set p = $PWD
  # cd lib && bmake -e -m $p/share/mk install
  # exit

* Build the GNU libs

  % ( cd gnu/lib ; bmake -e -m $p/share/mk dependall MKSHARE=no )

* Install the GNU libs

  # cd gnu/lib && bmake -e -m $p/share/mk install MKSHARE=no
  # exit

* Fix usr.bin/file/ so that mkmagic can built for the hosting machine

  Change the lines in usr.bin/file/apprentice.c that look like

	__RCSID("$NetBSD: ... ")


	#elsif defined(__NetBSD__)
	__RCSID("$NetBSD: ... ")

  Do the same for usr.bin/file/print.c.  Ideally, I suppose, the
  HOST_CC make variable could be set to drag in the compat parts
  of the tools/ directory.

  Next, change the fmttime() function in usr.bin/file/print.c
  so that it can be compiled & run on machines that don't
  support the USG/Unix-98/XSH5 daylight global variable.
  Change the line

		if (daylight)

		time_t tt = time(0);
		tm = localtime(&tt);
		if (tm->tm_isdst > 0)

  I'm not entirely sure this is equivalent to using the daylight global.
  If it is, though, I wonder if something like the following should go
  the tools compat library:

		#ifdef IDUNNOdaylight
		#define daylight __daylight__()
		int __daylight__(void)
			static int __daylight;
			static int initialized = 0;
			if (!initialized) {
			   time_t t = time(0);
			   tm = localtime(&t);
			   __daylight = tm->tm_isdst > 0;
			return __daylight;

* Fix usr.bin/kdump/Makefile.ioctl-c to use $(DESTDIR) includes

  By default, the list of ioctls that kdump knows how to decode
  is built using the shell script usr.bin/kdump/mkioctls.  mkioctls
  isn't being invoked correctly: it will use the native cc and it won't
  see DESTDIR.  So, change the first line of the commands updating
  ioctl.c in Makefile.ioctl-c from

      /bin/sh ${.CURDIR}/../kdump/mkioctls ${DESTDIR}/usr/include/sys/ioctl.h \
	    ${DESTDIR}/usr/include/sys/ioctl_compat.h > ioctl.c


      CC=$(CC) DESTDIR=$(DESTDIR) /bin/sh ${.CURDIR}/../kdump/mkioctls ${DESTDIR}/usr/include/sys/ioctl.h \

* Add HOST_CFLAGS to mk.conf to fix some incompatibilities w/ NetBSD

  Add the lines

      HOST_CFLAGS             += '-D__RCSID(x)='
      HOST_CFLAGS             += '-D__COPYRIGHT(x)='

  to mk.conf.  Ideally, we'd get these straight from the compat
  part of tools/, which could perhaps get put into

  These changes make one set of previous mods to src in usr.bin/file/
  unnecessary (though those mods don't hurt anything).

  Perhaps these mods should instead be made to HOST_CPPFLAGS.

* Fix more MACHINE vs. MACHINE_ARCH in sys/lkm/

  The references to MACHINE should really be MACHINE_ARCH.  Oddly,
  there are proper uses of MACHINE_ARCH scattered throughout the
  Makefiles in this hierarchy.

  The files that need fixing, named relative to sys/lkm/:


  Change all occurrences of {MACHINE} to {MACHINE_ARCH}.

* Fix some trickier MACHINE vs. MACHINE_ARCH in sys/lkm/

** sys/lkm compat/linux/ 

   This file contains a conditional that looks like it's trying to
   compensate for (historical?) problems when MACHINE_ARCH doesn't
   necessarily get set.  If sys/$(MACHINE)/$(MACHINE)/ exists,
   that's used to build assym.h.  Otherwise, it does the same check
   using $(MACHINE_ARCH).  This should change either to try $(MACHINE_ARCH)
   first, or to never try $(MACHINE).

   So, in the file sys/lkm/compat/linux/, change the lines

      .if exists($S/arch/${MACHINE}/${MACHINE}/
      .elif exists($S/arch/${MACHINE_ARCH}/${MACHINE_ARCH}/


      .if exists($S/arch/${MACHINE_ARCH}/${MACHINE_ARCH}/

   to ignore $(MACHINE) altogether.

** sys/lkm/exec/linux_elf/Makefile

   This file unconditionally sets -DEXEC_ELF64.  It seems to me
   that it should be changed to only set that for 64bit architectures.
   So, two changes:




   and change

	SRCS+=  linux_exec_elf64.c

	SRCS+=  linux_exec_elf64.c

   I haven't tried the build without this change to see if it's necessary,
   so somebody who knows the code should comment.  Also, it may be that
   -DEXEC_ELF32 should be defined *only* for 32-bit archs, I dunno.

* Delete reference to dbsym

  The dbsym executable doesn't seem to exist any longer.
  However, usr.sbin/Makefile thinks that it does.  Convince
  it otherwise by removing its use of the word "dbsym".

* Build it all

  % bmake -k -e -m $p/share/mk dependall

* Some builds fail

** The Don't Cares

*** Man pages are made with native nroff

    nroff should get into the tools dir so that man pages
    are built properly.  Some macros don't necessarily work
    on non-NetBSD systems.  Here's an example from make's
    man page:

        .SUFFIXES   Each source specifies a suffix to .

    Maybe using a shell script that points the native
    nroff at the NetBSD macros would work.

*** DHCP won't compile

    usr.sbin/dhcp/ sets DIST to point to dist/dhcp, which hoses
    redefinitions in some include files in usr.sbin/dhcp, eg, in the file

   The problem is with DIST set, includes first from $(DIST) and the
   .h files in the usr.sbin/dhcp hierarchy are skipped.

*** rpc.* servers from OpenBSD won't compile

    rpc.bootparamd, rpc.statd

    The problem here is that warnings are being treated as errors
    and __RCSID() magic isn't being used to squelch gcc's complaint
    about an unused "rcsid" variable.

*** usr.bin/groff/hpftodit/../../../dist/groff/src/utils/htpftodit/

    Hm.  The assembler doesn't like some comparisons generated from this file:

	 Error: Value of 4294967296 too large for field of 4 bytes at 1724

** The Do Cares

*** gnu/usr.bin/gcc won't compile

**** gnu/usr.bin/gcc/backend/Makefile needs help

     There's a bootstrapping problem in this Makefile.  It tries
     build .depend, but it can't do that until it first generates
     some .h and .c files from .md files.  The fix is to add the
     generated files to the DPSRCS variable.

     However, there's also trouble with building the native object
     files used by the programs that generate the .[ch] files.
     The fix I've got for this problem is hairy, so I won't
     go into it any great detail.  Basically, I've never had
     make let me down whenever every last underlying dependency
     is expressed in some rule.  So, the fix changes the .for loop
     to say that each gen* program depends on the .lo files, and
     each file built from an .md file depends on the gen* prog.

     See the patch in the Patches section, below.

*** gnu/usr.bin/gcc/f771/Makefile needs help

    The problem here is similar to the previous one.  This time,
    the fix is structured a bit better.  See the patch in the Patches

** The Might Cares

*** Assembly warnings

    I have not idea whether these warnings matter.

**** libc: indirect jmp without `*'

   Building libc generates warnings from the assembler like this

	{standard input}:950: Warning: indirect jmp without `*'
    in the following files


*** libc: using %pishfoo instead of %posh due to `b' suffix

    Assembling lib/libc/arch/i386/string/memcmp.S generates
    warnings like these:

	     Warning: using `%al' instead of `%eax' due to `b' suffix
	     Warning: using `%dl' instead of `%edx' due to `b' suffix

* Install it all

  I haven't made it this far.

  % su
  # bmake -e -m $p/share/mk _BUILD= install

* Patches

** gnu/usr.bin/gcc/backend/Makefile

Index: Makefile
RCS file: /cvsroot/gnusrc/gnu/usr.bin/gcc/backend/Makefile,v
retrieving revision 1.1
diff -u -r1.1 Makefile
--- Makefile	2001/06/18 16:18:34	1.1
+++ Makefile	2001/09/10 16:50:07
@@ -12,12 +12,16 @@
-.include <>
 # Independent generation programs.
+CLEANFILES+=	gengenrtl gencheck
+# Force these sources to be built b/4 .depend
+DPSRCS+=	genrtl.c genrtl.h tree-check.h
+DPSRCS+=	insn-attr.h insn-codes.h insn-config.h insn-flags.h tree-check.h
-CLEANFILES+=	gengenrtl genrtl.c genrtl.h \
-		gencheck tree-check.h
+.include <>
 genrtl.c genrtl.h: gengenrtl.c
 	${HOST_LINK.c} -o gengenrtl $>
@@ -26,32 +30,36 @@
 tree-check.h: gencheck.c
 	${HOST_LINK.c} -o gencheck $>
 	./gencheck >$@
-# Programs which depend on common sources.  The ${MAKE} dance
-# ensures that these programs regen their object files
-# whenever they are needed (so a build can be restarted on
-# another platform if desired).
+# The host progs use these libs
 LOFILES=	obstack.lo ${G_HOST_RTL:.o=.lo} ${G_HOST_PRINT:.o=.lo}
-lofiles: ${LOFILES}
-.if !target(.BEGIN) && !make(lofiles)
-	@rm -f ${LOFILES}
+# Progs which need to link w/ ${G_HOST_RTLANAL}
+NEEDRTLANAL=	genattr genattrtab
 .for f in attr.h attrtab.c codes.h config.h emit.c extract.c \
 	flags.h opinit.c peep.c output.c recog.c
-CLEANFILES+=	gen${f:R} insn-${f}
-insn-${f}: genrtl.h gen${f:R}.c
-	@cd ${.CURDIR} && ${MAKE} lofiles
-	${HOST_LINK.c} -o gen${f:R} ${>:M*.c} ${LOFILES}
+GENPROG = gen${f:R}
+CLEANFILES+=	${GENPROG} insn-${f}
+insn-${f}: ${GENPROG} ${G_md_file}
 	./gen${f:R} ${G_md_file} >$@
+	${HOST_LINK.c} -o $@ $>
+${GENPROG}.o: gen${f:R}.c rtl.h genrtl.h
+genextract.o genattrtab.o: insn-config.h
-insn-attrtab.c: ${G_HOST_RTLANAL:.o=.c}
-insn-extract.c: insn-config.h
 ${OBJS}: insn-attr.h insn-codes.h insn-config.h insn-flags.h tree-check.h
 .PATH: ${DIST}/gcc ${DIST}/libiberty ${G_out_file:H}

** gnu/usr.bin/gcc/f771/Makefile

Index: Makefile
RCS file: /cvsroot/gnusrc/gnu/usr.bin/gcc/f771/Makefile,v
retrieving revision 1.1
diff -u -r1.1 Makefile
--- Makefile	2001/06/18 16:18:35	1.1
+++ Makefile	2001/09/10 17:25:40
@@ -7,27 +7,21 @@
-.include "../Makefile.backend"
-# See comment in ../backend/Makefile for the reason for
-# this ${MAKE} dance.
-fini: fini.c proj.c
-	${HOST_LINK.c} -o $@ $>
-.if !target(.BEGIN) && !make(fini)
-	@rm -f fini
+GENFILES=	1t 2t fo io nq op ot
+.for f in $(GENFILES)
+DPSRCS+=	str-${f}.h str-${f}.j
-.for f in 1t 2t fo io nq op ot
-CLEANFILES+=	str-${f}.h str-${f}.j
+.include "../Makefile.backend"
-str-${f}.h str-${f}.j: str-${f}.fin fini.c proj.c
-	@cd ${.CURDIR} && ${MAKE} fini
+.for f in $(GENFILES)
+str-${f}.h str-${f}.j: str-${f}.fin fini
 	./fini ${>:M*.fin} str-${f}.j str-${f}.h
 ${OBJS}: str-${f}.h
+fini: fini.c proj.c
+	${HOST_LINK.c} -o $@ $>
 .PATH: ${DIST}/gcc/f