--sU4rRG038CsJurvk
Content-Type: multipart/mixed; boundary="d6iqOn7HZPWKXx18"
Content-Disposition: inline


--d6iqOn7HZPWKXx18
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Oct 23, 2001 at 10:30:29PM -0400, josh@ssimr.com wrote:
> I've asked about kernel builds from time to time and received some
> helpful advice.  I am going to be doing some kernel building and was
> wondering if there is a step by step guide out on somebody's web
> page. I'm both worried and interested in what features can be left out
> and supporting non-standard hardware.

I've got about fifteen NetBSD machines of various architectures for
a reasearch project at Swarthmore College.

Attached is the document I emailed the other people working on the
project including how to safely build a kernel and a full system
within our environment. (I'm sort of presupposing that you'll want
to do the latter having done the former.) It's got definite macppc
leanings, though (the subset of machines we were playing with at the
time were all macppcs).

Hope this helps...

--=20
       ~ g r @ eclipsed.net

--d6iqOn7HZPWKXx18
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="NetBSD-build.txt"
Content-Transfer-Encoding: quoted-printable

## In order to clean up the source on cygnus, I did:
rm -rf /usr/src
ftp ftp.netbsd.org
# get pub/NetBSD/NetBSD-release-1-5/tar_files/src/*.tar.gz
cd /usr
for i in src/*.tar.gz ; do echo $i ; tar xpzf $i ; done
# verify that that did the right thing
rm src/*.tar.gz	# we're done with them, and we want the space they're in

## In the future, the following should be sufficient (but falling
## back the the above will always work should things go wrong,
## presuming ftp.netbsd.org doesn't break):
cd /usr/src
make cleandir	# maybe overkill, but it's safer this way
cvs -d anoncvs@anoncvs.netbsd.org:/cvsroot update -r netbsd-1-5

## Especially if you're about to build a brand-new version of
## NetBSD, it's a good idea to build and install the new kernel
## (which will always be compatible with older userland; the
## contrary is NOT necessarily true) and boot the machine with it to
## make sure there are no basic problems there.
##
## To build just a kernel, you would do this with your newly-updated
## source tree:
cd /usr/src/sys/arch/macppc/conf
## Obviously this is /usr/src/sys/arch/<something else>/conf if
## you're building for some other architecture. We also have sparc
## and mac68k in the cluster at the time of writing; we're likely to
## have i386 and maybe sparc64 (UltraSparc as opposed to regular Sparc)
## in the forseeable future.
cp GENERIC NAME_OF_CHOICE
## NAME_OF_CHOICE, for kernels that will be used all over the
## cluster, has been SCINTILA for me. It is frequently the hostname
## of the machine for which the kernel was optimized (thus ELEANOR
## below, the name of my machine for Wasabi). It's pretty immaterial
## what you choose, short of the fact that it'll show up in uname -v
## output, so probably best to avoid vulgarities or deep, dark
## secrets. ;^>
${EDITOR} NAME_OF_CHOICE
## The GENERIC kernel configuration (the one built and installed by
## default on a new system) should run pretty well most of the time
## on most machines (laptops are the only major exception to this
## rule), but it's got a fairly large memory footprint since it
## includes a lot of stuff that you won't need on any given machine.
##=20
## For instance, the macppc GENERIC kernel detects three four
## different kinds of PCI busses:
##=20
## pci*  at bandit? bus ?
## pci*  at grackle? bus ?
## pci*  at uninorth? bus ?
## pci*  at ppb? bus ?
##=20
## Most of the machines we have in the cluster have a bandit PCI
## controller, a couple may have a grackle (I'm not even sure), and
## nothing but G3s or above have a uninorth. So far as I know, no
## Apple hardware shipped standard with a PCI controller on the
## other side of a ppb (a PCI to PCI bridge), which would be, say,
## an external case to hold a bunch of extra PCI cards (I believe
## Adaptec still makes some). It's plausible that PowerComputing
## macs shipped with these, but I'm not sure.
##=20
## The point is, look at what was detected in the kernel that was
## already booted on the machine (as I said, GENERIC if this was a
## fresh install) by using dmesg(8), and lose the stuff you don't
## need in the kernel by commenting it with a # character.
##=20
## It's probably also worth discussing the syntax of this file,
## although I find it's fairly self-evident after you read over it a
## bit.
##=20
## Lines that begin with "options" enable (or, if commented
## out, disable) specific additions to the kernel. This ranges from
## with which other operating systems--or older versions of
## NetBSD--you want to maintain binary compatibility to which
## subsystems should have descriptive tables of strings to spit out
## should errors occur compiled into the kernel.
##=20
## There is precisely one line which bgins with "config", which
## states where the root partition lives and what file system type
## it lives on. This is only worth pinning down specifically if
## you're building a kernel for exactly one machine and you may be
## adding SCSI devices at weird places, and will pin down the SCSI
## device you want to boot from to exactly the right place later on
## in the config file. Trust me, if you actually need to mess with
## this, you'll know it. But you probably don't need to mess with
## it.
##=20
## Then there are a whole bunch of device detection lines,
## containing two kinds of wild cards (? is filled in with exactly
## one other device, * is filled in with a number starting from 0
## and incrementing potentially infinitely). These look a whole lot
## like the output of dmesg(8), which makes sense since they're
## compiled (as strings) into the kernel, which then uses them in
## dmesg output. Unless you're pinning a specific device down at a
## specific place, you pretty much want to leave these alone or
## comment them out (if you're not using them). Particularly ripe
## for deletion are ethernet card chip sets that will never come
## anywhere near a given machine and USB devices on a machines that
## do not nor will ever have USB. Commenting these things out
## doesn't just make the kernel smaller post-compilation (which is
## good because it'll wire less RAM when it boots, not just because
## it'll take up less disk space, which is only a minimal
## difference), but also because compiling stuff you'll never use
## really wastes compile time.
##=20
## Now you're ready to actually begin compiling the kernel. In order
## to do this, you need a set of Makefiles with specifically the
## options you've selected in the kernel configuration you just
## edited. To get that, you run:
config NAME_OF_CHOICE
## Should there be any problems in your kernel config, they will
## *usually* be reported by config (though it doesn't catch
## everything, you have to do some pretty weird stuff to slip
## something by it). Should everything be okay, it will have created
## a directory at /usr/src/sys/arch/macppc/compile/NAME_OF_CHOICE.
## The output you want from config is a single line saying "Don't
## forget to make depend." Sage advice, for now we:
cd ../compile/NAME_OF_CHOICE
make depend && make
## This is important. You do NOT want to make (all) until you've
## made the depends. Things will break all over the place, and make
## it rather hard to clean things up (objects will have been
## generated inappropriately; the only really good way to clean up
## is to cd /usr/src && make cleandir, which will wipe out
## everything that's been built under /usr/src, which can slow
## things immensely down if you're compiling minor changes in a
## fairly static source tree).
##=20
## This will scroll a whole lot of build messages past the screen.
## In general, if something goes wrong here, it's a huge deal and
## time to use send-pr(1), so I don't usually bother redirecting it
## to an output file (as I do below for the full system build, which
## tend to be a bit more fragile as they're trying to do way more
## stuff, and problems are often the builder's fault, not the source
## tree's), though doing so might cause you less of a headache if
## something does go wrong.
##=20
## After the kernel has successfully been built (the last line of
## the build output will be a listing of the size of the new kernel
## just built), you'll want to reboot using it. To do that, do:
mv /netbsd /netbsd.old
## I usually actually preserve old kernels at, for instance,
## /netbsd-1.5-ELEANOR. If you still want whatever you've got at
## netbsd.old, obviously, don't overwrite it.
cp netbsd /
## Could mv here too, but it's nice to have a backup of what you
## just built too just in case things go catastrophically wrong.
shutdown -r now
## Hopefully, all will be well, and you'll come back up with a
## working system running your new kernel (check uname -v to be
## sure, but it should show up above the login prompt on the console
## too, though that's a bit less convenient if you're working
## remotely). If problems do occur, you'll need to get on the
## console, boot the machine and type something besides Enter during
## the 5 (default) second count down before the kernel boots. Then
## tell it to "boot netbsd.old" (or similar; boot -s is the way to get
## single user on NetBSD, btw, which is maybe generally valuable
## information) at the bootloader prompt. If this works, swell, but
## I'm not sure how well the bootloader knows how to deal with
## OpenFirmware-initialized SCSI (or IDE) disks just yet, so it may
## not. If so, just boot off a floppy from OpenFirmware (that's a
## whole other chunk of documentation either Rafa or I should
## write), mv your new kernel off somewhere to preserve it for later
## inspection, mv netbsd.old back to netbsd, and reboot again. This
## is why you saved the old kernel, obviously.

## Now then, presuming all of that worked, to build a whole NetBSD
## snapshot:
cd /usr/src
OUTPUT=3Dbuild-output${DATE}00
DESTDIR=3D/build RELEASEDIR=3D/release/${DATE} EXTRA_KERNELS=3DELEANOR make=
 release \
  > ${OUTPUT} 2>&1 & ; tail -f ${OUTPUT}
## This says build a full NetBSD installation at /build (${DESTDIR}),
## then pull all of that plus the standard install support stuff for
## the platform you're on together, and dump that in /release/${DATE}
## (${RELEASEDIR}).
##=20
## Note that ${DATE} is NOT defined usually... I always have it set to `date
## +%Y%m%d`. You may want to to. ${RELEASEDIR} will be completely
## cleaned out when you do a make release, and a given release isn't
## all that big on disk, so doing this is preferable to just using
## /release as the value for ${RELEASEDIR}. So, even if you're not
## going to follow my ${DATE} routine, at least do something to
## differentiate the builds. We can always blow away old ones if we run
## low on space.
##
## EXTRA_KERNELS contains a list (single quote them if you're listing
## more than one) to be included in the set. It's actually kind of
## silly of me to include the kernel specific to my machine for Wasabi
## here, (actually, I'd already built the kernel on the machine
## itself, so it's totally irrelevant), but the example's here to
## show how it's done. Actually, the only thing that cares about
## this environment variable is /usr/src/etc/Makefile, but the
## environment gets handed off to it by doing a make release, and
## it doesn't hurt anything to set this now.
##
## Also, MKOBJDIRS affects the build process (read the Makefile to
## see how). You want it set, which it is by default and by conscious
## decision in /etc/mk.conf on cygnus. Don't worry about it, but
## it's worth knowing it exists.
##=20
## If something goes wrong in the build, restart it as above, but
## with UPDATE=3D1 as well. (This will save the cleandir that is
## automatically done at the beginning of each build.) You'll
## probably want to change the 00 on the end of ${OUTPUT} (which
## isn't used by the NetBSD makefiles at all, btw, just by that command
## line, which is why it's not exported and not defined on the same
## command line... the Makefiles may actually use a value called
## ${OUTPUT} and mean something very different by it, so I avoid
## stomping on that), for subsequent builds. Incidentally, I use
## less ${OUTPUT} and just hit ^F in less, as I like being able to
## scroll back to check on things without having to ^C out of tail
## and start up less.
##
## Also, it's plausible that something minor could break which is
## repairable by cd'ing into the source directory that caused the
## breakage, examining what went wrong, and fixing it. There's nothing
## wrong with starting all over again from a practicality standpoint
## (assuming you *have* fixed the problem; otherwise it'll just happen
## again), but it can save some time to do the make in a subdir by
## hand, then back up directories making by hand, especially when you
## get close to the end of the release building (making the tarballs,
## whatever).

## Now then, we also need the X source built because we like being
## able to X forward from machines in the cluster. This step needs
## to be taken *after* having already built the regular source into
## ${DESTDIR}, and preferably after having already done a make
## release in /usr/src. You should, of course, use the same values
## for ${DESTDIR} and ${RELEASEDIR} that you did when building the
## standard system source.
##=20
## (X is divided so strictly from standard source both because there
## are many machines--especially ones which run NetBSD, like embedded=20
## systems--which never need X at all and also for licensing
## reasons.)
cd /usr/xsrc
make cleandir
cvs update -r netbsd-1-5 -dP
OUTPUT=3Dbuild-output${DATE}00
DESTDIR=3D/build RELEASEDIR=3D/release/${DATE} make release \
  > ${OUTPUT} 2>&1 & ; tail -f ${OUTPUT}

## After all of this, we should have a full NetBSD distribution
## snapshot in /release/${DATE}, which consists of a few checksum files
## and .../binary and .../installation directories. It's worth
## explaining how to install from one of these on all of the cluster,
## but that's maybe the subject of a separate piece of internal
## documentation.
##=20
## Be aware that you do NOT have to do all of this if all you're
## changing is the kernel on a system whose userland already matches
## that basic kernel version (that is, "1.5_BETA2" or whatever).
## There's no need to recompile all of userland if you're just
## hacking on kernel source a little bit (well, presuming you
## haven't changed any offsets in the kernel that things like top
## and ps depend on, but you probably don't want to go doing that so
## much anyway, and they probably don't use literal memory addresses
## anyhow) or changing options in a kernel config, but only when you
## are actively upgrading the whole system. if you're doing one of
## those first two things, you've read about one hundred lines too
## far. ;^>

--d6iqOn7HZPWKXx18--

--sU4rRG038CsJurvk
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (NetBSD)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjvWjeMACgkQ9ehacAz5CRovygCfWkn6Xu30I45k4pHCyQ/rYThk
AF0AoIWmyAP3DC0vdBHph8bYfuR9M+PK
=NVu4
-----END PGP SIGNATURE-----

--sU4rRG038CsJurvk--