Subject: experiences with NetBSD Live!
To: None <netbsd-users@netbsd.org>
From: Steve Bellovin <smb@research.att.com>
List: netbsd-users
Date: 07/11/2003 20:27:13
Per some previous postings, I was trying to build a bootable NetBSD CD. 
I was directed to the NetBSD Live! package.  After some work, I got 
things working.  I figured I'd post my experiences, for the benefit of 
everyone else.

First -- let me thank Joerg; without his work, I'd have had a *much* 
harder time getting anything working.  That said, I did have some 
trouble.

The biggest problem was with getting the boot "floppies" built.  The 
current 1.6 version has some interaction with (or dependency on) the 
tool chain; the impression I have is that it doesn't fully understand 
the new toolchain.  Somehow, I made it work once, but I don't know how; 
I was not able to make it work again.  (I was able to fix the toolchain 
build by changing the 'make' invocation to 'make install'.  But I 
couldn't get the floppies to build reliably.)

Fortunately, Jeremy Reed sent me the following simple script:

    dd if=/dev/zero of=myboot.fs count=5760
    vnconfig -t floppy288 -v -c /dev/vnd0d myboot.fs || exit
    disklabel -rw /dev/vnd0d floppy288 || echo disklabel problem $?
    newfs -B le -m 0 -o space -i 204800 -c 80 /dev/rvnd0a floppy288
    $DIRTREE/usr/mdec/installboot -v -f $DIRTREE/usr/mdec/biosboot.sym /dev/rvnd0a
    mount /dev/vnd0a $MOUNT
    gzip -v -c -9 $KERNEL > $MOUNT/netbsd || echo problem with gzip $?
    chmod a+x $MOUNT/netbsd
    df -k $MOUNT
    ls -la $MOUNT
    umount $MOUNT
    vnconfig -u vnd0d

That let me use any kernel I wanted with the CD, so long as it had

	config root on cd0a

in its config file.  (In fact, how to do that was my original question, 
but if I'd received that answer I wouldn't have learned about NetBSD Live!,
which I'd have regretted.)

Joerg's distribution does an excellent job of stripping out things that 
aren't needed on such a CD, a much better job than I had done.  Tarring 
and gzipping /var/db/pkg is a clever idea, though I'm not convinced it 
saves enough space to be worthwhile, especially given the comparatively 
small set of packages one can fit on a NetBSD Live! CD.  Similarly, the 
script generates a stripped, compressed /netbsd file; I undid that, 
because I want to be able to run things like 'netstat'.

I did use the script -- several times! -- to start building the 
distribution.  Eventually, I stopped using it and started changing 
things on my own, usually with the aid of chroot.  Running it from the 
beginning each time was too expensive, especially installing the 
packages I needed.  (I had my own specific package set that had to be 
on the installation.)  I think that the build process would have been 
simpler to use, simpler to customize, and simpler to debug if it were 
composed of several individual scripts, all invoked initially by a 
wrapper script.

For reasons I don't understand, several network services are on by 
default.  I'm not sure why remote login to a NetBSD Live! machine is 
useful; in any event, I disabled them.  The CD also tries to 
enable samba; since I didn't install that package, it's harmless for me.

A few invocations of 'tar' seemed to be missing -p, which caused 
erroneous permissions.  I had to fix that it one crucial instance -- 
the unpacking of mfs.tgz at boot-time -- but I think there are other 
problems of that nature.  In particular, I noticed that none of the 
executables were setuid or setgid.  Of course, that is probably my 
fault; as I said, I did a lot manually.  I fixed those problems on my 
CD.

The invocation in the boot-time script of xdm (and I think kdm and gdm, 
though I haven't tried those) seems wrong -- it specifies -nodaemon, 
which means that rc doesn't finish, which means that the alternate 
consoles don't seem to be enabled.  I booted the CD on two different 
machines; X worked on one, though not on the other.  I think the 
problem was the flat panel monitor; it just displayed "out of range".  
I haven't yet investigated how to fix that.  I couldn't try the 
alternate XFree86 3 servers, since I don't have any of those lying 
around.

I do need to experiment more with putting defaults in /cdconf, to 
minimize the number of interactions at boot time.  I didn't see ways to 
eliminate some of the interactions; I may try to fix that.

I did have to change the default prompt, since ksh -- my preferred 
shell -- doesn't handle the color-change escapes the way bash does.  
/usr/pkg/bin/bash was the shell for root, which was a problem for me 
since I didn't install that package.  (On the other hand, I also didn't 
use the provided scripts that analyze your package selection; I don't 
know if it would have warned me about bash (which I could tolerate) or 
if it would have insisted on samba, which I don't want.)

The notion of reading override defaults from a floppy is a clever one; 
unfortunately, it didn't work for me.  I haven't bothered to figure out 
why, though I should investigate that -- just running mkisofs is a very 
time-consuming part of the build process.  The warning in the 
documentation about how many test CDs you'll have to build is quite 
valid, though I was lucky --  CD-RWs worked for me, even though the 
documentation says that they aren't bootable.  

mfs.tgz is another clever idea, though an annoying one; often, I'd 
change something in /etc, but I'd forget to rebuild the compressed 
version.  I assume that the compressed version's purpose is to make it 
possible to have mfs-mounted versions of /etc, /dev, /var, and /home; I 
wonder if there's an easier way to do it, such as copying them at boot 
time to a temporary mfs file system, mounting the mfs version over the 
real directory, copying, and then discarding the temporary mfs version.
That would also save space on the CD.  Or perhaps more could be done 
with overlay mounts.  I don't know; I haven't tried this yet.  I did 
have to create another mfs file system -- amd doesn't work if /amd 
isn't writable. That one I had to figure out the hard way...

To sum up -- if you want a bootable NetBSD CD, I suggest starting with 
NetBSD Live!.  I hope that my notes help with customization or with 
building your own CDs.

		--Steve Bellovin, http://www.research.att.com/~smb (me)
		http://www.wilyhacker.com (2nd edition of "Firewalls" book)