I've offered this on the embsd list and the soekris-tech list
before.


Quoting Peter Seebach (seebs@plethora.net):
> I am toying with my old Compaq network appliance again, and I'm thinking of
> trying to build a "complete" installation which lives on a single CF card.
> How important is it that /usr be a separate filesystem?  I'm a bit strapped
> for space (I want X installed locally).  I can always give up and go back
> to a netboot kernel, but I think it'd be awfully neat if I could make
> everything run locally.

I'm a very strong advocate of making /usr separate because I
mount it read-only.  In fact, except for root, if it's got
a binary on it, it's RO.  If it's got data, it's mounted
noexec, nosuid, nodev.  Several reasons.  And I've built machines
where the binaries are on disks PINNED read-only (trojan that!).

But we're talking Compact Flash.

My soekris boxes have 1 large partition.  Mounted read-only.
I have a /mfs partition that's mounted, er, mfs.  It gets filled,
on boot, with an rsync from /mfs.PROTO/.

Then I have symlinks.
  /var/run  -> /mfs/var/run
  /tmp      -> /mfs/tmp
  /var/tmp  -> /mfs/var/tmp
  /etc/resolv.conf -> /mfs/etc/resolve.conf  (one is a DHCP client)

and so on.
  
It runs a DHCP server, wireless, IPSec endpoint, and a (secondary
and caching) BIND 9 server.

Tricks to readonly root include that terminals (/dev/ptyX and such)
need to be chown-able.  devfs would be ideal, hacks to libutil let
me move those devices to /dev/term/  (which is a tiny MFS populated
on boot with a MAKDEV script).


Syslogs go elsewhere or can also go to /mfs/log/ where newsyslog
rotates them at 1MB and keeps 2 generations.


It's kinda useful to use HALF the CF for your stuff.  Trick I learned
was that updates go into partition #2 (well, "d").  Boot into that,
if it works, fine move it info A.  If it fails, a simple reboot lets
you recover.

Friend is running FreeBSD at 8MB (crunchgen, special purpose box), I've
got a 256MB on my first CF (it was on sale, I needed a robust machine);
I've got a 16MB on others I've built.