At Fri, 15 May 2020 20:18:28 -0400, "Aaron B." <aaron%zadzmo.org@localhost> wrote: Subject: Re: NetBSD Jails > > - Processes can "see" each other; I have to be careful not to reuse > UID numbers. For example: if I build a chroot with an instance of nginx > that runs as UID 2505, and then deploy multiple copies of that chroot, > all of them can call kill(2) on a process in a different chroot. Perhaps all that's required is a tool which extracts the minimum required entries from the real /etc/master.passwd for each chroot? (and some way to maintain chroot copies?) (Another way would be a new service behind nsdispatch(3) which would provide access through the chroot, e.g. via a socket, to the shared /etc/master.passwd, though that would assume all chrooted programs use only the "standard" interfaces supported by nsdispatch(3).) > - All chroots share the same network stack. If I tell nginx to bind to > '0.0.0.0' or '::', the first instance will startup fine, the others > will fail with "address already in use." Well if you're chrooting multiple instances of the same service, isn't it obvious that each has to listen on one and only one specific address? If I understand correctly one could also route a subnet via a bridge interface to each chrooted service. Maybe a chrooted process should also be prevented from listening to a wildcard address? I've heard FreeBSD folks go on for days about how FreeBSD's "jails" make network management simpler, but I still don't have any real understanding of exactly what this means. The only thing that seems to be of interest is in allowing a chrooted "root" user(*) (e.g. to allow someone inside the jail to muck around with interface addresses, routes, etc.), but I would suggest that allowing root inside a chroot is a very very bad idea no matter what "jails" features you might think will protect you (i.e. I would never trust a chrooted root user, "jails" or no jails). (*) "chrooted root user" -- i.e. the "root" user in the jail can only do things (as superuser ID#0) to those resources, e.g. interfaces, routes, etc. that are delegated to the jail. > The wiki's projects list has a > clean solution to this particular point, which may or may not be within > scope of jails: > > https://wiki.netbsd.org/projects/project/virtual_network_stacks/ Virtual network stacks seem to be a rather complex solution looking for a problem -- i.e. in actual fact more of a problem looking for trouble. > - Some way to set per-chroot resource limits would be helpful. I can > manipulate ulimits, but that is basically driving screws with a hammer. > It's simply the wrong tool. Well here's where /etc/login.conf solves most problems for normal chroot environments, since only ordinary users should be running inside the chroot. (Or it could, if there were resource controls related to networking. :-)) For anything beyond that I'm pretty certain that a full virtual machine is the real solution. Personally I think full VMs are the K.I.S.S. of death for "containers" (especially once you see the nightmare of complexity that underlies them in the kernel, i.e. CGroups). I would much rather have the clearly structured overhead of the VM instead of the hidden overhead and excessive complexity of "containers", or even just some of the bits of them like virtual network stacks. All this said though I would note that perhaps re-engineering the whole network stack in the netgraph way (perhaps directly using netgraph[1]), provides some form of "virtualization" for network things in a clean and structured way. [1] https://en.wikipedia.org/wiki/Netgraph One thing I would like to have for making VMs easier to manage though is a filesystem that can be shared read-only as a block device -- i.e. through xbdback(4) to xbd(4), or the equivalent for other kinds of VMs. (I say this because I abhor the use of NFS to share to VM domUs.) -- Greg A. Woods <gwoods%acm.org@localhost> Kelowna, BC +1 250 762-7675 RoboHack <woods%robohack.ca@localhost> Planix, Inc. <woods%planix.com@localhost> Avoncote Farms <woods%avoncote.ca@localhost>
Attachment:
pgpiJWgVSMtjn.pgp
Description: OpenPGP Digital Signature