Thinking about "branes" for netbsd...

[Darren Reed <darrenr%netbsd.org@localhost>]

After spending some time thinking about what would be required
to implement branes as part of the SMP networking project,
I've put together the notes below. There are some aspects of
this where I'm not sure yet what the bike shed should look
like (e.g. managing branes) so if you would like to argue
about its colour, please construct it first.

At this point I don't know if it will turn into a formal project
proposal but I thought it would be wise to seek feedback from
others anyway to see if there are aspects of this problem that
I've missed or over designed.

Darren

Assumptions
===========
In the one line brief for branes, there's no stated requirements
about attributes such as their longevity. For instance, should a
brane exist for the duration of a process or longer? This proposal
assumes the former - that a brane is to be created without there
necessarily being any process that is assigned to it.

Phase 1 (Implement default brane)
=================================
* Analyse NetBSD networking stack to determine what data
 structures can or cannot be removed from global context
 - Include in this analysis what the required scope is for
   those that are currently global

* Design and implement structures that describe the external
 management interfaces for a network family/protocol.
e.g. create, start, shutdown, destroy
 - create may fail
 - start never fails
 - shutdown/destroy must never fail although cleanup and
   action may be delayed if resources are busy
 - required for each of inet, inet6, tcp, udp, icmp, etc,
   separately.
 - support an external source registering its own callbacks
   for execution..
   + after create, after start, before shutdown, before
     destroy, of the builtins

* Convert the use of global variables in NetBSD's networking
 to elements of a local structure that is allocated per
 protocol family and network protocol
 - use an array of network families for initialisation at
   boot/shutdown
 - struct protosw is no longer global
 - network protocols add themselves to the protosw as part
   of initialisation (network families have an array of network
   protocols that they initialise)


Phase 2 (Support multiple branes)
=================================
* Design interface to support managing multiple branes
 - network interfaces can only belong to one brane
 - will require the use of virtual interfaces to share
   one physical interface between multiple branes
 - a network interface name must be unique across all branes

* pfil updates
 - each brane will have its own pfil_head
 - it should be possible for a subscriber to either receive
   callbacks for all branes or just one

* Design and implement kernel interfaces to allow a process
 to determine which brane it is in
* Design and implement system call interface to support this
 management from userspace
 - allow the brane that a process sees to be changed from
   the primary brane (that created when the kernel boots)
   to a secondary brane if the correct privilege is held
 - do not allow processes to change their brane without
   privilege or if it is not the primary brane

* New tool to manage branes is required that does the following:
 - create brane
 - delete brane
 - list branes
 - assign a network interface to a brane
 - remove a network interface from a brane
 - this should potentially be a generic tool, such as nicctl,
   or branectl or ... that is dedicated to managing network
   interfaces and not IP, etc.
 - ifconfig is not an appropriate tool for managing branes
   or which brane an interface belongs to because it cannot
   see all branes. This means that "ifconfig -a" will not
   report all interfaces when branes are in use.

* Implement chroot(3) on top of a new system call that allows
 for either filesystem or networking "root" (or both) to be
 changed
 - modify chroot(8) to use new system call and allow for
   setting the brane in addition to or instead of the new
   root.
 - Comment: whilst it may be desirable to only change the
   brane, the most common use will be in addition to the
   filesystem root and rather than worry about permissions
   and ordering of these operations it would seem better to
   support changing both of them together.
 - build the system call using propvals to convey data rather
   than a special structure.
 - of course there are compatibility concerns here that will
   be taken care of in due course such as ensuring that old
   binaries with syscalls continue to work.

* For a brane to have its own loopback interface, it must be
 named uniquely, thus no brane could ever have lo0 ecxcept
 the default. To allow each brane to have its own lo0 requires
 further work - interface name aliasing.

Interface naming
================
For each brane to have a lo0 accessible within it requires that
each brane have its own network interface name space. This is
almost a separate project in its own right.

The first approach is to describe this name space as a logical
name space that sits above the network device name space. To
implement this requires that each brane would have its own
mapping table of network device names. This would be used by
the system to translate network interface names in ioctls that
work with "struct ifnet". This creates other problems as other
code inside the kernel would need to be aware of this mapping -
such as code that calls ifunit() - or the mapping is enforced
elsewhere as well.

A second alternative to this is to implement a logical network
interface abstraction that sits on top of the device name space.
That is, design and implement a "struct lifnet" (logical ifnet)
for NetBSD. This is a rather large exercise and its presence will
be felt in many applications that issue ioctls for "struct ifnet"
that ultimately are for unkown interfaces.

A third alternative would be for the "struct ifnet" to become the
logical network interface structure that is exposed through IP
and other interfaces. Under "struct ifnet" would be a new data
structure ("struct devnet" for example) that was used by all
network devices in lieu of "struct ifnet". There would be a strict
one to one relationship between "struct ifnet" and "struct devnet".
Initially, all of the data that is currently within "struct ifnet"
would stay there and it would serve only as a tool to support a
namespace that is globally unique (that of network devices) with
one or more "struct ifnet" sets inside branes on top of that.
In the fullness of time it might be appropriate to move some data
elements out of "struct ifnet" and into "struct devnet".

Out of scope
============
Beyond providing support to use the pfil interfaces to support
either global or local brane filtering, modifying any of the
firewalls to work in any capacity beyond filtering for the
default brane is out of scope for this project. The reason for
that is the decision on which behaviour model to implement and
support is something that should be made after considering
which security model to enforce. That type of decision making
is out of scope for this project as it is principally concerned
with supporting the creation and management of branes. Similarly,
discussing which colour to paint that particular bike shed should
wait until branes exist. In the case of per-brane filtering,
there are issues to consider such as how and when a rule set
might become associated with said brane.