Re: packet-classifier Summer of Code RFQ

[David Young <dyoung%pobox.com@localhost>]

On Thu, Mar 25, 2010 at 10:16:22AM -0400, Andy Pyles wrote:
> ---------
> In kernel API for registering packet classes
> --------
> 
> 
> Hello, I'm interested in the Summer of Code proposal for
> packet-classifier API. Below is a proposal that I have put together
> regarding this SOC project. Any comments provided will be greatly
> appreciated.
> Most of the RFQ here is a re-phrasing of proposal outlined here:
> http://www.netbsd.org/contrib/soc-projects.html#packet-classifying
> This is my understanding of the proposal, and may contain errors, in
> which case corrections are welcome.
> 
> 
> Request for Comment: packet-classifier API
> 
> Overview:
> Currently when a new class is added via pf/ALTQ / ALTQD it is added to
> an internal ALTQ data structure. Similarly, PF uses tags, it's own
> data structure for tagging packets.  Since PF and ALTQ are tightly
> coupled together, I propose a way to decouple PF and ALTQ.  For
> instance, suppose an alternative firewall would like to integrate with
> ALTQ? Currently there is not a viable way to accomplish this outside
> of the tight coupling that is done with PF. What is required is a
> common API that can be used by ALTQ, PF and network device drivers.
>     This common API is coined the "packet-classifier" API. The
> packet-classifier will store packet classes in a separate data
> structure. ALTQ and network device drivers can register new classes
> with the packet-classifier. An ethernet device driver could register
> two classes "hi_priority" and "low_priority" with the
> packet-classifier for instance if it contains transmit rings of
> different priority. When a register() is successful, the
> packet-classifier returns a token than can then be used for labeling a
> packet placed in an MBUF for instance that can be classified later.
> 
> The packet-classifier API will have the following function calls:
>      Token *packet_classifier_register(....); // register a new class
> returns Token
>      int packet_classifier_lookup(....);  // lookup classifier
>      int packet_classifier_destroy(Token); // destroy classifier with
> associated Token
>      int packet_classifier_replace(Token,.....); // replace classifier
> with associated Token, with updated         information.
>      TODO:  others??

Andy,

A couple of comments about your proposal:

1) Some of the ALTQ schedule types *may* be able to use a "packet
   extract," such as a src/dst port/addr tuple (or hash), that the
   packet filter has already calculated.  For example, a schedule may
   assign packets to different queues using the hash of the source port
   & address.

2) Each interface may need a mapping from one or more class-labels
   applied by the packet filter to service categories.  E.g., "bulk"
   may be the name of a packet classification, but "hipri" and "lopri"
   are NIC 1's service types, and "hi", "med", "lo" are NIC 2's service
   types.  The operator may desire for NIC 1 to assign service type
   "lopri" to "bulk" packets, and for NIC 2 to assign service type "med"
   to "bulk" packets.

BTW, these days I am also concerned with three problems that are
related to the packet-classifying one:

1) What flows go through my router or network interface, *right now*?
   How much traffic is in each flow?  What are the historical flows on
   this router/interface?

   Several programs purport to track this information, but none of them
   have met my needs.

2) Lots of bus-mastering NICs have both a transmit-descriptor ring that
   holds umpteen packets, and a transmission queue (struct ifnet member
   if_snd) that holds umpteen more.  The queue empties into the ring.
   The queue size is controllable by sysctl, but the ring size usually
   is not.  ISTM that *one* sysctl variable should control the maximum
   length of the ring plus the queue.

   A related problem is that ALTQ operates on the transmission queue,
   not on the ring+queue.  I don't think that ALTQ can be effective by
   prioritizing packets on the transmission queue if the ring can grow
   very long.

Dave

-- 
David Young             OJC Technologies
dyoung%ojctech.com@localhost      Urbana, IL * (217) 278-3933