Re: multiple rx/tx rings and interrupt delivery on newer nics

[Sepherosa Ziehau <sepherosa%gmail.com@localhost>]

On Wed, Apr 1, 2009 at 11:59 PM, Darren Reed <darrenr%fastmail.net@localhost> 
wrote:
> Sepherosa Ziehau wrote:
>>
>> On Thu, Jun 14, 2007 at 3:05 PM, Darren Reed 
>> <darrenr%fastmail.net@localhost> wrote:
>> > Allen Briggs wrote:
>> >>
>> >> ...
>> >> Different hardware has different kinds of packet classification
>> >> for incoming packets and probably different kinds of scheduling
>>
>> M$ defined two standard classification of incoming packets.  Intel has
>> several extension (like use addr/port tuple for UDP packets) in their
>> pcie NICs, however, I don't think it will get widely deployed.  As
>> about the multi-tx queues, AFAIK, all NICs support multi-tx queue
>> provided some mechanism to make all TX queues use same priority.
>
> I disagree.

mmm, which points do you disagree :)?  The description about the RX
side or the TX side?

>
> Everyone that wants to sell a NIC for use wit virtualisation is
> going to add support for delivery into rings based on IP address.

Do you mean by the hash of {faddr,laddr} pair?  All NICs support RSS
could do that.

>
>
> The latest releases of Solaris that you can download (Solaris
> Express Community Edition - not an officially supported product)
> can make full use of these NIC features for either queueing

Could you tell me which NIC (the chip/vendor) and which feature you
have mentioned above?  On RX side, I think the most common feature is
RSS, which is widely implemented in many modern NICs.  Broadcom's NICs
have kind of programmable filtering mechanism on RX side, I don't know
whether you mean that.  Currently in dragonfly, we only utilize RSS
feature of some NICs.

> packets up for zones or applications. It'll be in the next
> "release" of OpenSolaris. You can be sure that if Linux does
> not have something like that now, it will "soon" and the same
> for Microsoft. NetBSD can sit around, look at the ceiling,
> whistle dixie and pretend that it isn't relevant but nothing
> is going to stop the others striving to be "better".
>
> Intel isn't adding these features because people don't want

Could you describe a little bit more about "these features"?

> them or won't use them, if anything it is the exact opposite.
> They've got a single 10GB NIC and would like to turn that into
> 10 virtual 1GB NICs, etc.

Well, I have to say, I currently don't have a clear idea about how
SR-IOV works in Intel's relatively newer products (e.g. 82576, it is
1GB NIC tho), if by "these features" you mean SR-IOV (I think your
description is quite close to it)

>
> I can easily see this feature being used for routing (lets give
> all port 80 traffic its own set of tx/rx rings.) Wouldn't you
> rather be able to dedicate a descriptor ring or two for your ssh
> traffic than rely on ALTQ and the device driver for priority
> delivery? Or maybe you want specific rings for ssh and http
> because you're using bittorrent a lot and that uses effectively
> random addresses and ports?

Well, these are nice features.  It is definitely doable on TX side.
However, I don't have idea how you could program any currently
available NICs to do that on RX side.  I didn't take a close look at
NetExtremeII's RX filtering mechanism, maybe it could kinda do what
you have described?

>
> This capability will eventually work its way into cheaper NICs,
> if only in a very limited fashion, because people will want to
> run a virtual guest on the desktop using the motherboard NIC
> and to not have to suffer from the virtual guest "flooding"
> their NIC.

Yep, if one day they appeared in the NICs, they would be nice features
to support :)

Best Regards,
sephe

-- 
Live Free or Die