Re: Writing to multiple descriptors with one system call

[Sad Clouds <cryintothebluesky%googlemail.com@localhost>]

On Wed, 17 Mar 2010 11:03:03 -0500
Eric Haszlakiewicz <erh%nimenees.com@localhost> wrote:
 
> That it might, but I'm having a hard time seeing how you'd actually
> use it. Do you have a delay in each connection to allow for that
> connections response to be sent along with any other connections
> responses?  How do you figure out which connections are having the
> same data sent?  It seems like the overhead of coordinating all that
> might outweigh any benefit you get from fewer context switches.
> If you can flesh out the usage model it a bit, and maybe even write a
> sample program, it'd be easier to figure out whether this makes sense
> to do.  A simple user space wrapper function around the write() call
> would let you do everything except the performance testing.
> 
> eric

In the context of a busy web server, this would make a lot of sense.
This is because it is common for web server to send many small text
files to many concurrent clients. You could fit several of those files
in a socket's TCP send buffer.

On a busy server you cache the most frequently accessed files in
memory, so the data is ready to be sent down the TCP sockets.

1. Accepting connections

On busy web server it is likely that the listening socket will have
multiple entries on the completed connection queue (i.e. established
state). You call accept() in a loop until you get EWOULDBLOCK and since
these are new connections, the chances are all these sockets contain
HTTP GET requests for /index.html

Instead of calling write() 50 times for all 50 new sockets, you could
just call write2v() once.

2. Sending similar requests

When the server is handling large number of connections, there is a
pretty good chance that some of those connections will request the same
data for the same popular web resources. You have a queue of active
connections and every time you go in a loop servicing those
connections, you check your cache for valid data. Whenever you have a
cache hit, you mark it and aggregate multiple requests for the same
file into a single reply queue. Again, instead of calling write()
multiple times, you could issue a single system call to write a set of
buffers to multiple sockets.

This might make a big difference to the overall system time and
dramatically reduce load.