Re: Max concurrent connections and TCP port limits

[Steven Bellovin <smb%cs.columbia.edu@localhost>]

On Aug 24, 2010, at 9:24 31AM, Sad Clouds wrote:

> On Tue, 24 Aug 2010 07:47:44 -0400
> Steven Bellovin <smb%cs.columbia.edu@localhost> wrote:
> 
>> 
>> On Aug 24, 2010, at 6:38 11AM, Sad Clouds wrote:
>> 
>>> Hi I was pondering about the absolute maximum number of concurrent
>>> TCP connections a given machine can support. In the old days the
>>> limit was dictated by the CPU speed and/or available memory.
>>> 
>>> Well, these days it seems the limit is TCP's 16-bit port numbers. If
>>> you have a single NAT firewall, or a load balancer in front of many
>>> machines, how can it support more than 65536 concurrent TCP
>>> connections?
>>> 
>>> One possible (the only?) way is to bind multiple IP addresses to the
>>> same physical machine, then theoretically you can have a maximum of:
>>> 
>>> num_IP_addr * 2^16 concurrent TCP connections
>>> 
>>> Any idea if NetBSD TCP stack can handle such a case, i.e. can it
>>> reuse the same ephemeral port number with different IP addresses?
>>> 
>> When it's acting as a server, it can almost certainly handle that
>> with no problem whatsoever.  For client port numbers, I believe you
>> have to remember to set SO_REUSEADDR before you can bind to it.
>> 
>> A TCP connection is defined by the 4-tuple <local IP,local port,
>> remote IP, remote port>.  You can have very many connections to port
>> 80 if they're all coming from the different remote ports or remote
>> hosts.  Thus, you can also increase the number of IP addresses on
>> your front end box.
> 
> Well this is the scenario I was thinking:
> 
> Load balancer (LB) in front of a large cluster of servers. All client
> requests go via load balancer, which creates TCP connections to
> servers. The servers do heavy-duty processing and send their responses
> back to the clients, via load balancer.
> 
> LB __ S1
>  |__ S2
>  |__ S3
>  |__ S4
>  ...
>  |__ S1024
> 
> With SO_REUSADDR the load balancer should be able to establish
> unlimited number of concurrent TCP connections with clients (provided
> it has enough memory).
> 
> However the limit on the total number of concurrent TCP connections
> between the load balancer and each server is 65536. If each server is
> powerful enough to handle 500000 concurrent connections, the hardware
> will be underused.
> 
> I can think of just two solutions:
> 
> 1) Have each server bind multiple IP addresses in order to overcome
> TCP's 16-bit port limitation.

Right, and the load balancer should neither know or care that a given IP 
address is on the same machine as some other IP address it's routing traffic 
to.  It will just work.
> 
> 2) Develop application level protocol to multiplex multiple connections
> over a single TCP connection. This way load balancer creates one
> connection to each server and the servers demultiplex client requests.
> This is a bit like reinventing TCP.
> 
Very ugly -- don't go there.

Also 3) virtual machines on the servers, to permit more utilization of the 
hardware, and 4) multiple IP addresses on the load balancer.

I'd go with (1) -- it's the simplest, and shouldn't require any new code 
anywhere.


                --Steve Bellovin, http://www.cs.columbia.edu/~smb