Re: so_rerror

[christos%astron.com@localhost (Christos Zoulas)]

In article <cf28a475-f248-6f2d-c947-4e8313906375%marples.name@localhost>,
Roy Marples  <roy%marples.name@localhost> wrote:
>On 03/11/2018 15:33, Christos Zoulas wrote:
>> Hello,
>> 
>> Since the introduction of so_rerror tracking to detect receive
>> socket overflows, we have been trying to make other programs cope
>> by increasing their buffer sizes and avoid the error message flood.
>> We have taken this as far as it goes now, and still there are
>> pathological cases where the new behavior cannot easily be fixed
>> (one has to go and fix each program separately). One example is
>> when a program turns on debugging to syslog and logs quicker than
>> syslog can absorb. What happens then is that syslog and the program
>> keep spewing error messages about the socket overflow and do little
>> else...
>
>This is a double edged sword.
>If there's any issue logging messages, I want to know about it.
>I have no idea if the discarded message was pointless debug or someone 
>trying to breach my server. A good example is on one high traffic server 
>I have the bulk of text in /var/log/messages is from blacklistd - I 
>would be upset if any of that was silently discarded.
>Maybe we need an option to increase the size of syslogd buffers?

I understand, but there is only one channel now and debugging messages
can overwhelm (and usually do) others in terms of quantity.

>Or better yet, we need a way of dynamically increasing/descreasing the 
>buffers in the kernel to avoid this.

Sure that might help, but it does not help if the receiving size
keeps falling behind (and might end up with resource exhaustion in
the kernel).

>If you want this new global option to default to off (which I strongly 
>disagree with - sweeping issues under the carpet should not be a default 
>option), then can we also have a man page update to describe this 
>behaviour please.

Well, it has to default to off, since we keep finding new programs we
need to fix. The new behavior is "new" and old programs are not designed
with it in mind and since most OS's don't behave like this we can expect
them to break (we just modified BIND)...

Here's a new patch that includes the global sysctl, and the fix to
not call sorwakeup (thanks!)

christos

Index: sys/socket.h
===================================================================
RCS file: /cvsroot/src/sys/sys/socket.h,v
retrieving revision 1.128
diff -u -u -r1.128 socket.h
--- sys/socket.h	16 Sep 2018 20:40:20 -0000	1.128
+++ sys/socket.h	3 Nov 2018 18:51:15 -0000
@@ -132,6 +132,7 @@
 #define	SO_NOSIGPIPE	0x0800		/* no SIGPIPE from EPIPE */
 #define	SO_ACCEPTFILTER	0x1000		/* there is an accept filter */
 #define	SO_TIMESTAMP	0x2000		/* timestamp received dgram traffic */
+#define	SO_RERROR	0x4000		/* Keep track of receive errors */
 
 
 /*
Index: kern/uipc_socket.c
===================================================================
RCS file: /cvsroot/src/sys/kern/uipc_socket.c,v
retrieving revision 1.265
diff -u -u -r1.265 uipc_socket.c
--- kern/uipc_socket.c	3 Sep 2018 16:29:35 -0000	1.265
+++ kern/uipc_socket.c	3 Nov 2018 18:51:15 -0000
@@ -118,6 +118,7 @@
 
 extern const struct fileops socketops;
 
+static int	sooptions;
 extern int	somaxconn;			/* patchable (XXX sysctl) */
 int		somaxconn = SOMAXCONN;
 kmutex_t	*softnet_lock;
@@ -537,6 +538,7 @@
 	so->so_proto = prp;
 	so->so_send = sosend;
 	so->so_receive = soreceive;
+	so->so_options = sooptions;
 #ifdef MBUFTRACE
 	so->so_rcv.sb_mowner = &prp->pr_domain->dom_mowner;
 	so->so_snd.sb_mowner = &prp->pr_domain->dom_mowner;
@@ -1757,6 +1759,7 @@
 	case SO_OOBINLINE:
 	case SO_TIMESTAMP:
 	case SO_NOSIGPIPE:
+	case SO_RERROR:
 #ifdef SO_OTIMESTAMP
 	case SO_OTIMESTAMP:
 #endif
@@ -1958,6 +1961,7 @@
 	case SO_OOBINLINE:
 	case SO_TIMESTAMP:
 	case SO_NOSIGPIPE:
+	case SO_RERROR:
 #ifdef SO_OTIMESTAMP
 	case SO_OTIMESTAMP:
 #endif
@@ -2542,4 +2546,11 @@
 		       SYSCTL_DESCR("Maximum socket buffer size"),
 		       sysctl_kern_sbmax, 0, NULL, 0,
 		       CTL_KERN, KERN_SBMAX, CTL_EOL);
+
+	sysctl_createv(&socket_sysctllog, 0, NULL, NULL,
+		       CTLFLAG_PERMANENT|CTLFLAG_READWRITE,
+		       CTLTYPE_INT, "sooptions",
+		       SYSCTL_DESCR("Default socket options"),
+		       NULL, 0, &sooptions, 0,
+		       CTL_KERN, CTL_CREATE, CTL_EOL);
 }
Index: kern/uipc_socket2.c
===================================================================
RCS file: /cvsroot/src/sys/kern/uipc_socket2.c,v
retrieving revision 1.132
diff -u -u -r1.132 uipc_socket2.c
--- kern/uipc_socket2.c	3 Sep 2018 16:29:35 -0000	1.132
+++ kern/uipc_socket2.c	3 Nov 2018 18:51:16 -0000
@@ -509,8 +509,10 @@
 	KASSERT(solocked(so));
 
 	so->so_rcv.sb_overflowed++;
-	so->so_rerror = ENOBUFS;
-	sorwakeup(so);
+	if (so->so_options & SO_RERROR)  {
+		so->so_rerror = ENOBUFS;
+		sorwakeup(so);
+	}
 }
 
 /*