[src/trunk]: src/sys/opencrypto The decompressor in sys/net/zlib.c has a bug:...

[drochner <drochner%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/c639c2c8ea75
branches:  trunk
changeset: 762214:c639c2c8ea75
user:      drochner <drochner%NetBSD.org@localhost>
date:      Thu Feb 17 17:10:18 2011 +0000

description:
The decompressor in sys/net/zlib.c has a bug: It returns Z_BUF_ERROR after
a successful decompression in rare cases. A necessary but not sufficient
condition seems to be that the decompressed data end exactly at the end
of an allocated output buffer. (I can reproduce this reliably with
a userland program built against kernel zlib. Userland libz is much
newer and not affected.)
Since kernel zlib is based on an old version and heavily modified, I don't
dare to touch it. So catch this case in the wrapper.
Being here, reorder deflate/inflate error handling and add comments
to make understandable what is tested and why.

diffstat:

 sys/opencrypto/deflate.c |  26 ++++++++++++++++++--------
 1 files changed, 18 insertions(+), 8 deletions(-)

diffs (57 lines):

diff -r 0e67ff3f347c -r c639c2c8ea75 sys/opencrypto/deflate.c
--- a/sys/opencrypto/deflate.c  Thu Feb 17 17:07:55 2011 +0000
+++ b/sys/opencrypto/deflate.c  Thu Feb 17 17:10:18 2011 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: deflate.c,v 1.15 2011/02/16 19:08:57 drochner Exp $ */
+/*     $NetBSD: deflate.c,v 1.16 2011/02/17 17:10:18 drochner Exp $ */
 /*     $FreeBSD: src/sys/opencrypto/deflate.c,v 1.1.2.1 2002/11/21 23:34:23 sam Exp $  */
 /* $OpenBSD: deflate.c,v 1.3 2001/08/20 02:45:22 hugh Exp $ */
 
@@ -35,7 +35,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: deflate.c,v 1.15 2011/02/16 19:08:57 drochner Exp $");
+__KERNEL_RCSID(0, "$NetBSD: deflate.c,v 1.16 2011/02/17 17:10:18 drochner Exp $");
 
 #include <sys/types.h>
 #include <sys/malloc.h>
@@ -125,10 +125,22 @@
        for (;;) {
                error = decomp ? inflate(&zbuf, Z_SYNC_FLUSH) :
                                 deflate(&zbuf, Z_FINISH);
-               if (error != Z_OK && error != Z_STREAM_END)
+               if (error == Z_STREAM_END) /* success */
+                       break;
+               /*
+                * XXX compensate for two problems:
+                * -Former versions of this code didn't set Z_FINISH
+                *  on compression, so the compressed data are not correctly
+                *  terminated and the decompressor doesn't get Z_STREAM_END.
+                *  Accept such packets for interoperability.
+                * -sys/net/zlib.c has a bug which makes that Z_BUF_ERROR is
+                *  set after successful decompression under rare conditions.
+                */
+               else if (decomp && (error == Z_OK || error == Z_BUF_ERROR)
+                        && zbuf.avail_in == 0 && zbuf.avail_out != 0)
+                               break;
+               else if (error != Z_OK)
                        goto bad;
-               else if (zbuf.avail_in == 0 && zbuf.avail_out != 0)
-                       goto end;
                else if (zbuf.avail_out == 0) {
                        if (i == len) {
                                len += ZBUF;
@@ -146,11 +158,9 @@
                        buf[i].size = size;
                        zbuf.avail_out = buf[i].size;
                        i++;
-               } else
-                       goto bad;
+               }
        }
 
-end:
        result = count = zbuf.total_out;
 
        if (i != 1) { /* copy everything into one buffer */