kern/44539: opencrypto DEFLATE compression output is not correctly terminated

[M.Drochner%fz-juelich.de@localhost]

>Number:         44539
>Category:       kern
>Synopsis:       opencrypto DEFLATE compression output is not correctly 
>terminated
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Feb 09 18:40:00 +0000 2011
>Originator:     Matthias Drochner
>Release:        current
>Organization:
FZJ
>Environment:
NetBSD zelz27 5.99.44 NetBSD 5.99.44 (MIST+MP+MODS) #178: Wed Feb  9 18:35:07 
MET 2
011  
drochner@zelz27:/home/drochner/netbsd/work.src.usbdev/sys/arch/i386/compile/MI
ST+MP+MODS i386

>Description:
The code in opencrypto/deflate.c fails to set the Z_FINISH flag
to deflate() to tell that a particular input data chunk is the
last one. (It is a trivial case here because the input data
is a single contigous block in opencrypto.)
This makes that the compressed output doesn't have the flag set
which indicates the end to the decompressor. (For details, look at
http://www.gzip.org/zlib/rfc-deflate.html -- the "BFINAL"-Bit
is the missing one.)
Apparently, the decompressors in NetBSD's IPSEC implementations
(KAME and FAST_IPSEC) ate lazy enough to ignore this, but the
might be interoperability problems.

>How-To-Repeat:
-code inspection (the code in netinet6/ipcomp_core.c is correct)
-look at IPCOMP network packets sent by FAST_IPSEC
-run the test program which I'll send as mail attachment

>Fix:
(This also replaces the deprecated Z_PARTIAL_FLUSH in inflate().)
Change the calls to inflate/deflate to do:
   error = decomp ? inflate(&zbuf, Z_SYNC_FLUSH) :
                    deflate(&zbuf, Z_FINISH);

This is tested to interoperate with a NetBSD box running KAME IPSEC.