pkg/42969: Maximum memory calculation of archivers/xz is incorrect

[ryo_on%yk.rim.or.jp@localhost]

>Number:         42969
>Category:       pkg
>Synopsis:       Maximum memory calculation of archivers/xz is incorrect
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Mar 14 12:05:01 +0000 2010
>Originator:     Ryo ONODERA
>Release:        NetBSD 5.99.24
>Organization:
        
>Environment:
        
        
System: NetBSD hydrogen.elements.tetera.org 5.99.24 NetBSD 5.99.24 
(BRIGHTGIRL7) #5: Sat Mar 13 20:03:36 JST 2010 
root%hydrogen.elements.tetera.org@localhost:/usr/obj/sys/arch/i386/compile/BRIGHTGIRL7
 i386
Architecture: i386
Machine: i386
>Description:
On NetBSD/i386 current, commands of archivers/xz calculates the incorrect 
maximum limit
of memory usage.
The man page of sysctl(3) says sysctl(3) return 0 or -1, but present patch-aa 
and patch-ab
cannot catch return value = 0.
Then memory usage limit is minimized, and I cannot decompress large .tar.xz 
file,
for example, 
http://sourceforge.jp/projects/ume-font/downloads/22212/umefont_413.tar.xz/ .

>How-To-Repeat:
The man page of xz says the default value of max limit is 40% of physical 
memory.
Now, from my dmesg

total memory = 1013 MB
avail memory = 985 MB

and

% xz --long-help|tail -5
On this system and configuration, this program will use at maximum of roughly
12 MiB RAM and one thread.

Report bugs to <lasse.collin%tukaani.org@localhost> (in English or Finnish).
XZ Utils home page: <http://tukaani.org/xz/>


If apply following patches, 

% xz --long-help|tail -5
On this system and configuration, this program will use at maximum of roughly
405 MiB RAM and one thread.

Report bugs to <lasse.collin%tukaani.org@localhost> (in English or Finnish).
XZ Utils home page: <http://tukaani.org/xz/>

>Fix:
I does not understand the detail of xz source.
Please examine my patches.
http://gnats.netbsd.org/41963 describe this problem, I think.
But I cannot understand it completely, because of the lack of my knowledge.


diff -upNr patches.orig/patch-aa patches/patch-aa
--- patches.orig/patch-aa       2009-09-05 01:57:14.000000000 +0900
+++ patches/patch-aa    2010-03-14 20:36:06.000000000 +0900
@@ -7,7 +7,7 @@ $NetBSD: patch-aa,v 1.1 2009/09/04 16:57
        int cpus;
        size_t cpus_size = sizeof(cpus);
 -      if (!sysctl(name, &cpus, &cpus_size, NULL, NULL)
-+      if (sysctl(name, 2, &cpus, &cpus_size, NULL, 0) > 0
++      if (sysctl(name, 2, &cpus, &cpus_size, NULL, 0) >= 0
                        && cpus_size == sizeof(cpus) && cpus > 0)
                ret = (uint32_t)(cpus);
  #endif
diff -upNr patches.orig/patch-ab patches/patch-ab
--- patches.orig/patch-ab       2009-09-05 01:57:14.000000000 +0900
+++ patches/patch-ab    2010-03-14 20:36:06.000000000 +0900
@@ -7,7 +7,7 @@ $NetBSD: patch-ab,v 1.1 2009/09/04 16:57
        } mem;
        size_t mem_ptr_size = sizeof(mem.u64);
 -      if (!sysctl(name, 2, &mem.u64, &mem_ptr_size, NULL, NULL)) {
-+      if (sysctl(name, 2, &mem.u64, &mem_ptr_size, NULL, 0) > 0) {
++      if (sysctl(name, 2, &mem.u64, &mem_ptr_size, NULL, 0) >= 0) {
                // IIRC, 64-bit "return value" is possible on some 64-bit
                // BSD systems even with HW_PHYSMEM (instead of HW_PHYSMEM64),
                // so support both.


>Unformatted: