Subject: Re: gzip and amanda conflict?
To: Peter Seebach <seebs@plethora.net>
From: Mike M. Volokhov <mishka@apk.od.ua>
List: current-users
Date: 09/19/2005 09:54:31
On Sun, 18 Sep 2005 22:22:21 -0500
seebs@plethora.net (Peter Seebach) wrote:

> In message <b0459d5c050918200370e7f7b5@mail.gmail.com>, matthew sporleder write
> s:
> >I don't think this is new behavior.  I've known about it for at least
> >two-three years, anyway.
> 
> The new behavior isn't that gzip has problems on large files.  It's that, now,
> it indicates an error status so that Amanda trashes the backup.

AFAIU, there is gzip problem. Some time ago we have had switched out
from GNU gzip. Right now, please look at the following code fragment
(in_tot is a total size of bytes to compress):

gz_compress(...)
{
	off_t in_tot = 0;
	...        
	i = snprintf(outbufp, BUFLEN, "%c%c%c%c%c%c%c%c", 
		(int)crc & 0xff,
		(int)(crc >> 8) & 0xff,
		(int)(crc >> 16) & 0xff,
		(int)(crc >> 24) & 0xff,
		(int)in_tot & 0xff,
		(int)(in_tot >> 8) & 0xff,
		(int)(in_tot >> 16) & 0xff,
		(int)(in_tot >> 24) & 0xff);
	...
	if (in_tot > 0xffffffff)
		maybe_warn("input file size >= 4GB cannot be saved");
	...
}

I.e. there are no room for reporting about more than 2^32=4G data.
Later this also badly affects on "gzip -l" for files with uncompressed
size more than specified volume. So...

--
Mishka.