Subject: Re: pkg/15501: rsync development branch causes corruption with alpha server
To: NetBSD-current Discussion List <current-users@netbsd.org>
From: Manuel Bouyer <bouyer@antioche.lip6.fr>
List: netbsd-bugs
Date: 02/28/2002 12:16:34
On Tue, Feb 26, 2002 at 08:09:30PM -0500, Greg A. Woods wrote:
> I don't know anythin about the current stat of anoncvs -- I'm asking
> about rsync, which definitely is not OK yet.

Well, if the file you're trying is corrupted on the server, the cause may not
be rsync. rsync.netbsd.org is now running 2.3.3, and I checked,
bsd.pkg.defaults.mk,v is OK on the server.

> 
> > Did  you try 'rsync -c -W' ?
> 
> Hmmm.... '-c' isn't going to be very friendly to the server, but I guess
> I can try it once:
> 
>        -c, --checksum
>               This forces the sender to checksum all files  using
>               a  128-bit MD4 checksum before transfer. The check-
>               sum is then explicitly checked on the receiver  and
>               any  files of the same name which already exist and
>               have the same checksum and size on the receiver are
>               skipped.  This option can be quite slow.
> 
> Note that '-W' cannot do anything if there is no need to transfer any
> files in the first place:
> 
>        -W, --whole-file
>               With this option the incremental rsync algorithm is
>               not used and the whole file is sent as-is  instead.
>               The  transfer  may be faster if this option is used
>               when the bandwidth between the  source  and  target
>               machines  is  higher  than  the  bandwidth  to disk
>               (especially when the "disk" is actually a networked
>               file  system).   This  is the default when both the
>               source and target are on the local machine.
> 
> (and yes, I've confirmed this as much as I can with the rsync source,
> though there is some small room for error in my reading of this rather
> complex code).
> 
> 
> Note also that rsync.netbsd.org still fails Alan's test (see his reply
> in the PR database) even with '-c':
> 
> 	$ rsync -v -c -W rsync://rsync.netbsd.org/anoncvs/main/pkgsrc/mk/bsd.pkg.defaults.mk,v /tmp/foo,v
> 	/tmp/foo,v
> 	wrote 119 bytes  read 91924 bytes  26298.00 bytes/sec
> 	total size is 91784  speedup is 1.00
> 	$ rlog /tmp/foo,v > /dev/null                                     
> 	rlog: /tmp/foo,v:189: missing 'author' keyword
> 	rlog aborted
> 	$ 

This works for me, with rsync 2.4.8 locally.

> 
> As does rsync.jp.netbsd.org too:
> 
> 	$ rm -f /tmp/foo,v
> 	$ rsync -v -c -W rsync://rsync.jp.netbsd.org/anoncvs/main/pkgsrc/mk/bsd.pkg.defaults.mk,v /tmp/foo,v
> 	/tmp/foo,v
> 	wrote 119 bytes  read 84722 bytes  13052.46 bytes/sec
> 	total size is 84582  speedup is 1.00
> 	$ rlog /tmp/foo,v > /dev/null
> 	rlog: /tmp/foo,v:49: missing date
> 	rlog aborted
> 	$ 
> 
> Hmmm.... corrupted, but differently! (well it is a different size too)
> 
> > Can you try to rsync from anoncvs.fr.netbsd.org and tell us how it goes ?
> 
> First let's try to run Alan's test, with '-c', with that sever too.
> 
> 	$ rm -f /tmp/foo,v
> 	$ rsync -v -c -W rsync://rsync.fr.netbsd.org/anoncvs/main/pkgsrc/mk/bsd.pkg.defaults.mk,v /tmp/foo,v
> 	@ERROR: Unknown module 'anoncvs'
> 	rsync: connection unexpectedly closed (45 bytes read so far)
> 	rsync error: error in rsync protocol data stream (code 12) at io.c(140)
> 	$ 
> 
> oops -- maybe not.....
> 
> What's the correct incantation for rsync.fr, and why would it be different?

because I didn't have instructions on how this should be when I've set
up the mirror. Also, It's possible that I offered rsync access to
the anoncvs tree before it was available on rsync.netbsd.org

Try: rsync -v -c -W rsync://rsync.fr.netbsd.org/NetBSD-CVS/pkgsrc/mk/bsd.pkg.defaults.mk,v /tmp/foo,v

--
Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
--