Subject: Rework of the fetch target (was: CVS commit: pkgsrc/mk)
To: Frederick Bruckman <fredb@netbsd.org>
From: Bernd Ernesti <netbsd@arresum.inka.de>
List: tech-pkg
Date: 01/27/2002 17:01:38
Hi,
I am not exactly sure that it is this change, but doing a make fetch-list
is now a real mess.
Even if have the distfile, you see this kind of output, which makes it
impossible to see which distfiles you need to get:
pkgsrc/graphics/jhead> make fetch-list
test -d /home/temp/distfiles/ || /bin/mkdir -p /home/temp/distfiles/; cd /h=
ome/temp/distfiles/; unsorted_sites=3D"ftp://ftp.info-zip.org/pub/infozip/s=
rc/ ftp://ftp.icce.rug.nl/infozip/src/ ftp://uiarchive.cso.uiuc.edu/pub/p=
ackages/zip/src/ ftp://ftp.fu-berlin.de/pub/unix/tools/archivers/unzip/ ft=
p://ftp.netbsd.org/pub/NetBSD/packages/distfiles/ ftp://ftp.freebsd.org/pu=
b/FreeBSD/distfiles/"; sites=3D" $unsorted_sites"; file=3D"unzip542.t=
ar.gz"; bfile=3D"unzip542.tar.gz"; if [ "X" !=3D "X" ]; then =
for d in "" ; do if [ "X$d" =3D "X" -o "X$d" =3D "X/home/temp/distfiles"=
]; then continue; fi; if [ -f $d//$bfile ]; then echo "Using $d//$bfile"=
; /bin/rm -f $bfile; /bin/ln -s $d//$bfile $bfile; break; fi; =
done; fi; if [ ! -f $file -a ! -f $bfile -a ! -h $bfile ]; =
then echo "=3D> $bfile doesn't seem to exist on this system."; if [ ! -w =
/home/temp/distfiles//. ]; then echo "=3D> Can't download to /home/temp=
/distfiles/ (permission denied?)."; exit 1; fi; for site in =
$sites; do echo "=3D> Attempting to fetch $bfile from ${site}."; if /=
usr/bin/ftp ${site}${bfile} ; then if [ -n "" -a -f /home/source/pkgsrc/a=
rchivers/unzip/distinfo -a -f /home/temp/distfiles//$bfile ]; then alg=3D`=
/usr/bin/awk 'NF =3D=3D 4 && $2 =3D=3D "('$file')" && $3 =3D=3D "=3D" {prin=
t $1;}' /home/source/pkgsrc/archivers/unzip/distinfo`; if [ -z "$alg" ]; t=
hen alg=3DSHA1; fi; CKSUM=3D`/usr/pkg/bin/digest $alg < /home/temp/di=
stfiles//$bfile`; CKSUM2=3D`/usr/bin/awk '$1 =3D=3D "'$alg'" && $2 =3D=3D =
"('$file')" {print $4;}' </home/source/pkgsrc/archivers/unzip/distinfo`; i=
f [ "$CKSUM" =3D "$CKSUM2" -o "$CKSUM2" =3D "IGNORE" ]; then break; els=
e echo "=3D> Checksum failure - trying next site."; fi; elif [ ! -=
f /home/temp/distfiles//$bfile ]; then echo "=3D> FTP didn't fetch expecte=
d file, trying next site." ; else break; fi; fi done; =
if [ ! -f /home/temp/distfiles//$bfile ]; then echo "=3D> Couldn't =
fetch $bfile - please try to retrieve this"; echo "=3D> file manually into =
/home/temp/distfiles/ and try again."; exit 1; fi; fi;
test -d /home/temp/distfiles/jhead-1.5 || /bin/mkdir -p /home/temp/distfile=
s/jhead-1.5; cd /home/temp/distfiles/jhead-1.5; unsorted_sites=3D"http://ww=
w.sentex.net/~mwandel/jhead/ ftp://ftp.netbsd.org/pub/NetBSD/packages/distf=
iles/jhead-1.5/ ftp://ftp.freebsd.org/pub/FreeBSD/distfiles/jhead-1.5/"; s=
ites=3D" $unsorted_sites"; file=3D"jhead-1.5/jhead.zip"; bfile=
=3D"jhead.zip"; if [ "X" !=3D "X" ]; then for d in "" ; do if [ =
"X$d" =3D "X" -o "X$d" =3D "X/home/temp/distfiles" ]; then continue; fi; i=
f [ -f $d/jhead-1.5/$bfile ]; then echo "Using $d/jhead-1.5/$bfile"; /bin=
/rm -f $bfile; /bin/ln -s $d/jhead-1.5/$bfile $bfile; break; fi; =
done; fi; if [ ! -f $file -a ! -f $bfile -a ! -h $bfile ]=
; then echo "=3D> $bfile doesn't seem to exist on this system."; if [ ! -=
w /home/temp/distfiles/jhead-1.5/. ]; then echo "=3D> Can't download to=
/home/temp/distfiles/jhead-1.5 (permission denied?)."; exit 1; fi; =
for site in $sites; do echo "=3D> Attempting to fetch $bfile f=
rom ${site}."; if /usr/bin/ftp ${site}${bfile} ; then if [ -n "" -a -f /=
home/source/pkgsrc/graphics/jhead/distinfo -a -f /home/temp/distfiles/jhead=
-1.5/$bfile ]; then alg=3D`/usr/bin/awk 'NF =3D=3D 4 && $2 =3D=3D "('$file=
')" && $3 =3D=3D "=3D" {print $1;}' /home/source/pkgsrc/graphics/jhead/dist=
info`; if [ -z "$alg" ]; then alg=3DSHA1; fi; CKSUM=3D`/usr/pkg/bin/=
digest $alg < /home/temp/distfiles/jhead-1.5/$bfile`; CKSUM2=3D`/usr/bin/a=
wk '$1 =3D=3D "'$alg'" && $2 =3D=3D "('$file')" {print $4;}' </home/source/=
pkgsrc/graphics/jhead/distinfo`; if [ "$CKSUM" =3D "$CKSUM2" -o "$CKSUM2" =
=3D "IGNORE" ]; then break; else echo "=3D> Checksum failure - tryi=
ng next site."; fi; elif [ ! -f /home/temp/distfiles/jhead-1.5/$bfile =
]; then echo "=3D> FTP didn't fetch expected file, trying next site." ; e=
lse break; fi; fi done; if [ ! -f /home/temp/dis=
tfiles/jhead-1.5/$bfile ]; then echo "=3D> Couldn't fetch $bfile - please =
try to retrieve this"; echo "=3D> file manually into /home/temp/distfiles/j=
head-1.5 and try again."; exit 1; fi; fi;
Bernd
On Sun, Jan 06, 2002 at 10:03:24PM +0200, Frederick Bruckman wrote:
>=20
> Module Name: pkgsrc
> Committed By: fredb
> Date: Sun Jan 6 20:03:24 UTC 2002
>=20
> Modified Files:
> pkgsrc/mk: bsd.pkg.mk
>=20
> Log Message:
> Re-work the fetch target(s), as discussed on "tech-pkg". The primary
> motivation is to make the order that sites are hit more sane, and
> especially to prevent hitting sites that aren't ever expected to have
> the sought after file.
>=20
> Now, ${MASTER_SITE_OVERRIDE} is always hit first. If that fails, then
> the sites designated for that file, then ${MASTER_SITE_BACKUP}, are hit
> (by default), but with the order subject to ${MASTER_SORT_REGEX} and
> ${MASTER_SORT_AWK}. The "designated sites" are usually ${MASTER_SITES}
> for files in ${DISTFILES}, and ${PATCH_SITES} for files in ${PATCH_FILES}.
> However, defining a variable `${SITES_foo}' in the package "Makefile"
> overrides that for file "foo". [The use of ${MASTER_SITES_foo} and
> ${PATCH_SITES_foo}, which is currently only used by a couple of packages
> for the same purpose, is deprecated, and will be shortly unsupported.]
>=20
> Also eliminate redundancy in the do-fetch and fetch-list-one-pkg targets,
> by making them use a single, common macro (escaped with `:Q' in the
> fetch-list-one-pkg case), so "make fetch-list-one-pkg | sh" now does
> exactly the same thing as "make do-fetch".
>=20
>=20
> To generate a diff of this commit:
> cvs rdiff -r1.894 -r1.895 pkgsrc/mk/bsd.pkg.mk
>=20
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>=20
>=20