Subject: Re: Proposed changes to bsd.pkg.mk
To: Alistair Crooks <agc@pkgsrc.org>
From: David Burgess <burgess@mitre.org>
List: tech-pkg
Date: 04/13/2001 17:05:14
Alistair Crooks wrote:
> 
> 
> 1. Move md5 and patch-sum info out of the files/ subdir, thereby
> losing most of the files/ dirs from most packages.
> 
> 2.  Move the PLIST and DESCR and other binary package support files
> out of the pkg/ subdirectories into the package directory, thereby
> losing the pkg/ subdirs.

I don't know about this one...  I like having the PLIST and DESCR files
in their own directory.
> 
> 3.  Incorporate other information into the area which holds the digest
> info for distfiles, distpatches and patches.  In particular, I'd like
> the size of the distfiles and distpatches to be recorded.  This is not
> really for file integrity reasons (if the digest doesn't do that for
> you, it's not a very good digest), more for doing things like putting
> virtual quotas on sizes of distfile that can be downloaded without a
> warning.  And also for my personal use - I really would like to know
> how big a distfile is before I even contemplate downloading it over
> a dialup line.

We still need to have a mechanism for handling multi-architecture patch
files (like the i386 only patches on Apache?)  Segregating the patches
from the rest of the package is another area where the existing system
seems very clean to me.  In fact, I really like having pkg and patches
directories available.  It just seems to make a lot of sense to me.  Of
course, moving the files in pkg/ down one level won't make me jump off a
bridge or anything....

On the download size issue, it would certainly be nice to be able to say
"No, wait until the world has gone to bed to download the OpenMotif
file."  In fact, it would be really spiffy to be able to turn that on
and off in the /etc/mk.conf (FAST_DOWNLOAD= YES).

Of course, the next step after that is automated 'after hours' updates,
which would be followed by the possibility of a self aware package
system that keeps track of everything you have installed and updates it
for you as new stuff becomes available.  After that, it's singing, and
playing chess, and going to Jupiter, and "Dave, my circuits are
functioning normally..."

Oops - never mind; I need to take my medication.... :-)

> 
> Now whether the digests go in the pkg Makefile or in a separate
> distinfo file like FreeBSD are doing are another topic for discussion.
> 

I was out grabbing the package for saint and noticed this.  I have to
admit that it is a tempting idea.  It avoids mucking about in the
Makefile every time a patch changes, and does allow us to extend the
model for the package.

On the other end of the spectrum, I'd like to know what package dist
files I can delete.  As we work our way through the versions in the
packages, it would be nice to know that distfile $version-1 can be
deleted since we are now downloading a new version.  There are probably
clean ways to handle that now, but this would make it so that we could
have two or three valid 'old versions' around in case we need them for
something, but still help automate the cleanup of the distfiles
directory.  Just a musing.

> Personally, I'd prefer all digest, checksum and size and other
> ancillary information to go into a separate file. I'd like to
> avoid overloading the pkg Makefile.

I've been noodling on this all afternoon, and it might just be a good
idea.  Putting this stuff into the Makefile wasn't a bad idea - but
having a new file (especially one that we control the format of) could
be an easier solution.  It would reduce the inode count and get rid of
at least one directory, which I think is a Good Thing (tm).  We could
even extend the file to include the pkg/ file contents (if that turns
out to be a good idea).

> 
> Also, I believe that the number of files isn't necessarily the kicker.
> I think it also has a lot more to do with the proliferation of
> directories in pkgsrc, both when extracting from a tarfile, and when
> updating using cvs, due to synchronous writes of dirs, and cvs having
> to maintain separate CVS dirs for every directory it supports.

If I recall correctly, it wasn't the number of files that was the
problem, it was the number of inodes.  Every directory we can eliminate
or set of files we can consolidate reduces the inode count.  That gives
us more room to store more stuff.

> 
> Having said that, the COMMENT mods did speed up things just a smidgin,
> so I maybe wrong here.

If your machine is fast enough, you don't notice :-)

Dave Burgess