On Mon, Jul 28, 2003 at 08:00:38AM -0400, Steven M. Bellovin wrote:
> In message <20030728094622.GB19735@nef.pbox.org>, Alistair Crooks writes:
> >On Sun, Jul 27, 2003 at 05:25:03PM -0400, Steven M. Bellovin wrote:
> >> In message <Pine.NEB.4.53.0307272227490.199@forsaken.emlyn.absd.org>, David 
> >Bro
> >> wnlee writes:
> >> >On Sun, 27 Jul 2003, David Maxwell wrote:
> >> >
> >> >> On Sun, Jul 27, 2003 at 09:12:54AM -0400, Steve Bellovin wrote:
> >> >> > It's complaining that the new vulnerability list is shorter than the
> >> >> > old one.
> >> >>
> >> >> Corrected. The combination of a 0 size change by a 4 char-name committer
> >> >> after a 5-char name committer...
> >> >>
> >> >> One of these days I'll eliminate this size dependency scheme.
> >> >
> >> >	Maybe it could switch to 'wc -l'?
> >> >
> >> 
> >> What's needed is some assurance that (a) the file wasn't truncated on 
> >> download, and (b) that the file is newer than the previous one.  I 
> >> suggest sha1 for the former and a timestamp -- date +%s will do -- for 
> >> the latter.
> >
> >I disagree - you are suggesting two extra pieces of information to be
> >downloaded (and maintained by NetBSD developers).  I think the
> >suggestion of an EOF token is the best way to go, since the
> >information is held in the vulnerabilities file, there is no other
> >extraneous information to download, and two less things to worry about
> >when updating the vulnerabilities file.
> 
> They don't have to be maintained manually, of course.  And I've seen 
> far too many accidents involving wrong versions of files or truncated 
> files to be happy with no extra checks.

Yes, I agree that they should not be maintained manually.

It is problematic to include an sha1 digest and timestamp in the
vulnerabilities file itself, although it could be done (we do that for
the patches/* files right now), but this adds complexity to what is
now a simple process:

At the moment, when a developer adds a vulnerability to the file, it
is a simple edit, check in to cvs, and then the file is uploaded.

Using your scheme, how could the new sha1 digest and timestamp be
given to the person/program downloading the vulnerabilities file? 
Whilst it's nice to think about automatic SMTP delivery of a new sha1
digest, and timestamp, which then triggers a download of the
vulnerabilities file, I suspect that there are a number of places
which aren't SMTP-connected, and so would have to retrieve timestamp
and sha1 by other means.  If it is other means, how do you get around
the problem we have right now of being sure that the full file has
been downloaded without being compromised?

Whatever else, it is obvious that we need to buck up our ideas,
because no matter how large the capitals are which say that the file
must not shrink, it seems there are times when it happens (and this
morning's example was more of the "freak accident" than an "I forgot
the instructions").

David's suggestion about "wc -l" would be an excellent short-term
move, I think, but I'd like to sort this out properly long-term, and
I'm all ears for any suggestions that do not involve downloading
extra files. The current ideas that occur to me are:

1. an explicit EOF token placed in the file (from david@)
2. use "wc -l" to calculate lines rather than character size of file
   (from abs@)
3. download 1 file, which is a shar including vulnerabilities file,
   digest and/or timestamp.
4. send out via SMTP changes notices whenever the file has changed
5. add an "integrity check" file containing sha1 and digest (from smb@,
   discussed above).

Does anyone have any other ideas?

Thanks,
Alistair