Subject: Re: archives in mbox format?
To: Stefan 'Kaishakunin' Schumacher <stefan@net-tex.de>
From: J.P. Larocque <piranha@thoughtcrime.us>
List: netbsd-advocacy
Date: 08/21/2005 21:55:47
--qMm9M+Fa2AknHoGS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Sun, Aug 21, 2005 at 10:06:09PM +0200, Geert Hendrickx wrote:
> On Sun, Aug 21, 2005 at 09:37:26PM +0200, Stefan 'Kaishakunin' Schumacher wrote:
> > I guess it's no problem to make the mbox file available, since the
> > content of the list is already published on the net.
> 
> I think it would be a good idea to make the mbox files available for 
> http/ftp download (e.g. in monthly chunks), so people can jump in threads
> on lists they're not subscribed to.  

I attempted to download the full archives of a few lists, on a
semi-automated and rate-limited basis.  After a few archives were
e-mailed to me, they stopped being transmitted.  Attached is a message
I sent to what I thought would be the list server administrator,
without reply.  (I'll see if there's a more appropriate address to
send that message to later.)

My purpose for the archives is to present a web archive using archival
software I'm writing, as an alternative to the crude and nearly
unusable mail-index.netbsd.org.  Demo (on a non-NetBSD list):
http://ely.ath.cx/~piranha/mlforum/usagi-users/dynamic/

-- 
J.P. Larocque is <piranha@thoughtcrime.us> and <piranha@ely.ath.cx>
Encrypted/signed e-mail preferred; http://ely.ath.cx/~piranha/pgp
Fpr 5612 10A8 4986 2D85 A995  252B 4C02 5E02 F61D 2E61; ID 0xF61D2E61

--qMm9M+Fa2AknHoGS
Content-Type: message/rfc822
Content-Disposition: inline

Date: Mon, 1 Aug 2005 13:17:38 -0700
From: "J.P. Larocque" <piranha@thoughtcrime.us>
To: majordomo-owner@NetBSD.org
Subject: List archive retrieval
Message-ID: <20050801201732.GA21986@evanescence.ely.ath.cx>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.9i

Hello,

I've been retrieving list archive files from a few mailing lists:
tech-net, netbsd-users, and netbsd-advocacy.  I'm trying to get
complete archives, starting with these lists.

To avoid placing a large bandwidth burden on your mail server with my
requests, I've been issuing requests for archive files at 3 per hour
(1 per hour per list).  At about 2.5 million bytes per list archive
file, this averages to just over 2KB/s.  I hope you consider my
approach reasonable.

I noticed that I stopped receiving list archive files after 2005-07-31
07:41:33 UTC.  Requests for files were acknowledged with messages like
this, which were previously accompanied by the archive file requested.

	To: piranha@thoughtcrime.us
	From: majordomo@NetBSD.org
	Subject: Majordomo results: tech-net.0012
	Message-Id: <20050731080933.504AC63B11E@mail.netbsd.org>
	Date: Sun, 31 Jul 2005 08:09:33 +0000 (UTC)
	
	--
	
	>>>> get tech-net tech-net.0012
	List 'tech-net' file 'tech-net.0012'
	is being sent as a separate message.

To avoid an avalanche of pending responses, I've ceased sending
requests for each list every hour.

Every indication tells me that my mail server has not been receiving
the files, and that it's probably something on your end.  If you or
someone else intentionally stopped the process, I'd like to discuss
the issue and perhaps arrange some other way to receive the list
archives that would be less of a burden on your systems.  Otherwise, I
wanted to bring this problem to your attention to try getting it
resolved.

Thanks for your time,

-- 
J.P. Larocque is <piranha@thoughtcrime.us> and <piranha@ely.ath.cx>
Encrypted/signed e-mail preferred; http://ely.ath.cx/~piranha/pgp
Fpr 5612 10A8 4986 2D85 A995  252B 4C02 5E02 F61D 2E61; ID 0xF61D2E61

--qMm9M+Fa2AknHoGS--