Subject: Re: periodic(8) etc.
To: None <tech-userlevel@netbsd.org>
From: Bruce J.A. Nourish <bjan+tech-userlevel@bjan.net>
List: tech-userlevel
Date: 09/30/2003 23:18:55
On Wed, Oct 01, 2003 at 12:51:15AM -0400, Andrew Brown wrote:
> >There seems to be a number of issues with /etc/{daily,weekly,monthly}.
> >
> >misc/14825: asks for all the scripts to be unified, so that (for
> >example) setting rebuild_locatedb=YES in daily.conf would cause
> >the locatedb to be rebuild daily, rather than weekly. Greg Woods
> >then suggested that our whole framework should be replaced by
> >something like FreeBSD's periodic(8).
> 
> that's being looked at, but isn't a terribly pressing task, i'm
> afraid.

...which is why I'm volunteering to do it :-)

> >misc/11011: points out that when bringing a laptop up from several
> >days or more of bieng suspended, the daily/weekly scripts immediately
> >kick in and start several very battery-expensive operations (i.e.
> >find / ...). John Hawkinson replied, suggesting that the multiple
> >find / operations could probably be eliminated, and mentioned some
> >other problems with daily/weekly scripts on laptops.
> 
> ooh!  we already fixed that.  months ago, too.  no idea there was a
> pr.  i'll close it.
> [...]  
> okay, let's see...
> 
> 17029	interesting idea
> 20272	again, interesting idea
> 22148	hmm...this is only tangentially related
> 22230	ah, good point
> 7716	this goes back to solving the tricky find problem
> 11046	another reason to make things a bit more modular
> 17306	arguable
> 18628	another good point
> 
> >*My* beef with the current setup is this: I shut down my PC when I'm
> >not at home (most of the day) and asleep (a depressingly small
> >fraction of the night). As I do, however, usually make it to bed
> >before 3AM, /etc/{daily,weekly} never run.
> 
> you are a good boy.  i'm frequently still up at that point.  :)
> 
> >I've looked at FreeBSD's periodic(8), and it's rather more modular
> >and extensible than our framework. It does not, however, solve the
> >problem that I'm referring to. Slackware Linux does something
> >similar to FreeBSD, just not quite as well.
> >
> >The two solutions that I've seen are anacron (anacron.sf.net) and
> >a limited bash-script clone of anacron in Gentoo Linux. Basically,
> >for each periodic job, a timestamp is kept under /var. Several times
> >an hour, a script is run that checks the age of each timestamp
> >relative to the period of the corresponding job.
> 
> that's interesting, but i suspect it would end up irritating more
> people that it helped.  speaking for myself, i don't want a job
> kicking off in the middle of me doing something just because its
> gotten fed up with not being allowed to run for too long.

Regarding the system bieng in use when the daily/weekly are run: It's 
not ideal for me, either, but I'd rather they happened then than not 
at all. (And besides, naughty boy, if you stay up that late, the 3AM 
job will kick in whether you like it or not ;-).

And as to the motivation of the job, I can't imagine a better reason
for running a daily/weekly job than it bieng "fed up with not bieng
allowed to run for too long." The whole point of scheduled jobs is
that they run on schedule.

Seriously, though, your point about timing and system usage is well 
taken; but it is not easily solved. Time constraints are hard to
express in an intuitive and easily parsed way. Load restraints are
not terribly useful, because it is the load after the job has started
that you are trying to avoid.

> >Is there any consensus as to what would be required/unacceptable
> >in a new periodic implementation? I have lots of ideas and some
> >running code, but I want to get some input on this before I go
> >any further.
> 
> modular would be good.  being able to use the old config files as well
> as a new one (that listed when everything was to be run) would be
> good, too.  another aspect of modular would be the ability to run an
> arbitrary piece of it at any time (for example, sometimes i want to
> update my locate database in the middle of the afternoon because i'm
> fed up with all those erroneous results from stuff i know i just
> deleted).

I could do this all pretty easily with small changes to FreeBSD's 
periodic(8). Perhaps the best approach would be to port that 
framework, add in rcorder magic, and fix it to be backward compatible
with /etc/*.conf. 

As neither the jobs nor periodic care about how recently they've been 
run (or under what conditions), the problem of scheduling jobs on
machines with irregular uptime (or which are currently on batteries,
or whatever...) can then be separated out and solved/argued over at 
some other time (no pun intended).

Assuming someone were to implement a timestamp shell script, I would 
be more than happy if it were included in share/examples along with
a crontab that replicated the default configuration, using timestamps
rather than an absolute time. It would then be possible to switch
with two shell commands.

> unless you're dead set against it, perhaps you might want to set up
> daily to run out of cron @reboot as well?  or instead?

I wasn't aware of those cron knobs. The @reboot thing would be a
good workaround, at least for myself. Thanks :-)

P.S. Sorry about the subject line...
-- 
Bruce J.A. Nourish <bjan+public@bjan.net> http://bjan.net