Re: Rewrite pkg_comp for Summer of Code

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Tue, 6 Apr 2010 15:09:28 +0200
    From:        Julio Merino <jmmv84%gmail.com@localhost>
    Message-ID:  
<s2n6b2d1e191004060609ha83cb240y7c6757a7571ff85d%mail.gmail.com@localhost>

  | If you have ever used pkg_comp, you may have noticed that this is a
  | very useful tool,

I certainly agree with that, however ...

  | but the current implementation is "suboptimal"... to put it nicely.

I don't think it is nearly as bad as you're suggesting.  It has a bunch
of minor problems (as it is currently distributed) but those are mostly
easy to fix.

  | It is really hard to add new features due to the code
  | base not being modular enough

I actually disagree with that, it is quite cleanly written, and fairly
easy to add to - to which I can speak from experience, having done quite
a lot of modifications to pkg_comp code.

Almost anything can be made more modular, so almost anything can be criticized
for not being modular enough, and of course, you're free to criticize your
own code as much as you like - but personally I have no problem at all with
the overall structure of the code, it is easy to understand, and easy to
modify.

  | and, to make things worse (in my opinion), it is completely tied to NetBSD:

Yes, that it is, and if it were possible to make it work in other systems,
that would be an entirely worthwhile project.  Of course, the major issue
with this is that pkg_comp emulates (after building) a complete installation
of the host system - one of the most system dependent things that exists in
the various systems (in general, no-one cares about making their distribution
methods or formats anything like anyone else's - and really, not necessarily
even consistent from release to release, there's normally no need for that).

That's going to make it somewhat of a challenge to make this portable, at
the very least it is going to mean studying the release formats of all
potential target systems to learn how they can be extracted into a chroot
(or whatever the relevant host system has as an alternative).

  | Your goal is to rewrite pkg_comp in a real scripting language and, at

I would make that a 100% non-goal - pkg_comp is already written in the
best possible scripting language for its purpose.  If there is any function
at all for which sh(1) is ideally suited, it is for running programs
in controlled ways.  Since that is all pkg_comp does (there's no arithmetic,
no networking, no ...) sh(1) is the ideal tool for it.   What's more, it
is hard (well, unbelievable) to imagine pkgsrc ever running on a system
without a working sh - on the other hand, on many (perhaps most, and NetBSD
in particular) systems, pkgsrc is used to install every other scripting
language that the system runs (possibly except lua on some new NetBSD's,
I didn't keep up to date on where that debate ended...)   If pkgsrc were
written in any other language, there would be a chicken and egg bootstrap
problem, and pkg_comp couldn't be used for all pkgsrc operations, which it
can now, and that would be a huge nuisance.

  | the same time, make it portable.

That's always a worthwhile goal - how it can be accomplished here is
a challenge without doubt.   Note it isn't the code writing that's the
difficult part, it is the gaining of the knowledge as to how every
different system's releases need to be manipulated so they can be correctly
pseudo-installed.

  | Should you want to work on this summer project, apply right away!

If anyone decides to undertake this project, feel free to contact me, and
I'll send my current version of pkg_comp which has a lot of the little
niggles fixed and some new stuff added (but absolutely nothing related to
system portability.)

kre