Subject: [SoC] pkg_install rewrite
To: None <>
From: None <>
List: tech-pkg
Date: 05/07/2006 16:11:16
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Hi all,
attached is the proposal for the SoC rewrite proposal. I'll soon add a
follow up with more detailed information as discussed during pkgsrcCon.


Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="Proposal-0.2.txt"

Name: Joerg Sonnenberger


The pkgsrc project is currently using packaging tools derived from
FreeBSD's pkg_install. Since it is the backbone of the framework, any
difficulties to the maintainer can hinder the ability of pkgsrc to
blossom and expand.  

pkg_install in the current incarnation consists of tools to create,
register, install and remove packages. It also contains tools to query
and alter the state of the pkgsrc database. Frontends for package management
have to parse the output of the tools because no direct library interface

There's a serious lack of documentation for both the metadata
installed and the format of a binary package. The current code base was
not designed or built with security in mind. For example, it uses the
system() call throughout the code base. The existing code base is also
extremely difficult to audit.

The project

I hope to re-write pkg_install from scratch with a different set of
technical requirements as learned from daily interaction with pkgsrc.

The re-write should feature the following requirements:
- security conscious design/coding
- detailed documentation of all features
- versioned metadata that allows extensions to be added later without
having to re-build all packages in the system
- a modular architecture that helps writing frontend programs and
different access methods with minimal effort required
- sharing of code fragments for runtime management e.g. of fonts, info
files, configure files etc.

Time frame

First Part:
- Design of the file format and initial metadata set
- Design of the library API
- Design of the Command Line Interface
- Implementation of conformance tests
- Implementation of the unit tests for the library and CLI

1.5 months

Second Part:
- Implementation of the library interface
- Implementation of the Command Line Interface
- Refinement of the interfaces and tests based on feedback

1 month

Third Part:
- Implementation of conversion tools from old metadata and old
binary packages to the new formats
- Initial integration into the pkgsrc infrastructure

0.5 month

Experience related to the project

I have been working on pkgsrc since the Summer of 2005, gaining developer
status with commit rights in October. My work mostly focuses on
improving the DragonFly support in pkgsrc, which resulted in the
decision of the project to support pkgsrc as the official way to
manage third party software.

Due to this work, I became familiar with many packages and the
advantages and limitations of the current infrastructure. Before using
pkgsrc I had used a number of other packaging systems to varying
degree including Debian, Gentoo, SuSE, FreeBSD ports and OpenBSD

Prior experience and Open Source development

I have extensively designed programs in C, sh, PHP and Python. I am
reasonable familiar with Lisp, C++, Pascal and awk.

I have worked in various areas of the DragonFly Operating System,
covering the toolchain, userland programs, libraries and the system
kernel. In addition I have been writing and porting software to Linux,
AIX and other Unix-like systems.

During various summer jobs I have successfully implemented two online
shopping systems. This included using extensive database programming
skills and various system administration.

Academic CV

In 2002 I successfully finished my general education, gaining excellent
marks in all subjects. Soon afterwards I began studying Mathematics for a

Summer of Code 2005

For Summer of Code 2005 I proposed to implement a better 'bulk build'
framework for pkgsrc. Unfortunately, my application was rejected. This
project is still planned, however the requirements have changed to a
large degree due to gaining first hand experience from using the pkgsrc
framework on a daily basis.