Re: [GsoC] Add pkgsrc support to packagekit

[Sylvain Mora <sylvain.mora%googlemail.com@localhost>]

Le 4 avr. 2011 à 11:59, Julio Merino a écrit :

> On Wed, Mar 30, 2011 at 5:48 PM, Sylvain Mora
> <sylvain.mora%googlemail.com@localhost> wrote:
>> Hi,
>> 
>> Thanks you for your response.
>> 
>> Le 29 mars 2011 à 12:12, Julio Merino a écrit :
>>> 
>>> Having an API to interact with pkgsrc and binary packages would be
>>> wonderful.  Unfortunately, Python is not part of NetBSD.  If you write
>>> the library in Python, it means that we cannot use it in the places
>>> where we need it most: in sysinst or in pkg_install so that an
>>> out-of-the-box NetBSD installation could easily retrieve/install new
>>> packages and keep the system up to date.
>>> 
>>> Having said that, a Python API would work nicely as a prototype.  But
>>> you should investigate how this would play with the languages that are
>>> part of NetBSD (C, C++, shell and Lua).
>> 
>> The goal is not the create a full API for all the system, but a basic API to 
>> help to the creation of the backend. It is a layer which retrieves pkgsrc 
>> commands stream.
>>  It could be a good idea to create an API for C pkgsrc but I think it is for 
>> an other project.
> 
> Sorry, but I still don't understand where the need to use Python comes
> from.  Is PackageKit written in Python?  If not, how does writing this
> intermediate API in Python help the goal of adding pkgsrc support to
> PackageKit?
> 
> As mentioned, writing this API in C would be very useful, both for the
> PackageKit project and for NetBSD in general.  If we had such API,
> we'd (I think) very easily integrate pkgsrc support in sysinst, for
> example :-)  (In this case, using Lua would make sense though, because
> Lua is supported by NetBSD.)
> 
> (Given that the major goal of your project is to add PackageKit
> support, this API needn't be complete: such API only needs to provide
> the set of features required to implement the PackageKit backend.  But
> working on an API that can later be easily extended would be a nice
> idea.)

Python is not mandatory. 
I propose Python because I'm writing my API for Pkgin with it. I have the habit 
to manipulate string with Python and it's easier to make test, but it's not a 
problem to use C. 
C and Python are both my favourite languages.
Moreover if I have not to write a complete API for pkgsrc, it's another good 
point for using C.


>>>> Reading the Wiki I think I can make the same work with pkgsrc :
>>>> - Write an API for pkgsrc
>>>> - Use this API to write a backend
>>>> 
>>>> It may be possible to reuse part of the pkgin API for binary package 
>>>> management (or rewrite something equivalent for pkg_add).
>>>> 
>>>> The API will reproduce basics pkgin/pkg_add/pkgsrc commands and add useful 
>>>> methods like listing all outdated packages, listing installed packages in 
>>>> which package names, versions, etc are separate.
>>> 
>>> In my opinion, the library should seamlessly manage packages from
>>> pkgsrc and binary packages (this is quite tricky but certainly
>>> doable).  An example: install/update everything from binaries if they
>>> are up-to-date, or otherwise fall back to building from pkgsrc.
>> 
>> The wiki says that the user have the choice to fall back, but I'm agree with 
>> you. May be it's possible to make the choice with an option :
>> - automatically install with pkgsrc if the package is not available as binary
>> - ask to the user if he want to install the package with pkgsrc
>> - do nothing and say to the user how he can install the package
> 
> That sounds good.  A thing to keep in mind: when you say "ask", that
> should not be an interactive question.  The user must be able to
> pre-configure behavior.  Interactive questions during package
> installations are pretty annoying, specially on slow machines!

Totally agree with that.

> 
>>>> The harder part will be the capability to build package from source 
>>>> because I have to reproduce what did "make" in Python for retrieving 
>>>> dependencies etc (The backend allows to make package if it is not 
>>>> available as binary)
>>> 
>>> For this, you can just resort to calling make and parsing the results;
>>> this is slow but effective.  Alternatively, if the "summary" database
>>> exists, just parse that database to get much faster results; see
>>> pkg_summary(5).
>> 
>> Yes, for now I have two choice: parse the "make result" stream, or search by 
>> my own way into package files to retrieve deps.
> 
> You do not need to scan package files to retrieve dependencies.  Take
> a look at pkg_summary(5): this information is already available in a
> database that you can query efficiently.  This is most useful when
> retrieving dependencies from remote servers.  If the summary is not
> available, then yes, you can (and should!) scan package files
> directly.

Awaiting your reply, I have read information about "pkg_summary", and it is a 
mine of information about each package. So I guess I can't
ignore pkg_summary in the project.

> 
>>> Lastly, I'd recommend to extend your proposal a bit more by including
>>> a list of deliverables, a preliminary schedule with a set of major
>>> milestones, and more examples on how your proposed API would look
>>> like.
>> 
>> I expect to make that very clear in my head. Really know what you expect the 
>> project.
>> But if you really want a schedule, I can make an effort to do this very 
>> quickly.
> 
> Preparing an schedule will help you plan the project in your head
> clearly.  It will also let us see that the project has a rough plan
> and a set of milestones.  This is a very important thing to have, as
> it gives more value to your project proposal... and a detailed
> schedule means that your project has more chances of succeeding in the
> allowed time frame.

Okay, I began to make a schedule. I will finalize it during week end. It's 
difficult to determine the time
that I will pass in each part. I often change the values :)

>> For the API I don't know what type of example you want.
> 
> I think it'd be nice if you'd list the kind of operations (i.e. the
> public interface) you expect this API to support.  For example, start
> by detailing what exactly does PackageKit need for its backend, and
> then list how these PackageKit operations should translate to pkgsrc
> operations.  The more you try to define a specific API, the more you
> will realize things that you didn't realize in advance.

Backends do not have to be complete. If there are a missing function, 
Packagekit do not propose the action to the user.
I will make a sort of table with all PackageKit functions, hypotheticals API 
functions and the pkgsrc operations. 
It will be complete in the same time as the schedule.

Best regards,
Sylvain