tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: GSoC 2011 project proposal [Add kqueue support to GIO]

Hi Julio,

Thank your for your feedback! And sorry for a delayed answer.

As mentioned above, this does not only help the desktop; it helps
servers as well.  You may want to do a bit more research in this area
and explain your findings in your proposal, because it will affect
your design decisions.

Sure, file system notifications are important on servers as well. As
you have already mentioned, it is used by Mediatomb; it is also used
by some administrative tools like lsyncd, iwatch and probably Samba.

This aspect imposes some constraints on the kqueue backend and the
emulation layer - it should be more fast and more reliable.

  1. A set of patches for GIO;

Try to elaborate a bit on how you plan to integrate these upstream

It would be perfect if the original GIO developers will accept the
final patches to GIO's upstream :) I think we should contact them
before start. In the alternative case it is quite possible to have
NetBSD's own GIO branch for pkgsrc and to keep it in sync with the
original repository.

  2. [probably] Userspace inotify-over-kqueue library sources;

Elaborate on how you think this library should be distributed.  I.e.
as a separate software package, or bundled right into NetBSD.  Each
approach has its advantages/disadvantages, and you should explain what
these are in the proposal.

Distributing inotify emulation library as a separate package would be
flexible - it makes this library optional and users may choose, to
install it or not to install. But (for me) this approach has a general
disadvantage: it is a library, it is a dependency package and thus
all the packages that depend on it should be updated to include
this dependency. I think it would be simplier and straightforward to
include the emulation into the base system - I stand for simplicity :)

As for the compatibility library: take a look at ATF
(  It's what we are using in NetBSD
for our tests and this library should integrate seamlessly.

Great, I will study it.

Don't postpone tests until a late stage.  Develop tests as you go.
When you are writing code, you can see more easily what the tricky
code paths are. If you postpone the development of tests until a very later stage, you will still be able to write simple tests, but you may
not remember what the tricky cases are.  Also, having tests along the
way ensure that your code is easily testable and that you don't
introduce regressions early on.

Surely, I have planned to use unit tests at the implementation
stage :). At the late stage I had more generic tests in mind - testing
the introduced functionality in its consumer applications, such as

Also, in general, don't postpone reviews until late. It is nice if
you keep your mentor involved while you develop your code. Reviewing
a ton of code on the deadline is painful; reviewing the code
incrementally is much easier, and you may get some interesting
comments along the way!


Would be nice to provide a detailed description of what kqueue lacks
during the early stages of the project.  It might be the case that
it's impossible to extend kqueue to implement some important
functionality... in which case we could be looking at a different
project altogether (e.g. implement inotify in the kernel)!

I have found that kqueue does not have analogues to the following
inotify events:

1. IN_ACCESS        - File was read from
2. IN_OPEN          - File was opened
3. IN_CLOSE_WRITE   - File was closed (and was open for writing)
4. IN_CLOSE_NOWRITE - File was closed (and was not open for writing)
5. IN_MOVED_FROM    - File was moved away from a watched directory
6. IN_MOVED_TO      - File was moved into a watched directory

There is also the IN_UNMOUNT event. According to manual, kqueue has
the NOTE_REVOKE event to signal that the underlying fileystem was
unmounted, but I could not reach this behaviour. The fact that
kqueue requires an open file descriptor per entry complicates the
monitoring on removable devices.

Inotify also provides an additional information for several
notifications, i.e. for MOVED_FROM/MOVED_TO events it also supplies
a name of the file moved. Technically for kqueue this information
can be passed from kernel via (if to consider this field
as a pointer).

Determining the possibility of extending kqueue with this additional
functionality requires a deeper analysis, I have planned it on the
community bonding period.

Julio, may I already submit the updated proposal on the official GSoC

Best regards,

Home | Main Index | Thread Index | Old Index