-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

  * NetBSD and the Google "Summer of Code" 2006 Summary

This report is about the results of NetBSD's participation in Google's
"Summer of Code" 2006. After NetBSD was already part of the first GSoC
last year, we had another chance to introduce a number of students to
our operating system and get them paid by Google to work on projects
defined by the NetBSD developers.

The NetBSD Foundation's Communication executive comittee with the lead of
Jan Schaumann and Hubert Feyrer took steering the event again, including
organizing descriptions of possible projects, spawning discussions on the
NetBSD mailing lists, reviewing of more than one hundred project
proposals, assigning of mentoring NetBSD developers, setting up
communication and development infrastructure on SourceForce as well as
acting as a point of contact and communication for Google.

The projects and the students working on them were mentored by various
NetBSD developers -- see below for for a list of projects that we were
able to start, and the results stemming from the students' and mentors'
efforts:

  1. Project jffs: Support for journaling for FFS

     Developing student: Kirill Kuvaldin <Kirill.Kuvaldin@gmail.com>
     Mentoring NetBSD developer: Manuel Bouyer <bouyer@NetBSD.org>
     Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>

     Summary and goal: The lack of journaling in the Berkeley Fast File
     System (FFS) and the result of lengthy file system checks if the
     system crashes is one of the less desirable features in NetBSD. As a
     result, high hopes were set into this project to add journalling to
     FFS to allow fast system startup in case of crashes. The project
     proposal was to do journaling on a file operations level, and replay
     the various operations in case of a crash.

     Results: Let's tell the bad news first - this project didn't finish.
     After an initial design phase with some discussion about what the pros
     and cons of the various kinds of journaling are at about midterm, the
     student got the midterm payment, and further emails were not replied
     to. The project did not result in any code at all. Despite efforts of
     the mentoring NetBSD developers, only a rough outline of the intended
     work was proposed on the NetBSD mailing lists, with no further
     response or commits to the SourceForge CVS repository.

     The lesson to learn from this project from the NetBSD side is to offer
     as much of a detailed design plan as possible, to make it easier for
     students to decide if they can implement a given proposal, and to be
     less generous about offering midterm payment if we're not really
     confident a project is enough on-track to reach the final(!) goal.

  2. Project mips64: Support for MIPS64 ISA

     Developing student: LIU Qi <liuqi.sf@gmail.com>
     Mentoring NetBSD developer: Garrett D'Amore <gdamore@NetBSD.org>
     Mentoring NetBSD developer: Simon Burge <simonb@NetBSD.org>

     Summary and goal: NetBSD already runs on a number of 32- and 64bit
     machines, but on the MIPS platform it only supports 32bit machines so
     far. The goal of this project was to change this, and work on
     toolchain, cache, virtual memory and related userland changes that are
     necessary to get NetBSD working on 64bit MIPS machines.

     Results: Unfortunately this is the second (of two) project that failed
     this summer -- the student had to go to hospital in Beijing during the
     first half of the project, and the code written by the student at the
     start of the project never got committed to the SourceForge CVS
     repository.

     Six weeks after the end of the Summer of Code, the student was
     released from hospital and told us that he intends to work on the
     project. Given that the primary mentor moved on to other projects, a
     shift in the mentoring NetBSD developer will happen, and we are
     looking forward for progress.

  3. Project ppcg5: PowerPC G5 support in NetBSD

     Developing student: Yevgeny Binder <yevbee@comcast.net>
     Mentoring NetBSD developer: Allen Briggs <briggs@NetBSD.org>
     Mentoring NetBSD developer: Garrett D'Amore <gdamore@NetBSD.org>

     Summary and goal: The goal of this project was to also add more
     hardware support for NetBSD, in this case the target was the G5
     PowerPC CPU and peripheral hardware as found in Apple iMac G5,
     PowerMac G5 and Xserve machines. The original scope of the project as
     proposed by the student and agreed on by the NetBSD mentors was
     changed when, right at the start of the project, someone posted that
     they had basically done this project out of the sights of the open
     source community and was ready to contribute it. As a result, the
     project's goals were adjusted to still offer a win for NetBSD and
     meeting the student's goals for learning.

     Results: After the initial change in direction, the project worked
     together with the party that did the initial port, and work was done
     do move from serial console to an OpenFirmware based console,
     debugging and optimisation in the cache and memory mapping code as
     well as in the serial driver to integrate it with NetBSD's
     machine-independent code framework. Yevgeny is currently in the
     process of getting a NetBSD developer account (AKA "a commit bit") to
     integrate his Summer-of-Code projects -- both from this year and from
     last year, to add HFS+ support and be able to boot from a Mac
     partition.

  4. Project congest: Improved Writing to Filesystem Using Congestion
     Control

     Developing student: Sumantra R. Kundu <sumantra@gmail.com>
     Mentoring NetBSD developer: Bill Studenmund <wrstuden@NetBSD.org>
     Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>

     Summary and goal: In a multiuser/multitasking environment, many
     processes can issue read/write to a single disk at once, reaching a
     point where the underlying hardware cannot service requests fast
     enough and resulting in convestion. Effects of such congestion are
     unequal sharing of disk bandwidth between processes as well as
     unpredictable read and write delays. The goal of this project was to
     introduce an interface to the NetBSD virtual-memory system (UVM) that
     detects congestion and prevents it, as well as benchmarking the system
     and providing documentation.

     Results: There are several possible approaches to the problem of
     detecting and preventing congestion, and the approach of observing a
     process' write behaviour and acting upon it was implemented within
     NetBSD's UVM system, and first benchmarks showed that that penalty of
     multiple writers was reduced from 42% to 15%. For the immediate
     future, the project will keep a status of "work in progress" to get
     better understanding of the dynamics of the page daemon and UVM, and
     to serve as framework for further research on how to detect and
     prevent contestion.

  5. Project ecn: TCP ECN support

     Developing student: Rui Paulo <rpaulo@gmail.com>
     Mentoring NetBSD developer: Kentaro A. Kurahone <kurahone@NetBSD.org>
     Mentoring NetBSD developer: Allen Briggs <briggs@NetBSD.org>
     Mentoring NetBSD developer: Matt Thomas <matt@NetBSD.org>

     Summary and goal: Explicit Congestion Notification as defined in RFC
     3168 is one of several TCP congestion control algorithm to make
     delay-sensitive low-traffic applications perform smoother. It works by
     having routers indicating possible upcoming congestion situations,
     which allows to avoid dropped packets by having the sender adjusting
     the transmission window size.

     Results: A working end-host ECN implementation was delivered by the
     student, along with test results that show the improved timing
     behaviour. The work was integrated into NetBSD already and will be
     part of NetBSD 5.0.

  6. Project ipsec6: Fast_ipsec and ipv6

     Developing student: Degroote Arnaud <alwe.maindargent@gmail.com>
     Mentoring NetBSD developer: Sam Leffler <sam@NetBSD.org>
     Mentoring NetBSD developer: Christos Zoulas <christos@NetBSD.org>
     Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>

     Summary and goal: NetBSD currently has two IPsec implementations,
     ipsec(4) that came with KAME's IPv6 implementation but that lacks
     support for crypto hardware, and fast_ipsec(4) which supports crypto
     hardware but lacks support for IPv6. The goal of this project was to
     add IPv6 support to fast_ipsec(4).

     Results: Both tunnel and transport mode for authentication (AH) and
     encrypted security payload (ESP) over IPv6 were added, compressed
     transport (IPcomp) was fixed to work with both IPv4 and IPv6, and TCP
     MD5 checksums work. We plan to integrate the work into NetBSD as soon
     as possible.

  7. Project pkg_install: pkg_install rewrite for pkgsrc

     Developing student: Joerg Sonnenberger <joerg@bec.de>
     Mentoring NetBSD developer: Dieter Baron <dillo@NetBSD.org>
     Mentoring NetBSD developer: Johnny C. Lam <jlam@NetBSD.org>
     Mentoring NetBSD developer: Alistair Crooks <agc@NetBSD.org>
     Mentoring NetBSD developer: Thomas Klausner <wiz@NetBSD.org>

     Summary and goal: The core tools of pkgsrc, the NetBSD Packages
     Collection, have gone through a number of updates and maintenance
     cycles with added features on many sides, which led to the current
     state of the tools. The goal of this project is to re-collect all the
     requirements and perform a complete redesign and rewrite plus
     documentation.

     Results: The student working on this project had previous experience
     from working on pkgsrc for NetBSD and DragonflyBSD, and it seems that
     the routine maintenance work he's conducting on these platforms turned
     out as a challenge for this project, keeping all the balls in the air
     and still proceeding. The results include a design plan with a new
     scheme for wildcard handling, partial code to add and create binary
     packages, but also a number of unresolved areas in the area of package
     registration that need further work. Given Joerg's involvement in the
     pkgsrc project we are looking forward to see him persue his ideas
     further.

     The overall impression from this project was, similar to that of the
     Journaling for FFS (jffs) project, that too much time was needed to
     clean up or specify the design, which left too little time for
     implementation, testing and writing documentation.

  8. Project mbuf: Improving the mbuf API and implementation

     Developing student: Pavel Cahyna <pavel.cahyna@gmail.com>
     Mentoring NetBSD developer: Martin Husemann <martin@NetBSD.org>
     Mentoring NetBSD developer: Matt Thomas <matt@NetBSD.org>

     Summary and goal: The mbuf functions and macros provide an easy and
     consistent way to handle a networking stack's memory management needs.
     A drawback of the current API is that it's easy to write code with
     incorrect assumptions, which will work in most but not all cases,
     which is leading to situations that are very hard to debug. Goal of
     the project was to improve the existing mbuf API, but in a way that
     was explicitely not adding new user-visible features, but to improve
     the current implementation and make the network stack more reliable.

     Results: The mtod() macro was replaced by new mbuf accessor routines
     that encourage correct handing of noncontiguous mbuf chains and
     read-only mbuf storage and code correctness. Handling of read-only
     storage was implemented by using the "const" qualifier, using the
     compiler-provided checking. A start was made to convert existing code
     to the new API and constifying it in the process. As a result, the
     input path of IPv4 and IPv6 protocols is now safe with respect to
     read-only mbuf storage, another side-effect of this work could be
     zero-copy I/O in the NetBSD/Xen backend driver. Legacy protocols such
     as XNS, ISO or Appletalk were left as future work. These could also
     benefit from the documentation written during the project and
     published on the project's homepage, currently hosted on SourceForge.
     The project is expected to be soon integrated to the NetBSD mainline.

In summary, the results of this year's Google "Summer of Code" were a
success for the NetBSD project as well as for the participating students.
Even with two failed projects, the work achieved during this summer will
have a noticable impact on the NetBSD project, its code structure,
networking and platform support, even if they are not directly user
visible in all cases.

Things we have learned as mentoring project is that we have to invest more
work into details of project proposals, outlining design choices and
offering decisions that would possibly delay projects if left open.
Benefits for our (prospective) students and developers will be clearer
guidance and more time for actual implementation, testing, debugging and
last but not least documentation.

Another experience was that having a direct, interactive, and informal
form of contact with the student -- and using it to just say, "how's it
going?" if a mentor hadn't heard anything from a student for a couple of
days -- made a big difference. It gives the student a chance to ask
questions, explain the current issues, and for the mentor to work through
some of the issues with the student in a fairly interactive fashion. Some
of our mentors and student used AOL IM (via Adium and GAIM), which was
sufficient.

Finally, we would like to thank Google for offering us this great
opportunity and support, and Google's Leslie Hawthorn, Chris DiBona and
Greg Stein as well as their whole support team working behind the scenes,
for their organization and encouragement, all of the NetBSD developers who
volunteered to act as mentors during the summer, and also the students who
invested a lot of sweat blood and tears into their projects, and NetBSD.
Thank you!


  - Hubert Feyrer
    The NetBSD Foundation


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (NetBSD)

iD8DBQFFGmZx9PcicYLJuikRAukxAKCidlVBtVdswMQqfZfGMKLu6kNl2wCdF/4B
1WQeYgVf/8FWTdj7Al83Y/8=
=uD6i
-----END PGP SIGNATURE-----