Subject: NetBSD and the Google "Summer of Code" 2006 Summary
To: None <netbsd-announce@NetBSD.org>
From: Hubert Feyrer <hubertf@NetBSD.org>
List: netbsd-announce
Date: 09/27/2006 14:04:56
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
* NetBSD and the Google "Summer of Code" 2006 Summary
This report is about the results of NetBSD's participation in Google's
"Summer of Code" 2006. After NetBSD was already part of the first GSoC
last year, we had another chance to introduce a number of students to
our operating system and get them paid by Google to work on projects
defined by the NetBSD developers.
The NetBSD Foundation's Communication executive comittee with the lead of
Jan Schaumann and Hubert Feyrer took steering the event again, including
organizing descriptions of possible projects, spawning discussions on the
NetBSD mailing lists, reviewing of more than one hundred project
proposals, assigning of mentoring NetBSD developers, setting up
communication and development infrastructure on SourceForce as well as
acting as a point of contact and communication for Google.
The projects and the students working on them were mentored by various
NetBSD developers -- see below for for a list of projects that we were
able to start, and the results stemming from the students' and mentors'
efforts:
1. Project jffs: Support for journaling for FFS
Developing student: Kirill Kuvaldin <Kirill.Kuvaldin@gmail.com>
Mentoring NetBSD developer: Manuel Bouyer <bouyer@NetBSD.org>
Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>
Summary and goal: The lack of journaling in the Berkeley Fast File
System (FFS) and the result of lengthy file system checks if the
system crashes is one of the less desirable features in NetBSD. As a
result, high hopes were set into this project to add journalling to
FFS to allow fast system startup in case of crashes. The project
proposal was to do journaling on a file operations level, and replay
the various operations in case of a crash.
Results: Let's tell the bad news first - this project didn't finish.
After an initial design phase with some discussion about what the pros
and cons of the various kinds of journaling are at about midterm, the
student got the midterm payment, and further emails were not replied
to. The project did not result in any code at all. Despite efforts of
the mentoring NetBSD developers, only a rough outline of the intended
work was proposed on the NetBSD mailing lists, with no further
response or commits to the SourceForge CVS repository.
The lesson to learn from this project from the NetBSD side is to offer
as much of a detailed design plan as possible, to make it easier for
students to decide if they can implement a given proposal, and to be
less generous about offering midterm payment if we're not really
confident a project is enough on-track to reach the final(!) goal.
2. Project mips64: Support for MIPS64 ISA
Developing student: LIU Qi <liuqi.sf@gmail.com>
Mentoring NetBSD developer: Garrett D'Amore <gdamore@NetBSD.org>
Mentoring NetBSD developer: Simon Burge <simonb@NetBSD.org>
Summary and goal: NetBSD already runs on a number of 32- and 64bit
machines, but on the MIPS platform it only supports 32bit machines so
far. The goal of this project was to change this, and work on
toolchain, cache, virtual memory and related userland changes that are
necessary to get NetBSD working on 64bit MIPS machines.
Results: Unfortunately this is the second (of two) project that failed
this summer -- the student had to go to hospital in Beijing during the
first half of the project, and the code written by the student at the
start of the project never got committed to the SourceForge CVS
repository.
Six weeks after the end of the Summer of Code, the student was
released from hospital and told us that he intends to work on the
project. Given that the primary mentor moved on to other projects, a
shift in the mentoring NetBSD developer will happen, and we are
looking forward for progress.
3. Project ppcg5: PowerPC G5 support in NetBSD
Developing student: Yevgeny Binder <yevbee@comcast.net>
Mentoring NetBSD developer: Allen Briggs <briggs@NetBSD.org>
Mentoring NetBSD developer: Garrett D'Amore <gdamore@NetBSD.org>
Summary and goal: The goal of this project was to also add more
hardware support for NetBSD, in this case the target was the G5
PowerPC CPU and peripheral hardware as found in Apple iMac G5,
PowerMac G5 and Xserve machines. The original scope of the project as
proposed by the student and agreed on by the NetBSD mentors was
changed when, right at the start of the project, someone posted that
they had basically done this project out of the sights of the open
source community and was ready to contribute it. As a result, the
project's goals were adjusted to still offer a win for NetBSD and
meeting the student's goals for learning.
Results: After the initial change in direction, the project worked
together with the party that did the initial port, and work was done
do move from serial console to an OpenFirmware based console,
debugging and optimisation in the cache and memory mapping code as
well as in the serial driver to integrate it with NetBSD's
machine-independent code framework. Yevgeny is currently in the
process of getting a NetBSD developer account (AKA "a commit bit") to
integrate his Summer-of-Code projects -- both from this year and from
last year, to add HFS+ support and be able to boot from a Mac
partition.
4. Project congest: Improved Writing to Filesystem Using Congestion
Control
Developing student: Sumantra R. Kundu <sumantra@gmail.com>
Mentoring NetBSD developer: Bill Studenmund <wrstuden@NetBSD.org>
Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>
Summary and goal: In a multiuser/multitasking environment, many
processes can issue read/write to a single disk at once, reaching a
point where the underlying hardware cannot service requests fast
enough and resulting in convestion. Effects of such congestion are
unequal sharing of disk bandwidth between processes as well as
unpredictable read and write delays. The goal of this project was to
introduce an interface to the NetBSD virtual-memory system (UVM) that
detects congestion and prevents it, as well as benchmarking the system
and providing documentation.
Results: There are several possible approaches to the problem of
detecting and preventing congestion, and the approach of observing a
process' write behaviour and acting upon it was implemented within
NetBSD's UVM system, and first benchmarks showed that that penalty of
multiple writers was reduced from 42% to 15%. For the immediate
future, the project will keep a status of "work in progress" to get
better understanding of the dynamics of the page daemon and UVM, and
to serve as framework for further research on how to detect and
prevent contestion.
5. Project ecn: TCP ECN support
Developing student: Rui Paulo <rpaulo@gmail.com>
Mentoring NetBSD developer: Kentaro A. Kurahone <kurahone@NetBSD.org>
Mentoring NetBSD developer: Allen Briggs <briggs@NetBSD.org>
Mentoring NetBSD developer: Matt Thomas <matt@NetBSD.org>
Summary and goal: Explicit Congestion Notification as defined in RFC
3168 is one of several TCP congestion control algorithm to make
delay-sensitive low-traffic applications perform smoother. It works by
having routers indicating possible upcoming congestion situations,
which allows to avoid dropped packets by having the sender adjusting
the transmission window size.
Results: A working end-host ECN implementation was delivered by the
student, along with test results that show the improved timing
behaviour. The work was integrated into NetBSD already and will be
part of NetBSD 5.0.
6. Project ipsec6: Fast_ipsec and ipv6
Developing student: Degroote Arnaud <alwe.maindargent@gmail.com>
Mentoring NetBSD developer: Sam Leffler <sam@NetBSD.org>
Mentoring NetBSD developer: Christos Zoulas <christos@NetBSD.org>
Mentoring NetBSD developer: Thor Lancelot Simon <tls@NetBSD.org>
Summary and goal: NetBSD currently has two IPsec implementations,
ipsec(4) that came with KAME's IPv6 implementation but that lacks
support for crypto hardware, and fast_ipsec(4) which supports crypto
hardware but lacks support for IPv6. The goal of this project was to
add IPv6 support to fast_ipsec(4).
Results: Both tunnel and transport mode for authentication (AH) and
encrypted security payload (ESP) over IPv6 were added, compressed
transport (IPcomp) was fixed to work with both IPv4 and IPv6, and TCP
MD5 checksums work. We plan to integrate the work into NetBSD as soon
as possible.
7. Project pkg_install: pkg_install rewrite for pkgsrc
Developing student: Joerg Sonnenberger <joerg@bec.de>
Mentoring NetBSD developer: Dieter Baron <dillo@NetBSD.org>
Mentoring NetBSD developer: Johnny C. Lam <jlam@NetBSD.org>
Mentoring NetBSD developer: Alistair Crooks <agc@NetBSD.org>
Mentoring NetBSD developer: Thomas Klausner <wiz@NetBSD.org>
Summary and goal: The core tools of pkgsrc, the NetBSD Packages
Collection, have gone through a number of updates and maintenance
cycles with added features on many sides, which led to the current
state of the tools. The goal of this project is to re-collect all the
requirements and perform a complete redesign and rewrite plus
documentation.
Results: The student working on this project had previous experience
from working on pkgsrc for NetBSD and DragonflyBSD, and it seems that
the routine maintenance work he's conducting on these platforms turned
out as a challenge for this project, keeping all the balls in the air
and still proceeding. The results include a design plan with a new
scheme for wildcard handling, partial code to add and create binary
packages, but also a number of unresolved areas in the area of package
registration that need further work. Given Joerg's involvement in the
pkgsrc project we are looking forward to see him persue his ideas
further.
The overall impression from this project was, similar to that of the
Journaling for FFS (jffs) project, that too much time was needed to
clean up or specify the design, which left too little time for
implementation, testing and writing documentation.
8. Project mbuf: Improving the mbuf API and implementation
Developing student: Pavel Cahyna <pavel.cahyna@gmail.com>
Mentoring NetBSD developer: Martin Husemann <martin@NetBSD.org>
Mentoring NetBSD developer: Matt Thomas <matt@NetBSD.org>
Summary and goal: The mbuf functions and macros provide an easy and
consistent way to handle a networking stack's memory management needs.
A drawback of the current API is that it's easy to write code with
incorrect assumptions, which will work in most but not all cases,
which is leading to situations that are very hard to debug. Goal of
the project was to improve the existing mbuf API, but in a way that
was explicitely not adding new user-visible features, but to improve
the current implementation and make the network stack more reliable.
Results: The mtod() macro was replaced by new mbuf accessor routines
that encourage correct handing of noncontiguous mbuf chains and
read-only mbuf storage and code correctness. Handling of read-only
storage was implemented by using the "const" qualifier, using the
compiler-provided checking. A start was made to convert existing code
to the new API and constifying it in the process. As a result, the
input path of IPv4 and IPv6 protocols is now safe with respect to
read-only mbuf storage, another side-effect of this work could be
zero-copy I/O in the NetBSD/Xen backend driver. Legacy protocols such
as XNS, ISO or Appletalk were left as future work. These could also
benefit from the documentation written during the project and
published on the project's homepage, currently hosted on SourceForge.
The project is expected to be soon integrated to the NetBSD mainline.
In summary, the results of this year's Google "Summer of Code" were a
success for the NetBSD project as well as for the participating students.
Even with two failed projects, the work achieved during this summer will
have a noticable impact on the NetBSD project, its code structure,
networking and platform support, even if they are not directly user
visible in all cases.
Things we have learned as mentoring project is that we have to invest more
work into details of project proposals, outlining design choices and
offering decisions that would possibly delay projects if left open.
Benefits for our (prospective) students and developers will be clearer
guidance and more time for actual implementation, testing, debugging and
last but not least documentation.
Another experience was that having a direct, interactive, and informal
form of contact with the student -- and using it to just say, "how's it
going?" if a mentor hadn't heard anything from a student for a couple of
days -- made a big difference. It gives the student a chance to ask
questions, explain the current issues, and for the mentor to work through
some of the issues with the student in a fairly interactive fashion. Some
of our mentors and student used AOL IM (via Adium and GAIM), which was
sufficient.
Finally, we would like to thank Google for offering us this great
opportunity and support, and Google's Leslie Hawthorn, Chris DiBona and
Greg Stein as well as their whole support team working behind the scenes,
for their organization and encouragement, all of the NetBSD developers who
volunteered to act as mentors during the summer, and also the students who
invested a lot of sweat blood and tears into their projects, and NetBSD.
Thank you!
- Hubert Feyrer
The NetBSD Foundation
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (NetBSD)
iD8DBQFFGmZx9PcicYLJuikRAukxAKCidlVBtVdswMQqfZfGMKLu6kNl2wCdF/4B
1WQeYgVf/8FWTdj7Al83Y/8=
=uD6i
-----END PGP SIGNATURE-----