GSoC 2011 project proposal [Add kqueue support to GIO]

[Dmitry Matveev <me%dmitrymatveev.co.uk@localhost>]

Hello everybody!

I am a 5th year student of Nizhny Novgorod State Technical University,
Russia.

Please excuse me for CCing tech-kern@ list in this message, I have
asked some questions regarding the project in tech-pkg@ but there was
no answer.

Here is my proposal for "Add kqueue support to GIO" GSoC project. Any
comments are welcome.


What is the goal of the project?
--------------------------------

The goal of the project is to add kqueue support to GIO. GIO, the Glib
IO layer, currently uses inotify interface to monitor changes in the
filesystem, so it does work on Linux and does not work on NetBSD. The
importance of the file monitoring grows with the evolution of modern
desktop environments. Semantic desktops, local file indexers and even
usual file managers, such as Nautilus, use it to provide an updated
information to user. GIO is used in many applications, and providing
a full-featured file monitoring to it will significantly increase
the end-user convenience as well as application compatibility.

If the time will permit (I assume it will), it would be also possible
to implement an userspace library that will emulate the inotify
interface on the top of kqueue. It will help applications that do not
use GIO but use inotify directly.


What will be the deliverables of the project?
---------------------------------------------

In the form in which I see the project, the final deliverables are
expected to be:
   1. A set of patches for GIO;
   2. [probably] Userspace inotify-over-kqueue library sources;
   3. Documentation files describing general design decisions;
   4. A set of sample programs to ensure that is the introduced
      functionality does work (tests).


Milestones and schedule
-----------------------

1. Community bonding period
   * Getting familiar with GIO file monitoring implementation
   * Researching the way to isolate it from inotify
   * Probably studying kqueue/inotify implementations on the kernel
     side
   * Discussing the ways of implementation with mentor.

2. May 23 - June 23: the core work
   * Isolating GIO from inotify with a compile-time abstration level
   * Implementing the kqueue backend

3. June 24 - July 10: wrap up for this milestone
   * Reviews
   * Bug fixes
   * Tests

4. July 11 - August 1: the rest of the work
   * Implementing the user-space inotify interface over kqueue

5. August 2 - August 7: offline
   * Probably I will miss this days

6. August 8 - August 22: final stage
   * Reviews
   * Bug fixes
   * Tests
   * Delivery


Some implementation notes
-------------------------

Obviously my project will use kqueue(). The abstraction layer for GIO
will be implemented as a compile-time Strategy pattern.

There is a tricky moment with inotify-over-kqueue emulation. The
problem is that inotify provides some types of notifications that
kqueue does not - i.e. IN_MOVED_FROM/IN_MOVED_TO with the additional
information and so on. For it I am going to use the standard file
system routines such as readdir(), stat() and so on.

However the frequent usage of these system calls will result in a
negative impact on the performance, so I consider the possibility to
extend kqueue to support additional events. This moment (as well as
others :)) is open for discussion.


About me
--------
I use NetBSD for an year, it is the primary OS on my home laptop and
I am completely happy with it.

I have been working as a VoIP programmer since 2008 and left it in
2010. It is 2+ years of strong enterprise programming experience on
C, C++ for embedded platforms, such as VxWorks and WinMobile. We have
used SVN for source control, currently I am enjoying git for it.

Besides C and C++, I have studied Smalltalk and Common Lisp for fun.
I think that it allows to see a problem under different points of view
and pick a solution in the most appropriate way with any language. It
is always good.

In 2010-2011 I have completed a course of Linux system programming by
Auriga, where we have studied Linux kernel basics and general OS
development concepts. Currently I am continuing studying this sphere
by myself.

At the same time I have freelanced some projects on oDesk, you can
find some positive feedback here:

    http://www.odesk.com/users/~~31f3b99a3b8e3e81

Regarding my NetBSD experience, I have been working on GNU Smalltalk
to make it work well on NetBSD. Within this activity I have discovered
a bug in the NetBSD kernel and proposed a fix:

    http://mail-index.netbsd.org/tech-net/2011/02/17/msg002457.html

Also recently I have started a tech blog and NetBSD is the one of the
major topics there:

    http://dmitrymatveev.co.uk/blog?tag=netbsd

With best regards,
Dmitry