Re: GSoC 2011 project proposal [Add kqueue support to GIO]

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Sun, Apr 3, 2011 at 12:35 PM, Dmitry Matveev 
<me%dmitrymatveev.co.uk@localhost> wrote:
> Hi Julio,
>
> Thank your for your feedback! And sorry for a delayed answer.
>
>> As mentioned above, this does not only help the desktop; it helps
>> servers as well.  You may want to do a bit more research in this area
>> and explain your findings in your proposal, because it will affect
>> your design decisions.
>
> Sure, file system notifications are important on servers as well. As
> you have already mentioned, it is used by Mediatomb; it is also used
> by some administrative tools like lsyncd, iwatch and probably Samba.
>
> This aspect imposes some constraints on the kqueue backend and the
> emulation layer - it should be more fast and more reliable.
>
>>>  1. A set of patches for GIO;
>>
>> Try to elaborate a bit on how you plan to integrate these upstream
>
> It would be perfect if the original GIO developers will accept the
> final patches to GIO's upstream :) I think we should contact them
> before start.

Definitely.  I'm sure they won't have objections to implementing this,
but contacting them in advance is a good idea.  Another thing to keep
in mind is to see what progress has FreeBSD done in this area.  They
often do much more work in the desktop front, and it wouldn't surprise
me if they have some implementation of this already.

> In the alternative case it is quite possible to have
> NetBSD's own GIO branch for pkgsrc and to keep it in sync with the
> original repository.

Uh... yes.  We can (and do) keep custom patches in pkgsrc.
Unfortunately, maintaining big patches like the one we are discussing
here is a big PITA.  Things get easily out of sync with upstream, as
the upstream maintainers have no incentive in keeping our custom
changes working.

>>>  2. [probably] Userspace inotify-over-kqueue library sources;
>>
>> Elaborate on how you think this library should be distributed.  I.e.
>> as a separate software package, or bundled right into NetBSD.  Each
>> approach has its advantages/disadvantages, and you should explain what
>> these are in the proposal.
>
> Distributing inotify emulation library as a separate package would be
> flexible - it makes this library optional and users may choose, to
> install it or not to install. But (for me) this approach has a general
> disadvantage: it is a library, it is a dependency package and thus
> all the packages that depend on it should be updated to include
> this dependency. I think it would be simplier and straightforward to
> include the emulation into the base system - I stand for simplicity :)

As other people have mentioned, distributing it as a separate package
means that other BSD systems could easily benefit from it.  Keep in
mind that this does not preclude NetBSD from importing this library
into the base system (see, e.g. atf).

Also, this kind of "compatibility libraries" can be transparently
managed by pkgsrc.  pkgsrc is able to, e.g. install this library iff
the system does not support inotify, and when the library is
installed, make the packages use it instead of the native inotify API.
 We do this for other compat stuff, such as
pkgsrc/pkgtools/libnbcompat.

[...]
> Determining the possibility of extending kqueue with this additional
> functionality requires a deeper analysis, I have planned it on the
> community bonding period.

Agreed.  The analysis above is pretty good; include it in your proposal!

> Julio, may I already submit the updated proposal on the official GSoC
> portal?

Sure, don't block on me for submitting your proposal.  During past
years, it was possible to submit an application multiple times to do
corrections/extensions; I guess this is still the case this year,
isn't it?

Thanks,

-- 
Julio Merino / @jmmv