CVS commit: [netbsd-6] src

["Martin Husemann" <martin%netbsd.org@localhost>]

Module Name:    src
Committed By:   martin
Date:           Sun Aug 12 13:13:21 UTC 2012

Modified Files:
        src/lib/libperfuse [netbsd-6]: debug.c ops.c perfuse.c perfuse_if.h
            perfuse_priv.h subr.c
        src/lib/libpuffs [netbsd-6]: dispatcher.c puffs.3 puffs.h puffs_ops.3
        src/sys/fs/puffs [netbsd-6]: puffs_msgif.c puffs_msgif.h puffs_sys.h
            puffs_vfsops.c puffs_vnops.c
        src/usr.sbin/perfused [netbsd-6]: msg.c perfused.c

Log Message:
Pull up following revision(s) (requested by manu in ticket #438):
        lib/libperfuse/perfuse_priv.h: revision 1.31
        sys/fs/puffs/puffs_msgif.h: revision 1.80
        sys/fs/puffs/puffs_vnops.c: revision 1.171
        lib/libpuffs/puffs_ops.3: revision 1.31
        sys/fs/puffs/puffs_vnops.c: revision 1.172
        sys/fs/puffs/puffs_vnops.c: revision 1.173
        sys/fs/puffs/puffs_vnops.c: revision 1.174
        usr.sbin/perfused/perfused.c: revision 1.24
        sys/fs/puffs/puffs_sys.h: revision 1.80
        sys/fs/puffs/puffs_sys.h: revision 1.81
        sys/fs/puffs/puffs_sys.h: revision 1.82
        lib/libperfuse/subr.c: revision 1.19
        lib/libperfuse/perfuse.c: revision 1.30
        sys/fs/puffs/puffs_msgif.c: revision 1.90
        sys/fs/puffs/puffs_msgif.c: revision 1.91
        sys/fs/puffs/puffs_msgif.c: revision 1.92
        lib/libperfuse/ops.c: revision 1.59
        lib/libpuffs/puffs.3: revision 1.53
        lib/libperfuse/debug.c: revision 1.12
        lib/libpuffs/puffs.3: revision 1.54
        sys/fs/puffs/puffs_vnops.c: revision 1.167
        sys/fs/puffs/puffs_msgif.h: revision 1.79
        usr.sbin/perfused/msg.c: revision 1.21
        sys/fs/puffs/puffs_vfsops.c: revision 1.102
        sys/fs/puffs/puffs_vfsops.c: revision 1.103
        sys/fs/puffs/puffs_vfsops.c: revision 1.105
        lib/libpuffs/puffs.h: revision 1.123
        lib/libperfuse/perfuse_if.h: revision 1.20
        lib/libperfuse/perfuse.c: revision 1.29
        lib/libpuffs/dispatcher.c: revision 1.42
        lib/libpuffs/dispatcher.c: revision 1.43
- Fix same vnodes associated with multiple cookies
The scheme used to retreive known nodes on lookup was flawed, as it only
used parent and name. This produced a different cookie for the same file
if it was renamed, when looking up ../ or when dealing with multiple files
associated with the same name through link(2).
We therefore abandon the use of node name and introduce hashed lists of
inodes. This causes a huge rewrite of reclaim code, which do not attempt
to keep parents allocated until all their children are reclaimed
- Fix race conditions in reclaim
There are a few situations where we issue multiple FUSE operations for
a PUFFS operation. On reclaim, we therefore have to wait for all FUSE
operation to complete, not just the current exchanges. We do this by
introducing node reference count with node_ref() and node_rele().
- Detect data loss caused by FAF
VOP_PUTPAGES causes FAF writes where the kernel does not check the
operation result. At least issue a warning on error.
- Enjoy FAF shortcut on setattr
No need to wait for the result if the kernel does not want it. There is
however an exception for setattr that touch the size, we need to wait
for completion because we have other operations queued for after the
resize.
- Fix fchmod() on write-open file
fchmod() on a node open with write privilege will send setattr with both mode
and size set. This confuses some FUSE filesystem. Therefore we send two FUSE
operations, one for mode, and one for size.
- Remove node TTL handling for netbsd-5 for simplicity sake. The code
still builds on netbsd-5 but does not have the node TTL feature anymore.
It works fine with kernel support on netbsd-6.
- Improve PUFFS_KFLAG_CACHE_FS_TTL by reclaiming older inactive nodes.
The normal kernel behavior is to retain inactive nodes in the freelist
until it runs out of vnodes. This has some merit for local filesystems,
where the cost of an allocation is about the same as the cost of a
lookup. But that situation is not true for distributed filesystems.
On the other hand, keeping inactive nodes for a long time hold memory
in the file server process, and when the kernel runs out of vnodes, it
produce reclaim avalanches that increase lattency for other operations.
We do not reclaim inactive vnodes immediatly either, as they may be
looked up again shortly. Instead we introduce a grace time and we
reclaim nodes that have been inactive beyond the grace time.
- Fix lookup/reclaim race condition.
The above improvement undercovered a race condition between lookup and
reclaim. If we reclaimed a vnode associated with a userland cookie while
a lookup returning that same cookiewas inprogress, then the kernel ends
up with a vnode associated with a cookie that has been reclaimed in
userland. Next operation on the cookie will crash (or at least confuse)
the filesystem.
We fix this by introducing a lookup count in kernel and userland. On
reclaim, the kernel sends the count, which enable userland to detect
situation where it initiated a lookup that is not completed in kernel.
In such a situation, the reclaim must be ignored, as the node is about
to be looked up again.
Fix hang unmount bug introduced by last commit.
We introduced a slow queue for delayed reclaims, while the existing
queue for unmount, flush and exist has been renamed fast queue. Both
queues had timestamp for when an operation should be done, but it was
useless for the fast queue, which is always used to run an operation
ASAP. And the timestamp test had an error that turned ASAP into "at next
tick", but nobody what there to wake the thread at next tick, hence
the hang. The fix is to remove the useless and buggy timestamp test for
fast queue.
Rename slow sopreq queue into node sopreq queue, to refet the fact that
is only intended for postponed node reclaims.
When purging the node sopreq queue, do not call puffs_msg_sendresp(), as
it makes no sense.
Fix race condition between (create|mknod|mkdir|symlino) and reclaim, just
like we did it between lookup and reclaim.
Missing bit in previous commit (prevent race between create|mknod|mkdir|symlink
and reclaim)
Bump date for previous.
New sentence, new line; remove trailing whitespace; fix typos;
punctuation nits.
Add PUFFS_KFLAG_CACHE_DOTDOT so that vnodes hold a reference on their
parent, keeping them active, and allowing to lookup .. without sending
a request to the filesystem.
Enable the featuure for perfused, as this is how FUSE works.
Missing bit in previous commit (PUFFS_KFLAG_CACHE_DOTDOT option to avoid
looking up ..)


To generate a diff of this commit:
cvs rdiff -u -r1.10.2.1 -r1.10.2.2 src/lib/libperfuse/debug.c
cvs rdiff -u -r1.50.2.5 -r1.50.2.6 src/lib/libperfuse/ops.c
cvs rdiff -u -r1.25.2.1 -r1.25.2.2 src/lib/libperfuse/perfuse.c
cvs rdiff -u -r1.18.2.1 -r1.18.2.2 src/lib/libperfuse/perfuse_if.h
cvs rdiff -u -r1.25.2.3 -r1.25.2.4 src/lib/libperfuse/perfuse_priv.h
cvs rdiff -u -r1.15.2.2 -r1.15.2.3 src/lib/libperfuse/subr.c
cvs rdiff -u -r1.38.2.2 -r1.38.2.3 src/lib/libpuffs/dispatcher.c
cvs rdiff -u -r1.49.2.1 -r1.49.2.2 src/lib/libpuffs/puffs.3
cvs rdiff -u -r1.119.4.2 -r1.119.4.3 src/lib/libpuffs/puffs.h
cvs rdiff -u -r1.29.4.2 -r1.29.4.3 src/lib/libpuffs/puffs_ops.3
cvs rdiff -u -r1.89 -r1.89.8.1 src/sys/fs/puffs/puffs_msgif.c
cvs rdiff -u -r1.77.8.1 -r1.77.8.2 src/sys/fs/puffs/puffs_msgif.h
cvs rdiff -u -r1.78.8.1 -r1.78.8.2 src/sys/fs/puffs/puffs_sys.h
cvs rdiff -u -r1.100.8.1 -r1.100.8.2 src/sys/fs/puffs/puffs_vfsops.c
cvs rdiff -u -r1.163.2.3 -r1.163.2.4 src/sys/fs/puffs/puffs_vnops.c
cvs rdiff -u -r1.20 -r1.20.2.1 src/usr.sbin/perfused/msg.c
cvs rdiff -u -r1.22.2.1 -r1.22.2.2 src/usr.sbin/perfused/perfused.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.