Konrad Schroder suggested a VOP_UPCALL a while back. I've been thinking
about it, and I think it'd be a good thing. I haven't started coding it,
but I'd like input. Also, I'd like to discuss how we can use it to resolve
PR 4315.

The two uses I can see now are:

1) vgone on the underlying vnode. We need to blow away all upper vnodes
first (they might need the lower one to clean up).

2) removing the underlying vnode from the name space. In this case, we
want the upper layer to get rid of the nodes soon ("fast death," clarified
below).

In both of these cases, something's happening on the lower vnode and the
upper layers need to react.

To do this, I propose we add both a queue head and a queue entry pointer
to struct vnode (probably TAILQ). The queue head would be the list head
for all vnodes stacked above this vnode, while the entry pointer would be
for the list of vnodes above the vnode this one is above.

Access to the list would be controled by the v_interlock of the vnode with
the list head. To touch the queue entry in a vnode, you need to lock the
interlock on the underlying vnode. 

vn_upcall(vp, command, void *) would walk the list of over-stacked vndoes,
lock a node, call VOP_UPCALL on it, and itterate (taking care that the
list might change while in the VOP_UPCALL() but then again it might not).

VOP_UPCALL(vp, command, void *) would call vn_upcall on its vp, and then
process the command for its node.

Thoughts on VOP_UPCALL?


Fast death:

There's an open PR (4315) for where someone had a umapfs over an nfs
mount. He deleted a file in the nfs layer, and instead it turned into a
.nfsBLAH file, which wasn't going anywhere. Until the umap layer was
unmounted. 

The problem was that all layered fs's keep a reference to the underlying
vnode, even if the layered vnode is on the free list. When the file was
deleted, nfs saw it was still in use so it renamed it to .nfsBLAH (the
exact name is in the PR). So even though the file was not in use, because
the layered vnode was on the free list (not yet having been revoked),
the .nfsBLAH file persisted.

I suggest we add a new vnode flag, tentetivly named VDELETED (name
suggestions appreciated), which indicates that the vnode should be put on
the front of the free list, rather than the end, when the usage count goes
to 0.

When a leaf fs vnode is unlinked, the resulting VOP_UPCALL() would set
this flag (VDELETE or whatever). Then when the upper vnode goes on the
free list, it will rapidly be reclaimed, at which point the references to
the nfs node will disapear.