Subject: Re: kern/36331: MP deadlock resembling ufs_hashlock deadlock
To: None <kern-bug-people@netbsd.org, gnats-admin@netbsd.org,>
From: Andrew Doran <ad@netbsd.org>
List: netbsd-bugs
Date: 05/17/2007 17:05:05
The following reply was made to PR kern/36331; it has been noted by GNATS.
From: Andrew Doran <ad@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Sverre Froyen <sverre@viewmark.com>
Subject: Re: kern/36331: MP deadlock resembling ufs_hashlock deadlock
Date: Thu, 17 May 2007 18:00:47 +0100
On Mon, May 14, 2007 at 09:05:00PM +0000, perseant@netbsd.org wrote:
> I've been chasing down what I had thought was an LFS bug until I isolated
> it last night, and it looks more like a multiprocessing locking bug to me.
> Any thoughts on what is going on here would be helpful.
On the code itself: there are two locks around the inode hash. ufs_hashlock
serializes insertion of inodes into the hash (locks the code path).
ufs_ihash_lock is taken by readers to prevent the hash from changing (locks
the data structure). One thread is trying to lock in this order:
ufs_hashlock -> vnode lock
The other thread is I think acquring them in the opposite order. There is
contention and so they deadlock waiting for each other to release a lock.
> vget(cf50a014,10002,94,0,c03c1250) at netbsd:vget+0x74
> ^^^^^^^^
> ufs_ihashget(4,2,0,2,c03c8734) at netbsd:ufs_ihashget+0x94
> ffs_vget()
I think the solution is to never lock the vnode with ufs_hashlock held. One
way is to change ufs_ihashget so that if the 'flags' argument is zero (not
LK_EXCLUSIVE or something else), it should not try to lock the vnode. It
should just drop ufs_ihash_lock and return the vnode address to indicate
whether or not there is now an entry in the hash. Then we could change
ffs_vget to look like this:
retry:
/* If it's already in the hash, we win. */
if ((*vpp = ufs_ihashget(dev, ino, LK_EXCLUSIVE)) != NULL)
return (0);
/* Allocate a new vnode/inode. */
if ((error = getnewvnode(VT_UFS, mp, ffs_vnodeop_p, &vp)) != 0) {
*vpp = NULL;
return (error);
}
ip = pool_get(&ffs_inode_pool, PR_WAITOK);
/*
* If someone beat us to it, put back the freshly allocated
* vnode/inode pair and retry.
*/
mutex_enter(&ufs_hashlock);
if (ufs_ihashget(dev, ino, 0) != NULL) {
mutex_exit(&ufs_hashlock);
ungetnewvnode(vp);
pool_put(&ffs_inode_pool, ip);
goto retry;
}
I can see about making this change but I don't have full Internet access
at the moment. It will be at least a few days before that changes.
Andrew