Subject: Re: another parallel-make race
To: segv <segv@netctl.net>
From: Chuck Silvers <chuq@chuq.com>
List: current-users
Date: 09/13/2005 07:25:52
--sm4nu43k4a2Rpi4c
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Mon, Sep 12, 2005 at 09:38:51PM +0100, segv wrote:
> On Mon, 12 Sep 2005 21:32:52 +0100
> Patrick Welche <prlw1@newn.cam.ac.uk> wrote:
> 
> > On Mon, Sep 12, 2005 at 12:48:13PM +0100, segv wrote:
> > > On Mon, 12 Sep 2005 11:04:39 +0100
> > > Patrick Welche <prlw1@newn.cam.ac.uk> wrote:
> > > 
> > > 
> > > > From my point of view the problem "went away"... I didn't actually track
> > > > anything down..
> > > 
> > > I don't think it went away, I have just updated src and xsrc trees for
> > > netbsd-current. Running build.sh with -j 4 results in the following errors.
> > > Running with -j 1 "fixes" the problem.
> > 
> > That's what I used to get - I just tried again with -j6 and today's cvs,
> > and didn't hit the race condition, though I did reproducibly before :-/
> > 
> > Cheers,
> > 
> > Patrick
> > 
> 
> Well I ran 'cvs update' today as well. I was building on a dual Pentium 3, I
> guess this triggers the errors. Try building it on a SMP machine, maybe you'll
> be able to reproduce the errors.


nevermind, I see that luke fixed the race I was complaining about
the same day as his previous post.

the race that you're still seeing is different.  we end up with multiple
instances of pax operating on same destination at the same time,
and pax is dumb about trying to create a directory where a directory
already exists.  in this case, pax removes the directory and then creates
a new directory in the same place, which can cause problems if another pax
is trying to create something in that directory while it's gone.

please try the attached patch (which both fixes the above problem and
updates pax's copy of mkpath() with the race-avoiding version from mkdir.c).

-Chuck

--sm4nu43k4a2Rpi4c
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="diff.pax-race"

Index: src/bin/pax/file_subs.c
===================================================================
RCS file: /cvsroot/src/bin/pax/file_subs.c,v
retrieving revision 1.53
diff -u -p -r1.53 file_subs.c
--- src/bin/pax/file_subs.c	24 Apr 2005 01:24:57 -0000	1.53
+++ src/bin/pax/file_subs.c	13 Sep 2005 14:19:09 -0000
@@ -465,6 +465,17 @@ node_creat(ARCHD *arcn)
 				}
 			}
 			res = mkdir(nm, file_mode);
+			if (res < 0 && errno == EEXIST) {
+				/*
+				 * Something's in the way.
+				 * If it's a directory, say we're done.
+				 */
+				if (lstat(nm, &sb) == 0 &&
+				    S_ISDIR(sb.st_mode)) {
+					res = 0;
+					break;
+				}
+			}
 
 badlink:
 			if (ign)
Index: src/bin/pax/options.c
===================================================================
RCS file: /cvsroot/src/bin/pax/options.c,v
retrieving revision 1.89
diff -u -p -r1.89 options.c
--- src/bin/pax/options.c	29 Jun 2005 02:21:27 -0000	1.89
+++ src/bin/pax/options.c	13 Sep 2005 14:19:10 -0000
@@ -690,7 +690,7 @@ pax_options(int argc, char **argv)
 		--argc;
 		dirptr = argv[argc];
 		if (mkpath(dirptr) < 0)
-			pax_usage();
+			exit(1);
 		/* FALLTHROUGH */
 	case ARCHIVE:
 	case APPND:
@@ -1335,16 +1335,27 @@ mkpath(path)
 		done = (*slash == '\0');
 		*slash = '\0';
 
-		if (stat(path, &sb)) {
-			if (errno != ENOENT || mkdir(path, 0777)) {
-				tty_warn(1, "%s", path);
-				return (-1);
+		if (mkdir(path, 0777) < 0) {
+			/*
+			 * Can't create; path exists or no perms.
+			 * lstat() path to determine what's there now.
+			 */
+			int	sverrno;
+
+			sverrno = errno;
+			if (lstat(path, &sb) < 0) {
+				/* Not there; use mkdir()s error */
+				syswarn(1, sverrno,
+					"Cannot create directory %s", path);
+				return -1;
+			}
+			if (!S_ISDIR(sb.st_mode)) {
+				/* Is there, but isn't a directory */
+				syswarn(1, ENOTDIR,
+					"Cannot create directory %s", path);
+				return -1;
 			}
-		} else if (!S_ISDIR(sb.st_mode)) {
-			syswarn(1, ENOTDIR, "%s", path);
-			return (-1);
 		}
-
 		if (!done)
 			*slash = '/';
 	}

--sm4nu43k4a2Rpi4c--