Re: variable MAKEOBJDIR inside of Makefile

[David Holland <dholland-tech%netbsd.org@localhost>]

On Thu, Feb 06, 2025 at 07:00:55PM -0800, Greg A. Woods wrote:
 > > The core problem is that chdir changes the meaning of relative paths,
 > > and (a) make lives on paths and (b) relative paths are strongly
 > > preferred to absolute paths so that the location of your checkout
 > > doesn't get entangled into the build.
 > 
 > Well, yes, I think that was probably the idea in the first place, and I
 > think that part of the design seems very good.
 > 
 > > This makes the whole idea (of having an OBJDIR you chdir to) suspect.
 > 
 > So, I disagree quite with that claim.

Ok...?

 > > It's perfectly reasonable to not chdir but still put your build output
 > > into an object directory; tools that don't work correctly in that
 > > style are broken should be fixed regardless. This is what I'd
 > > recommend if designing a new build system, or new build tools.
 > 
 > I don't think the problem is with the tools -- as far as I know there
 > hasn't been such a problem with mainline tools used in Makefiles for
 > decades, though perhaps if you include any scripts defined within a
 > Makefile as a "tool", well, yeah, but there's the rub -- either trying
 > to teach Make to manipulate just and only the "right" paths within a
 > rule to prepend .OBJDIR gets mighty ugly mighty fast -- it's just plain
 > inelegant.

Nah. It's easy to write $(.OBJDIR)/$(OBJ): $(SRC) and $(CC) $(CFLAGS)
-c $(SRC) -o $@. It's much messier to have the implicit chdir to
$(.OBJDIR) and then write $(OBJ): $(SOMEWHERE)/$(SRC) and $(CC)
$(CFLAGS) -c $(SOMEWHERE)/$(SRC). Maybe $(.PARSEDIR) is good enough
for $(SOMEWHERE), maybe it's not.

Also, if you don't chdir but just place the output where you want it,
the pathnames displayed in compiler diagnostics are the same ones that
work in your shell or editor. With the chdir behavior, this only works
if the pathnames get absolutized, and that has a significant negative
impact on the legibility of the messages.

I dunno about you but I'd far rather see
   ../../../../ufs/lfs/lfs_bio.c:170: Parse error
than
   /home/dholland/projects/netbsd/trees/lfs/src/sys/arch/amd64/compile/LFSTEST/../../../../ufs/lfs/lfs_bio.c:170: Parse error

especially if there's a whole pile of errors and not just one. (And it
can get a lot worse if anything in there decides it ought to expand
symlinks.)

 > Even with the ability to give non-local pathnames to tools "we" have
 > been "taught" to write make rules such that the default is to assume
 > products go in the "current" directory.

Except that it's a lot better not to. Unless you have tools that can't
be told where to place their output, and as I was saying if any of
those still exist they should just get themselves fixed.

 > And that's true across all "make" variants I've ever encountered.

No make variant or mode other than this chdir behavior of bmake has a
preference.

Well, I guess that's not entirely true. There are some problems with
suffix rules. But there are problems with suffix rules as soon as you
have anything other than sources _and_ objects in the current
directory. .PATH is supposed to fix that, but it's at best a bandaid.

 > So I think it's far more elegant and "clean" to transform sources
 > to have the .CURDIR prefix.

And I don't, and I say so from the perspective of having set up many
complex builds. Including ones that build to separate output trees.

 > The kicker though is that while Make does know about _a_ target product
 > for each rule, it doesn't necessarily know about any intermediate or
 > secondary products created by a given rule (nor does it always know
 > about sibling products made by the same rule, e.g. by yacc, though
 > arguably that's a bug in the Makefile).  All those things are likely
 > going to end up in the current working directory no matter how hard you
 > try to convince Makefile writers to change their ways.

Those are the tool problems I was talking about.

 > Make can also safely always assume those sources can be
 > found relative to the directory where it was invoked (.CURDIR).  (And we
 > mostly have well known ways to find other "hidden" dependencies that
 > might be used by a rule, such as header files, etc. and we even have a
 > well known pseudo-target to do this: "make depend")

Are you suggesting that make inspect the contents of recipes for
things it ought to insert $(.CURDIR) or $(.PARSEDIR) in front of? That
will never work.

 > Well I think the first main problem that it is hard, next to impossible
 > given the current implementation, especially of MAKEOBJDIRPREFIX, to
 > specify a relative directory structure that, say, might be beside the
 > source tree, especially when there are subdirectories in the source
 > tree.  Related is the problem of having a relative MAKEOBJDIRPREFIX that
 > works even if you invoke "make" from deeper in such a multi-level source
 > tree.

Yes, because it all has to be magic. If you write your recipes so they
place things in an explicit output dir, it's not magic and you can set
it up however you want.

All you need to put the build output in a parallel objects tree is 
a TOP=../../.. (for the right number of ..'s) in each makefile;
everything else you need can readily be derived from that.

In an ideal world maybe make would know how to figure that on its own,
but given that complex projects often have multiple notions of the top
(e.g. in netbsd src and src/sys both play that role for different
purposes) it's probably not actually a good idea.

 > Having to have a separate initial "make obj" step, and having
 > everything go horribly wrong if you forget it, is a major botch.

That I agree fully with.

 > Personally I would _never_ have .OBJDIR be relative to within each
 > source directory (in multi-directory source trees) either -- always in a
 > separate sub-tree, though possibly within the top level of the source
 > directory.  Of course it would also be nice if it were easier to write
 > one top-level Makefile for a multi-directory source tree, and maybe even
 > have Make look for the Makefile up the parent directory chain until it
 > finds one (e.g. how git looks for its repo).

That will never work except for toy projects small enough to put
everything in a single makefile. (And even then it will be
problematic; all the paths in the makefile will be relative to where
it is, and you're running it from somewhere else, so it'll have to
read your mind at length to know how to adjust the recipes.)

 > Also the name .CURDIR is rather unfortunately confusing.  .SRCDIR might
 > have been less ambiguous.

It becomes confusing as well if you have sources in multiple places.

-- 
David A. Holland
dholland%netbsd.org@localhost