Re: Summary of implicit DEPENDS problem

[Jonathan Perkin <jperkin%mnx.io@localhost>]

* On 2023-11-25 at 20:42 GMT, Greg Troxel wrote:

> It strikes me that inclusion guards could check;
>
>  has this been included before, with at least this value of DEPMETHOD
>
> so that including it once with build and once with full would do the
> processing the second time.
>
> That might require some fixes, but conceptually it seems like the right
> thing.

One problem with this approach is that DEPMETHOD is not set by default 
prior to the first inclusion, and so a later test may have a different 
guard but refer to the same state.

A simpler fix that may work for many cases that I thought about this 
morning is what I just committed to vala, i.e.:

  lang/vala/buildlink3.mk:
   +.if ${BUILDLINK_DEPMETHOD.vala:U:Mfull}
    .include "../../devel/glib2/buildlink3.mk"
    .include "../../devel/libltdl/buildlink3.mk"
    .include "../../graphics/graphviz/buildlink3.mk"
   +.endif

If a bl3 is included for the sole purpose of a build dependency, which 
is usually just to run some tools, then I think the for the vast 
majority of situations you are not going to need the implicit 
dependencies.  They should only be required when linking, and if we're 
linking against this libraries from this package then it should not be a 
"build" dependency.

Another idea I just considered is that we move away from having packages 
set BUILDLINK_DEPMETHOD.foo=build themselves, make that a pkglint error, 
and instead have explicit bl3 files that should be used for the "build" 
case that do not include other bl3 files.  For example lang/vala/tool.mk 
that would have to duplicate a few bits like BUILDLINK_API_DEPENDS, but 
would contain any logic required to run the tools and no other includes.

This wouldn't work for the gtk-doc case though.  We may just need to do 
something custom there involving BUILD_DEPENDS and hacking together a 
pkg-config setup manually.

> >   * Adds IMPLICIT_DEPENDS and IMPLICIT_BUILD_DEPENDS, ensuring that the
> >     depends file includes them too.
>
> I don't quite follow this from your email.  It sounds like
> IMPLICIT_DEPENDS is something that is a full dep of something that is a
> full dep, and IMPLICIT_BUILD_DEPENDS is a full dep of something that is
> a build dep.  Things that are build deps only of either are not
> relevant.  Is that what you mean?  (I'm not at all sure I got that
> right.)

Yes.  An implicit dependency is anything .include'd in a buildlink3 of a 
direct dependency.  IMPLICIT_DEPENDS are those where the top-level bl3 
DEPMETHOD is "full", IMPLICIT_BUILD_DEPENDS where they are "build".  For 
completeness sake there is also the chance that a top-level bl3 is set 
to "full" but along its tree a DEPMETHOD is set to "build", in which 
case that dep (and its children) will be IMPLICIT_BUILD_DEPENDS.

Ultimately they should be effectively the same as DEPENDS and 
BUILD_DEPENDS, which is why I chose the same nomenclature, but are 
implicit rather than explicit.

> > As per the commit message, this isn't ready for integrating into
> > pkgsrc yet though.  Due to problem #3 we see occasional false
> > positives if an indirect dependency is tagged as "build" when it is
> > later correctly tagged as "full".
>
> So check-shlib will complain, even though things are written in a way
> that should be ok.  But they aren't actually ok due to #3?

Yep.

> > I'm happy to keep this in my tree for now and commit workarounds as I
> > find them, as I don't think other folks would be happy with any false
> > positives.
>
> Is it sensible to think about integrating with a flag to enable, so
> people who want to stmuble over these things and fix/add-workaround can?
>
> Or do we think that misordering depends from what feels natural as a
> workaround is icky, and we should refrain from doing that and instead
> fix the inclusion guard bug?

I think that if I at least expose and fix most of the larger issues 
while I have this enabled in my SmartOS and macOS builds, we will then 
have a better understanding of the problem and may be able to avoid 
having to do any radical overhauls.  We can then consider integrating it 
with a flag once we're happy it's the right approach.

--
Jonathan Perkin   -   mnx.io   -   pkgsrc.smartos.org
Open Source Complete Cloud   www.tritondatacenter.com