Subject: Re: Fetch distfile to different local filename
To: None <tech-pkg@netbsd.org>
From: Amitai Schlair <schmonz@schmonz.com>
List: tech-pkg
Date: 03/04/2002 12:34:18
"Frederick Bruckman" <fredb@immanent.net> wrote:


> On Sun, 3 Mar 2002, Amitai Schlair wrote:
>
> > "Amitai Schlair" <schmonz@schmonz.com> wrote:
> >
> > > This does exactly what I expect -- downloads the man page into
> > > ${DIST_SUBDIR} -- but unlike the binaries, there's only one man
page,
> > > and it seems wasteful (and confusing) to have nine distinfo
entries for
> > > one file simply because it could wind up in one of nine places.
>
> But no one would have to download more than one file, and you could
> set up hard links on ftp.netbsd.org so the one file would be available
> to every platform in whatever directory they require.

I hadn't even gotten to thinking about mirroring. It's good to know that
this could work. However, in addition to the manual page there's also a
text file, an HTML file, and two .zip archives of HTML files. Multipled
by nine architectures (possibly more -- I'm hoping to persuade Perforce
to make builds for more NetBSD platforms), that's a lot of redundant
links and distinfo entries to make. Especially considering that this
only becomes an issue because of some other problem that was weakly
worked around.

> > I couldn't think of any other way around it, so I made a quick
change to
> > bsd.pkg.mk:
> >
> > http://netbsd.schmonz.com/patches/fetch-distfile-local-filename.diff
> >
> > This lets a package say, for example,
> >
> > DISTFILE_LOCAL.remotename.tar.gz= localname.tar.gz
> >
> > ...which keeps my Perforce package sane. I'm quite surprised that
there
> > hasn't already been some package that needed to be able to do this!
>
> ...but yet there has not, so why not just override the do-fetch target
> in the package "Makefile"? If the idea turns out to have applicability
> to other packages, then it's time to think about putting it in
> bsd.pkg.mk.

I agree with the principle. After further thought, I still think the
change properly belongs in bsd.pkg.mk, for several reasons:

1) There are only two packages that use FETCH_BEFORE_ARGS, and they both
do it like so:

    FETCH_BEFORE_ARGS=-o ${DISTNAME}${EXTRACT_SUFX}

They need this because /usr/bin/ftp otherwise guesses wrong at what the
local filename should be (there's a bunch of cruft in the query string
that throws it off). The intent of that usage is not precisely
equivalent to mine, but it's basically doing what I'm doing.

2) I'm working on several Perforce-related packages: the client, the
server, and the web interface. All three have the problem that the
distfiles for different architectures are stored in different
directories, but have the same filename. (There's at least one more such
package that I'll probably want to look at, too.)

3) It seems logical to me that the the feature of optionally storing
downloaded files with different names ought to belong to the generalized
fetching code.

4) The changes don't seem overly complex. I'm not very familiar with the
underlying code, but I was able to do my little thing in about half an
hour. (Granted, it's incomplete.)

- Amitai