On Fri, Apr 13, 2001 at 06:35:25PM +0930, Berndt Josef Wulf wrote:
> G'day,
> 
> I like to submit a proposal of changes to bsd.pkg.mk that will help to
> drastically reduce the number of small files in the pkgsrc filesystem.
> 
> I propose to have the md5 and patchsum information for each package
> to be appended to the Makefile in the top directory of the package.
> The information would look similar to that below:
> 
> Makefile:
> ...
> #
> # Begin of Checksum Data
> #
> #[DCS] SHA1 (vim-5.7-src.tar.gz) = 71e263e9072118e5e56dad77a4bfbaed363bf591
> #[DCS] SHA1 (vim-5.7-rt.tar.gz) = c60c2996547e8b3939cbe744f7d2520b47a9060f
> #[PCS] SHA1 (patch-aa) = 1f38e5291e4803abeed50ffa403ec897ffb856bd
> #[PCS] SHA1 (patch-zb) = c929da81e94120a6540b6a503d40dd9f9ffc83ff
> 
> The advantages are a drastic reduction in the number of small files
> in the packages source tree and speed improvements during
> cvs update and building process.
> 
> There are only three package that to my knowledge wouldn't work
> without some minor modification to the Makefile. These are
> vim-*, nethack-* and ap-jk. I've since modified and rebuilt vim
> without any problems. 
> 
> The diff file can be found at
> ftp://ftp.netbsd.org/home/wulf/bsd.pkg.mk.diff
> 
> There may be other ways to implement this, but sofar nobody else has
> come forward to take it on and I took the liberty to take the
> first step.
> 
> What do you think? As always, I entertain your comments and
> suggestions.

OK, long term picture of what I'd like to see happen to our package
directories:

1. Move md5 and patch-sum info out of the files/ subdir, thereby
losing most of the files/ dirs from most packages.

2.  Move the PLIST and DESCR and other binary package support files
out of the pkg/ subdirectories into the package directory, thereby
losing the pkg/ subdirs.

3.  Incorporate other information into the area which holds the digest
info for distfiles, distpatches and patches.  In particular, I'd like
the size of the distfiles and distpatches to be recorded.  This is not
really for file integrity reasons (if the digest doesn't do that for
you, it's not a very good digest), more for doing things like putting
virtual quotas on sizes of distfile that can be downloaded without a
warning.  And also for my personal use - I really would like to know
how big a distfile is before I even contemplate downloading it over
a dialup line.

Now whether the digests go in the pkg Makefile or in a separate
distinfo file like FreeBSD are doing are another topic for discussion. 

Personally, I'd prefer all digest, checksum and size and other
ancillary information to go into a separate file. I'd like to
avoid overloading the pkg Makefile.

Also, I believe that the number of files isn't necessarily the kicker. 
I think it also has a lot more to do with the proliferation of
directories in pkgsrc, both when extracting from a tarfile, and when
updating using cvs, due to synchronous writes of dirs, and cvs having
to maintain separate CVS dirs for every directory it supports.

Having said that, the COMMENT mods did speed up things just a smidgin,
so I maybe wrong here.

My opinions,
Alistair