Re: pbulk using tmpfs

[Jan Danielsson <jan.m.danielsson%gmail.com@localhost>]

On 04/20/13 23:47, iMil wrote:
[---]
> In order to speed up builds, I would like to know if I
> could benefit from tmpfs by mounting the WRKOBJDIR as a tmpfs
> directory.
> 
> In particular, while most packages will fit into a couple of
> megabytes, some of them (firefox, libreoffice, openoffice,
> koffice and such) take a couple of gigabytes to get built.
> 
> My question is: how will tmpfs behave? If I mount the WRKOBJDIR
> without any argument, it will "use", if I understand correctly
> the manpage, all of the available memory. Does this mean I will
> end up in a situation were the system memory will be filled by
> the build or is there some kind of smart mechanism within tmpfs
> which will "overflow" somewhere?

   I use pkg_comp (a locally modded version of it, which configures the
work directory onto a tmpfs directory), and I don't know what pbulk is
(the name implies the basic idea though).

   I have been using tmpfs-mounted directories for work directories on
two of my systems for well over a year. I know one shouldn't trust top
at all, etc, but watching the build hog more and more memory, it's "all"
returned as the build completes and I run "make clean-depends" and/or
"make clean".

   One of the systems has 16GB ram, and on that I build both pkgsrc
packages and NetBSD itself on the tmpfs (read: object work dir only).
The other system has 2GB RAM (and 0.5G swap), and on that I only build
pkgsrc on tmpfs.

   With regards to packages with gigantic dependencies: I don't know how
pbulk works, but I have a script which goes through a file from top to
bottom and builds the packages listed in it, and for each
completed/failed build cleans up the workdir. I've strategically placed
common big near-leaf node dependencies at the top of the file. I doubt I
could build all packages on the system with 2GB ram if I wouldn't have
sorted the file that way. The swap is used, albeit little of it, on this
particular system, even when I build firefox. But it should be noted
that I have built all dependencies separately before building it.

   I've seen a few scary reports about problems with tmpfs, but I
haven't encountered any problems despite having rebuilt my packages
and/or NetBSD probably - on average - once a week.

-- 
Kind regards,
Jan Danielsson