Re: Memory usage of crunchgenned binary vs shared

[David Laight <david%l8s.co.uk@localhost>]

On Thu, Feb 28, 2008 at 11:37:30AM +0000, Stephen Borrill wrote:
> In an embedded project, I have kernel with an internal md root. I am then 
> using compressed vnd images, one of which includes standard shared 
> libraries (e.g. libc). By default, all the standard scripting tools (sh, 
> awk, sed, rm, cp, etc.) are all part of the monolithic crunchgenned 
> binary. As such they have a vsz of around 2.5MB when running. I've 
> noticed that in short memory situations that they are sometimes killed 
> causing scripts to fail in odd ways. This lead me to think that 
> installing the shared versions in the vnd and symlinking to them might 
> save memory.
> 
> However, every time I think about the logic of this I come up with a 
> different answer :-)
> 
> My current belief is that using the shared version will actually make 
> things worse. The md block cannot be run in place as they are in a 
> read-only place in the kernel memory so they are copied out. This is done 
> by block. In a crunchgenned binary the whole binary (and therefore all 
> links to it) will therefore be copied once and thus take up 2.5MB of 
> memory (over and above the RAM already used by its read-only image in the 
> kernel). If the shared version is used, the buffer cache will have the 
> whole of libc (which is likely to be there anyway because it'll be used 
> by other things) plus the shared binary versions of each individual 
> executable. I'm not sure how this maps to per-process memory usage though.
> 
> Could any VM experts give me a definitive answer here? :-)

Definitive answers are hard to come by!

Firstly the whole of the crunched binary or libc won't be copied into
memory, only the accessed pages will be copied.  In both cases all the
code (and readonly data) is not modified so can be disarded and reread
from the 'disk' at any time.

Secondly only the accessed data and bss will occupy real memory, so
when you run a crunchgenned binary you only need memory pages for the
data accessed by the specific binary (although it might get spread
over more pages than in the non-crunched, shared library version).

The crungenned binary has a slight advantage in the the non-PIC code
is smaller, and that the 'libc' it treats as 'shared' only contains
the functions that the binaries need - and without you having to
manually determine what can be removed.

But if you are likely to page in any part of the full dynamic libc, then
running crunchgenned utilities can only result in more than one copy
of parts of libc being in memory.

        David

-- 
David Laight: david%l8s.co.uk@localhost