:
:Matthew Dillon <dillon@apollo.backplane.com> writes:
:
:>     If you don't have the disk necessary for a standard overcommit model to
:>     work, you definitely do not have the disk necessary for a non-overcommit 
:>     model to work.
:
:I'd _really_ like to know how you figure this.
:
:text    data    bss     dec     hex     filename
:45024   4096    392     49512   c168    /bin/cat
:311264  12288   9900    333452  5168c   /bin/sh
:212960  4096    28492   245548  3bf2c   inetd.static
:458720  12288   73716   544724  84fd4   sendmail.static
:
:None of these are particularly huge.  Dynamically linked binaries
:inflate things somewhat, of course, but even there:
:
:442336  12288   35780   490404  77ba4   /usr/lib/libc.so.12.40
:
:the data and bss per library just aren't that huge.
:
:Of course, in addition to the sizes of the data and bss sections, you
:need to make sure you've got enough extra for dynamically allocated
:data, and for stack pages, and for a few other tidbits of dynamically
:allocated storage, but the end result is a system which, even without
:overcommit, can fit into a reasonable amount of backing store.

    The text size of a program is irrelevant, because swap is never
    allocated for it.  The data and BSS are only relevant when they
    are modified.

    The only thing swap is ever used for is the dynamic allocation of memory.
    There are three ways to do it:  sbrk(), mmap(... MAP_ANON), or
    mmap(... MAP_PRIVATE).

    Dynamic allocation of memory can occur under a huge number of 
    conditions.  The actual physical allocation of dynamic memory - what is
    known as a copy-on-write - cannot be predicted from the potential
    allocation of memory.  The most obvious example of this is a fork().

    There is a lot of hidden 'potential' VM that you haven't considered.
    For example, if the resource limit for a process's stack is 8MB, then
    the process can potentially allocate 8MB of stack even though it may
    actually only allocate 32K of stack.  When a process forks, the child
    process can potentially modify just about every single non-text page that
    was owned by the parent process, causing a copy-on-write to occur.
    The dynamic potential can run into the megabytes but most child processes
    only modify a small percentage of those pages.

:So, I just went back and looked at my mail folder for this NetBSD
:mailing list (tech-userlevel), which has about a week and a half's
:worth of messages.
:
:Nowhere did I see what amounts to anything other than hand-waving
:claims that you'll have to allocate much, much more backing store than
:you currently need to, and claims that that's unacceptable for general
:purpose computing environments.  If you have a more specific analysis
:that you'd like me/us to read, please point us at it more specifically.

    You are welcome to bring up real-life situations as examples.  That
    is what I do.  But look who is doing the hand-waving now?

:* not all the world's a general purpose computing environment,

    Which is meaningless handwaving.  Again, you are welcome to point out
    your own real-life situations.

:* while you certainly need to allocate more backing store than you
:would with overcommit, it's _not_ ridiculously more for most
:applications(+), and, finally,

    Based on what?  I am basing my information on the VM reservation made
    by programs, and I am demonstrating specific points showing how those
    reservations occur.  For example, the default 8MB resource limit for
    the stack segment.

    I am also demonstrating the level of control that is possible with
    resource limits.  If you are doing an embedded system you can easily
    guarentee memory utilization even in an overcommit model by properly 
    setting the memory resource limits for each process.  If the system
    did not have any swap, I would do this for every single program I
    run on that system.

    If you are going to argue the point, work out the numbers and present
    them.

    I had to deal with a reservation model on our old SGI's running 5.3
    for almost a year.  I know what I'm talking about and I can point to
    real-life cases that demonstrate it.  Certainly there are many different
    situations... you are welcome to bring up other real-life situations
    as examples.

:* even if you are not willing to pay that price, there _are_ people
:who are quite willing to pay that price to get the benefits that they
:see (whether it's a matter of perception or not, from their
:perspective they may as well be real) of such a scheme.

    Quite true.  In the embedded world we preallocate memory and shape
    the programs to what is available in the system.  But if we run out
    of memory we usually panic and reboot - because the code is designed
    to NOT run out of memory and thus running out of memory is a catastrophic
    situation.

    It is not appropriate to test every single allocation for a failure...
    that results in bulky code and provides no real benefit since the
    code is already designed to not overcommit the embedded system's memory.
    If it does, you reboot.  The most critical embedded systems, such as
    those used in satellites, avoid dynamic allocation as much as possible
    in order to guarentee that no memory failures occur.  This works under
    UNIX as well.  If you run your programs and touch all your potentially
    modifiable pages you have effectively reserved the memory.

:page from the system, it's almost certainly going write something to
:it, and, while there are undoubtedly a few pages that aren't written
:to, they are by far the majority.  And, of course, once the page has
:been written, it's no longer reserved, it's committed.  8-)

    Swap is used solely for dynamically modified pages of data that is not
    otherwise backed (e.g. not backed by a file, for example).

:I would honestly love to know: what do you see huge numbers of
:reserved pages being reserved for, if they're not actually being
:committed, by 'average' UNIX applications (for any definition of
:average that excludes applications which do memory based computation
:on sparse dasta).
:
:cgd
:-- 
:Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html

    Stack, hidden VM objects after a multi-way fork, MAP_PRIVATE memory maps,
    private memory pools (e.g. for web servers, java VM's, and so forth).

					-Matt
					Matthew Dillon 
					<dillon@backplane.com>