On 1999/07/14 at 11:17pm +0900, "Daniel C. Sobral" <dcs@newsguy.com> wrote:
>Ladavac Marino wrote:
>>
>>         This topic has been trashed to death a few months ago.  There is no
>> win-win situation in presence of processes which allocate a lot of memory
>> without actually using it (read: your typical FORTRAN library).
>
>This is not about just Fortran libraries.

OK, I'll bite. I'm curious as to why Fortran's library has been used as
an example here. Some years ago I worked on a Fortran compiler; I wrote
its library as well as its front end. The targets were (then)
high-performance graphics cards, with little memory, no MMU, no swap, and
certainly no sparse objects in the intrinsics. (Are you referring to user
libraries that try to provide 'large enough' fixed-size arrays? One can't
stop people from doing silly things. One *can* try to stop these silly
things from interfering with non-silly things.)

This thread originated with C's library, however, not with Fortran's --
in particular with malloc(), which is defined by the ANSI standard to
either return NULL, or usable memory. Killing the program later for
trying to use successfully malloc()'d memory (what the standard would
call 'undefined behavior') isn't an option. If a program that calls only
standard C functions can die when it uses malloc()'d memory then, as I
read the document, the system does not support standard C. In standard C,
malloc() returns something functionally equivalent to committed memory,
or it returns NULL.

On a system that has a writable file system, a standard-conforming
version of malloc() could presumably be written by having malloc() create
a file, make it non-sparse, and mmap() it. This seems unattractive.

>Imagine a reasonably big
>program, like Netscape or Emacs, of which you usually just use a
>subset of features. There can easily be many megabytes of code and
>data in them you never actually use, or you don't _usually_ use
>(like the people who use emacs like it was vi :). Without
>overcommit, you need to allocate all that memory for the code, no
>matter whether you end up using it or not. With overcommit, there is
>no such problem.

Code, static data, and not-yet-written writable data should be backed by
the executable file, not by swap space, so unused code and tables should
not be a problem.

>And that's not all, either.
>
>Hell, people, if overcommit was not useful, everybody wouldn't be
>doing it.

If a program wants uncommitted space, it should request it by means
beyond the standard C library functions (e.g. mmap() with MAP_NORESERVE).
A program that does so can then deal with the consequences (e.g. a signal
when unavailable space is touched), or die, as the case may be. Innocent
bystanders should not be hit.

(Of course, programs that use standard C functions like malloc() should
be prepared to deal with the consequences of *that*, but the consequences
should be the ones defined by the C standard, i.e. a NULL return from
malloc().)

Stack is more interesting. There might be a place for a global overcommit
switch. I think I'd be happier with a scheme in which stack the first
page or first few pages are committed (so that reasonable programs will
never run into trouble) and remaining stack is over-/un-committed by
default, along with means for unusual programs to commit (and/or test
commitability of) subsequent pages.

--
Kevin Schoedel <schoedel@kw.igs.net>
"Sorry, Opus. I only have NetBSD on my computer." "Darn it. There *has* to
be a place where I can use some bad software." -- 'User Friendly' 99/06/14