[cc list trimmed because it was getting ... insane, and it's not like
this is a critical point. It's just beating up a topic which has been
beaten up by many others.]

Matthew Dillon <dillon@apollo.backplane.com> writes:
>     Thus it makes little sense to try to disallow overcommit.  It gains you
>     absolutely nothing, and forces you to waste huge amounts of disk space.
>
>     To anyone who is paranoid about it:  Fine, then just allocate an insane
>     amount of swap and forget about it.  It would be no more then you would
>     have to allocate anyway if we were to actually disallow overcommits!  But
>     with overcommits allowed, your box will never come close to using that
>     much swap.

This may be a decent answer for the workstation world, but it's not so
good for more restricted systems.  Further, your claim that
disallowing overcommit gains you absolutely nothing is simply false.

It gains you two things (which are at least immediately obvious to me):

* Certain knowledge that (if the system is implemented correctly) the
  system will never have to kill a process (or take similar corrective
  action) due to overcommit, and that attempts to allocate more backing
  store resources than are present will fail.

  If the programs you're using do reasonable error handling when out-of
  memory situations crop up, then this can be a very useful propery to
  have if you're building a reliable system.  It's not necessarily a
  matter of gigs of swap space.  It means that you know that you can
  safely run in, say, 64MB or memory, with _no_ swap, if your programs
  don't overallocate ridiculously.

  The only alternative to avoiding overcommit that seems plausibly
  enforceable by the OS/system designer is using resource limits and
  then planning based on the maximum resource limits and number of
  processes you allow.  However, that results in a ridiculously loose
  upper bound and a fair bit of extra work tuning the limits.

  The obvious workstation example is a bit grotesque: 1GB max datasize
  by default, maxproc 532, and you need > .5TB to be _sure_ that no
  process will get killed for lack of resources.

  A more realistic embedded system might allow 64 processes, two of
  which need 10MB each and the rest of which need 300K-1MB each.  If you
  do this naively, you end up "needing" 640MB to guarantee safely.  If
  you tweak a bunch of things with some special-purpose resource
  limiting code, you can get it down to 96MB or so to guarantee safety.
  Only by going to a _great_ deal of extra effort -- which could be
  avoided by just disabling overcommit -- you can't be sure the system
  will operate safely in, say, 48MB or even 64MB, both of which may well
  be adequate.

  Using resource limits to solve this problem is ... not the right
  answer (even if various systems implemented resource limits
  correctly).

* protection against bogosity.

  I may run a system in which all of the processes are effectively
  unlimited (i.e. have huge resource limits), but I know within a
  tolerance what the actual expected usage of the system is.

  If a program mallocs (or equivalently allocates with mmap) <a huge
  amount> it should be told up front that  that's not possible (or
  that it is), and filling that memory in later simply should not
  result in its death or in the death of another process.  This
  becomes especially important when the process making the bogus
  request is a "critical" process or when the other process that
  happens to be killed is a critical process.


Certain knowledge about system operation, and deterministinc system
behaviour in the face of programming bugs is exactly what the desire
for overcommit is about.

Overcommit avoidance may not be useful for your particular uses of
these UNIX-like systems.  However, if you think that it's not useful
to anybody who uses them (or that people who think it's useful are
deluding themselves 8-), then you're sorely mistaken and have a
... very wrong-headed attitude about why people find such features
useful.


cgd
-- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.