kern/49267: uvm should be able to advise apps on memory usage

[dholland%eecs.harvard.edu@localhost]

>Number:         49267
>Category:       kern
>Synopsis:       uvm should be able to advise apps on memory usage
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 09 20:35:01 +0000 2014
>Originator:     David A. Holland
>Release:        NetBSD 7.99.1 (20140819)
>Organization:
>Environment:
System: NetBSD macaran 7.99.1 NetBSD 7.99.1 (MACARAN) #21: Tue Aug 19 20:08:43 EDT 2014 dholland@macaran:/usr/src/sys/arch/amd64/compile/MACARAN amd64
Architecture: x86_64
Machine: amd64
>Description:

There are a lot of apps (both in and out of NetBSD base) that can
adjust their memory usage to prevailing conditions. sort(1) is a
simple example; but a lot of other programs contain caches or other
variable-sized entities whose size can be set more or less
arbitrarily.

Currently programs that want to pick the sizes of such things have no
way to figure out what their memory footprint should be. The best they
can do is look at UVM stats and try to guess something based on the
number of free and inactive pages and maybe add some voodoo based on
the number of cache pages. But application programmers don't in
general know how to do this, and they shouldn't be in the business of
trying to understand VM internals anyhow -- and figuring numbers like
this is a black art, like tuning page thresholds is, and having every
random app using its own weird ideas doesn't benefit anyone.

Most apps more likely just look at the size of physical memory and
assume they should be using all of it, or if they come from the Linux
world, assume by rote that file pages are actually free pages, or just
allocate until rlimits stop them.

UVM should provide a number for this based on the amount of memory
currently in use, the number of processes going, and recent memory
pressure behavior. Even if it's wrong it's probably still an
improvement on random stuff made up by apps.

>How-To-Repeat:
n/a
>Fix:

Figure out what number to report; then add it to uvmexp reporting; or
possibly better, give it its own sysctl.