Re: Free memory pages estimation for process use

[Nicolas Joly <njoly%pasteur.fr@localhost>]

On Thu, Jul 26, 2012 at 06:47:30PM -0400, matthew sporleder wrote:
> On Thu, Jul 26, 2012 at 2:59 PM, Nicolas Joly <njoly%pasteur.fr@localhost> 
> wrote:
> >
> > Hi,
> >
> > While testing free memory consumable requests with OGS (Open Grid
> > Scheduler), i noticed that free memory reported by a NetBSD host was
> > amazingly low ... preventing its use.
> >
> > By example, on a 8GB NetBSD/amd64 machine running -current :
> >
> > njoly@lanfeust [gridscheduler/source]> qhost
> > HOSTNAME                ARCH         NCPU  LOAD  MEMTOT  MEMUSE  SWAPTO  
> > SWAPUS
> > -------------------------------------------------------------------------------
> > lanfeust.sis.pasteur.fr nbsd-x86_64     8  0.09    7.8G    7.7G   40.0G     
> > 0.0
> >
> > At the same time 'vmstat -s'report the following:
> >
> >      4096 bytes per page
> >         8 page colors
> >   2032144 pages managed
> >     10496 pages free
> >   1247833 pages active
> >    616160 pages inactive
> >         0 pages paging
> >      1510 pages wired
> >      8937 zero pages
> >         1 reserve pagedaemon pages
> >        40 reserve kernel pages
> >     10717 anonymous pages
> >   1848644 cached file pages
> >      6142 cached executable pages
> > [...]
> >
> > Looking into the OGS code i found that memory/swap values are fetched
> > with VM_UVMEXP2 sysctl :
> >
> >   mib[0] = CTL_VM;
> >   mib[1] = SGE_VM_UVMEXP;
> >   size   = sizeof(uvmexp);
> >
> >   sysctl(mib, sizeof(mib)/sizeof(int), &uvmexp, &size, NULL, 0);
> >
> >   /* Memory */
> >   mem_info->mem_total = ((double) uvmexp.npages * uvmexp.pagesize) / (1024 
> > * 1024);
> >   mem_info->mem_free  = ((double) uvmexp.free   * uvmexp.pagesize) / (1024 
> > * 1024);
> >
> >   /* Swap */
> >   mem_info->swap_total = ((double) uvmexp.swpages * uvmexp.pagesize) / 
> > (1024 * 1024);
> >   mem_info->swap_free = (((double) uvmexp.swpages - uvmexp.swpginuse) * 
> > uvmexp.pagesize) / (1024 * 1024);
> >
> > But mem_free does only take uvmexp.free member into account, which
> > seems wrong to me. At least uvmexp.inactive should be added ...
> >
> > Is it correct ? Anything else i missed ?
[...]
> I look at free memory like this:
> https://github.com/msporleder/cons/blob/master/free/free.c
> but it looks like you're going the same thing.
> 
> Doesn't linux have the same problem where file caching takes away from
> total "free" space?

At least for OGS, the linux code parse /proc/meminfo and use
MemFree+Buffers+Cached to count free memory.

I'll have a look and try to do the same for NetBSD,

  uvmexp.free + uvmexp.filepages

Thanks.

-- 
Nicolas Joly

Biology IT Center
Institut Pasteur, Paris.