Re: zfs pool behavior - is it ever freed?

[Greg Troxel <gdt%lexort.com@localhost>]

mlelstv%serpens.de@localhost (Michael van Elst) writes:

> tnn%NetBSD.org@localhost (Tobias Nygren) writes:
>
>>There exists ZFS code which hooks into UVM to drain memory -- but part
>>of it is ifdef __i386 for some reason. See arc_kmem_reap_now().
>
> That's an extra for 32bit systems (later code replaced __i386 with
> the proper macro) where kernel address space is much smaller.

Sure, but I don't see why it shouldn't always be hooked up.

The upstream code ends up setting

  arc_c_min: 1/32 of memory.  I think this is "amount below which arc
  will not be pushed, even under memory pressure".

  arc_c: 1/8 of memory.  I think this is "target size of ARC"

  arc_c_max: all memory except 1GB.  I think this is "size above which
  ARC will be hard prohibited from growing".

The idea that arc_c_max is so big is only reasonable if there is an
effective, known-working mechanism to free memory under pressure.

I'm in the middle of trying to document what the variables mean and
rationalize the size calculation, and will post a diff fairly soon.

With tnn@'s patch, I am seeing

  $ sysctl kstat.zfs.misc.arcstats|egrep size
  kstat.zfs.misc.arcstats.size = 48786184
  kstat.zfs.misc.arcstats.compressed_size = 36969472
  kstat.zfs.misc.arcstats.uncompressed_size = 185070080
  kstat.zfs.misc.arcstats.overhead_size = 1348096
  kstat.zfs.misc.arcstats.hdr_size = 10160672
  kstat.zfs.misc.arcstats.data_size = 0
  kstat.zfs.misc.arcstats.metadata_size = 38317568
  kstat.zfs.misc.arcstats.other_size = 307944
  kstat.zfs.misc.arcstats.anon_size = 16384
  kstat.zfs.misc.arcstats.mru_size = 28901888
  kstat.zfs.misc.arcstats.mru_ghost_size = 87785472
  kstat.zfs.misc.arcstats.mfu_size = 9399296
  kstat.zfs.misc.arcstats.mfu_ghost_size = 97047552
  kstat.zfs.misc.arcstats.l2_size = 0
  kstat.zfs.misc.arcstats.l2_asize = 0
  kstat.zfs.misc.arcstats.l2_hdr_size = 0

but also vmstat -m

  In use 1445870K, total allocated 3323556K; utilization 43.5%

so I think either ARC use is not being accounted for correctly or there
is substantial non-ARC use.   But, my system is stable with 6G of RAM
under Xen. (Which is good because nvmm requires newish CPUs and I can't
run it.)


Also, I don't understand "ghost" yet.