Re: event counting vs. the cache

[David Laight <david%l8s.co.uk@localhost>]

On Thu, Jan 17, 2013 at 03:43:13PM -0600, David Young wrote:
> It's customary to use a 64-bit integer to count events in NetBSD because
> we don't expect for the count to roll over in the lifetime of a box
> running NetBSD.
> 
> I've been thinking about what these wide integers do to the cache
> footprint of a system and wondering if we shouldn't make a couple of
> changes:
> 
> 1) Cram just as many counters into each cacheline as possible.
>    Extend/replace evcnt(9) to allow the caller to provide the storage
>    for the integer.
> 
>    On a multiprocessor box, you don't want CPUs sharing counter
>    cachelines if you can help it, but do cram together each individual
>    CPU's counters.

Actually, if the counter can be placed in the same area as some other
driver data, then it will typically already be in the cache.
This is probably most important for things that are changed very often
(like ethernet byte and packet counts).
Having error counts in different cache lines probably isn't that important.

This does mean that the evcnt(9) interface is completely the wrong one!
It looks like 8 + 6 x sizeof (void *) bytes per counter - so every increment
is (more or less) guaranteed to be a cache line miss.

        David

-- 
David Laight: david%l8s.co.uk@localhost