Re: Adding pool_cache_invalidate_local() to pool_cache(9) API

[Jason Thorpe <thorpej%shagadelic.org@localhost>]

On Oct 4, 2009, at 5:00 PM, Jean-Yves Migeon wrote:

> Hi list,
> 
> I would like to introduce a new function to the pool_cache(9) API, namely 
> pool_cache_invalidate_local() (see [1] for patch and documentation).
> 
> Basically, it is similar to pool_cache_invalidate(), except that it 
> invalidates the per-CPU caches for the current CPU executing the calling 
> thread.
> 
> The reason for why I would like to have such a function is to add support for 
> Xen save/migrate/restore functionality under PAE. Reasons follow.

Maybe I'm missing something ... how is this ever safe to use?  An object can be 
allocated from one CPU's cache and freed to another.  The per-CPU cache is 
simply an optimization, and it seems very wrong to expose this implementation 
detail to consumers of the API.


> 
> ========================
> For evident security reasons, Xen tracks page types, especially those
> dealing with the VM system (page tables).
> 
> During save/migrate/restore, Xen snapshots the current VM state.
> Unfortunately, it has no intrinsic knowledge of how the domain uses its
> page tables. In NetBSD's case, L2 page tables are allocated from a
> pool_cache. When they are recycled, they are not necessarily destroyed,
> but put back in the pool_cache, so Xen still consider them as L2 page
> although they are not currently used "as is".
> 
> As some of you probably know, Xen has a hard time with alternative
> recursive mappings between page tables. We have to clear them before
> save, and set them back to their value after resume. Iterating through
> pmaps is sufficient for i386 and amd64, but adding PAE support
> introduced a L2 shadow page that needs some handling too. This L2 shadow
> page is obtained through a pool_cache, this is why I need a routine to
> clean up both global and per-CPU caches without necessarily having to go
> through pool_cache_destroy for CPU-bound objects.
> ========================
> 
> The patch was compiled and tested for MONOLITHIC and XEN3_DOM0 for 32 bits 
> (running through a ./build.sh without crashing), and only compile tested for 
> amd64. Sadly, I don't have a multi-core (or CPU) x86 at my disposal, so I am 
> limited in my testing about this change. Should not be much different from 
> the current code, anyway.
> 
> 
> Opinions, reviews, are welcomed.
> 
> [1] http://www.netbsd.org/~jym/pc_invalidate_local.diff
> 
> -- 
> Jean-Yves Migeon
> jeanyves.migeon%free.fr@localhost

-- thorpej