Re: tracking P->V for unmanaged device pages

[Masao Uebayashi <uebayasi%gmail.com@localhost>]

- struct vm_physseg is MI representation of a continuous physical
address region.  It is currently allocated only for main memories, but
apertures should have one.
- struct vm_page is MI representation of memory pages that have
backing stores; IOW, a state between memory page and its backing
store.
- PV is state that a physical page address is mapped to a virtual address.

Considering these, PV should be kept in struct vm_physseg, instead of
struct vm_page, ideally.  PV should be also an MI struct, like struct
vm_pv.

(ISTR yamt@ had an idea to link PVs from vm_map_entries for obvious reasons.)

If you once have such a right design, you no longer need to have a
separate API to track PV, because pmap_enter() is given sufficient
information to decide whether PV is tracked or not; P/V addresses and
flags telling how to map the new virtual address (cache enabled, PAT
used, etc.).

On Thu, Mar 26, 2015 at 9:13 PM, Taylor R Campbell
<campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
> Various DRM graphics drivers, including Intel, Radeon, and Nouveau,
> sometimes need to unmap all virtual mappings of certain physical
> pages for which there is no struct vm_page.  The issue is explained in
> detail here:
>
> https://mail-index.netbsd.org/tech-kern/2014/07/23/msg017392.html
>
> It's not desirable to simply add struct vm_pages on a freelist that
> uvm_pagealloc ignores, because struct vm_page is large (120 bytes on
> amd64, for example), most of it is unnecessary for P->V tracking, and
> the physical regions that need P->V tracking are large (hundreds of
> megabytes, or gigabytes).
>
> The attached patch implements the following extension to pmap(9) on
> x86 and uses it in DRM[*].  The implementation uses a linear list of
> pv-tracked ranges, since it is expected to be short (one to three
> elements).  The list is managed with pserialize(9) so it adds no
> locking overhead to existing pmap operations that need to look up
> entries in it.
>
> core@ discussed the problem and asked that this approach be marked as
> an interim solution, because not everyone was happy about it but
> nobody had an obviously better idea.
>
> Objections?
>
>
> void    pmap_pv_init(void);
>
>   Initialize the pmap_pv(9) subsystem.  Called by uvm_init.
>
> void    pmap_pv_track(paddr_t startpa, paddr_t size)
>
>   Do pv-tracking for the unmanaged pages in [startpa, startpa + size).
>   Called by a driver on initialization to register device pages.  Can
>   be done only once for any given [startpa, startpa + size) range,
>   with no overlapping allowed.  The range must be page-aligned.
>
> void    pmap_pv_untrack(paddr_t startpa, paddr_t size)
>
>   Stop doing pv-tracking for the pages in [startpa, startpa + size).
>
> void    pmap_pv_protect(paddr_t pa, vm_prot_t prot)
>
>   Reduce page protection of pv-tracked unmanaged page at pa to prot.
>   pa must be page-aligned.
>
>
> [*] This predictably fixes some rendering issues but seems to expose
> another bug causing the X server to crash more frequently, which is
> under investigation now.