Re: Utility of PV on non-obsolete hardware

["Mathew, Cherry G." <cherry%zyx.in@localhost>]

On 1 June 2019 6:12:02 PM GMT+02:00, Maxime Villard <max%m00nbsd.net@localhost> wrote:
>Le 02/06/2019 à 01:27, Cherry G. Mathew a écrit :
>> Maxime Villard <max%m00nbsd.net@localhost> writes:
>> 
>>> I hope we will be able to retire XenPV sooner than later. We have
>too many
>>> shades of x86 (between native-32, native-32-pae, native-64,
>XenPV-32-pae,
>>> XenPV-64, XenPVHVM*), and it is becoming impossible to understand
>what the
>>> code really does. Meanwhile, the x86 architecture is also getting
>increasingly
>>> complex, and there are planned future changes that will modify the
>paging
>>> logic. If we ever want to support any of that, we will either have
>to retire
>>> certain configurations, or split the code and give a separate pmap
>(among
>>> other things) to the different configurations.
>>>
>>> This would ease maintenance, which would ease support for future
>features
>>> and also the already existing ones.
>>>
>>> For example the other day I was playing with tprof, and noticed that
>about 10%
>>> of the TLB misses we get during a build.sh are caused by the
>recursive PTE
>>> slot. If we were using the direct map, almost all of these misses
>would vanish.
>>> Should I add another ton of #ifdefs in all directions just to get
>native-64 to
>>> use the direct map in order to reduce TLB misses?
>>>
>>> In having so many configurations that share the same code, we just
>end up
>>> using the least performant options to accommodate everyone. This is
>not really
>>> smart, it's even completely dumb.
>> 
>> I have previously suggested this - I think the pmaps for native and
>PV
>> should diverge. I would also recommend relooking nvmm pmap changes
>from
>> this PoV so that the divergence is clear and methodical. If you lead
>the
>> excision for nvmm, I will attempt to follow the pattern. This will
>also
>> force rethinking the level of abstractions we currently use for
>PVops.
>> 
>> Unfortunately (for several reasons) I cannot lead this process right
>now
>> but again, if you take the initiative, I am happy to review and test
>> changes.
>
>Even then, I don't see any reason to keep XenPV, given that
>XenPVHVM/XenPVH
>will cover everything. And I don't see any reason to go through the
>effort of
>splitting everything in order to keep XenPV for ... no other reason
>than the
>sake of keeping XenPV.
>
>Also, I think it is no surprise that our PV implementation is really
>slow.
>After ten years we still don't have SMP support, there are a lot of
>gross
>splvm() calls that are hard to justify, there are many hypercalls we
>don't
>combine, there is a lot of cache false sharing in the structures, and
>there
>are in short many things that overall betray a glaring lack of any kind
>of
>optimization in our PV code.
>
>Should we also mention stability, some things were broken for years
>before I
>came in and fixed them, and some others are still broken; I don't see
>the
>point in perpetuating this endless fixing. Not to mention the security
>issues
>caused by the fact that the kernel runs in ring3 on amd64...
>
>Overall, I wouldn't be surprised if Xen decided to drop PV completely,
>and I
>think we should drop it once XenPVH is functional.

small correction - we've had PV SMP for domU since 2012 at least. yes, its not the most efficient, yes you've fixed a few things, yes there are inherent inefficiencies in the model (every kernel/usermode boundary is TLB flush, for eg: and we don't use PG_G to help temper this) - however, as long as there are existing users, I don't see why we should not keep support. 

there a few reasons I would personally like to keep PV:

1. It makes virtualization available to h/w without VTx
2. It demonstrates a low level PA<->VA management approach that is educational and perhaps useful on other archs  we support without XEN if we provide hypervisor support.
3. it provides a completely isolated VA space  situation for the kernel without inherent h/w support for this - which will keep any assumptions about shared kernel/usermode VA space for eg: in check. 

However 2. and 3. are irrelevant if the userbase for 1. doesnt exist - and that's a call that the postmaster needs to make.

HTH.
-- 
Sent from my phone. Please excuse my brevity.