Re: X server in dom0: Bad VBT signature

[Gianluca Guida <glguida%tlbflush.org@localhost>]

On Tue, Sep 04, 2012 at 09:25:15PM +0200, Manuel Bouyer wrote:
> On Tue, Sep 04, 2012 at 03:32:23PM +0200, Gianluca Guida wrote:
> > On Tue, Sep 04, 2012 at 02:29:10PM +0200, Manuel Bouyer wrote:
> > > On Tue, Sep 04, 2012 at 02:12:39PM +0200, Gianluca Guida wrote:
> > > > This is only partially true. Xen also gives us a memory map through
> > > > memory_op hypercall (for dom0, it actually gives us something pretty
> > > > similar to the host e820).
> > > > With this information, we can map the I/O space and other non-RAM pieces
> > > > of memory to identity mapping that access the actual physical memory of
> > > > the host (I am talking about dom0 case here, clearly).
> > > 
> > > And what do you do with ram that Xen did give us at these addresses ?
> > > remap it somewhere else ?
> > 
> > My memory isn't helping me here, as the last time I looked at this code
> > in the Linux kernel was in 2009, and then I've stopped working on Xen
> > for almost two years, but:
> > 
> > IIRC Linux's dom0 kernel gives back the memory to Xen. I will look at
> 
> Ha yes, that's another possibility.
> 
> > the details later. It should probably be possible to remap the memory in
> > some other part, though it might be messy as we'll have to carefully
> > handle the various holes in the e820 map.
> > 
> > 
> > > The identity mapping isn't even needed, if the address is not in a
> > > ram segment it's a machine address.
> > > 
> > > But such a hack doesn't solve the underlying problem which is,
> > > basically, that under Xen, physical and machine addresses are
> > > different address spaces.
> > > 
> > 
> > I have trouble following you here. We seem to see the problem from a very
> > different points of view. And I fail to understand why this should be a
> > hack.
> 
> The real problem is that X, unlike in-kernel drivers, can't express
> that it wants to access device memory space and not plain RAM. Splitting
> the /dev/mem space in RAM and non-RAM segments is only a workaround for
> that. This workaround works for x86, but one could immagine a hardware
> where this woudln't work.

At the beginning of my introduction to this thread, I said that I was
probably missing a major design choice of NetBSD's Xen implementation. I
finally found, after probably too many emails but enough sleep, what
this design choice is.
NetBSD is way more natural towards Xen, separating guest memory (PA)
from machine accessible memory (MA). I kind of like this, I am pretty
sure this ended up with a more clean implementation.

So, the problem we are facing now, though, is that /dev/mem isn't really
meant for this scheme. It is meant for architectures with a single,
uniquely addressable memory called "physical memory" that contains both
RAM and I/O / weird BIOSes / unaddressable memory. We don't have that in
the xen port.

As opposed to the NetBSD situation, Linux dom0 kernel has a rather
different approach. To quote Konrad -- I didn't realize it was on CC --
that's the situation there:

On Tue, Sep 04, 2012 at 03:33:03PM -0400, Konrad Rzeszutek Wilk wrote:
> The end result is that the the P2M tree mirrors the E820 and we the
> 'ram' pages are only in the E820_RAM regions.

This means they indeed have in kernel what /dev/mem is trying to access.
They are mimicking the host memory layout. This gives them a way to
avoid major modification in the kernel, which is a real problem there.


While I guess I am not in the position to define architecturale choices
in this kernel, I see two options:

I  - We full switch to the Linux dom0 model of having a single memory
address for everything. It might needs a lot of rethought of the way
NetBSD implements Xen though, and TBH one of the reasons I am interested
in BSDs support for Xen is that of avoiding Xen to be a "one OS pony",
and different models do help Xen to be generic and OS-agnostic.

II - We give to /dev/mem what it's searching for, that is a E820-like
map of the host memory. This might mean to decode through a map modeled
after the E820 map what the guest is trying to access, be it MMIO or
RAM.
This of course requires choices, e.g. how do we know where the RAM
mapped at a PA is mapped into the /dev/mem physical memory layout, and
requires a address translation in the /dev/mem driver.

There is of course The Other Solution, which is to hack until X works,
and probably many others, but right now the innate guilty sense I bring
with me of speaking and writing too much is kicking in so I'll wait for
comments.

Thanks, this has been a very useful thread for me.

Gianluca