Re: [Milkymist-devel] [Milkymist port] virtual memory management

[Yann Sionneau <yann.sionneau%gmail.com@localhost>]

Le 04/06/13 00:23, David Holland a écrit :
> On Mon, Jun 03, 2013 at 10:11:14PM +0200, Yann Sionneau wrote:
>   >> There are essentially three ways to do this.  Which one you chose depends
>   >> on the hardware.
>
> There's also a second issue you're overlooking, which is that you need
> a way to access the user address space from the kernel.
>
>   > >1) Turn off the MMU on exception
>   >
>   > That sounds like the better thing to do from my point of view, I
>   > don't see any big downside apart from having to duplicate a few
>   > pointers in a few structures to have both virtual and physical
>   > addresses of some data structures like PCB, trapframes, page
>   > tables.
>   >
>   > Is there any big downside I don't see there?
>   > If not, I would pick this solution to implement the virtual memory
>   > subsystem.
>
> There are two different architectures here: one is to run the entire
> kernel with the MMU disabled; the other is to disable the MMU only for
> the TLB refill handler. (And maybe for the entry points of other
> exceptions too, or maybe not, but that's relatively unimportant.)
>
>   > >2) Keep parts of the address space untranslated
>   >
>   > I could modify the MMU to do that, but I would prefer keeping the
>   > entire 4 GB address space for user space :)
>
> In this case you cannot set up the conventional scheme where the
> kernel is mapped into the upper portion of all processes' address
> spaces. You will then need to implement copyin/copyout using special
> instructions or some other hardware-supported scheme to allow
> accessing the user address space via the MMU while the MMU is
> either disabled or mapping the kernel address space (as per my
> previous paragraph).
>
> I hope some such scheme exists, or, since I gather you're designing
> the MMU as well as the pmap, you're willing to implement some such
> scheme.
I would like to stay in conventional scheme as much as possible, with 
the less modifications possible to the original lm32 core (the one 
without MMU).
So I guess that I will end up doing the usual 1GB/3GB ( is that what is 
done in NetBSD? ) virtual memory layout.
>   > >3) Lock important pages into the TLB
>   >
>   > [...]
>
> That is in general a bad idea. With a poorly conceived MMU it might be
> necessary, but if you're inventing the MMU as well you don't need to
>
>   > >So you can implement your VM subsystem several different ways.  Just
>   > >remember that on a TLB-only machine you need to make sure the MMU handlers
>   > >can access the page tables, either bypassing the MMU or using some trick
>   > >that does not result in recursive TLB faults.
>   >
>   > In my case, I guess I need to keep the MMU off while looking-up the
>   > page table and to make sure I have physical address pointers in PCB
>   > and in trap frames. I also need to have access to physical address
>   > of curlwp (which therefore would give me access to physical address
>   > of curpcb).
>
> The TLB refill handler should only be a dozen or so instructions long.
> You want to avoid going and looking at things like curlwp and curpcb.
> All you should need is the base (physical) address of the page table,
> which I'd encourage providing a hardware register to hold.
I can easily add a CSR (Control and Status Register) to the LM32 core to 
hold the
physical address of the page table.
This register would not be used at all by the hardware but would only be 
there as a storage register for the software to store and load page 
table phys address.
Writing a new value to the register could even trigger a TLB flush.
> Also remember to consider the layout of your TLB entries from the
> point of view of loading them from a page table using as few
> instructions as possible.
I would need at least 2 or 3 instructions to load a TLB entry, which is 
kind of slow but I will try to avoid at least doing bit mask and shifts.
> Also in general it seems to me like you ought to stop for a bit and
> nail down the overall design of the MMU and the virtual memory
> environment before you go charging around implementing pmap code.
> This was the basis of the questions I asked in my other email, except
> you hadn't yet explained you were designing the MMU.
Well, I admit that I jumped into the code to get my hands dirty a little 
bit too early I suppose.
I sent an e-mail to the mailing list when I realized that indeed a lot 
of things were not clear to me regarding the low level architecture of 
such a Milkymist port.
I think I needed exactly the kind of advice the few of you have given to 
me and the few points you put bold on. So thanks again :)

I will sit and think about all of this!

To add a little bit of background, the MMU has been designed with 
"simplicity" in mind.
It was designed to be the simplest MMU "possible", so all the missing 
features are by-design.
Hardware (even in FPGA) is really tough to make (to test, and to debug). 
So the "KISS" rule very much applies to CPU design.
The goal of this project was to add the minimum bits possible to LM32 
CPU to:

1°) not break anything
2°) not diminish its rather good performance
3°) add virtual memory support and memory protection so that it can be 
used by a modern OS and run *BSD or Linux or others

Indeed the MMU is *very* software assisted and therefore the resulting 
performances will not be light speed, but the CPU runs at 80 to 100 MHz 
for now which is not very fast anyway. The value of Milkymist SoC is not 
in its CPU performance but in its openness that allows anyone to learn 
and study from its design and in the performance of its hardware 
accelerated peripherals: 2D graphics accelerator, Programmable FPU and 
the fact that it does not occupy a lot of space (LUTs, SRAM) in the FPGA 
which gives a lot of room for improvements and adding many more 
peripherals, controllers and accelerators for your own purposes.

Best regards and thanks again :)

--
Yann Sionneau