Subject: Alchemy wired memory regions
To: None <port-evbmips@netbsd.org, port-mips@netbsd.org>
From: Garrett D'Amore <garrett_damore@tadpole.com>
List: port-mips
Date: 10/20/2005 11:12:34
I've been thinking about the memory map for Alchemy parts, and I want to
post my proposal out here for review. Please let me know your thoughts
on it.
Alchemy parts have 32 TLB entries. Each entry can arrange for mappings
of up to 32MB.
PCI and PCMCIA addresses on Au15XX boards at least are well above the
32-bit address space.
Here's what I'm thinking:
For PCI configuration space accesses, I can use a single wired entry of
32MB, and move that wired entry on pci_conf_XXX accesses when I need
to. I've already written this and works great. It consumes 1 wired
entry if PCI is configured into the kernel.
For PCI memory accesses via the bus_space framework (excluding mmap) I
am planning on using 32 MB wired entries. For most purposes, this means
if you have any devices configured in your kernel that have registers in
PCI memory space, you are going to consume one more wired entry. (For
most purposes, one wired entry will cover pretty much all of the devices
you have on your system.)
For PCI I/O accesses, I can use the same kind of logic as the PCI
configuration space (one shared entry that moves), but in reality a
single 32MB entry should cover all the needs without having to move it.
This also consumes a wired entry, but only if you have I/O space devices
configured in the kernel and on your bus.
For PCI framebuffers that do mmap (and any other similiar devices), they
get a non-wired TLB entry as per normal.
PCMCIA also has three address spaces, unfortunately. My treatment of
these would be the same as for PCI, I think, where memory space gets
wired entries on demand, and I/O and ATTR space each have a single
shared wired entry.
It is also true that for access that do not have BUS_SPACE_LINEAR
mappings, I could have a single system wide shared mapping, that is
moved as needed under protection of a lock. While this will work, I
wonder if it will hurt performance more to move this around, as I
suspect that we may find ourselves flip-flopping back and forth across
two or more devices in different parts of the address space.
In a fully configured system, with stuff in all address spaces, the
worst case looks like it will typically be about 6 wired entries (plus
the one that NetBSD uses for context switching). If I try to mitigate
this by sharing, I can save 3 wired entires, at the cost of more
frequent changes to the TLB.
I guess in theory it is possible to avoid the wired entries altogether
and use ordinary TLB entries, but this looks a lot harder, and I'm not
sure what the implications of trying to support register access from
interrupt context are likely to be. Frankly, that's an avenue I don't
want to explore yet, unless someone feels a compelling need for it.
(And I think not having wired spaces will impact performance too, as
most devices will be interrupting frequently I think, thereby
prematurely forcing out TLB entries at random that might otherwise get
to stay.)
Thanks in advance for your thoughts on this.
-- Garrett