Subject: Re: port-mips/31915: provide centralized wired_map logic
To: None <tsutsui@netbsd.org, gnats-admin@netbsd.org,>
From: Garrett D'Amore <garrett_damore@tadpole.com>
List: netbsd-bugs
Date: 10/26/2005 16:54:01
The following reply was made to PR port-mips/31915; it has been noted by GNATS.
From: "Garrett D'Amore" <garrett_damore@tadpole.com>
To: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Cc: gnats-bugs@NetBSD.org, port-mips@NetBSD.org
Subject: Re: port-mips/31915: provide centralized wired_map logic
Date: Wed, 26 Oct 2005 09:53:03 -0700
Izumi Tsutsui wrote:
>In article <20051025205402.7B93A63B8C2@narn.netbsd.org>
>garrett_damore@tadpole.com wrote:
>
>
>
>> Here are the diffs. Again, I've not conditionalized the inclusion of
>> mips3_wired_map.c, but if folks feel strongly enough about it I can
>> certainly do so.
>>
>>
>
>IMO, currently only arc port uses wired map, so it might be better to
>have some proper option (like options MIPS3_WIRED_MAP etc.).
>
>
That sounds reasonable. The evbmips port (or at least some kernels in
it) will be using it when I get my PCI support committed. I need it for
PCI and PCMCIA spaces on the Alchemy parts.
>
>
>> Also, the ARC port could probably make use of my logic in
>> mips3_wired_map.c, but I wanted to minimize change (and I don't have an
>> ARC system to test against, anyway, so all I can do is verify the compile.)
>>
>>
>
>I think the MI wired map function should also handle independent
>(non-contiguous) PA ranges for TLB_LO0 and TLB_LO1.
>arc port requires it to map PCI memory and I/O space at the early
>bootstrap stage, and it could save wired TLB entries in other case.
>
>
An interesting idea. I agree it would save entries. Right now I am
assuming that you can nearly always use a full 32MB of mapping, but
obviously that isn't always the case. (E.g. the PCI configuration space
entry only really needs a single 4KB window.) The complexity arises in
trying to use these "efficiently".
However, if I make the logic such that we are using 16MB granular
entries, I can pretty much hide the details of the fact that there are
really 2 mappings per entry from the caller.
>I also have some more quick comments:
>
>- VA region for wired map should be defined in machine dependent
> <machine/vmparam.h> and it should be managed by extent(9) etc.
>
>
Agreed. Right now my use defines it in a machine dependent header (not
vmparam.h, but that sounds like a good place for it!) and it does use an
extent. I've intentionally taken the route of *not* exposing this
detail to the MI wired mapping logic that I submitted.
>- pagemask for wired maps should be defined in machine dependent headers,
> or maybe it should be passed from callers.
>
>
I don't like passing it from callers, because I don't want it to be
variable. (This would make the code a lot more complex to make sure it
was correct and you didn't have overlapping maps, etc.) But defining
it in the machine dependent header seems reasonable.
>Currently I use the attached wired_map.[ch] for ews4800mips ports,
>though it lacks some sanity checks and unmap (or remap) functions.
>
>Maybe we should discuss more about APIs for these wired map functions.
>
>
This looks pretty darn close to the mapping logic that was in the ARC
port. I haven't run a diff to examine your deltas, but it has a lot
more of the logic in it then mine does -- i.e. managment of the extents,
and such. I was foisting the address space management issues on the
caller, and just providing bare-bones logic for establishing the mappings.
Is there a belief that we need to have centralized logic for managing
these address spaces? Or is my approach of having the caller figure it
out better. (E.g. in my Alchemy port I have the PCI code set up its own
extent for the address spaces, and it knows what the physical to logical
mappings are.)
I find that having the caller set it up works well for me, because the
caller can do things to establish very simple translations. For
example, on my Alchemy port:
* 0xD0000000 - 0xDFFFFFFF == PCI memory virtual address space
I then set up the physical memory translation so that all I have to do
to go from physical to virtual is OR in the upper bits (in this case
physical addresses are 0x4D0000000 - 0x4DFFFFFFF (yes, those are 36-bit
addresses).
I have different rules for PCI I/O and configuration space.
In order to establish these simple mappings, the caller needs to have
some basic control over the VA space (i.e. I use different extents for
PCI memory, IO, and configuration space.)
I suppose this is inefficient use of the limited VA space, but I find it
simpler. (I never have to inquire of the wired map to figure the VA->PA
mapping.)
-- Garrett
>---
>Izumi Tsutsui
>
>
>---
>/* $NetBSD$ */
>
>/*-
> * Copyright (C) 2005 Izumi Tsutsui.
> * Copyright (C) 2000 Shuichiro URATA. All rights reserved.
> *
> * Redistribution and use in source and binary forms, with or without
> * modification, are permitted provided that the following conditions
> * are met:
> * 1. Redistributions of source code must retain the above copyright
> * notice, this list of conditions and the following disclaimer.
> * 2. Redistributions in binary form must reproduce the above copyright
> * notice, this list of conditions and the following disclaimer in the
> * documentation and/or other materials provided with the distribution.
> * 3. The name of the author may not be used to endorse or promote products
> * derived from this software without specific prior written permission.
> *
> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
> * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
> * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
> * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
> * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> */
>
>#include <sys/cdefs.h>
>__KERNEL_RCSID(0, "$NetBSD$");
>
>#include <sys/param.h>
>#include <sys/systm.h>
>#include <sys/malloc.h>
>#include <sys/extent.h>
>
>#include <uvm/uvm_extern.h>
>
>#include <mips/pte.h>
>
>#include <machine/locore.h>
>#include <machine/vmparam.h>
>
>#include <ews4800mips/ews4800mips/wired_map.h>
>
>#define EWS4800MIPS_TLB_WIRED_ENTRIES 8 /* upper limit */
>
>static int ews4800mips_enter_wired(vaddr_t va, paddr_t pa0, paddr_t pa1,
> uint32_t pg_size);
>
>static struct wired_map_entry {
> paddr_t pa0;
> paddr_t pa1;
> vsize_t size;
> vaddr_t va;
>} wired_map[EWS4800MIPS_TLB_WIRED_ENTRIES];
>
>static int nwired;
>static struct extent *wired_map_ex;
>static long wired_map_ex_storage[EXTENT_FIXED_STORAGE_SIZE(8) / sizeof(long)];
>
>void
>ews4800mips_init_wired_map(void)
>{
>
> nwired = 0;
> wired_map_ex = extent_create("wired_map",
> VM_MIN_WIRED_MAP_ADDRESS, VM_MAX_WIRED_MAP_ADDRESS, M_DEVBUF,
> (void *)wired_map_ex_storage, sizeof(wired_map_ex_storage),
> EX_NOWAIT);
> if (wired_map_ex == NULL)
> panic("ews4800mips_init_wired_map: can't create extent");
>}
>
>static int
>ews4800mips_enter_wired(vaddr_t va, paddr_t pa0, paddr_t pa1, uint32_t pg_size)
>{
> struct tlb tlb;
> psize_t size;
>
> if (nwired >= EWS4800MIPS_TLB_WIRED_ENTRIES) {
>#ifdef DIAGNOSTIC
> printf("ews4800mips_enter_wired: wired entry exausted.\n");
>#endif
> return ENOMEM;
> }
>
> size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_size);
> va = (va + (size - 1)) & ~(size - 1);
>
> wired_map[nwired].va = va;
> wired_map[nwired].pa0 = pa0;
> wired_map[nwired].pa1 = pa1;
> wired_map[nwired].size = size;
>
> /* Allocate new wired entry */
> mips3_cp0_wired_write(MIPS3_TLB_WIRED_UPAGES + nwired + 1);
>
> /* Map to it */
> tlb.tlb_mask = pg_size;
> tlb.tlb_hi = mips3_vad_to_vpn(va);
> if (pa0 == 0)
> tlb.tlb_lo0 = MIPS3_PG_G;
> else
> tlb.tlb_lo0 = mips3_paddr_to_tlbpfn(pa0) | \
> MIPS3_PG_IOPAGE(PMAP_CCA_FOR_PA(pa0));
> if (pa1 == 0)
> tlb.tlb_lo1 = MIPS3_PG_G;
> else
> tlb.tlb_lo1 = mips3_paddr_to_tlbpfn(pa1) | \
> MIPS3_PG_IOPAGE(PMAP_CCA_FOR_PA(pa1));
> mips3_TLBWriteIndexedVPS(MIPS3_TLB_WIRED_UPAGES + nwired, &tlb);
>
> nwired++;
>
> return 0;
>}
>
>/* Allocate new wired entries */
>int
>ews4800mips_map_wired(vaddr_t va, paddr_t pa0, paddr_t pa1, uint32_t pg_size)
>{
> psize_t size;
> int error;
>
> size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_size);
> va = (va + (size - 1)) & ~(size - 1);
>
> if (va < VM_MIN_WIRED_MAP_ADDRESS ||
> va + size > VM_MAX_WIRED_MAP_ADDRESS) {
>#ifdef DIAGNOSTIC
> printf("ews4800mips_enter_wired: invalid va range.\n");
>#endif
> return EINVAL;
> }
>
> error = extent_alloc_region(wired_map_ex, va, size, EX_NOWAIT);
> if (error) {
>#ifdef DIAGNOSTIC
> printf("ews4800mips_enter_wired: cannot allocate region.\n");
>#endif
> return error;
> }
>
> error = ews4800mips_enter_wired(va, pa0, pa1, pg_size);
>
> return error;
>}
>
>vaddr_t
>ews4800mips_map_wired_region(paddr_t pa, int size, uint32_t pg_mask)
>{
> vaddr_t va, rva;
> vsize_t off;
> psize_t pg_size;
> int error;
>
> pg_size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_mask);
> off = pa & WIRED_ENTRY_OFFMASK(pg_size);
> pa &= ~(paddr_t)WIRED_ENTRY_OFFMASK(pg_size);
> size += off;
>
> if ((size + WIRED_ENTRY_SIZE(pg_size) - 1) / WIRED_ENTRY_SIZE(pg_size) >
> EWS4800MIPS_TLB_WIRED_ENTRIES - nwired) {
>
>#ifdef DIAGNOSTIC
> printf("ews4800mips_map_wired_region(0x%lx, 0x%lx)"
> ": %d is not enough\n", pa + off, size - off,
> EWS4800MIPS_TLB_WIRED_ENTRIES - nwired);
>#endif
> return 0; /* free wired TLB is not enough */
> }
>
> error = extent_alloc(wired_map_ex, size, WIRED_ENTRY_SIZE(pg_size), 0,
> EX_NOWAIT, &va);
> if (error) {
>#ifdef DIAGNOSTIC
> printf("ews4800mips_map_wired_region: can't allocate region\n");
>#endif
> return 0;
> }
> rva = va;
>
> while (size > 0) {
> ews4800mips_enter_wired(va, pa, pa + pg_size, pg_mask);
> pa += WIRED_ENTRY_SIZE(pg_size);
> va += WIRED_ENTRY_SIZE(pg_size);
> size -= WIRED_ENTRY_SIZE(pg_size);
> }
>
> return rva + off;
>}
>---
>
>---
>/* $NetBSD$ */
>
>/*-
> * Copyright (C) 2005 Izumi Tsutsui.
> * Copyright (C) 2000 Shuichiro URATA. All rights reserved.
> *
> * Redistribution and use in source and binary forms, with or without
> * modification, are permitted provided that the following conditions
> * are met:
> * 1. Redistributions of source code must retain the above copyright
> * notice, this list of conditions and the following disclaimer.
> * 2. Redistributions in binary form must reproduce the above copyright
> * notice, this list of conditions and the following disclaimer in the
> * documentation and/or other materials provided with the distribution.
> * 3. The name of the author may not be used to endorse or promote products
> * derived from this software without specific prior written permission.
> *
> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
> * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
> * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
> * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
> * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> */
>
>#define WIRED_ENTRY_SIZE(pg_size) ((pg_size) * 2)
>#define WIRED_ENTRY_OFFMASK(pg_size) (WIRED_ENTRY_SIZE(pg_size) - 1)
>
>void ews4800mips_init_wired_map(void);
>int ews4800mips_map_wired(vaddr_t va, paddr_t pa0, paddr_t pa1,
> uint32_t pg_size);
>vaddr_t ews4800mips_map_wired_region(paddr_t pa, int size, uint32_t pg_size);
>---
>
>