Re: constraints on rbus_min_start?

["Steven M. Bellovin" <smb%cs.columbia.edu@localhost>]

On Sun, 06 Jan 2008 10:13:38 -0500
nathanw%MIT.EDU@localhost (Nathan J. Williams) wrote:

> "Steven M. Bellovin" <smb%cs.columbia.edu@localhost> writes:
> 
> >> The only way to determine the used addresses is to enumerate all
> >> the devices. That is potentially dangerous since you have to
> >> rewrite the BARs with a base address of 0 in order to determine
> >> the size. Presumably the NetBSD PCI device grope code will have
> >> done this during boot and remembered the addresses that are in use.
> >
> > Which should, then, be in some data structure somewhere...
> 
> NetBSD doesn't normally do this on desktop boxes; we assume that the
> firmware has done it. The option to probe and allocate all the BARs
> ourself (PCI_NETBSD_CONFIGURE) is more commonly used on embedded
> systems with minimal or PCI-unaware firmware. 
> 
> If PCI_CONFIG_DUMP is enabled, we will read the existing values and do
> the read/write/read/rewrite dance to size the BAR but restore the
> firmware-programmed value; I don't think that data is saved anywhere
> but the dmesg output.

I'm not sure which firmware you mean -- individual devices?  the BIOS?
-- but the answer seems to be that (a) NetBSD generally doesn't know
where PCI devices live in memory address space; (b) if it finds out, it
doesn't store the information anywhere; (c) there isn't a good
alternative to guessing to locate the Cardbus space; (d) guessing won't
work for machines with a lot of RAM; (e) for now, at least, I'm SOL if
I choose to run amd64, because fixing this is a fair amount of work...

Also, from what I understand of the hardware (not much), the hardware
is automatically declining to pass some amount of address space to the
actual RAM, devoting it instead to PCI devices.  This amount ranges
from .5GB to .75GB.  On machines with 3GB or less, 3GB is a reasonable
starting point for the Cardbus allocation, since other PCI devices will
live higher up.  On machines with 4GB or more, actually mapping the
hole is necessary.  (Aside: presumably, there's some amount of address
space that can't be used even on desktop or server machines, which
means that NetBSD has to be aware of the location of the hole or it
couldn't support 4GB machines.  While I'm aware of problems with large
memory on i386, I thought it was supported on amd64.  Am I wrong?)

                --Steve Bellovin, http://www.cs.columbia.edu/~smb