Re: constraints on rbus_min_start?

["Steven M. Bellovin" <smb%cs.columbia.edu@localhost>]

On Thu, 17 Jan 2008 07:57:52 +0100
Bernd Ernesti <netbsd%lists.veego.de@localhost> wrote:

> On Fri, Jan 11, 2008 at 09:26:52PM +0000, Steven M. Bellovin wrote:
> > On Tue, 8 Jan 2008 02:05:58 +0000
> > "Steven M. Bellovin" <smb%cs.columbia.edu@localhost> wrote:
> > 
> > > On Sun, 06 Jan 2008 10:56:55 -0500
> > > "Jared D. McNeill" <jmcneill%invisible.ca@localhost> wrote:
> [..]
> 
> > > Diagnostic output I added says that the size of the rbus area
> > > was .75GB, starting at 3.25GB.  In other words, it overlaps the
> > > entire PCI hole. Is that right?
> > > 
> > I changed my RBUS_MIN_OPTION to 3GB; that let the EVDO card be
> > recognized properly.  The compact flash card worked better -- it
> > recognized wdc2 and an atabus controller.  It did not recognize wd2,
> > not did it recognize anything when I removed the card.  I tried
> > inserting the EVDO card it the same slot; that wasn't noticed.
> > However, when I removed the EVDO card, it decided that the CF card
> > had been removed.
> 
> How will the rbus_machdep.c code work when you have more then 2 or 3GB
> of ram?

I think it will actually help.

First, I think it's the same as the current code for machines in the
1GB-3GB size range.  3GB doesn't work at all today; that's my problem.
I don't know what will happen with machines in the 3GB-4GB range -- the
PCI hole is *somewhere* in that range, and it's a hardware feature that
will deflect certain addresses from going out to RAM.  If you know the
start of the hole, you can allocate some space manually and limit its
size, but I don't know how to figure out what space to allocate -- see
below.  If you do know a suitable chunk, my code should help, as it
will let you limit the extent of the rbus allocation. Anything larger
than 4GB should have the same properties as 4GB machines, since
addresses over 32 bits aren't relevant here.

I agree we need a better solution.  Unfortunately, there are two
issues.  First, I don't know enough about the amd64 PCI bus
architecture to write it.  Second, it requires a fair amount of work
elsewhere in autoconf to keep track of what parts of the PCI hole have
already been allocated, and probably to ask the hardware how big the
hole is and where it's located.  Per previous discussion in this
thread, that isn't done anywhere in NetBSD.  Some of it is typically
done by the BIOS in desktop machines, but not in embedded machines.

So -- my proposed fix makes life better for some people but doesn't
really solve the problem.  That said, I think it's worth doing because
it does help in some situations and never makes life worse.
> 
> > Since the differing behavior made me suspect problems with the
> > range 3.25-3.5GB, I added code for a new kernel option, RBUS_SIZE.
> > It doesn't seem to have helped with the CF card; given that EVDO was
> > helped by the change in RBUS_MIN_OPTION, I'm inclined to commit the
> > (attached) change, at least until someone who understands the amd64
> > architecture can write the correct code.
> > 
> > Comments?
> 
> It strikes me that we have now two rbus_machdep.c files with small
> differences. Yes, this is port-amd64, but I first looked in the i386
> version and that had some line offsets so it couldn't be that version
> which you modified.
> 
> Can't we move it to sys/arch/x86/x86 and use some ifdefs?

My original intent was to do just that.  But note the comments in the
i386 version about how it wasn't tested on 3GB machines.  I tried using
3GB as the value and ran into trouble, which is what led to my current
code.

Right now, the two routines are heading in different directions.  The
i386 code uses a heuristic based on RAM size.  Note the comment that
some older machines can't handle large addresses at all, so the
heuristic might really be necessary.  The amd64 code is going to have
to deal with enumeration of the PCI bus space, which may not work on
these older i386s, unless we have a good way of knowing how much
address space they can decode.

Note that I'm speaking only about the part of the code handling
rbus_min_start.  I have no idea about the similarity of the rest of the
file.

I think the right thing to do is to wait until we've really solved the
problem.  Then we can see if that code works on i386s, and in
particular on old i386s.  At that point, we'll know if it pays to move
the two together.  I'll note that in the comments I'm going to write....
> 
> And it would be nice to have some comments in the source file for new
> code what it does.
> 
Good point.  I also have to write man page text.


                --Steve Bellovin, http://www.cs.columbia.edu/~smb