On Wed, Apr 07, 2004 at 04:24:26PM -0800, Erik E. Fair wrote:
> Just out of curiosity, does this change if one compiles the kernel 
> with -mcpu=21164a ?

No. This would solve the problem if the compiler refused to generate the
required Byte and Word granularity instructions. However, this gets taken
care of by an explicit assembler directive in
/sys/arch/alpha/pci/cia_bwx_bus_io.c.

The source of the problem is that the code in /sys/arch/alpha/pci/cia.c
uses BWX only if four conditions are satisfied:

	a) BWX isn't disabled by setting CIA_PCI_USE_BWX and
	   CIA_BUS_USE_BWX to zero in the kernel config and not setting
	   CIA_PYXIS_FORCE_BWX to one.

	b) CNFG_BWEN is enabled in the CIA's CNFG register.

	c) The CPU is an EV5.

	d) ALPHA_AMASK_BWX is enabled in the CPU's capabilities mask.

Check d) is done by checking the global variable cpu_amask (see
/sys/arch/alpha/alpha/cpu.c) which doesn't get initialised before
cpuattach() time (and from cpu_hatch(), but I believe only for secondary
CPUs). When cia_init() is called from the platform's cons_init(),
cpuattach() hasn't run yet and cpu_amask is zero -- so check d) always
fails and the bus_space call-vectors and memory windows are set up to use
SWIZ. The fact that cia_init() is called again later at attach time doesn't
help because the bus_space call-vectors and memory windows that decide
whether BWX extensions get used at all only get initialised once. Moreover,
the message shown at boot time and in dmesg is potentially misleading as it
might suggest BWX is being used when actually it isn't. As far as I can
tell from looking at the code without running it, this concerns platforms
DEC 1000A, DEC 550, DEC EB164 (mine), DEC KN20AA and DEC KN300.

Three solutions come to mind:

	1) Up to revision 1.54 of /sys/arch/alpha/pci/cia.c, the check for
	   EV5 and BWX was done "manually" in cia_init() by calling
	   alpha_implver() and alpha_amask(). Reverting to this procedure
	   works for me.

	2) If 1) is deemed unaesthetic now that the cpu_amask variable
	   exists, an alternative solution would be to reinitialise the
	   bus_space call-vectors and memory windows every time cia_init()
	   is called (which is at most twice). Every time except the first
	   time this involves freeing "extent" structures allocated in the
	   process.

	3) Yet another way would be to initialise cpu_amask before console
	   init time, instead of from cpu_announce_extensions().

If there is a preference for one of these options, I will implement and
send-pr it right away. ;)

cu Jo

PS: I hope I didn't mention the CIA too many times.

-- 
La perfection est atteinte non quand il ne reste rien à ajouter, mais quand
il ne reste rien à enlever.  (Antoine de Saint-Exupéry)