RE: Interrupt mask on multi-processor machine

[Tim Rightnour <root%garbled.net@localhost>]

On 08-Feb-2008 KIYOHARA Takashi wrote:
> I have not experience to programing on multi-processor machine. Shall I
> need atomic operation to interrupt mask on bebox?
># However, cpu1 is not supported now... X-<

Well, I can't directly answer your question as it applies to bebox, as I am
only minimally familiar with the bebox interrupt controller.  However, my
understanding of it is that it is somewhat similar to the openpic in nature, so
I will describe how that works to you.

Basically, the openpic has 3 main modes of operation:

1) Direct all interrupts to a single CPU.
2) Direct specific interrupts to a specific CPU.
3) Direct all interrupts to both cpus in a "this cpu is less busy" manner.

>From what I understand of the bebox architecture, the bebox interrupt
controller provides for 1 and 2, and it might have some kind of minimal support
for 3, but in a less intelligent manner.

So.. the easy way to avoid the problem while first bringing up SMP, is to mask
off all interrupts on cpu1, and direct them all to cpu0.  I believe the bebox
has a mask set for each CPU.

Now, when you try to attempt some form of mixed delivery (option 3), you can
start running into problems depending on the nature of the controller. If it
were to deliver a single interrupt to both cpu's simultaneously, that would be
bad.  Additionally, if you need to mask a given IRQ off on both CPUs to avoid
having it deliver them to the other cpu, then yes, you would need to use atomic
ops for that.  Basically, if you ever need to touch the other cpu's mask from
the cpu that is processing the interrupt, you will need atomic ops.

My suspicion is that the nature of the 603 CPU and cache coherency, will make
SMP on the bebox difficult, but not impossible.  I would avoid the problem with
interrupts as long as possible by just sending them all to cpu0, until you have
everything else worked out.  As long as you never deliver interrupts to cpu1,
you shouldn't need to do any kind of atomic ops in there, as interrupt code
will never be called from cpu1.

Note that even if you eventually get some form of mixed delivery working, you
will have to avoid processing i8259 interrupts on cpu1 entirely, as that will
definately cause chaos if two cpu's simultaneously attempt to manipulate the
8259s.

As for the cache coherency thing, I suspect that you will have to add a bunch
of cache flush operations all throughout the tree.  If that ends up being the
case, please protect them with some kind of ifdef, like PPC_603SMP, so that
other arches/cpus aren't affected by the slowdown.  (Unless of course the bebox
has magical hardware to do this for you, I'm not familiar enough with it to say)

---
Tim Rightnour <root%garbled.net@localhost>
NetBSD: Free multi-architecture OS http://www.netbsd.org/
Genecys: Open Source 3D MMORPG: http://www.genecys.org/