Re: Does the KA630 have a TB?

[Johnny Billquist <bqt%softjar.se@localhost>]

On 2023-09-24 16:55, Mouse wrote:
> > Should not matter.  You should be able to use any size TB.
>
> If the OS uses it correctly, yes.  If.

Agreed. :-)

> > > I suspect [the TB's] size matters because that NetBSD version
> > > actually does run on the KA630, without the TBIA/TBIS usage, whereas
> > > when provided with an unlimited-size TB [...] it fails - and I've
> > > added enough debugging code to be confident that it's failing
> > > because the TB holds stale entries.
> > I think you are wrong here.
>
> Possibly.  See below for more.
>
> > > > And I have a hard time thinking that NetBSD would depend on any
> > > > specific size of the TB.
> > > I don't think it does, not on any specific size.  I think it, that
> > > version, just depends on its size and replacement policy being such
> > > that that stale entries get replaced before they cause trouble.
> > That would be crazy and broken.
>
> Yes.  I don't for a moment think it _deliberately_ depends on that.  I
> think someone just missed writing the relevant mtpr() calls and it took
> a while for anyone to notice, presumably because all the tests anyone
> did were on systems (whether hardware or simulated) which either lacked
> a TB or provided one which ended up flushing stale entries before they
> caused trouble.  Ragge added some TBIA calls in April 2000; see rev
> 1.78 of the VAX pmap.c.

Oh. I had missed that you were running on such an old version.

Speaking of that, 1.85 is also fun... And I *hope* it's not that one 
that bit you...

> > So, I seriously doubt that there is even some hidden dependency on
> > some size.  And the actual size is rather different on different
> > VAXen that NetBSD runs on.
>
> Certainly.  But how many different models has 1.4T/vax been tested on?
> Personally, don't think I've ever tried it on anything but a uVAX2,
> which Antonio Carlini said has a _tiny_ TB (8 bytes).

Well. The 11/750 was what Ragge was initially using, and was the first 
system I also used, back at 0.9.

Actually, I don't know if I've ever run it on an uVAX2. But by the year 
2000 I think I had been running it on an 8650 a while.

I had also been running on a 3500 for a few years, and fought a problem 
with double pagefaults on the CVAX. They don't exactly work as the VARM 
say they should in some situation. I can't remember the details now, but 
it should be in the code somewhere...

> > But basically, we're talking about the TB now. It's automatically
> > populated, but in some situations software needs to explicitly flush
> > it.
>
> Right.  NetBSD 1.4T fails to flush it in at least some circumstances
> when it should.  Read pmap_enter() in -current and in 1.4T and note the
> lack of a TBIA in some codepaths in the 1.4T version.  (There are a few
> other pmap calls which are affected as well.)
>
> > But from what you are saying, it sounds like you are hitting problems
> > where none of those situations are going on, right?
>
> It _appears_, from what little I've found, to be a case of taking a
> write fault, updating the pmap, failing to invalidate, and then
> returning to userland.  (I suspect the write fault is may be COW case,
> but I haven't done the digging to be certain.  I am inclined to doubt
> it's just a write-tracking fault, because the PFN changes.)

Yeah. A copy on write means you also get a changed page table, so the TB 
for that page needs to be invalidated.

> > So, my thinking are context switches, which also affect the TB, since
> > you get completely new page tables.  [...] LDPCTX implicitly should
> > force a TB flush.
>
> Yes.  The third-last thing my ldpctx implementation does is to clear
> the whole TB.  (The second-last and last are pushing the PC and PSL on
> the (new) stack in preparation for REI.)

Good.

> > So my question to you now is: are you flushing the TB when that
> > instruction is executed?
>
> Yes.  Also, there is no ldpctx (or svpctx, not that that's very
> relevant) happening between the TB entry being created and it later
> being discovered to be stale - I did a run with tracing cranked all the
> way up, so it logged _everything_.

Well. You might be correct in that NetBSD just lacked some flushes then. 
I didn't realize you were running something as old as 1.4T.

But that version had definitely been run on other hardware than just 
uVAX2. And it sounds amazing that the problem would have been mostly 
unnoticed... But then again, Ragge did throw in those additional flushes 
at some point for a reason.

> I've now modified my NetBSD/1.4T enough that it appears to work with
> the large TB.  I just added more uses of the TBIA, based mostly on the
> delta between sys/arch/vax/vax/pmap.c 1.77 and 1.78, as modified by the
> delta from 1.84 to 1.85; I need to grab a copy of that file from
> -current and make sure I have appropriate analogs of all the TBIA/TBIS
> calls it does.

Cool. Sounds like that was the problem, then.

> I am pleased to find that it appears to be relatively effective.  I
> just booted my emulator single-user and force-fscked its disk.  After
> that, printing TB stats gives
>
> Translation buffer statistics:
> Hits 792967524 misses 3419573 (99.5706%), enters 3089291
> Level 1: 6/4096 (0.146484%)
> Level 2: 72/12288 (0.585938%)
> Clears: all 27910, one 0
>
> The difference between misses and enters corresponds to attempted
> accesses that don't reach RAM, either because they fail or because they
> go to devices or other things outside RAM - my TB doesn't cache
> anything that doesn't point to main memory.  (The "Level 1" and "Level
> 2" numbers are current-state info, not long-term statistics, so they're
> pretty meaningless for purposes of this email.)

I was wondering about that one. Why not cache entries outside main 
memory, by the way?

> I find it interesting that this NetBSD doesn't use TBIS at all.

Agreed. Seems like it would potentially be clever to just do TBIS when 
you update one page.

  Johnny

--
Johnny Billquist                  || "I'm on a bus
                                  ||  on a psychedelic trip
email: bqt%softjar.se@localhost             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol