Re: NetBSD/sgimips 10.0 RELEASE on SGI Challenge S: INSTALL32_IP2x works, GENERIC32_IP2x kernel panics on boot after install

[Tim McIntosh <tmcintos%eskimo.com@localhost>]

> On May 21, 2024, at 12:26 AM, <iwama%t3.rim.or.jp@localhost> <iwama%t3.rim.or.jp@localhost> wrote:
> I had a similar problem on NetBSD-9.3.
> A similar problem occurred on O2 with NetBSD-9.4.

Thanks for this information!

> I get around this problem by putting the kernel in IRIX's / partition and starting it using sash.

Interesting. It looks like I still have sash in the volume header, but I no longer have any IRIX partitions on this machine, unfortunately.

> I suspect that there is a problem with the bootloader.


I rolled back to the 5.0.2 boot loader and it appears to exhibit the same problems with the problematic kernels. E.g.

>>> boot
> 
> NetBSD/sgimips 5.0.2 Bootstrap, Revision 1.5
> (builds%b7.netbsd.org@localhost, Sat Feb  6 21:26:53 UTC 2010)
> 
> devopen: scsi(0)disk(3)rdisk(0)partition(0) type scsi file netbsd.gdb
> 3446288+113248 [243344+233661]=0x3d9ca4
> 
> Exception: <vector=UTLB Miss>
> Status register: 0x2<IPL=8,MODE=KERNEL,EXL>
> Cause register: 0x30008008<CE=3,IP8,EXC=RMISS>
> Exception PC: 0x0, Exception RA: 0x881eadb4
> exception, bad address: 0x0
>  Saved user regs in hex (&gpda 0xa8740e48, &_regs 0xa8741048):
>  arg: a8740000 0 5f 10
>  tmp: a8740000 883c0000 388c 1 88400be4 8832c15c 883bb960 1fae
>  sve: a8740000 0 0 0 0 0 0 0
>  t8 a8740000 t9 0 at 0 v0 0 v1 0 k1 883b0000
>  gp a8740000 fp 0 sp 0 ra 0
> 
> PANIC: Unexpected exception
> 
> [Press reset or ENTER to restart.]


It is able to boot the old kernel that I still had on the local disk, which turned out to be a 5.1.2 kernel that I built myself (I must have done this to work around the problem that I had booting the 5.1.2 install image back in 2012):

> NetBSD 5.1.2 (GENERIC32_IP2x) #0: Mon May  7 04:43:09 PDT 2012
>        root%hoth.astro.net@localhost:/usr/src/sys/arch/sgimips/compile/GENERIC32_IP2x

Of course, that kernel now fails to mount the updated root file system, presumably due to incompatible FFS changes since 5.1.2.

But if this is indeed a boot loader issue, it seems that it must be a longstanding bug that is sensitive to something about the kernel image that has changed since 5.1.2.

Thanks again,

Tim McIntosh


> On May 21, 2024, at 12:26 AM, <iwama%t3.rim.or.jp@localhost> <iwama%t3.rim.or.jp@localhost> wrote:
> 
> Hi all,
> 
> A similar problem occurred on O2 with NetBSD-9.4.
> 
>> hinv
>                  System: IP32
>               Processor: 400 Mhz R12000, with FPU
>    Primary I-cache size: 32 Kbytes
>    Primary D-cache size: 32 Kbytes
>    Secondary cache size: 2 Mbytes
>             Memory size: 448 Mbytes
>                Graphics: CRM, Rev C
>                   Audio: A3 version 1
>               SCSI Disk: scsi(0)disk(2)
>              SCSI CDROM: scsi(0)cdrom(4)
>               SCSI Disk: scsi(1)disk(2)
>               SCSI Disk: scsi(1)disk(3)
>> version
> 
> 
> PROM Monitor (BE)
> Tue Oct 22 10:58:00 PDT 2002 
> VERSION 4.18
> O2 R5K/R7K/R10K/R12K
> IRIX 6.5.x IP32prom IP32PROM-v4
> 
>> boot -f scsi(1)disk(2)rdisk(0)partition(8)boot scsi(1)disk(2)rdisk(0)partition(0)netbsd
> 54464+1792 entry: 0x80002000
> 
> NetBSD/sgimips 9.4 Bootstrap, Revision 1.5 (Sat Apr 20 13:32:22 UTC 2024)
> 
> devopen: scsi(1)disk(2)rdisk(0)partition(0) type scsi file netbsd
> 7738672+131920 [396528+388532]=0x8419b0
> 
> Exception: <vector=Normal>
> Status register: 0x2<IPL=8,MODE=KERNEL>
> Cause register: 0x8008<CE=0,IP8,EXC=RMISS>
> Exception PC: 0x0, Exception RA: 0x8048a508
> Read TLB miss exception, bad address: 0x0
> Saved user regs in hex (&gpda 0x81061838, &_regs 0x81061a38):
> arg: 81070000 46 49 2
> tmp: 81070000 305e 5be6 80842bfc 80842bfc 8083dd5c 80842bfc 4
> sve: 81070000 0 0 0 0 0 0 0
> t8 81070000 t9 0 at 0 v0 0 v1 0 k1 807c0000
> gp 81070000 fp 0 sp 0 ra 0
> 
> PANIC: Unexpected exception
> 
> [Press reset or ENTER to restart.]
> -- 
> Yoshihiko Iwama
> 
> -----Original Message-----
> From: iwama%t3.rim.or.jp@localhost <iwama%t3.rim.or.jp@localhost> 
> Sent: Tuesday, May 21, 2024 3:42 PM
> To: 'port-mips%NetBSD.org@localhost' <port-mips%NetBSD.org@localhost>
> Subject: RE: NetBSD/sgimips 10.0 RELEASE on SGI Challenge S: INSTALL32_IP2x works, GENERIC32_IP2x kernel panics on boot after install
> 
> Hi all,
> 
> I had a similar problem on NetBSD-9.3.
> 
>>> hinv
>                  System: IP22
>               Processor: 200 Mhz R4400, with FPU
>    Primary I-cache size: 16 Kbytes
>    Primary D-cache size: 16 Kbytes
>    Secondary cache size: 1024 Kbytes
>             Memory size: 256 Mbytes
>                Graphics: Indy 24-bit
>               SCSI Disk: scsi(0)disk(1)
>               SCSI Disk: scsi(0)disk(2)
>               SCSI Disk: scsi(0)disk(3)
>              SCSI CDROM: scsi(0)cdrom(6)
>                   Audio: Iris Audio Processor: version A2 revision 4.1.0
>>> boot
> 
> NetBSD/sgimips 9.3 Bootstrap, Revision 1.5 (Thu Aug  4 15:30:37 UTC 2022)
> 
> devopen: scsi(0)disk(3)rdisk(0)partition(0) type scsi file netbsd.ecoff
> 4929840+127440=0x4b3930
> [   1.0000000] [ Kernel symbol table invalid! ]
> [   1.0000000] phys segment: 0xe000 @ 0x8002000
> [   1.0000000] adding 0xe000 @ 0x8002000 to freelist 0
> [   1.0000000] phys segment: 0x730000 @ 0x8010000
> [   1.0000000] adding 0x58000 @ 0x8010000 to freelist 0
> [   1.0000000] adding 0x22c000 @ 0x8514000 to freelist 0
> [   1.0000000] phys segment: 0xf800000 @ 0x8800000
> [   1.0000000] adding 0xf800000 @ 0x8800000 to freelist 0
> [   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
> [   1.0000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
> [   1.0000000]     2018, 2019, 2020, 2021, 2022
> [   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
> [   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
> [   1.0000000]     The Regents of the University of California.  All rights reserved.
> 
> [   1.0000000] NetBSD 9.3 (GENERIC32_IP2x) #0: Thu Aug  4 15:30:37 UTC 2022
> [   1.0000000] mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/sgimips/compile/GENERIC32_IP2x
> [   1.0000000] total memory = 256 MB
> [   1.0000000] (768 KB reserved for ARCS)
> [   1.0000000] avail memory = 245 MB
> [   1.0000000] WARNING: module error: built-in module sequencer can't find builtin dependency `midi'
> [   1.0000000] WARNING: module error: built-in module sequencer prerequisite midi failed, error 2
> [   1.0000000] mainbus0 (root): SGI-IP22 [SGI, 6909c4ea], 1 processor
> [   1.0000000] cpu0 at mainbus0: MIPS R4400 CPU (0x460) Rev. 6.0 with MIPS R4010 FPC Rev. 0.0
> [   1.0000000] cpu0: 48 TLB entries, 16MB max page size
> [   1.0000000] cpu0: 16KB/16B direct-mapped L1 instruction cache
> [   1.0000000] cpu0: 16KB/16B direct-mapped write-back L1 data cache
> [   1.0000000] cpu0: 1024KB/128B direct-mapped write-back L2 unified cache
> [   1.0000000] int0 at mainbus0 addr 0x1fbd9880
> [   1.0000000] int0: bus 100MHz, CPU 200MHz
> [   1.0000000] imc0 at mainbus0 addr 0x1fa00000: revision 3
> [   1.0000000] gio0 at imc0
> [   1.0000000] newport0 at gio0: SGI NG1 (board revision 1, cmap revision 5, xmap revision 5, vc2 revision 0), depth 24
> [   1.0000000] wsdisplay0 at newport0 kbdmux 1
> [   1.0000000] hpc0 at gio0: SGI HPC3 (onboard)
> [   1.0000000] zsc0 at hpc0 offset 0x59830
> [   1.0000000] zstty0 at zsc0 channel 1 (console i/o)
> [   1.0000000] zstty1 at zsc0 channel 0
> [   1.0000000] pckbc0 at hpc0 offset 0x59840
> [   1.0000000] pckbd0 at pckbc0 (kbd slot)
> [   1.0000000] wskbd0 at pckbd0 mux 1
> [   1.0000000] pms0 at pckbc0 (aux slot)
> [   1.0000000] wsmouse0 at pms0 mux 0
> [   1.0000000] sq0 at hpc0 offset 0x54000: SGI Seeq 80c03
> [   1.0000000] sq0: Ethernet address 08:00:69:09:c4:ea
> [   1.0000000] wdsc0 at hpc0 offset 0x44000: WD33C93B (20.0 MHz clock, BURST DMA, SCSI ID 0)
> [   1.0000000] wdsc0: microcode revision 0x0d, Fast SCSI
> [   1.0000000] scsibus0 at wdsc0: 8 targets, 8 luns per target
> [   1.0000000] haltwo0 at hpc0 offset 0x58000: HAL2 revision 4.1.0
> [   1.0000000] audio0 at haltwo0: playback
> [   1.0000000] audio0: slinear_be:16 2ch 48000Hz, blk 4096 bytes (21.3ms) for playback
> [   1.0000000] spkr0 at audio0: PC Speaker (synthesized)
> [   1.0000000] wsbell at spkr0 not configured
> [   1.0000000] pi1ppc0 at hpc0 offset 0x59800
> [   1.0000000] pi1ppc0: capabilities=0x8<PS2>
> [   1.0000000] ppbus0 at pi1ppc0
> [   1.0000000] ppbus0: No IEEE1284 device found.
> [   1.0000000] lpt0 at ppbus0: port mode = 0x1<COMPATIBLE>
> [   1.0000000] button0 at hpc0 offset 0x59850
> [   1.0000000] dsclock0 at mainbus0 addr 0x1fbe0000
> [   1.0000000] ioc0 at mainbus0 addr 0x1fbd9800: rev 0, machine Indy (Guinness), board rev 0
> [   1.0000030] scsibus0: waiting 2 seconds for devices to settle...
> [   1.0687213] sysctl: log 0x27bdffd8 root mismatch (0x8850c150)
> [   1.2677407] pid 0(system): trap: cpu0, TLB miss (load or instr. fetch) in kernel mode
> [   1.2677407] status=0x4ff03, cause=0x8, epc=0x88428100, vaddr=0
> [   1.2677407] tf=0x88003c40 ksp=0x88003ce0 ra=0x88335f28 ppl=0x8850c81c
> [   1.2677407] kernel: TLB miss (load or instr. fetch) trap
> Stopped in pid 0.1 (system) at  88428100:       lbu     t0,0(a0)
> db> q
> panic: utlbmod: 0: no pte
> [   1.2677407] cpu0: Begin traceback...
> [   1.2677407] pid -2013251120 not found
> [   1.2677407] cpu0: End traceback...
> [   1.2677407] kernel: breakpoint trap
> Stopped in pid 0.1 (system) at  88075e30:       jr      ra
>               bdslot: nop
> db>
> 
> 
>>> boot
> 
> NetBSD/sgimips 9.3 Bootstrap, Revision 1.5 (Thu Aug  4 15:30:37 UTC 2022)
> 
> devopen: scsi(0)disk(3)rdisk(0)partition(0) type scsi file netbsd
> 4929840+127440 [285568+277831]=0x55ca10
> 
> Exception: <vector=UTLB Miss>
> Status register: 0x2<IPL=8,MODE=KERNEL,EXL>
> Cause register: 0x30008008<CE=3,IP8,EXC=RMISS>
> Exception PC: 0x0, Exception RA: 0x882f12cc
> exception, bad address: 0x0
> Local I/O interrupt register 1: 0x80 <VR/GIO2>
> Local I/O interrupt register 2: 0x4 <>
> Saved user regs in hex (&gpda 0xa8740e48, &_regs 0xa8741048):
> arg: a8740000 5f 0 2
> tmp: a8740000 27c0 41ef 88578cc8 88578cc8 88575028 88578cc8 4
> sve: a8740000 0 0 0 0 0 0 0
> t8 a8740000 t9 0 at 0 v0 0 v1 0 k1 88510000
> gp a8740000 fp 0 sp 0 ra 0
> 
> PANIC: Unexpected exception
> 
> [Press reset or ENTER to restart.]
> 
> I suspect that there is a problem with the bootloader.
> I get around this problem by putting the kernel in IRIX's / partition and starting it using sash.
> -- 
> Yoshihiko Iwama
> 
> -----Original Message-----
> From: port-mips-owner%NetBSD.org@localhost <port-mips-owner%NetBSD.org@localhost> On Behalf Of Tim McIntosh
> Sent: Tuesday, May 21, 2024 2:21 PM
> To: Michael <macallan1888%gmail.com@localhost>
> Cc: port-mips%NetBSD.org@localhost
> Subject: Re: NetBSD/sgimips 10.0 RELEASE on SGI Challenge S: INSTALL32_IP2x works, GENERIC32_IP2x kernel panics on boot after install
> 
>> On May 20, 2024, at 6:36 PM, Tim McIntosh <tmcintos%eskimo.com@localhost> wrote:
>> 
>>> What I would do in your position is:
>>> - setup netboot - IIRC it's really just bootp/dhcp, load kernel via tftp, mount root via nfs.
>>> - build an INSTALL32_IP2x kernel, try to netboot it
>>> - INSTALL* is just GENERIC* with a bunch of 'no *' statements to strip
>>> it down. Enable a few of them, try again. If my theory is correct it
>>> should start crashing when the kernel image hits a certain size
>>> - you probably don't want all the stuff disabled in INSTALL32_IP2x, if
>>> you get to an image that's small enough to work and contains what you
>>> need, I'd run with that for the time being.
> 
> I don’t have a viable netboot environment currently, but what I was able to do to transfer files to the local disk was boot the installation image and then use FTP to transfer files via a temporary Python-based FTP server running on my development machine. Tedious, but it works.
> 
> So I built a modified INSTALL32_IP2x kernel with the following changes:
> 
>> --- a/sys/arch/sgimips/conf/INSTALL32_IP2x
>> +++ b/sys/arch/sgimips/conf/INSTALL32_IP2x
>> @@ -8,11 +8,11 @@ include       "arch/sgimips/conf/GENERIC32_IP2x"
>> makeoptions    COPTS="-Os -mmemcpy"
>>  # Enable the hooks used for initializing the root memory-disk.
>> -options         MEMORY_DISK_HOOKS
>> -options         MEMORY_DISK_IS_ROOT     # force root on memory disk
>> -options         MEMORY_DISK_SERVER=0    # no userspace memory disk support
>> -options         MEMORY_DISK_ROOT_SIZE=6600 # size of memory disk in blocks (3300k)
>> -options         MEMORY_DISK_RBFLAGS=RB_SINGLE   # boot in single-user mode
>> +#options         MEMORY_DISK_HOOKS
>> +#options         MEMORY_DISK_IS_ROOT     # force root on memory disk
>> +#options         MEMORY_DISK_SERVER=0    # no userspace memory disk support
>> +#options         MEMORY_DISK_ROOT_SIZE=6600 # size of memory disk in blocks (3300k)
>> +#options         MEMORY_DISK_RBFLAGS=RB_SINGLE   # boot in single-user mode
>>  # shrink kernel since ARC BIOS seems to have 8MB limit
>> options        FFS_NO_SNAPSHOT
> 
> I tried loading the ELF debug version and it appears to panic in the same way:
> 
>>>> boot
>> 
>> NetBSD/sgimips 10.0 Bootstrap, Revision 1.5 (Thu Mar 28 08:33:33 UTC 2024)
>> 
>> devopen: scsi(0)disk(3)rdisk(0)partition(0) type scsi file netbsd.gdb
>> 3446288+113248 [243344+233661]=0x3d9e04
>> 
>> Exception: <vector=UTLB Miss>
>> Status register: 0x2<IPL=8,MODE=KERNEL,EXL>
>> Cause register: 0x30008008<CE=3,IP8,EXC=RMISS>
>> Exception PC: 0x0, Exception RA: 0x881eadb4
>> exception, bad address: 0x0
>> Saved user regs in hex (&gpda 0xa8740e48, &_regs 0xa8741048):
>> arg: a8740000 0 5f 10
>> tmp: a8740000 883c0000 388c 1 88400d44 8832c15c 883bb960 1fae
>> sve: a8740000 0 0 0 0 0 0 0
>> t8 a8740000 t9 0 at 0 v0 0 v1 0 k1 883b0000
>> gp a8740000 fp 0 sp 0 ra 0
>> 
>> PANIC: Unexpected exception
>> 
>> [Press reset or ENTER to restart.]
> 
> I’ll have to try the ECOFF variant and attempt debugging when I get some more time to investigate this.
> 
> Thanks,
> Tim