Re: -current amd64 does not boot on huge machine (80 cores, RAM 1TB)

[Nicolas Joly <njoly%pasteur.fr@localhost>]

On Wed, Oct 05, 2011 at 11:56:09PM +0200, Nicolas Joly wrote:
> On Wed, Oct 05, 2011 at 11:36:38PM +0200, Zafer Aydo?an wrote:
> > 2011/10/3 Nicolas Joly <njoly%pasteur.fr@localhost>:
> [...]
> > > We just got, at work, a new toy ... This is a Supermicro SuperServer
> > > 5086B-TRF[1] machine, with 80 cores and RAM 1TB. Unfortunately, i
> > > cannot boot -current amd64 on it.
> [...]
> > Very nice Rack with 8 * 10 cores !
> > I'm not sure, but I think NetBSD can't boot > 32 cpu's at the moment.
> 
> Yep, that's right. Now that we have machines with more than that,
> priority fixing it will increase in the future ;)
> 
> > I heard rumours about a patch flying around, that will boot 255 cpu's.
> > Anyway. It would be nice to get this thing running!
> 
> I finally got it to boot ... but with both SMP :( and ACPI disabled
> (and coretemp via userconf which paniced the machine) on a non
> DIAGNOSTIC kernel (dmesg attached). Once done, we are missing the
> 3Ware 9750 Series RAID controller to have some disk to install on.

Forgot the full dmesg.

-- 
Nicolas Joly

Projects and Developments in Bioinformatics
Institut Pasteur, Paris.

> boot -12c
[...]
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 5.99.56 (GENERIC) #0: Mon Oct  3 13:44:13 CEST 2011
        
njoly%lanfeust.sis.pasteur.fr@localhost:/local/src/NetBSD/obj.amd64/sys/arch/amd64/compile/GENERIC
total memory = 1023 GB
avail memory = 994 GB
userconf: configure system autoconfiguration:
uc> disable coretemp
[191] coretemp* disabled
uc> exit
Continuing...
mainbus0 (root)
cpu0 at mainbus0/local/src/NetBSD/src/sys/arch/x86/x86/mtrr_i686.c: FIXME: more 
than 8 MTRRs (10)
: Intel(R) Xeon(R) CPU E7- 8870  @ 2.40GHz, id 0x206f2
ipmi0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1
pchb0 at pci0 dev 0 function 0: vendor 0x8086 product 0x3407 (rev. 0x22)
ppb0 at pci0 dev 1 function 0: vendor 0x8086 product 0x3408 (rev. 0x22)
ppb0: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci1 at ppb0 bus 1
ppb1 at pci0 dev 3 function 0: vendor 0x8086 product 0x340a (rev. 0x22)
ppb1: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci2 at ppb1 bus 2
ppb2 at pci0 dev 7 function 0: vendor 0x8086 product 0x340e (rev. 0x22)
ppb2: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0: vendor 0x10b5 product 0x8648 (rev. 0xbb)
ppb3: PCI Express 2.0 <Upstream Port of PCI-E Switch>
pci4 at ppb3 bus 4
ppb4 at pci4 dev 4 function 0: vendor 0x10b5 product 0x8648 (rev. 0xbb)
ppb4: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci5 at ppb4 bus 5
ppb5 at pci4 dev 5 function 0: vendor 0x10b5 product 0x8648 (rev. 0xbb)
ppb5: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci6 at ppb5 bus 6
ppb6 at pci4 dev 8 function 0: vendor 0x10b5 product 0x8648 (rev. 0xbb)
ppb6: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci7 at ppb6 bus 7
ppb7 at pci4 dev 9 function 0: vendor 0x10b5 product 0x8648 (rev. 0xbb)
ppb7: PCI Express 2.0 <Downstream Port of PCI-E Switch>
pci8 at ppb7 bus 8
vendor 0x13c1 product 0x1010 (RAID mass storage, revision 0x05) at pci8 dev 0 
function 0 not configured
vendor 0x8086 product 0x342d (interrupt system, interface 0x20, revision 0x22) 
at pci0 dev 19 function 0 not configured
vendor 0x8086 product 0x342e (interrupt system, revision 0x22) at pci0 dev 20 
function 0 not configured
vendor 0x8086 product 0x3422 (interrupt system, revision 0x22) at pci0 dev 20 
function 1 not configured
vendor 0x8086 product 0x3423 (interrupt system, revision 0x22) at pci0 dev 20 
function 2 not configured
vendor 0x8086 product 0x3438 (interrupt system, revision 0x22) at pci0 dev 20 
function 3 not configured
vendor 0x8086 product 0x342f (interrupt system, interface 0x20, revision 0x22) 
at pci0 dev 21 function 0 not configured
uhci0 at pci0 dev 26 function 0: vendor 0x8086 product 0x3a37 (rev. 0x00)
uhci0: interrupting at irq 11
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 26 function 1: vendor 0x8086 product 0x3a38 (rev. 0x00)
uhci1: interrupting at irq 10
usb1 at uhci1: USB revision 1.0
uhci2 at pci0 dev 26 function 2: vendor 0x8086 product 0x3a39 (rev. 0x00)
uhci2: interrupting at irq 5
usb2 at uhci2: USB revision 1.0
ehci0 at pci0 dev 26 function 7: vendor 0x8086 product 0x3a3c (rev. 0x00)
ehci0: interrupting at irq 5
ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2
usb3 at ehci0: USB revision 2.0
uhci3 at pci0 dev 29 function 0: vendor 0x8086 product 0x3a34 (rev. 0x00)
uhci3: interrupting at irq 5
usb4 at uhci3: USB revision 1.0
uhci4 at pci0 dev 29 function 1: vendor 0x8086 product 0x3a35 (rev. 0x00)
uhci4: interrupting at irq 11
usb5 at uhci4: USB revision 1.0
uhci5 at pci0 dev 29 function 2: vendor 0x8086 product 0x3a36 (rev. 0x00)
uhci5: interrupting at irq 5
usb6 at uhci5: USB revision 1.0
ehci1 at pci0 dev 29 function 7: vendor 0x8086 product 0x3a3a (rev. 0x00)
ehci1: interrupting at irq 5
ehci1: companion controllers, 2 ports each: uhci3 uhci4 uhci5
usb7 at ehci1: USB revision 2.0
ppb8 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0x90)
pci9 at ppb8 bus 9
vga0 at pci9 dev 1 function 0: vendor 0x102b product 0x0532 (rev. 0x0a)
wsdisplay0 at vga0 kbdmux 1
drm at vga0 not configured
pcib0 at pci0 dev 31 function 0: vendor 0x8086 product 0x3a16 (rev. 0x00)
piixide0 at pci0 dev 31 function 2: Intel 82801JI Serial ATA Controller (ICH10) 
(rev. 0x00)
piixide0: using irq 11 for native-PCI interrupt
atabus0 at piixide0 channel 0
atabus1 at piixide0 channel 1
ichsmb0 at pci0 dev 31 function 3: vendor 0x8086 product 0x3a30 (rev. 0x00)
ichsmb0: interrupting at irq 5
iic0 at ichsmb0: I2C bus
piixide1 at pci0 dev 31 function 5: Intel 82801JI Serial ATA Controller (ICH10) 
(rev. 0x00)
piixide1: using irq 11 for native-PCI interrupt
atabus2 at piixide1 channel 0
atabus3 at piixide1 channel 1
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
attimer0 at isa0 port 0x40-0x43
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
attimer0: attached to pcppi0
uhub0 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2 at usb2: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub3 at usb3: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub4 at usb4: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub5 at usb5: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub6 at usb6: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub7 at usb7: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
umass0 at uhub7 port 1 configuration 1 interface 0
umass0: vendor 0x07ab product 0xfcb4, rev 2.00/0.00, addr 2
scsibus0 at umass0: 2 targets, 1 lun per target
cd0 at scsibus0 target 0 lun 0: <Slimtype, DVD A  DS8A3L, YP51> cdrom removable
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex, playback, capture
boot device: cd0
root on cd0a dumps on cd0b
uhidev0 at uhub0 port 1 configuration 1 interface 0
uhidev0: American Megatrends Inc. Virtual Keyboard and Mouse, rev 1.10/1.00, 
addr 2, iclass 3/1
ukbd0 at uhidev0
wskbd0 at ukbd0 mux 1
uhidev1 at uhub0 port 1 configuration 1 interface 1
root file system type: cd9660
uhidev1: American Megatrends Inc. Virtual Keyboard and Mouse, rev 1.10/1.00, 
addr 2, iclass 3/1
ums0 at uhidev1: 3 buttons
wsmouse0 at ums0 mux 0
warning: no /dev/console
uhidev2 at uhub4 port 2 configuration 1 interface 0
uhidev2: Tangtop USBPS2, rev 1.10/0.01, addr 2, iclass 3/1
ukbd1 at uhidev2
wskbd1 at ukbd1 mux 1
uhidev3 at uhub4 port 2 configuration 1 interface 1
uhidev3: Tangtop USBPS2, rev 1.10/0.01, addr 2, iclass 3/1
ums1 at uhidev3: 3 buttons and Z dir
wsmouse1 at ums1 mux 0
Created tmpfs /dev (1490944 byte, 2880 inodes)
[...]