RE: problems when multiple mfi cards in system?

["Scott Burns" <Scott.Burns%SeQent.Com@localhost>]

I don't have any experience with this particular configuration but as a
starting point you may want to try updating both cards firmware as there are
newer versions for each.

        mfi0: PERC H810 Adapter version 21.0.1-0132
        Latest: 21.2.0-0007, A04

        mfi1: PERC 6/i Integrated version 6.2.0-0013
        Latest: 6.3.1-0003, A14


Scott...
-----Original Message-----
From: tech-kern-owner%NetBSD.org@localhost 
[mailto:tech-kern-owner%NetBSD.org@localhost] On
Behalf Of Mark Davies
Sent: September-19-14 12:29 AM
To: tech-kern%netbsd.org@localhost
Subject: problems when multiple mfi cards in system?

I have a Dell PERC H810 pcie card and an external disk array attached to it.
If I put this into a desktop 6.1_STABLE/amd64 system I can happily read and
write terabytes from the external array.  However if I move the card to a
Dell poweredge r610 (which also has an internal PERC 6/i) and try reading
from the external array, after a few minutes the entire machine just freezes
solid.  There doesn't seem to be any problem reading from the internal
disks.

Any ideas how to identify whats failing here and how to fix it?

dmesg for the r610 is below.


cheers
mark

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 6.1_STABLE (GENERIC) #25: Mon Jun  9 12:44:09 NZST 2014
        
mark%turakirae.ecs.vuw.ac.nz@localhost:/local/SAVE/6_64.obj/src/work/6/src/sys/arch/am
d64/compile/GENERIC
total memory = 8182 MB
avail memory = 7929 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100 Dell Inc.
PowerEdge R610
mainbus0 (root)
cpu0 at mainbus0 apid 16: Intel(R) Xeon(R) CPU           E5504  @ 
2.00GHz, id 0x106a5
cpu1 at mainbus0 apid 18: Intel(R) Xeon(R) CPU           E5504  @ 
2.00GHz, id 0x106a5
cpu2 at mainbus0 apid 20: Intel(R) Xeon(R) CPU           E5504  @ 
2.00GHz, id 0x106a5
cpu3 at mainbus0 apid 22: Intel(R) Xeon(R) CPU           E5504  @ 
2.00GHz, id 0x106a5
ioapic0 at mainbus0 apid 0: pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0 apid 1: pa 0xfec80000, version 20, 24 pins
acpi0 at mainbus0: Intel ACPICA 20110623
acpi0: X/RSDT: OemId <DELL  ,PE_SC3  ,00000001>, AslId <DELL,00000001>
acpi0: SCI interrupting at int 9
timecounter: Timecounter "ACPI-Safe" frequency 3579545 Hz quality 900
hpet0 at acpi0: high precision event timer (mem 0xfed00000-0xfed00400)
timecounter: Timecounter "hpet0" frequency 14318180 Hz quality 2000 WHEA
(PNP0C33) at acpi0 not configured SPK (PNP0C01) at acpi0 not configured
attimer1 at acpi0 (TMR, PNP0100): io 0x40-0x5f irq 0 COMA (PNP0501) at acpi0
not configured COMB (PNP0501) at acpi0 not configured MBIO (PNP0C01) at
acpi0 not configured NIPM (IPI0001) at acpi0 not configured
MBI1 (PNP0C01) at acpi0 not configured
PEHB (PNP0C02) at acpi0 not configured
VTD (PNP0C02) at acpi0 not configured
ipmi0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0: vendor 0x8086 product 0x3403 (rev. 
0x13)
ppb0 at pci0 dev 1 function 0: vendor 0x8086 product 0x3408 (rev. 
0x13)
ppb0: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled, rd/line, wr/inv ok
bnx0 at pci1 dev 0 function 0: Broadcom NetXtreme II BCM5709 1000Base- T
bnx0: Ethernet address 00:22:19:60:83:95
bnx0: interrupting at ioapic1 pin 4
brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
bnx1 at pci1 dev 0 function 1: Broadcom NetXtreme II BCM5709 1000Base- T
bnx1: Ethernet address 00:22:19:60:83:97
bnx1: interrupting at ioapic1 pin 16
brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
ppb1 at pci0 dev 3 function 0: vendor 0x8086 product 0x340a (rev. 
0x13)
ppb1: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled, rd/line, wr/inv ok
bnx2 at pci2 dev 0 function 0: Broadcom NetXtreme II BCM5709 1000Base- T
bnx2: Ethernet address 00:22:19:60:83:99
bnx2: interrupting at ioapic1 pin 0
brgphy2 at bnx2 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
bnx3 at pci2 dev 0 function 1: Broadcom NetXtreme II BCM5709 1000Base- T
bnx3: Ethernet address 00:22:19:60:83:9b
bnx3: interrupting at ioapic1 pin 10
brgphy3 at bnx3 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
brgphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
ppb2 at pci0 dev 7 function 0: vendor 0x8086 product 0x340e (rev. 
0x13)
ppb2: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci3 at ppb2 bus 4
pci3: i/o space, memory space enabled, rd/line, wr/inv ok
ppb3 at pci0 dev 9 function 0: vendor 0x8086 product 0x3410 (rev. 
0x13)
ppb3: PCI Express 2.0 <Root Port of PCI-E Root Complex>
pci4 at ppb3 bus 5
pci4: i/o space, memory space enabled, rd/line, wr/inv ok
mfi0 at pci4 dev 0 function 0: Dell PERC H810 Adapter
mfi0: interrupting at ioapic1 pin 8
mfi0: PERC H810 Adapter version 21.0.1-0132
mfi0: logical drives 1, 1024MB RAM, BBU type BBU, status good
scsibus0 at mfi0: 64 targets, 8 luns per target vendor 0x8086 product 0x342e
(interrupt system, revision 0x13) at pci0 dev 20 function 0 not configured
vendor 0x8086 product 0x3422 (interrupt system, revision 0x13) at pci0 dev
20 function 1 not configured vendor 0x8086 product 0x3423 (interrupt system,
revision 0x13) at pci0 dev 20 function 2 not configured
uhci0 at pci0 dev 26 function 0: vendor 0x8086 product 0x2937 (rev. 
0x02)
uhci0: interrupting at ioapic0 pin 17
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 26 function 1: vendor 0x8086 product 0x2938 (rev. 
0x02)
uhci1: interrupting at ioapic0 pin 18
usb1 at uhci1: USB revision 1.0
ehci0 at pci0 dev 26 function 7: vendor 0x8086 product 0x293c (rev. 
0x02)
ehci0: interrupting at ioapic0 pin 19
ehci0: BIOS has given up ownership
ehci0: EHCI version 1.0
ehci0: companion controllers, 2 ports each: uhci0 uhci1
usb2 at ehci0: USB revision 2.0
ppb4 at pci0 dev 28 function 0: vendor 0x8086 product 0x2940 (rev. 
0x02)
ppb4: PCI Express 1.0 <Root Port of PCI-E Root Complex>
pci5 at ppb4 bus 3
pci5: i/o space, memory space enabled, rd/line, wr/inv ok
mfi1 at pci5 dev 0 function 0: Dell PERC 6/i integrated
mfi1: interrupting at ioapic0 pin 16
mfi1: PERC 6/i Integrated version 6.2.0-0013
mfi1: logical drives 1, 256MB RAM, BBU type BBU, status good
scsibus1 at mfi1: 64 targets, 8 luns per target
uhci2 at pci0 dev 29 function 0: vendor 0x8086 product 0x2934 (rev. 
0x02)
uhci2: interrupting at ioapic0 pin 21
usb3 at uhci2: USB revision 1.0
uhci3 at pci0 dev 29 function 1: vendor 0x8086 product 0x2935 (rev. 
0x02)
uhci3: interrupting at ioapic0 pin 20
usb4 at uhci3: USB revision 1.0
ehci1 at pci0 dev 29 function 7: vendor 0x8086 product 0x293a (rev. 
0x02)
ehci1: interrupting at ioapic0 pin 21
ehci1: EHCI version 1.0
ehci1: companion controllers, 2 ports each: uhci2 uhci3
usb5 at ehci1: USB revision 2.0
ppb5 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 
0x92)
pci6 at ppb5 bus 6
pci6: i/o space, memory space enabled
vga0 at pci6 dev 3 function 0: vendor 0x102b product 0x0532 (rev. 
0x0a)
wsdisplay0 at vga0 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
drm at vga0 not configured
ichlpcib0 at pci0 dev 31 function 0: vendor 0x8086 product 0x2918 (rev.
0x02)
timecounter: Timecounter "ichlpcib0" frequency 3579545 Hz quality 1000
ichlpcib0: 24-bit timer
ichlpcib0: TCO (watchdog) timer configured.
piixide0 at pci0 dev 31 function 2: Intel 82801I Serial ATA Controller
(ICH9) (rev. 0x02)
piixide0: bus-master DMA support present
piixide0: primary channel configured to native-PCI mode
piixide0: using ioapic0 pin 23 for native-PCI interrupt
atabus0 at piixide0 channel 0
piixide0: secondary channel configured to native-PCI mode
atabus1 at piixide0 channel 1
isa0 at ichlpcib0
tpm0 at isa0 iomem 0xfed40000-0xfed44fff irq 7: device 0x4a100000 rev 0x4e
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
attimer1: attached to pcppi0
acpicpu0 at cpu0: ACPI CPU
acpicpu0: C1: FFH, lat   1 us, pow  1000 mW
acpicpu0: C3: FFH, lat  96 us, pow   350 mW
acpicpu0: P0: FFH, lat  10 us, pow 80000 mW, 1995 MHz
acpicpu0: P1: FFH, lat  10 us, pow 66000 mW, 1862 MHz
acpicpu0: P2: FFH, lat  10 us, pow 55000 mW, 1729 MHz
acpicpu0: P3: FFH, lat  10 us, pow 45000 mW, 1596 MHz
coretemp0 at cpu0: thermal sensor, 1 C resolution
acpicpu1 at cpu1: ACPI CPU
coretemp1 at cpu1: thermal sensor, 1 C resolution
acpicpu2 at cpu2: ACPI CPU
coretemp2 at cpu2: thermal sensor, 1 C resolution
acpicpu3 at cpu3: ACPI CPU
coretemp3 at cpu3: thermal sensor, 1 C resolution
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
timecounter: Timecounter "TSC" frequency 1995099040 Hz quality 3000
scsibus0: waiting 2 seconds for devices to settle...
scsibus1: waiting 2 seconds for devices to settle...
uhub0 at usb3: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhub2 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
uhub3 at usb5: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub3: 4 ports with 4 removable, self powered
uhub4 at usb4: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub4: 2 ports with 2 removable, self powered
uhub5 at usb2: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub5: 4 ports with 4 removable, self powered
sd0 at scsibus0 target 0 lun 0: <DELL, PERC H810, 3.13> disk fixed
sd0: fabricating a geometry
sd0: 8382 GB, 8583168 cyl, 64 head, 32 sec, 512 bytes/sect x
17578328064 sectors
sd0: fabricating a geometry
sd0: GPT GUID: e8829692-32f1-11e4-863d-180373e73b4d
dk0 at sd0: e8829699-32f1-11e4-863d-180373e73b4d
dk0: 17578327967 blocks at 64, type: ffs
sd0: tagged queueing
sd1 at scsibus1 target 0 lun 0: <DELL, PERC 6/i, 1.22> disk fixed
sd1: fabricating a geometry
sd1: 557 GB, 571136 cyl, 64 head, 32 sec, 512 bytes/sect x 1169686528
sectors
sd1: fabricating a geometry
sd1: tagged queueing
uhub6 at uhub5 port 3: vendor 0x0424 product 0x2514, class 9/0, rev
2.00/0.00, addr 2
uhub6: multiple transaction translators
uhub6: 3 ports with 3 removable, self powered
uhidev0 at uhub2 port 1 configuration 1 interface 0
uhidev0: Avocent Dell 03R874, rev 1.10/1.00, addr 2, iclass 3/1
ukbd0 at uhidev0
uhidev1 at uhub0 port 2 configuration 1 interface 0
uhidev1: Avocent USB Composite Device-0, rev 1.10/0.00, addr 2, iclass
3/1
ukbd1 at uhidev1
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev2 at uhub2 port 1 configuration 1 interface 1
uhidev2: Avocent Dell 03R874, rev 1.10/1.00, addr 2, iclass 3/1
uhidev2: 3 report ids
ums0 at uhidev2 reportid 1: 5 buttons and Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev2 reportid 2: input=2, output=0, feature=0
uhid1 at uhidev2 reportid 3: input=1, output=0, feature=0
wskbd2 at ukbd1 mux 1
wskbd2: connecting to wsdisplay0
uhidev3 at uhub0 port 2 configuration 1 interface 1
uhidev3: Avocent USB Composite Device-0, rev 1.10/0.00, addr 2, iclass
3/1
ums1 at uhidev3: 3 buttons and Z dir
wsmouse1 at ums1 mux 0
atapibus0 at atabus0: 2 targets
cd0 at atapibus0 drive 0: <TSSTcorp DVD+/-RW TS-L633C, l1M3456789KLMNOP,
D150> cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5
(Ultra/100)
cd0(piixide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using
DMA) Kernelized RAIDframe activated
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex, playback, capture boot device: sd1 root on sd1a
dumps on sd1b root file system type: ffs
/var: replaying log to disk
/usr: replaying log to disk
mfi1: normal state on 'mfi1:0' (online)
mfi0: normal state on 'mfi0:0' (online)
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)
ipmi0: version 2.0 interface KCS iobase 0xca8/8 spacing 4
/local: replaying log to disk
/am/state-opera/home1: replaying log to disk
/am/state-opera/x-home: replaying log to disk
/am/state-opera/vol/scratch: replaying log to disk
/scratch: replaying log to disk
/backup/X: replaying log to disk
Accounting started