It boots! Here's the config, I had the Northbridge Hardware Memory Hole Enabled.
---

include "arch/amd64/conf/GENERIC_ACPI"

#ident  "H8DCE-$Revision: 1.5 $"

maxusers        64

# don't use MPBIOS on H8DCE machines!

acpi0   at mainbus0
no options      MPBIOS
#no options     MPBIOS_SCANPCI          # already off in GENERIC_ACPI
#no options     ACPI_SCANPCI

#no options     PCI_INTR_FIXUP          # PCI interrupt routing via ACPI
#no options     ACPI_ACTIVATE_DEV
#no options     ACPICA_PEDANTIC

options         DEBUG_MEMLOAD   # for debugging Basemem/extmem settings
                                 # depends on pci mem hole setting in bios

no options      POWERNOW_K8     # doesn't seem to do anything

#
# Because gcc omits the frame pointer for any -O level, the line below
# is needed to make backtraces in DDB work.
#
makeoptions     COPTS="-O2 -fno-omit-frame-pointer"
options         DDB     # in-kernel debugger
options         DDB_HISTORY_SIZE=512    # enable history editing in DDB
makeoptions     DEBUG="-g"      # compile full symbol table

---

With this setup, the disks are not blinding. I did

dd if=/dev/rwd0a of=/dev/null bs=10k

and systat vmstat shows me about 16Mbytes/sec of throughput, with fairly high 
percentage of sys time (68%).

If I disable the hardware memory hole ... it boots! The disks seem to transfer 
about the same speed.

Maybe not a great test?

Here's the dmesg. I'll try to build an MP kernel, on the box, next.

 >> NetBSD/amd64 BIOS Boot, Revision 3.3
 >> (cak@vern.landsonar.com, Mon Aug 28 17:38:47 PDT 2006)
 >> Memory: 632/65535 k
 >  boot netbsd-H8DCE
booting hd0a:netbsd-H8DCE
6821864+856888+335248 [463872+301502]=0x960448
BIOS MEMORY MAP (10 ENTRIES):
     addr 0x0  size 0x9fc00  type 0x1
     addr 0x9fc00  size 0x400  type 0x2
     addr 0xe6000  size 0x1a000  type 0x2
     addr 0x100000  size 0xb7ed0000  type 0x1
     addr 0xb7fd0000  size 0xe000  type 0x3
     addr 0xb7fde000  size 0x22000  type 0x4
     addr 0xfec00000  size 0x1000  type 0x2
     addr 0xfee00000  size 0x1000  type 0x2
     addr 0xff780000  size 0x880000  type 0x2
     addr 0x100000000  size 0x145000000  type 0x1
loading 0x7000-0x9f000 (0x7-0x9f)
loading 0xa73000-0x1000000 (0xa73-0x1000)
loading 0x1000000-0xb7fd0000 (0x1000-0xb7fd0)
loading 0x100000000-0x245000000 (0x100000-0x245000)
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006
     The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
     The Regents of the University of California.  All rights reserved.

NetBSD 4.99.1 (H8DCE) #4: Tue Aug 29 10:21:07 PDT 2006
         cak@vern.landsonar.com:/usr/src/CURRENT/src/sys/arch/amd64/compile/H8DCE
total memory = 8143 MB
avail memory = 7848 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
timecounter: selected timecounter "i8254" frequency 1193182 Hz quality 100
mainbus0 (root)
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Dual Core AMD Opteron(tm) Processor 270, 2000.06 MHz
cpu0: features: f7dbfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features: f7dbfbff<PGE,MCA,CMOV,PAT,PSE36,MPC,NOX,MMXX,MMX>
cpu0: features: f7dbfbff<FXSR,SSE,SSE2,HTT,LONG,3DNOW2,3DNOW>
cpu0: calibrating local timer
cpu0: apic clock running at 199 MHz
cpu0: 16 page colors
cpu1 at mainbus0: apid 1 (application processor)
cpu1: not started
cpu2 at mainbus0: apid 2 (application processor)
cpu2: not started
cpu3 at mainbus0: apid 3 (application processor)
cpu3: not started
ioapic0 at mainbus0 apid 4 (I/O APIC)
ioapic0: pa 0xfec00000, version 11, 24 pins
ioapic1 at mainbus0 apid 5 (I/O APIC)
ioapic1: pa 0xdfefc000, version 11, 24 pins
acpi0 at mainbus0: Advanced Configuration and Power Interface
acpi0: fixed-feature power button present
timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
timecounter: selected timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
ACPI-Fast 24-bit timer
ACPI Object Type 'Processor' (0x0c) at acpi0 not configured
ACPI Object Type 'Processor' (0x0c) at acpi0 not configured
ACPI Object Type 'Processor' (0x0c) at acpi0 not configured
ACPI Object Type 'Processor' (0x0c) at acpi0 not configured
PNP0A03 [PCI/PCI-X Host Bridge] at acpi0 not configured
PNP0000 [AT Interrupt Controller] at acpi0 not configured
PNP0200 [AT DMA Controller] at acpi0 not configured
PNP0100 [AT Timer] at acpi0 not configured
PNP0B00 [AT Real-Time Clock] at acpi0 not configured
PNP0800 [AT-style speaker sound] at acpi0 not configured
PNP0C04 [Math Coprocessor] at acpi0 not configured
PNP0501 [16550A-compatible COM port] at acpi0 not configured
PNP0501 [16550A-compatible COM port] at acpi0 not configured
PNP0700 [PC standard floppy disk controller] at acpi0 not configured
PNP0400 [Standard LPT printer port] at acpi0 not configured
PNP0C02 [Plug and Play motherboard register resources] at acpi0 not configured
PNP0C02 [Plug and Play motherboard register resources] at acpi0 not configured
PNP0303 [IBM Enhanced (101/102-key, PS/2 mouse support)] at acpi0 not configured
PNP0C02 [Plug and Play motherboard register resources] at acpi0 not configured
PNP0C02 [Plug and Play motherboard register resources] at acpi0 not configured
PNP0C01 [System Board] at acpi0 not configured
PNP0A03 [PCI/PCI-X Host Bridge] at acpi0 not configured
acpibut0 at acpi0 (PNP0C0C-170): ACPI Power Button
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
PNP0C0F [PCI interrupt link device] at acpi0 not configured
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
NVIDIA nForce4 Memory Controller (miscellaneous memory, revision 0xa3) at pci0 
dev 0 function 0 not configured
pcib0 at pci0 dev 1 function 0
pcib0: NVIDIA nForce4 PCI-ISA bridge (rev. 0xa3)
NVIDIA nForce4 SMBus (SMBus serial bus, revision 0xa2) at pci0 dev 1 function 1 
not configured
ohci0 at pci0 dev 2 function 0: NVIDIA nForce4 USB Host Controller (rev. 0xa2)
ohci0: interrupting at ioapic0 pin 20 (irq 9)
ohci0: OHCI version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: NVIDIA OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 10 ports with 10 removable, self powered
ehci0 at pci0 dev 2 function 1: NVIDIA nForce4 USB2 Host Controller (rev. 0xa3)
ehci0: interrupting at ioapic0 pin 21 (irq 9)
ehci0: BIOS has given up ownership
ehci0: EHCI version 1.0
ehci0: companion controller, 4 ports each: ohci0
usb1 at ehci0: USB revision 2.0
uhub1 at usb1
uhub1: NVIDIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 10 ports with 10 removable, self powered
auich0 at pci0 dev 4 function 0: nForce4 AC-97 Audio
auich0: interrupting at ioapic0 pin 22 (irq 9)
auich0: ac97: Avance Logic ALC650 codec; 20 bit DAC, 18 bit ADC, Realtek 3D
auich0: ac97: ext id 5c7<AC97_22,LDAC,SDAC,CDAC,SPDIF,DRA,VRA>
viaide0 at pci0 dev 6 function 0
viaide0: NVIDIA nForce4 IDE Controller (rev. 0xf2)
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at ioapic0 pin 14 (irq 14)
atabus0 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel interrupting at ioapic0 pin 15 (irq 15)
atabus1 at viaide0 channel 1
viaide1 at pci0 dev 7 function 0
viaide1: NVIDIA nForce4 Serial ATA Controller (rev. 0xf3)
viaide1: bus-master DMA support present
viaide1: primary channel wired to native-PCI mode
viaide1: using ioapic0 pin 20 (irq 9) for native-PCI interrupt
atabus2 at viaide1 channel 0
viaide1: secondary channel wired to native-PCI mode
atabus3 at viaide1 channel 1
viaide2 at pci0 dev 8 function 0
viaide2: NVIDIA nForce4 Serial ATA Controller (rev. 0xf3)
viaide2: bus-master DMA support present
viaide2: primary channel wired to native-PCI mode
viaide2: using ioapic0 pin 21 (irq 5) for native-PCI interrupt
atabus4 at viaide2 channel 0
viaide2: secondary channel wired to native-PCI mode
atabus5 at viaide2 channel 1
ppb0 at pci0 dev 9 function 0: NVIDIA nForce4 PCI Host Bridge (rev. 0xa2)
pci1 at ppb0 bus 3
pci1: i/o space, memory space enabled
nfe0 at pci0 dev 10 function 0: ioapic0 pin 22 (irq 11), address 00:30:48:57:7f:b8
makphy0 at nfe0 phy 1: Marvell 88E1111 Gigabit PHY, rev. 2
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 
1000baseT-FDX, auto
ppb1 at pci0 dev 13 function 0: NVIDIA nForce4 PCIe Host Bridge (rev. 0xa3)
pci2 at ppb1 bus 2
pci2: memory space enabled, rd/line, wr/inv ok
ppb2 at pci0 dev 14 function 0: NVIDIA nForce4 PCIe Host Bridge (rev. 0xa3)
pci3 at ppb2 bus 1
pci3: i/o space, memory space enabled, rd/line, wr/inv ok
vga0 at pci3 dev 0 function 0: NVIDIA GeForce 6200TC (rev. 0xa1)
vga0: WARNING: ignoring 64-bit BAR @ 0x14
vga0: WARNING: ignoring 64-bit BAR @ 0x1c
wsdisplay0 at vga0 kbdmux 1
wsmux1: connecting to wsdisplay0
pchb0 at pci0 dev 24 function 0
pchb0: Advanced Micro Devices AMD64 HyperTransport configuration (rev. 0x00)
pchb1 at pci0 dev 24 function 1
pchb1: Advanced Micro Devices AMD64 Address Map configuration (rev. 0x00)
pchb2 at pci0 dev 24 function 2
pchb2: Advanced Micro Devices AMD64 DRAM configuration (rev. 0x00)
pchb3 at pci0 dev 24 function 3
pchb3: Advanced Micro Devices AMD64 Miscellaneous configuration (rev. 0x00)
pchb4 at pci0 dev 25 function 0
pchb4: Advanced Micro Devices AMD64 HyperTransport configuration (rev. 0x00)
pchb5 at pci0 dev 25 function 1
pchb5: Advanced Micro Devices AMD64 Address Map configuration (rev. 0x00)
pchb6 at pci0 dev 25 function 2
pchb6: Advanced Micro Devices AMD64 DRAM configuration (rev. 0x00)
pchb7 at pci0 dev 25 function 3
pchb7: Advanced Micro Devices AMD64 Miscellaneous configuration (rev. 0x00)
isa0 at pcib0
lpt0 at isa0 port 0x378-0x37b irq 7
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0 mux 1
wskbd0: connecting to wsdisplay0
attimer0 at isa0 port 0x40-0x43: AT Timer
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker (CPU-intensive output)
sysbeep0 at pcppi0
pcppi0: attached to attimer0
ioapic0: enabling
ioapic1: enabling
timecounter: Timecounter "TSC" frequency 2000124980 Hz quality 800
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
auich0: measured ac97 link rate at 47998 Hz, will use 48000 Hz
audio0 at auich0: full duplex, mmap, independent
Kernelized RAIDframe activated
atapibus0 at atabus0: 2 targets
cd0 at atapibus0 drive 1: <LITE-ON DVDRW SHW-160P6S, , PS0A> cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 4 (Ultra/66)
cd0(viaide0:0:1): using PIO mode 4, Ultra-DMA mode 4 (Ultra/66) (using DMA)
wd0 at atabus2 drive 0: <WDC WD800JD-23LSA0>
wd0: drive supports 16-sector PIO transfers, LBA48 addressing
wd0: 76324 MB, 155072 cyl, 16 head, 63 sec, 512 bytes/sect x 156312576 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd0(viaide1:0:0): using PIO mode 4, Ultra-DMA mode 6 (Ultra/133) (using DMA)
wd1 at atabus3 drive 0: <WDC WD4000YR-01PLB0>
wd1: drive supports 16-sector PIO transfers, LBA48 addressing
wd1: 372 GB, 775221 cyl, 16 head, 63 sec, 512 bytes/sect x 781422768 sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd1(viaide1:1:0): using PIO mode 4, Ultra-DMA mode 6 (Ultra/133) (using DMA)
boot device: wd0
root on wd0a dumps on wd0b
root file system type: ffs
Tue Aug 29 11:15:59 PDT 2006
swapctl: adding /dev/wd0b as swap device at priority 0
Checking for botched superblock upgrades: done.
Starting file system checks:
/dev/rwd0a: file system is clean; not checking
Setting tty flags.
Setting sysctl variables:
Starting network.
Hostname: myrtle.landsonar.com
Configuring network interfaces: nfe0.
Adding interface aliases:
Starting dhclient.
Internet Systems Consortium DHCP Client V3.0.3
Copyright 2004-2005 Internet Systems Consortium.
All rights reserved.
For info, please visit http://www.isc.org/products/DHCP

Listening on BPF/nfe0/00:30:48:57:7f:b8
Sending on   BPF/nfe0/00:30:48:57:7f:b8
Sending on   Socket/fallback
DHCPREQUEST on nfe0 to 255.255.255.255 port 67
DHCPREQUEST on nfe0 to 255.255.255.255 port 67
DHCPACK from 10.0.1.1
bound to 10.0.100.30 -- renewal in 254 seconds.
Building databases...
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)
Starting syslogd.
Checking for core dump...
savecore: no core dump
Setting date via ntp.
Mounting all filesystems...
Clearing /tmp.
Checking quotas: done.
Starting virecover.
Starting local daemons:.
Updating motd.
Starting ntpd.
Starting sshd.