>Number:         30124
>Category:       kern
>Synopsis:       FAST_IPSEC: "ping -s 3000" trips assertion failure
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue May 03 21:15:00 +0000 2005
>Originator:     Nathan J. Williams
>Release:        NetBSD 3.99.3 (2005-05-02)
>Organization:
	Wasabi Systems, Inc.
>Environment:
	
	
System: NetBSD marvin-the-martian.nathanw.com 3.99.3 NetBSD 3.99.3 (MARVIN) #94: Mon May 2 21:52:17 EDT 2005 nathanw@marvin-the-martian.nathanw.com:/nbsd/src/sys/arch/i386/compile/MARVIN i386

System: NetBSD mac-g4.nathanw.com 3.99.3 NetBSD 3.99.3 (G4) #85: Tue May  3 15:59:03 EDT 2005 nathanw@marvin-the-martian.nathanw.com:/nbsd/src/sys/arch/macppc/compile/G4 macppc

Architecture: powerpc
Machine: macppc
>Description:

Configure a kernel with FAST_IPSEC.

Set up ESP to another machine with a simple /etc/ipsec.conf (fed to setkey -c):

add 10.1.0.15 10.1.0.5 esp 1236 -E rijndael-cbc 0x79d06d135aadaba411ee0663fbcf969bc0137e91b0677e39; 
add 10.1.0.5 10.1.0.15 esp 1237 -E rijndael-cbc 0x92a933b4621cd5599d53834bdf3012d22cf460f8589f7166; 
spdadd 10.1.0.15 10.1.0.5 any -P out ipsec esp/transport//use;

Ping the other machine with a packet with more than 1940 data bytes:

ping -c 1 -s 1941
(1940 doesn't cause a problem; 1941, 1942, and 3000 all cause problems)


panic: kernel diagnostic assertion "remain < (256 - ((size_t)(unsigned long)(&(((struct _mbuf_dummy *)0)->M_dat.M_databuf))))" failed: file "../../../../netipsec/ipsec_mbuf.c", line 245
Stopped in pid 15661.1 (ping) at        netbsd:cpu_Debugger+0x18:       lwz     r
11, r1, 0x0
db> t
0xd56bba00: at panic+0x19c
0xd56bba90: at __assert+0x3c
0xd56bbac0: at m_makespace+0x404
0xd56bbb00: at esp_output+0x29c
0xd56bbb60: at ipsec4_process_packet+0x1d8
0xd56bbbe0: at ip_output+0x550
0xd56bbce0: at rip_output+0x15c
0xd56bbd70: at rip_usrreq+0x270
0xd56bbdb0: at sosend+0x2d8
0xd56bbe10: at sendit+0x170
0xd56bbe80: at sys_sendto+0x64
0xd56bbed0: at syscall_plain+0xe0
0xd56bbf40: user SC trap #133 by 0xeff37b24: srr1=0xf032
            r1=0xffffd890 cr=0x44000024 xer=0 ctr=0xeff37b1c
db> 

The network interface is:
wm0 at pci1 dev 4 function 0: Intel i82545GM 1000BASE-T Ethernet, rev. 4
wm0: interrupting at irq 25
wm0: Ethernet address 00:04:23:b2:30:90
makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 5
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto

wm0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=3f80<TSO4>
        enabled=0
        address: 00:04:23:b2:30:90
        media: Ethernet autoselect (1000baseT full-duplex,flowcontrol,rxpause,txpause)
        status: active
        inet 10.1.0.15 netmask 0xffffff00 broadcast 10.1.0.255

The problem doesn't occur with FAST_IPSEC removed.

With des-cbc instead of rijndael-cbc, the first problematic size
option is -s 1949.

The problem does not seem to appear on the recieve side; pinging with
1950-byte packets from the other machine works fine. This doesn't make
a lot of sense to me; perhaps it has to do with how the mbufs are
structured on an incoming packet vs. a packet constructed by the ping
command?

The problem was reproducable on an i386 system, also with a wm
interface. A crash dump was generated; GDB produced the following
trace on the dump:

(gdb) target kcore netbsd.0.core
panic: kernel %sassertion "%s" failed: file "%s", line %d
#0  0x00000000 in ?? ()
(gdb) where
#0  0x00000000 in ?? ()
#1  0xc04e3000 in ?? ()
#2  0xc028afb1 in cpu_reboot (howto=256, bootstr=0x0)
    at ../../../../arch/i386/i386/machdep.c:752
#3  0xc02214b0 in panic (
    fmt=0xc03d1280 "kernel %sassertion \"%s\" failed: file \"%s\", line %d")
    at ../../../../kern/subr_prf.c:242
#4  0xc0334ddc in __assert (t=0xc036daf9 "diagnostic ", 
    f=0xc0397420 "../../../../netipsec/ipsec_mbuf.c", l=245, 
    e=0xc0397460 "remain < (256 - ((size_t)(unsigned long)(&(((struct _mbuf_dummy *)0)->M_dat.M_databuf))))") at ../../../../../../lib/libkern/__assert.c:45
#5  0xc013cfef in m_makespace (m0=0xc150db00, skip=20, hlen=24, off=0xcc3abc0c)
    at x86/intr.h:160
#6  0xc0140809 in esp_output (m=0xc150db00, isr=0xc17e8280, mp=0x0, skip=20, 
    protoff=9) at ../../../../netipsec/xform_esp.c:742
#7  0xc013de23 in ipsec4_process_packet (m=0xc150db00, isr=0xc17e8280, 
    flags=38, tunalready=0) at ../../../../netipsec/ipsec_output.c:486
#8  0xc0115ed8 in ip_output (m0=0xc150db00)
    at ../../../../netinet/ip_output.c:765
#9  0xc0117e90 in rip_output (m=0xc150db00) at ../../../../netinet/raw_ip.c:374
#10 0xc0118380 in rip_usrreq (so=0xc15076c0, req=9, m=0xc150db00, 
    nam=0xc1518100, control=0x0, p=0xcc3644d4)
    at ../../../../netinet/raw_ip.c:631
#11 0xc02383e3 in sosend (so=0xc15076c0, addr=0xc1518100, uio=0xcc3abea4, 
    top=0xc150db00, control=0x0, flags=0, p=0xcc3644d4)
    at ../../../../kern/uipc_socket.c:886
#12 0xc023c4f5 in sendit (p=0xcc3644d4, s=3, mp=0xcc3abf14, flags=0, 
    retsize=0xcc3abf5c) at ../../../../kern/uipc_syscalls.c:538
#13 0xc023c308 in sys_sendto (l=0xcc367190, v=0xcc3abf64, retval=0xcc3abf5c)
    at ../../../../kern/uipc_syscalls.c:420
#14 0xc02931db in syscall_plain (frame=0xcc3abfa8)
    at ../../../../arch/i386/i386/syscall.c:161
(gdb) f 5
#5  0xc013cfef in m_makespace (m0=0xc150db00, skip=20, hlen=24, off=0xcc3abc0c)
    at x86/intr.h:160
160                     Xspllower(nlevel);
(gdb) p/x *m0
$10 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, mh_data = 0xcb03e838, 
    mh_owner = 0x0, mh_len = 0x7b1, mh_flags = 0x9000003, 
    mh_paddr = 0x3ed25b00, mh_type = 0x1}, M_dat = {MH = {MH_pkthdr = {
        rcvif = 0x0, tags = {slh_first = 0x0}, len = 0x7b1, csum_flags = 0x0, 
        csum_data = 0x0, segsz = 0x79f0b8c0}, MH_dat = {MH_ext = {
          ext_buf = 0xcb03e800, ext_free = 0x0, ext_arg = 0xc03f5b60, 
          ext_size = 0x800, ext_type = 0xbbd604b8, ext_nextref = 0xc150db00, 
          ext_prevref = 0xc150db00, ext_un = {extun_paddr = 0x3ecbc800, 
            extun_pgs = {0x3ecbc800, 0x5ee88aa0, 0x66792ac4, 0xb17b1c0f, 
              0x67174405, 0x40cf25d9, 0x735eaa4f, 0xcd16cff4, 0x7d2605f6, 
              0x3510fc1d, 0x9fc8df02, 0x3d155d19, 0xedeb128c, 0xed154e05, 
              0x78a0cad8, 0x6d06d05e, 0x4828d5fa}}, ext_ofile = 0xc0376d65, 
          ext_nfile = 0x0, ext_oline = 0x1b2, ext_nline = 0x0}, MH_databuf = {
	.....

More information from the crash dump can be provided on request.

>How-To-Repeat:
See above.

>Fix:

Unknown. Fix the "XXX code doesn't handle clusters XXX" in ipsec_mbuf.c?