IPsec: stack problems

[Maxime Villard <max%m00nbsd.net@localhost>]

Hi,
I'm a little concerned about the stack usage in the IPsec code. Note that what
I'm talking about here occurs _after_ authentication.

Typically, when an IPv4-AH packet is received, the code path is:

	ip_input
	(*pr_input) = ipsec_common_input
	ah_input
	crypto_dispatch
	[several crypto functions are called]
	ah_input_cb
	ipsec4_common_input_cb
	(*pr_input) = depends on the packet

These functions are nested, so the stack consumption grows on each call.
The main issue is that the call to pr_input at the end could put us back into
ah_input, if the AH-encapsulated payload is itself an AH-encapsulated payload.
If the third payload is again AH, we're again back in ah_input, and so on.

It is not complicated to end up with:

	ip_input
	(*pr_input) = ipsec_common_input
	ah_input
	crypto_dispatch
	[several crypto functions are called]
	ah_input_cb
	ipsec4_common_input_cb
	(*pr_input) = ipsec_common_input
		ah_input
		crypto_dispatch
		[several crypto functions are called]
		ah_input_cb
		ipsec4_common_input_cb
		(*pr_input) = ipsec_common_input
			ah_input
			crypto_dispatch
			[several crypto functions are called]
			ah_input_cb
			ipsec4_common_input_cb
			(*pr_input) = ipsec_common_input
				ah_input
				crypto_dispatch
				[several crypto functions are called]
				ah_input_cb
				ipsec4_common_input_cb
				(*pr_input) = ah_input
					ah_input
					crypto_dispatch
					[several crypto functions are called]
					ah_input_cb
					ipsec4_common_input_cb
					(*pr_input) = ah_input
						...

At some point a stack overflow occurs and the kernel crashes. We could fix
this by adding, in ipsec4_common_input_cb (and in the IPv6 equivalent):

	if (prot == IPPROTO_AH || prot == IPPROTO_ESP ||
	    prot == IPPROTO_IPCOMP) {
		goto bad;
	}

But I think this wouldn't solve the problem in tunnel mode. You could have
prot = IPPROTO_IPV4, and encapsw4.pr_input = ipsec_common_input, so the issue
is still there.

I think we need to check the packet's history against the IPsec configuration,
probably with a tag...

Maxime