Note: There was a bad value `' for the field `Class'.
	It was set to the default value of `sw-bug'.

>Number:         36285
>Category:       lib
>Synopsis:       openssl RC4 bug?
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon May 07 11:25:00 +0000 2007
>Originator:     Martin Husemann
>Release:        NetBSD 4.99.17
>Organization:
>Environment:
System: NetBSD night-porter.duskware.de 4.99.17 NetBSD 4.99.17 (PORTER) #6: Sat Apr 14 23:43:03 CEST 2007 martin@night-porter.duskware.de:/usr/src/sys/arch/i386/compile/PORTER i386
Architecture: i386
Machine: i386
>Description:

I was playing with a program doing some crypto stuff and, apparentley random,
ran into crashes that I could not explain.

I managed to reproduce it with a small sample program.

>How-To-Repeat:

Compile and run this program:

cc -O2 -l crypto test.c
./a.out

watch it dump core.

--8<--
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/pem.h>
#include <openssl/err.h>

static void testfun(void)
{
	EVP_MD_CTX ctx;
	EVP_CIPHER_CTX cctx;
	static unsigned char buf[2048];
	unsigned char tmpkey[EVP_MAX_KEY_LENGTH];
	static const unsigned char input[2048] = "slkfdjasdfkahfkjhasfhaskdfn";
	int outlen;

	EVP_DigestInit(&ctx, EVP_sha1());
	EVP_DigestUpdate(&ctx, "0123456789abcdef", 16);
	EVP_DigestUpdate(&ctx, "test", 4);
	EVP_DigestFinal(&ctx, tmpkey, NULL);
	EVP_DecryptInit(&cctx, EVP_rc4(), tmpkey, NULL);
	EVP_DecryptUpdate(&cctx, buf, &outlen, input, 588);
	memset(tmpkey+5, 0, 11);
	EVP_DecryptFinal(&cctx, buf + outlen, &outlen);
	EVP_DecryptInit(&cctx, EVP_rc4(), tmpkey, NULL);
	EVP_DecryptUpdate(&cctx, buf, &outlen, input, 588);
	EVP_DecryptFinal(&cctx, buf + outlen, &outlen);
}

int main(int argc, char **argv)
{
	size_t tryCnt = 0;
	int dying = 0;

	for (;;) {
		if ((sizeof(long) == 8 && tryCnt == 32605)
		    || (sizeof(long) == 4 && tryCnt == 65006)) {
			printf("trying: %zu\n ... and now *boom*\n", tryCnt);
			dying = 1;
		} else if (dying || (tryCnt % 1000) == 0) {
			printf("trying: %zu\r", tryCnt);
			fflush(stdout);
		}
		testfun();
		tryCnt++;
	}
	return 0;
}
-->8--

>Fix:
This might be a bug in the test program - if so, I'd like to hear about it.