Tiny allocations alignment requirements vs. jemalloc

[Martin Husemann <martin%duskware.de@localhost>]

Hey folks,

the recent switch to newer jemalloc broke a lot architectures and we are
still investigating the various fallout.

I remembered some subtle hack I added to our gcc some years ago to catch
one of the possible configuration failures at compile time, and unfortunately
it has been last valid back when we used jemalloc 1.0.

The hack is at the end of src/external/gpl3/gcc/dist/gcc/tree-ssa-ccp.cc:

it tries to mimic jemalloc configuration and find the minimum alignement
jemalloc provides, and then CTASSERTs that against the gcc configured
equivalent MALLOC_ABI_ALIGNMENT.

This is the comment explaining it:

#if defined(__NetBSD__) && defined(NETBSD_NATIVE)
/*
 * This is a big, ugly, temporary hack:
 *    http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59958
 * To make sure we have configured all our targets correctly, mimic the
 * #ifdef cascade from src/lib/libc/stdlib/jemalloc.c here and compile
 * time assert that the value matches gcc's MALLOC_ABI_ALIGNMENT here.
 */

The gist of that bugzilla ticket is that C90 requires full 
MALLOC_ABI_ALIGNMENT for any allocation from malloc(), no matter how
small it is (the answer to C90 defect report #075 clarified that), and
later C99 repeats that and clarifies further (DR #445).

However, jemalloc seems to think differently:

 * Now, let's start considering the first few size classes. Two extra constants
 * come into play here: LG_QUANTUM and SC_LG_TINY_MIN. LG_QUANTUM ensures
 * correct platform alignment; all objects of size (1 << LG_QUANTUM) or larger
 * are at least (1 << LG_QUANTUM) aligned; this can be used to ensure that we
 * never return improperly aligned memory, by making (1 << LG_QUANTUM) equal the
 * highest required alignment of a platform. For allocation sizes smaller than
 * (1 << LG_QUANTUM) though, we can be more relaxed (since we don't support
 * platforms with types with alignment larger than their size). To allow such
 * allocations (without wasting space unnecessarily), we introduce tiny size
 * classes; one per power of two, up until we hit the quantum size. There are
 * therefore LG_QUANTUM - SC_LG_TINY_MIN such size classes.

(this is from src/external/bsd/jemalloc/dist/include/jemalloc/internal/sc.h)

I read this as: we *must* verify that for all our architecture configurations
LG_QUANTUM == SC_LG_TINY_MIN (it is clear that LG_QUANTUM varies by 
architecture, while I am not totally sure what value SC_LG_TINY_MIN will have,
my best guess is: always 3).

I would suggest to add a CTASSERT() somewhere in jemalloc to catch violations
(and I am pretty sure we currently will see some, eg. mips n32 or n64 where
I think LG_QUANTUM will be 4 but SC_LG_TINY_MIN will be 3).

And I would also suggest to remove the (now ineffective) gcc change, or
replace it with something better - ideally including jemalloc headers
directly to get LG_QUANTUM, then assert that against MALLOC_ABI_ALIGNMENT
and rely on the jemalloc build to verify SC_LG_TINY_MIN vs. LG_QUANTUM.


Am I missing something?

Martin