Re: Proposal: Restore malloc(9) interface

[Masanobu SAITOH <msaitoh%execsw.org@localhost>]

Hi.

On 2023/12/31 7:44, Taylor R Campbell wrote:
> I propose to deprecate the kmem(9) interface and go back to the
> malloc(9) interface.
> 
> 
> 1. The main difference is that the malloc(9) interface enables
>    attribution of memory usage: how many bytes have been used for this
>    purpose vs that purpose, partitioned by named malloc tags like
>    M_MBUF or M_ACPI.  The conversion to the kmem(9) interface lost all
>    this valuable diagnostic information.
> 
>    I've personally spent probably dozens of hours over the last year
>    or two puzzling over `vmstat -m' output to guess which subsystem
>    might be leaking memory based on allocation sizes and which
>    kmem-NNNNN pool looks fishy.  This is extremely frustrating and a
>    huge waste of time to recover information we used to gather and
>    report systematically.

I think adding malloc type again is good thing.

> 2. A secondary difference is reduced diffs from FreeBSD and OpenBSD
>    drivers if we use malloc(9).
> 
> 3. A small difference is that kmem(9) distinguishes legacy allocation
>    from interrupt context, kmem_intr_alloc/free, from front ends that
>    forbid that, kmem_alloc/free.
> 
>    I'm not sure this has provided much valuable diagnostic
>    information, but it has provided a lot of frustrating crashes.  If
>    we want the same frustrating crashes we could introduce an M_INTR
>    flag which is mandatory when calling malloc from interrupt context.

Current kmen_(intr_)alloc()'s return address is usually aligned with
CACHE_LINE_SIZE. (Note that the smallest number is KMEM_ALIGN(== 8)
and the max is PAGE_SIZE.)
On kern_malloc(), the memory is first allocated with kmem_intr_alloc()
and put the malloc_header to the head. As a result, the return address
usually aligned with lower than CACHE_LINE_SIZE (at least on amd64).

I'm glad if the new malloc(9) API usually returns pointer aligned
with CACHE_LINE_SIZE or more.

> Note: I am NOT proposing any substantive changes to the implementation
> of the allocator -- I'm just proposing that we go back to the old
> _interface_, using the new pool-cache-based _implementation_, and to
> add lightweight per-CPU, per-tag usage counting to the malloc and free
> paths.
> 
> Nor am I suggesting changing anything about uvm_km(9), pool_cache(9),
> or anything else -- just changing kmem_alloc(N, KM_[NO]SLEEP) back to
> malloc(N, T, M_NOWAIT/WAITOK) and kmem_free(P, N) back to free(P, T),
> or possibly free(P, T, N) like OpenBSD does.
> 
> Thoughts?
> 
> 
> I asked for rationale for the kmem(9) interface last year, and none of
> the answers gave any compelling reason to have changed interfaces in
> the first place or to finish the conversion now:
> 
> https://mail-index.netbsd.org/tech-kern/2022/10/29/msg028498.html
> 
> As far as I can tell, we just spent umpteen hundreds of hours on
> engineering effort over the last decade to convert various drivers and
> subsystems from malloc(9) to kmem(9), in exchange for the loss of
> valuable diagnostic information about leaks, for increased cost to
> porting drivers, and for crashes when old subsystems newly converted
> to kmem(9) still allocate from interrupt context.

-- 
-----------------------------------------------
                SAITOH Masanobu (msaitoh%execsw.org@localhost
                                 msaitoh%netbsd.org@localhost)