Tangential, but the complexity and over-head of the malloc abstraction is precisely what made me very intrigued by "stack-only" languages, such a recently published Goose. There is something very elegant and exciting about the idea of (almost) completely getting rid of it. Just as modern programming languages have developed good ergonomics that replaces a garbage collector, the same could hopefully be done to replace heap-allocated memory.

This is mostly describing a kind of obsolete allocator that uses the brk to adjust a single contiguous heap range. This is basically irrelevant with virtual memory and mmap. The good malloc impls (jemalloc, mimalloc) don't do this.

If you want to study how modern allocators work I'd recommend something like mimalloc, which has a reasonably approachable codebase. No brk/sbrk in sight, btw (or boundary tags, or splitting/merging...).

When getting into wasm when it was new and had clang support, I wrote my own libc and malloc, which was so a great learning all these low level functions and tricks (free lists for example), I also like that I can read wasm quite well, and tooling is good to just return a .wat and read it.

mental reminder to finish my history of malloc writeup. It's almost 20 llm-less pages with context and examples. goes through general purpose malloc like dlmalloc, mimalloc, tcmalloc, jemalloc and others, while also mentioning some purpose-specific ones like scudo.

I found this one very well explained time ago, for those looking for more details: https://sploitfun.wordpress.com/2015/02/10/understanding-gli...

Good learning resource for this (and related) topics: https://cs61.seas.harvard.edu/site/2026/

I highly recommend reading Professional Linux Kernel Architecture by Wolfgang Mauerer, which describes the Linux memory management system in great detail.

I tried this on OpenBSD using ktrace / kdump. I added some utrace calls and had to increase the amount allocated significantly to get the mmap calls to show up. obsd just calls mmap with prot_read|prot_write meaning pages may be read/written.

This phrack article on malloc is highly recommended:

Vudo malloc tricks by Michel "MaXX" Kaempf (2001)

https://phrack.org/issues/57/vudo-malloc-tricks

As a rough rule of thumb: if you want smallest latency for small allocs use malloc not mmap, and for largest allocs prefer an ANON mmap. The blog post goes into more fractal detail, in general.

This shows you how you could "secure" memory code in C by using more robust implementations of these functions. But it doesn't show you that you don't even need them anyway. I think in my entire career I've only used dynamic memory in C twice. The rest of the time I use functions with static allocations, buffers, unrolled loops, etc. Maybe it's paranoia, or habits from embedded programming, or being a control freak... But my C programs don't get memory leaks or crash from memory issues.

malloc has terrible interface.

It has no way to specify custom alignment. So, if more than 8 or 16-bytes alignment is needed, custom user-code tricks are necessary (allocate more than necessary, round up).

It has no way to report how many bytes are actually allocated. Sometimes it may be useful, like if I reserve 9 bytes for a string and underlying allocation granularity is 16 bytes, I want to know that I can increase string size up to 16 bytes without calling malloc again.

It's a single entry point allowing no allocation size customizations. So, if I allocate objects of fixed size, I still need to pass a dynamic size parameter to malloc and it needs to perform branching in case if it does have some optimizations for particular sizes.

free has no size parameter, which is a mistake. This forces the allocator to maintain size values somewhere inside, which is often an unnecessary overhead, since user code already knows size (like for fixed-size allocations or containers storing explicit capacity).

With interface fixes described above a malloc implementation can be much more performant and (sometimes) even easier to use.

Malloc() is sloooow. Even more if you also use free(). Consider allocating a single array and scratching there if possible.

allocating memory, usually.

   One of  my pet  peeves is  when people,  trying to  make a
   point, use code  that doesn't compile.  I  mean, it's fine
   when you're sketching  out code on a white  board.  But on
   an  article you've  had  time to  prepare,  it's a  little
   annoying.

   Take for instance, this code fragment:

   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..  }

   I'm pretty sure what they //meant// to write was:

   .. #include <stdlib.h>
   ..
   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..     return( 0 );
   ..  }

   I  understand why  the  author produced  the  code on  top
   instead of the code at the bottom.  The code on the bottom
   includes  features extraneous  to  the conversation.   The
   include  directive  and  return statement  only  serve  to
   distract from the point the author is making.

   If you read far enough down,  you'll see he has links to a
   broken GitHub  repo presumably  with code  fragments.  (or
   maybe  GitHub itself  is  broken today  or  it decided  it
   didn't  like me  trying to  access the  repo from  an AT&T
   fiber line.)

   I'm sure when GitHub decides to behave and serve bits like
   it's supposed to, there will be compilable code so readers
   can understand  the author's intent within  the context it
   was intended to be viewed.

   Which is to say:

   1. It's probably  okay to have only code  fragments in the
      main part  of your  discussion.  Please provide  a note
      that  there is  code which  shows your  snippet in  the
      context  of a  compilable program.   And put  that note
      somewhere near the code fragment.

      Or... consider putting  a note at the  beginning of the
      article saying  something like "Compilable  versions of
      code  fragments  in this  article  are  available at  a
      GitHub  repository  referenced  at the  bottom  of  the
      page."

   2. Do  not depend on  GitHub to hold your  code fragments.
      You do not  know when it will be down  or which readers
      it will randomly decide to block.

   3.  I really  do  think  you should  include  more of  the
      "layout"  code.   When   reading  "The  Address  Space"
      section,   it  was   unclear  where   the  initialized,
      uninitialized and local came from.  I initially thought
      the  author was  using a  version of  C I  had not  yet
      encountered that defined these symbols.

   4.  Since  the code  uses  ``sbrk()``,  the author  should
      probably have  cautioned readers  the code  isn't POSIX
      compliant, but should compile  and run on Linux systems
      using gcc.   There's a  comment about  demonstrating it
      with  glibc, but  I think  that's not  a strong  enough
      warning since you can  compile programs with clang that
      are intended to link against glibc.
      
      I doubt there  are many QNX users out  there who expect
      random code from  the internet to run  on their systems
      without alteration, but it  seems good form to describe
      the environment you are targeting.

      Now I'm wondering if this is expected to work on Darwin
      as well as Linux.

   5.  It  might  be  a  decent  idea  to  order  the  SHOW()
      statements in  the code in "The  Address Space" section
      to follow  the order  of the diagram  at the  bottom of
      that section:

      .. SHOW( "local (stack)", &local );
      .. SHOW( "printf (libc)", printf );
      .. SHOW( "program break", sbrk(0) );
      .. SHOW( "malloc(16) (heap)", malloc(16) );
      .. SHOW( "uninitialised (bss)", &uninitialised);
      .. SHOW( "initialised (data)", &initialised );
      .. SHOW( "main(text)", main );

   6.  In  the  "Chunks  and  Headers"  section,  it  may  be
      interesting to mention what the flags in the allocation
      headers actually mean.

   7. glibc does late fill,  right?  I can't remember.  If it
      does,  it might  be  useful to  mention  in the  Demand
      Paging section since it affects the program logic.
   
   Now that I've yelled at the  cloud a bit, allow me to say,
   I enjoyed the article.  Things I liked:

   1. The author  used the exact strace command  they used to
      produce  the  output  discussed  in  the  text  of  the
      article.

   2. The graphics are quite nice.

   3. Despite grousing about  fragments being fragments, they
      do a good job of illustrating points made in the text.

   EDIT: After complaining about  GitHub not being available,
   it's finally come back.  The web interface is still borked
   for me, but  I can clone the repo mentioned  at the bottom
   of the page and view the local copy of the source.

[flagged]

A thing worth adding for anyone reading: in long-running PHP processes (workers, queue runners, Laravel Octane), the overhead of malloc/free patterns compounds in ways the request/response model hides.

A page handler that leaks 2KB per request is invisible under php-fpm (the process dies on the next request anyway) but becomes a slow memory creep in a queue worker that runs for hours. The fix usually isn't "call unset more" — it's recognizing which objects Zend's refcount can't collect without a full GC cycle (circular references, mostly), and forcing `gc_collect_cycles()` at known-safe points.

Took me a while to realize that in production.

[flagged]

[flagged]

[dead]

[flagged]

like many C functions you must remember that C is not low level, you won’t learn how memory is allocated by inspecting malloc() anymore than you would learn how characters are printed to the screen by reading printf() or how a file is written by reading write().

you will only learn about the thin or heavy wrapper that C is responsible for, the real magic is happening in the kernel and the device firmware

If the DRAM scarcity continues, not much.