<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Memory - Tag - fds-board</title><link>https://blog.fdsboard.com/tags/memory/</link><description>Memory - Tag - fds-board</description><generator>Hugo -- gohugo.io</generator><language>en</language><lastBuildDate>Tue, 04 Aug 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://blog.fdsboard.com/tags/memory/" rel="self" type="application/rss+xml"/><item><title>452 MB and Climbing: A Second glibc malloc Bug Behind WavePy's OOM Kills</title><link>https://blog.fdsboard.com/python/glibc-mmap-threshold-tile-encoder-oom/</link><pubDate>Tue, 04 Aug 2026 00:00:00 +0000</pubDate><author>Jinze(Jerry) Zhou</author><guid>https://blog.fdsboard.com/python/glibc-mmap-threshold-tile-encoder-oom/</guid><description><![CDATA[<h2 id="1-summary">1. Summary</h2>
<p>Three long-running encoder processes got OOM-killed at 20-30 GB despite disciplined <code>del</code> +
<code>gc.collect()</code> after every single unit of work. The Python heap was clean — the leak was one
layer down, in glibc&rsquo;s allocator. By default glibc doesn&rsquo;t use a fixed <code>mmap</code> threshold; it
raises the threshold every time a large <code>mmap</code>&rsquo;d chunk is freed, on the bet that a similarly
large allocation is coming again soon. For a workload that repeatedly allocates and frees many
<em>differently-sized</em>, short-lived, multi-megabyte buffers, that bet is wrong: allocations get
progressively demoted onto a heap arena that never gives pages back to the OS. The fix is one
line — <code>mallopt(M_MMAP_THRESHOLD, N)</code> pins the threshold and disables the adaptive behavior for
the rest of the process&rsquo;s life. This applies to any long-running process (Python or otherwise)
that cycles through many moderately-large, short-lived buffers — image tiles, video frames, ML
batches, whatever the buffers happen to be.</p>]]></description></item></channel></rss>