Dear Lazyweb

,
Nothing is perfect, but on Linux you can at least improve it and tune it.

I just ran into memory problems on my development box - I had tons of applications open, and since this is a laptop I prefer to have swap disabled.

I then started to build bind9 for a sarge backport, which meant that a long sequence of C compiler processes were launched, each allocating some memory, working with it, terminating, next.

Being really at the limit of the available memory, the system became unresponsive - and that behavious is really bad… I thought that sooner or later it will really run out of memory, and decided to watch some TV in the meantime. One hour later when I returned, it was still torturing my HD and still unresponsive.

Apparently the compile process has been all the time just below the memory limit, causing the kernel to remove pages from memory and load them back from the disk later, which slowed everything down just horribly.

So now I’m searching for a way to prevent such situations. The traditional ulimit approach won’t help - no single process was responsible for using all the memory, but each one had its reasonable share.

So dear lazyweb, is there a way to tell the kernel to reserve a share of the memory (lets say, 64 MB) for buffers and caches only? I can live with out-of-memory errors a bit early (and the just enable swap), but a massive slowdown as described above (where the kernel is apparently reading some data from the HD, then forgetting it again and replacing it with the next) must not happen that easily…

Right now I fear that you could bring down any Linux system easily with a “slow” forkbomb where each forked process allocates 100 MB and starts writing some random data to its memory, forking a new child every 50 MB or so. Neither the OOM killer, ulimit nor traditional forkbomb detectors will be able to counter that, since already a couple of processes will cause the system to become completely IO-bound. Oh, and I have no idea how Windows would react on that either. So don’t assume any OS is better, just because I say Linux isn’t doing perfectly well here.

If you have any tips for me, send me an email.