So I had some server unpleasantness that I noticed after I posted my 2014 Goals post. The site just froze and logins were barely possible.

I immediately SSH’d into the prgmr.com server, but was greeted by an extremely slow console.  I actually had to access the VPS Xen configuration menu and do a hard shutdown of the instance.  When I rebooted and accessed the server through the out-of-band console, I was getting out of memory errors galore.

The culprit?  A combination of MySQL, ClamAV and Amavisd.

After disabling ClamAV and Amavisd, I was able to stabilize the server, but the 1GB RAM VPS was still dangerously low on memory with 10-12 MySQL threads that I couldn’t clear.

So I took the unprecedented move to rebuild the server.  Two days later I am running fine again.  This time, I was a little bit careful of the installed services–with an emphasis on pruning unneeded services.

Hopefully I won’t ever have to do it again.

Tai