Poofy Goof Rant-O-Matic

At work we're finally retiring the in-house mail server. There's only so hard you can push a 450MHz PIII, only so much tuning you can do with a 384MB RAM cap, and only so much time you can spend on the care and feeding of something that should have never been your responsibility anyway.

I feel like I should have a small wake after I power it down, and reflect on the 3.3 million messages (over 48TiB worth) it passed while it was alive(?) over roughly the last six years. Now it's Google's problem, and while I'm not a fan of outsourcing mail to faceless corporations, and will maintain mail delivery for myself at work, I can't complain too much, since I no longer bear the responsibility of keeping mail running.

How did I get into this situation in the first place? Like a lot of sysadmin jobs, it started innocently enough.

When I arrived at my current employer in 1999, I was basically waved into a dust-covered cubicle, and left to fend for myself, computer-wise. The sysadmin at the time had fallen into the position mostly as an offshoot of his database and accounting responsibilities. The machines were still hand-builds for the most part, and the network was almost exclusively Novell-based. Even Kris had gotten into the build-it-yourself action, as she had put together an FIC (PA-2013?) socket-7 cyrix machine for the head of marketing a couple years before I arrived.

I ended up piecing together a machine from spare parts with a Rise R407E motherboard and a 133MHz AMD 5x86-P75 CPU. I had been working with NetBSD at home on my DECStations at the time, and decided to try it on the PC. It went smoothly and before long I had the network configured and X running. I pulled my mail down via POP with fetchmail.

It was pretty obvious that the old clunker wasn't going to cut it for actual development work, so a few weeks later Kris grabbed the company credit card and we went to the corner PC shop, and bought a more up-to-date machine, a 500MHz K6-2. I set it up to dual-boot windows 98 and Redhat 6(?). I kept the NetBSD box running for my home directory and to check mail on, since I figured the new box might be switching between Linux and windows.

Two machines? File-service between the two? Eek, I'm already a sysadmin. When Kaben joined the project, I folded his machine into the little two-machine fold, and got YP up and running. Our first working debugger supported Linux, (and ran fine under NetBSD's Linux emulation,) and Kaben and I shared it until we got some home-built BDM hardware working with gdb. Additional developers soon joined the project, and to support them, I had to put together some remote debugging boxes. The first one was a copy of debian that had been shoehorned onto a pair of 40MB drives, but I eventually got the BDM driver ported to NetBSD, and the later debugging boxes used that. A retired Pentium-133 that had previously been the engineering Novell server was dusted off and pressed into service as a backup server and source code repository. A retired Pentium-200 turned into our bugs database.

In late 2002, a number of people were fired due to tight financial situation and poor job performance, including our accountant-cum-sysadmin, and I became the sysadmin by default. Luckily Patrick was around to take care of the windows machines, and I started to migrate infrastructure pieces to NetBSD.

Mail was my top priority due to the continual growth of spam. By mid-2003 I had started doing some testing with the new mail server and was forwarding messages from the external windows-based mail server to the internal one, which was still located under Kaben's desk in engineering, even after he had left. Users were migrated one-by-one until I was convinced the new server wasn't going to fall over. (single-bit errors on IDE? been there.) The mail server didn't get racked up and moved to the machine room until late 2004.

Other machines were migrated to and through NetBSD as well. The Netgear router was replaced with a K6-II. The OS/2 webserver was replaced with a Pentium Pro. After Novell was flushed from the network, parts from the two Novell machines were combined to create a dual-CPU PII-300 file server, which was able to collapse services from a few other machines.

The mail server will be turned off next week. What's next on my list?