Projects: unixadminbot, Log::Slogger, File::Valet

So .. in this post (post #323 in case that link is flawed) I referenced my unixadminbot project for illustration, and someone else in the thread expressed interest in using it.

The problem is that the original unixadminbot is the intellectual property of The Internet Archive, and is embarrassingly horrible code anyway. I've been meaning to do a "cleanroom re-implementation" of it anyway, so I could clean up the code, generalize it a bit, and open source it for the world to enjoy (or point at and laugh).

No time like the present, right? I opened up my cleanroom unixadminbot project and remembered I hadn't been working on it because it has dependencies on another project which also hasn't been published yet -- Log::Slogger, a structured logging system for distributed environments, inspired in part by Google's Dapper and the quite excellent Log::Log4perl project.

So I left unixadminbot alone and developed Log::Slogger for a while instead. Maybe if I stay focused on it for a while (as much as paid-work will let me) I can get it published to CPAN and it will make sense to work on unixadminbot some more.

Before that, though, a co-denizen of #slackware-help on IRC noticed a couple of bugs in another of my CPAN modules, File::Valet, and I felt the need to address that sooner rather than later. Got version 1.03 pushed out earlier today, which works around the unit test problems and fixes the Errno version requirement problem.

So that at least is progress. Yay progress!

The idea of unixadminbot is that it automates some of the tasks normally performed by junior system administrators -- find problems and fix them. It goes beyond monitoring by not just detecting issues but also taking corrective measures.

This also distinguishes it from "configuration management systems" like Chef and Puppet in that it is forever vigilant and proactive, not just performing tasks when the sysadmin requests it.

Unixadminbot will incorporate fail2ban-like features (noticing security threats and shutting them out) Nagios-like features (flagging problems for human attention), init-like features (respawning services, but less stupidly) and CMS-like features (configuring the system under centralized direction). It will also be able to detect low-memory conditions and kill the "right" process before the OOM-killer kills the wrong process.

The original unixadminbot (aka "crutchcron" -- yeah, I was really bad at names) hard-coded all of its functionality and grew organically. Every time a new problem cropped up in The Internet Archive's data cluster, more code was grafted into the program to detect the condition and apply a fix. It got every bit as messy as that sounds.

I'm trying to avoid that messiness by generalizing the code, so features can be provided via a plugin system. I'm also trying to avoid some of the design problems that make Chef and Puppet such a pain in the arse -- the overly-abstracted and disjoint model, the chaotic "recipes", the awkward declarative goal representation.

My objective is to keep it concrete, flexible and imperative, allowing use of arbitrary code but with clear guidelines. I'm also talking with my sysadmin friends who work with (suffer under) Chef and Puppet. They're short on suggestions, but tell me exactly what they don't like, which is helpful in its own right.

I'm trying to make something that wouldn't make their jobs miserable. Perhaps that's a low bar to set, but perhaps that means I can implement a good solution.

But first I need to finish Log::Slogger! Much to do!