Ahh! Ive been hacked somehow .. And I want to catch them if they try it again.
You should not consider that a game to play unless you know you won't be a threat to other boxen on the LAN/internet (and since you got your box compromised, and this maybe sounds harsh, I'd say: it's a game WAY out of your league). Instead perform the three R's: repartition, reformat and re-install from scratch to get your box back to a trusted state. If you don't know how, see the Linux - Security forum's thread "FAQ: Security references": for more info or post in the Linux - Security forum.
IMNSHO you shouldn't use Portsentry or rely on it too much: all it does is react to packets tripping a port.
Need a stupid example? Say you configured Portsentry for tripping on port UDP/31337. Hell yeah, it'd trip each time someone tries the port, but what does this tell you? Zilch. Zilch. Zilch. (Besides that it would be a sad thing to do as BO server doesn't run on Linux.)
Now if you had something that scrubs packets, like Snort does, then you'd have a rule something like this: "
alert udp $EXTERNAL_NET any -> $HOME_NET 31337 (msg:"BACKDOOR BackOrifice access"; content: "|ce63 d1d2 16e7 13cf 39a5 a586|";)", which would only react to packets having a specific payload that correllates it with BO usage. See the difference?
From the Linux - Security forum's thread "FAQ: Security references": for more info see "Snort and PortSentry compared":
http://www.linux.ie/articles/portsen...rtcompared.php