i set up a virtual server for a public entity and it had a fatal crash in which it could not recover so i did not look at the log files this first time. i reinstalled everything all over again and upon testing it froze. this time it recovered from re-boot and i went looking in kernal, syslog etc... looking for a clue and nothing major lead to this freeze. since i was in /var/log i decided to just look in the auth.log file and to my surprise their was a million refused ssh connections from default system user accounts like root, oracle, postgre, admin etc.... they came from 3 different ip's. the shear barrage of them plus running several Vm's leads me to think the freeze was an unintentional DDoS. I know of at least one person who is disgruntled and used to work on this project and has the server ip or at least the range of the public ip's we'd be using. they also know we would be using uncommon port #'s to connect to the VM's via ssh and the "attack" is concentrating on uncommon ports for ssh access.
i am no security expert but is it safe to say this is a hack attempt. again they use one user say root and attmpt a 100+ logins then move to another user like postgre etc....

I couldn't say if it's an unusual crack attempt or DDoS attack, but I would suggest you install fail2ban to lock out the attackers. Out of the box, it's a bit lenient, so you'll want to toughen it up by reducing the number of attempts that trigger a ban, increase the lookback time, and increase the ban time. You'll also need to modify the sshd trap in your jail specification file since you are using non-standard ports.

Furthermore ... why are you permitting "passwords" to be used with ssh? Each user should have their own unique digital certificate, and when that disgruntled person had been shown the door, his or her certificate should have been revoked.

1 members found this post helpful.

thanks for the mitigation ideas i will do that as soon as my supervisor looks over the info i sent him. i doubt it was a DDoS because why use ssh login attempts as you know they are logged. there is no real reason to do it that way. i think the DDoS was accidental because at the time of the attempts(approx 4per second) we were testing the load of multiple VM's remotely. i know the server could handle the VM load but with the added 4per second login attempts i think a DoS occured. i didn't really mean DDoS just a DoS. these attempts are coming from various cloud server providers not even a bot-net so this tells me someone has a bunch of free accounts with companies less reputable than AWS who don't need actual credit cards or identification to sign up for a free trial/account. then they are just proxying these logins through them swithcing every cpl hundred attempts. it seems like an amateur attempt but i do suspect an insider or ex-insider because it is obvious to me they are using the port range we plannned for the ssh forwarding on the VM's. plus they attempted logins with the username oracle and we used vbox headless on this server. i know their are better solutions than vbox before anyone blasts me for that but it is a low end solution for non-critical use plus the phpvirtualbox interface is easy for the ppl using these VM's to understand since they are not IT ppl.

i know sundialsvc, but there is a reason for using passwords. i don't want to go into too much detail on a public forum as to the reason but what i can say is the ppl using these VM'S, which are not production machines, change constantly and ease of access is actually the priority. i can't give a key or certificate to the amount of ppl using the VM's given they always change. the host is key login but the VM's are not and the forwarding is done by vbox on the host. the gateway is not involved and there are legitimate reasons for that. one is we were not allowed to touch the gateway or ask them to add a bunch of forwarding rules for us. it is in a DMZ and their production network is protected by that fact and several other security measures. i know this doesn't make much sense but if i told you exactly what they are for you would understand.
BTW it is not part of a domain and the users not part of the business. i am not on their IT staff and have no access to their other equipment. i was given 1 machine and told this is all i get and it is stand alone in a DMZ and no configuration except forwarding to the host machine will be done on their end. also there is literally nothing on this machine. the person doing this is a jackass and is just trying to be disruptive to a project he bitched about because instead of making do with the limitations we were given all he could do was whine. so now one can only assume he is trying to make it fail soleley because he couldn't make it work

Still an Illegal act, what's the Boss say.....?

Hi,

I suggest you to disable root login for ssh. So noone will be able to get root access via ssh. Now create an user for ssh login and use su to login as root. Also change the port number of ssh.

yea i actually have it that way now just 1 sudoer can log into the host no other users needed on that just me. i shutdown the server till monday when i can work on it offline and install fail2ban and perhaps tweak ip tables as well. the VM's are a different story i can disable it but the users have root access on their own VM so i can't garauntee they won't change it on there. the purpose of these VM's is just for ppl to learn linux administration there is no data on them but every cpl months new ppl will have root accounts on them to learn. it is however unlikely they will change it since their user is a sudoer. only i have access to the host and i keep snapshots of their machines.

i agree that it is still an illegal attempt. i am not sure what action they will take. i am just a "contractor" on this one project. i did report to the guy that hired me to do it and i've known him for 2yrs but he might be tight lipped as to the action they take because it is their companies business not mine i don't work there. he agree's with what i showed him as for the guy i refer to as a suspect he is the previous guy that attempted this job before me he was not my guy either. i'm sure he is thinking it is him to but don't know if he would outright say that to me. so far he just said good job catching it. i will talk to him monday when i see him in person.

I agree, Good Job Catching It!

Ditto... Maybe this will help make you permanent...

The normal & safer way is to junk the old VM when finished with and clone/spin up a new clean one for each new learner.
That way junk & dangerous stuff doesn't accumulate & everyone starts a from known (good) state, which makes it easier to help them.

that is exactly the plan. actually they are all cloned from one image, and i keep that image aside for emergency. i also keep a snapshot of the 30 or so clean VM's before the ppl add their own accounts on day one(they all only have an instructor account to start). this way when the new class rolls around i just get rid of all the snapshots except that first clean one with only the instructor account then hand those out. day one they login to their VM via instructor account and learn to create an account for themselves then another snapshot is taken and that is what they use. when their done, their personal snapshots are deleted, and we start from the clean one again, etc....

i was wondering if anyone knows how the ACL on the cisco device got bypassed to do this. after submitting this issue to the IT guys they certainly were interested considering the ACL allows for very few ports and obviously the rest is implicit deny, and the ports being attacked are implied deny ports. they showed me the ACL for the this box and i can only assume they configured it on the right interface. reminder i don't have permissions to access IT's equipment so i can't look into this personally. the only thing i can find, again assuming they put this ACL on the right interface, is that some ASA devices are vulnerable to ACL bypass under certain rare circumstances. from what i read it is unclear exactly which versions or upgrade cause this behavior. apparently it appears to function as normal but the implicit deny function doesn't work and it uses the default interface security levels to make decisions not the ACL's implicit deny. has anyone ever seen this or have more info on exactly which circumstance creates the bug. it is not an exploit as it can not be forced but is a bug that cisco acknowledges but it's unclear on how it appears. besides that any other ideas how my box is even getting these requests on ports blocked by a cisco ASA devices ACL?