LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 12-27-2011, 11:44 AM   #1
algorim
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Rep: Reputation: Disabled
Ubuntu server / overtake / dos attack


Hello everyone,

I am new to this forum, but I'll jump right into my problem to prevent boring you about my history.

My virtual Server recently got shutted down with the hint of the provider, that there has been noticed a DoS-Attack outgoing from my Server to another.

Now. The situation is, that this server has rly no presence in the web. It is just unknown, except to a few of my fellows.
I'm rly wondering how this server even can be a victim of an overtake, because this would need some affort.
There is no software, which could possibly be harmful in the way a trojan or sth.

So sb. needed to take advantage out of nowhere.

(1)I'd like to hear if you, experts, can imagine somebody just fall over the server, which is completly unknown and uninteresting, and hack it.


(2)My second interest is, how to chase the method the Attacker took.
I'm assuming the attacker just got a ssh login to a user i created, with a password, which was indeed rly weak. If the server is up again, i hope the ssh log is activated, so i can check the logins.

Btw, the Attacks have had the following format:
17:13:51.792097 IP XXX.XXX.XXX.XXX:53769 > XXX.XXX.XXX.XXX:113: UDP, length 1

(3)Next question would be, which rights an attacker needs to perform such an attack. The user, I created, had no write and execute permissions.

So at the very end I want to ask, if you have any advice for me, that i rly need to take care of.

It is a base ubuntu system with the following software running:
Apache
MySql
Git
Gitosis


Sorry for my bad english, i hope you dont suffer that much.

Thanks for your help.

Greetings

algorim
 
Old 12-27-2011, 02:55 PM   #2
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,623

Rep: Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695Reputation: 2695
Security

No system is secure that depends upon ignorance on the part of the threat population.

You are being hosted, and the host company has a presences and WILL have its domains scanned. The average life without threat on the internet is between 20 and 180 seconds for windows machines, roughly ten times that for *nix machines (depending upon domain).

There is no clear way to tell if your machine was actually taken-over (it may have been the hosting companies machine, or a spoofer fooling it), but let us assume it was. SSH was a possible avenue, but so was Apache, so was MySQL. All are good and getting better, but you need to follow the latest security advice on running them safely or they may as well be open doors.

I would load a new server and secure it BEFORE populating it with applications and data. Make sure that your passwords are adequate, that no ports are open that do not need to be open, that no service runs with higher authority than the minimum required, and that you have a process scanning the logs for threat behavior often. Get it to current patch level, and have a planned schedule to keep it current.

I would also install clam-av and run a system scan daily, and run something like rootkithunter on a daily scheduled to alert in case something got past. If you have an intrusion detection system that does all or part of this, so much the better.

If that data on the system has any value at all, I would ensure that I have a schedule of full and incremental backups: frequency depending upon how fast it changes. (I use weekly full backups with daily incremental backups, and copy the latest version of each off-site when it is done. Where I used to work they did full system backups quarterly and after every major upgrade, but backed up changing data more often.)

There are many other suggestions that could be made, but these should be a good start at preventing successful incursions and recovering from any that succeed.

Last edited by wpeckham; 12-27-2011 at 03:26 PM.
 
Old 12-28-2011, 10:43 AM   #3
algorim
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks for your answer,

i am aware of being scanned all day, but i didnt know that there are even automatic attempts to break passwords of an uninteresting server. this break in was like an answer to creating the user (git). Of Course it could have been a sleeping script, but i didnt notice anything before i created the user.

I'll follow any of ur advices as many others to not get into this again. I think the main issue was, that i didnt monitor the server as much as i need to.

I read about several programs to do various automatic scans and alerts, but i'd like to hear which logs u take care of. I just need some hints to inform myself about the way it should be done.
 
Old 12-29-2011, 06:52 AM   #4
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by algorim View Post
Hello everyone,

I am new to this forum, but I'll jump right into my problem to prevent boring you about my history.

My virtual Server recently got shutted down with the hint of the provider, that there has been noticed a DoS-Attack outgoing from my Server to another.

Now. The situation is, that this server has rly no presence in the web. It is just unknown, except to a few of my fellows.
I'm rly wondering how this server even can be a victim of an overtake, because this would need some affort.
There is no software, which could possibly be harmful in the way a trojan or sth.
Can you please spell your words out, in full. I don't know whether those 'rly's are relay, really, railway or, as you would put it, sthlse. we'll make allowances for your English, and try to cope, but what you are doing is not trying and making it more difficult for everyone who reads this thread, and is against the site rules.

Lecture (well, friendly request) over.


Quote:
Originally Posted by algorim View Post
(1)I'd like to hear if you, experts, can imagine somebody just fall over the server, which is completly unknown and uninteresting, and hack it.
Yes. There is a limited range of ip adressess and everyone knows what it is, so you can't hide. Everyone (everyone) gets 'trying the door handles' attacks. Some people will get lots, some will get only a few, but everyone will get some. The only thing to do is to try to be secure. Trying to be someone who is not noticed just isn't security, it is buying a lottery ticket and relying on winning.

Quote:
Originally Posted by algorim View Post
(2)My second interest is, how to chase the method the Attacker took.
I'm assuming the attacker just got a ssh login to a user i created, with a password, which was indeed rly weak. If the server is up again, i hope the ssh log is activated, so i can check the logins.

Btw, the Attacks have had the following format:
17:13:51.792097 IP XXX.XXX.XXX.XXX:53769 > XXX.XXX.XXX.XXX:113: UDP, length 1
Which attack? Is this the putative attack on your server, or is it a packet from your server which you believe is being used to attack someone else's server?

There is nothing there about ssh. ssh is a common vector for attacks, but we have no evidence that ssh is in any way involved in this attack, could you please tell us what has convinced you that ssh was involved?

Quote:
Originally Posted by algorim View Post
(3)Next question would be, which rights an attacker needs to perform such an attack. The user, I created, had no write and execute permissions.
  • If you are asking 'what rights does an attacker need to send packets to my machine' the answer is none, exactly. therefore, you have to take measures to ensure that whatever packets are sent to your machine, the machine stays safe.
  • If you are asking 'if the attacker gets in as root, what extra rights do they need?' then the answer is, once they have root, they have everything, that is why root must be protected.
  • If you are asking 'if the attacker only got in as an ordinary user, what extra rights...?' well, that is probably an irrelevant question, as, if they did get in as an ordinary user (and you do not present any evidence that this is what happened), the first thing that they would have tried is to get to access the root account. And, once they are in as root, then you want to read the answer about root, above, but it is again all over.

Quote:
Originally Posted by algorim View Post
So at the very end I want to ask, if you have any advice for me, that i rly need to take care of.
We want facts. Give us facts. Your suspicions or assumptions aren't facts.
  1. Read this. There are many ways of configuring ssh, and you need to know the possibilities before you choose one.
  2. Choose one and implement it. Don't make mistakes
  3. Read the security stickies.

Right now, I'd suggest that you consider (as part of your consideration of ssh) something like 'fail2ban'; you might be able to get more out of it than just protection for ssh, so that might be good value, but I'd be the last to suggest that any one single measure would make you completely secure (...apart from the general 'security in depth is better...' point, if you spend all you time securing ssh, you will still be every bit as vulnerable as you are now to all of the attacks that don't involve ssh)


Quote:
Originally Posted by algorim View Post
about my history.

My virtual Server recently got shutted down with the hint of the provider, that there has been noticed a DoS-Attack outgoing from my Server to another.
More detail; are they right or wrong? What evidence is there (they should be able to give you some actual evidence, rather than just saying 'there was a problem'. Is it something that you can get under control (as a temporary measure) with an iptables block? Is it something that you could put an iptables 'block' in for, even when you get to a final situation?

Quote:
Originally Posted by algorim View Post

(2)My second interest is, how to chase the method the Attacker took.
I'm assuming the attacker just got a ssh login to a user i created, with a password, which was indeed rly weak. If the server is up again, i hope the ssh log is activated, so i can check the logins.
Maybe, maybe not. While it certainly a (very, very, very) bad idea to have ssh with a weak password, right now, it could be ssh or something else. In any case, prohibit root ssh login. Consider 'passwordless' or enforce strong passwords, consider only allowing a few named users and not the obvious user names (root, guest...).

Actually, use the logging facilities. You need to know when attacks are in progress, rather than waiting until someone has got in, and for that, you need to know earlier that you are under attack. You may want to look at logwatch.

The bottom line is that you need to expect to be under attack - you won't be disappointed - and take appropriate measures. 'Hiding' doesn't work.
 
1 members found this post helpful.
Old 12-29-2011, 08:10 AM   #5
algorim
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Well, thanks for your detailed answer.

You got me at bit wrong.

My Provider sends me a log of udp packets outgoing from my server to another, so my server is acting as an attacker. Which means my server is already compromised, and got shutted down by the provider.

The log says that my server sends a huge amount of packets with length 1 port 113 over udp to the same server within a millisecond. This is/was pretty much an evidence for me, that this is an attack, since I read that the 113 port is likely used to do such an attack.


So my questions were about:

Which gap did the attacker use, to compromise my server.
To get to that I want to know which rights does he need to send packets (atleast)
How could I chase the method.

About my guess to ssh I wrote that it is just like an answer to creating the user, but I really dont know if it is this thing, because I cant read a single log at the moment.
About securing ssh. I already read a bunch of things about it. I think i will make the passwords stronger (wow), disable root login and implement publickey identification, since i need this for git anyways.


One of the main issues i already stumbled over or more like it was punched me in the face, is that I thought I didnt need much further configuration to the distro my provider initialised. Sorry for that.

The additional software I installed (apache, mysql, git) was pretty much up to date and i had an eye on configure it securely.

Quote:
We want facts. Give us facts. Your suspicions or assumptions aren't facts.
I can't give you more facts. Thats everything i have at the time.

My question was pointed in the direction of general linux security points. I can understand if you tell me google it, i already did ofcourse, but if you have a guide which is really good in your opinion, give to me please. ^^

I'll take care everything you gave me already.

Thanks for the logwatch hint, i thought about such a thing and thought about to code a logmailer myself, but this is even better. ^^
 
Old 12-29-2011, 02:09 PM   #6
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Moved: This thread is more suitable in Linux - Security and has been moved accordingly to help your thread/question get the exposure it deserves.
 
Old 12-30-2011, 04:12 AM   #7
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by algorim View Post

My Provider sends me a log of udp packets outgoing from my server to another, so my server is acting as an attacker. Which means my server is already compromised, and got shutted down by the provider.

The log says that my server sends a huge amount of packets with length 1 port 113 over udp to the same server within a millisecond. This is/was pretty much an evidence for me, that this is an attack, since I read that the 113 port is likely used to do such an attack.

...I can't give you more facts. Thats everything i have at the time.
I was specifically wanting you to post the log (or, a few lines from it) that your provider has sent you. You might want to obfuscate IP addresses, eg, something like

111.222.333.444 -> xxx.xxx.xxx.444

to try to cover up exact addresses, but still allowing us to see what is going on; note, that this is only useful for working out what temporary counter-measures can be taken, rather than anything for the big picture, but it still could be useful, so I don't know why you are avoiding doing it.

Quote:
Originally Posted by algorim View Post

The log says that my server sends a huge amount of packets with length 1 port 113 over udp to the same server within a millisecond. This is/was pretty much an evidence for me, that this is an attack, since I read that the 113 port is likely used to do such an attack.
http://www.unidata.ucar.edu/support/.../msg00983.html
http://www.dslreports.com/faq/225
http://en.wikipedia.org/wiki/Ident
http://isc.sans.edu/port.html?port=113

Quote:
Originally Posted by algorim View Post
So my questions were about:

Which gap did the attacker use, to compromise my server.
To get to that I want to know which rights does he need to send packets (atleast)
How could I chase the method.

About my guess to ssh I wrote that it is just like an answer to creating the user, but I really dont know if it is this thing, because I cant read a single log at the moment.
About securing ssh. I already read a bunch of things about it. I think i will make the passwords stronger (wow), disable root login and implement publickey identification, since i need this for git anyways.
You want to know what 'gap' (presumably, gap in your security provisioning) that the attacker used. Honestly, there is no way that you can tell from the evidence here. What you can say is that ssh was (effectively) wide open, and if you configured it the same way again, it would eventually, or maybe a lot sooner, be exploited.

What cannot be said, on this evidence, is whether there are other weaknesses that would also be easily exploitable. I'm sorry, but if there is an assumption to be taken, we have to assume that there are other problems, until there is evidence that this is not the case.

Quote:
Originally Posted by algorim View Post
One of the main issues i already stumbled over or more like it was punched me in the face, is that I thought I didnt need much further configuration to the distro my provider initialised. Sorry for that.
Well, this is a common mistake. A lot of people assume that because some operating system can be secure, that their installation will be secure. There is a lot more to it than that.

Quote:
Originally Posted by algorim View Post
The additional software I installed (apache, mysql, git) was pretty much up to date and i had an eye on configure it securely.
I did a google search on the terms "apache security guide", and most of the first few hits looked good, superficially. Not sure about git, but I assume that if I do a similar search, I'll get a similar level of results.

You have to read this part of the security stickies, mentioned earlier. In particular, the section "Compromise, breach of security, detection" is very relevant, right now. I'm not sure that you'll be able to follow the CERT guide rigorously, because, when you get your server back, it will already have been trampled all over by your provider, and all you'll find is evidence of their dirty thumbprints all over everything you want to lok at. It would be nice if this wasn't the case, but I suspect that it will be, because your provider will be more concerned with re-imaging your server quickly, than leaving you a system on which you can do forensics, which would need something of a lighter touch (and a bit of thought).

OK, I'm now going to repeat myself:
Everything that you really need is in, or is linked from, http://www.linuxquestions.org/questi...erences-45261/ and its accompanying articles. You can't read/do it all at once, so you want to start on the big hitters; SANS, CERT, OWASP and the stuff that is directly relevant to post-incident clear up and hardening your newly-provisioned server. I have already pointed you in that direction, and there is a lot to do, so you need to make a start on it now.
 
1 members found this post helpful.
Old 01-03-2012, 12:50 PM   #8
Noway2
Senior Member
 
Registered: Jul 2007
Distribution: Gentoo
Posts: 2,125

Rep: Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781
Quote:
Which gap did the attacker use, to compromise my server.
One of the things that LQ Security can and will help with is performing a detailed investigation to answer this question. One of the key differences between LinuxQuestions.org and many other forums in this regard is that, as Salasi has repeatedly mentioned, we approach the investigation with an evidence based approach. Asking questions like, "what method could they have used" leads to assumptions and is counterproductive.

If you wish to perform an investigation into this matter, the first thing to do is read the CERT Intruder Detection Checklist. Before you even begin that, your system should be isolated. Since it does not appear as though you have physical access to the machine, you should put up a firewall to allow SSH connections from YOU only via a trusted source. Once you have secured the machine you may begin with your investigation.

In addition to the steps in the checklist, please obtain the output of the following commands:
Code:
    /bin/ps acxfwwwe 2>&1;
    /usr/sbin/lsof -Pwln 2>&1
    netstat -anpe 2>&1
    lastlog 2>&1; last 2>&1; who -a 2>&1
One of the key things that you will do, which is part of the check list is perform a detailed analysis of your logs. More often than not, your log files will contain key pieces of information related to a break in or break in attempt. The tool logwatch can assist with this process by sifting through the logs and reporting interesting pieces of information. Remember that while a useful tool, it is no substitute for an eyeball analysis which tends to pick up different items.

Code:
 logwatch --detail High --service All --range All --archives --numeric --save /path/to/logwatch.log
One VERY important thing that you will need to determine in your investigation is whether or not your root account or a sufficiently privileged sudo level account has been compromised. Unfortunately, if you have lost root security, you will be unable to trust the machine again and it will need to be cleaned and rebuilt. Doing so, should ONLY be done after you have sufficiently determined the cause of the compromise. Simply re-installing and changing a few passwords is likely to result in your visitor returning, only this time they know that you are more aware.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I secure my server from DoS attack ? neel.gurjar Linux - Server 9 06-25-2009 07:34 AM
DoS attack? port 22 templeton Linux - Security 1 11-11-2008 03:48 PM
is this a Dos Attack?? xtremeclones Linux - Security 8 09-27-2006 01:40 AM
detecting a DOS attack ignus Linux - Security 4 07-29-2004 02:17 PM
Are we under DOS attack? sarmadys Linux - Security 2 02-06-2002 09:41 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 08:32 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration