LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 02-18-2013, 03:56 AM   #1
accessthecloud
LQ Newbie
 
Registered: Apr 2011
Location: Glasgow, Scotland
Distribution: CentOS 5.5, Plesk 9-10
Posts: 9

Rep: Reputation: 0
Question Help in stopping DNS requests (DNS Amplification)


I lease a few servers from 1and1. Up until the beginning of last week, everything was functioning normally.

Starting last week one of my servers started receiving a massive increase in DNS traffic. There were between 100-200 requests per second with logs containing similar information to the following lines:
Code:
Feb 17 06:22:37 serv01 named[2563]: client xxx.xxx.xxx.xx#60797: query (cache) 'isc.org/ANY/IN' denied
Feb 17 08:23:37 serv01 named[2563]: client xx.xx.xxx.xx#53: query (cache) 'ripe.net/ANY/IN' denied
After some research I discovered that during a recent upgrade to Plesk, the DNS recursion settings were changed (by the upgrade script) to Any host! This resulted in my server acting as an open DNS answering all requests. This I changed immediately.
Further investigation identified huge log files and a massive amount of traffic. The traffic all requesting the above information, while spoofing the source ip. Blocking individual addresses required constant monitoring.

In order to reduce the log file size and CPU time, I implemented a set of iptables rules designed to:
  1. Limit the number of connections to UDP 53
  2. Match the strings; ripe.net and isc.org (discovered using Wireshark)

I have successfully managed to drop all these requests at the firewall and releasing the strain on my servers' I/O and CPU time, however using iptraf and 'iptables -vnL --line-numbers' the traffic is still continuing with the following statistics after 24 hours:

Code:
num   pkts bytes target     prot opt in     out     source               destination
1      22M 1387M            udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: SET name: DNSLF side: source
2      21M 1382M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: UPDATE seconds: 1 hit_count: 6 name: DNSLF side: source
3    82190 5269K            udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: SET name: DNSHF side: source
4     4522  289K DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: UPDATE seconds: 7 hit_count: 20 name: DNSHF side: source
5    4797K  316M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

1     1927  118K DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|076c796e6c656f6e03636f6d00|" ALGO name bm TO 65535 udp dpt:53
2    2052K  135M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|0472697065036e657400|" ALGO name bm TO 65535 udp dpt:53
3    2371K  152M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|00000000000103697363036f726700|" ALGO name bm TO 65535 udp dpt:53
4        0     0 DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|00000000010372697065036e657400|" ALGO name bm TO 65535 udp dpt:53
5    3576K  276M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
Although my firewall rules are working, I would prefer to find a more permanent method to block/reject. Anybody have any advice?

My server specs:
AuthenticAMD, AMD Phenom(tm) II X6 1055T Processor (6 CPU)
16GB RAM
1TB HDD
100Mbps/sec full duplex
Hardware Firewall
Linux 2.6.32-279.5.1.el6.x86_64 #1 SMP Tue Aug 14 23:54:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Plesk 11.0.9 (fully up-to-date)
'iptables' and 'ossec-hids'
8 IP addresses
authoritative name server (12 zones)

Thanks in advance,

Mark
 
Old 02-18-2013, 09:30 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,017
Blog Entries: 54

Rep: Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764
Quote:
Originally Posted by accessthecloud View Post
After some research I discovered that during a recent upgrade to Plesk, the DNS recursion settings were changed (by the upgrade script) to Any host! This resulted in my server acting as an open DNS answering all requests. This I changed immediately.
0) I hope you filed a bug report? 1) Changed how (as in allowing now recursion from where exactly: details)?


Quote:
Originally Posted by accessthecloud View Post
In order to reduce the log file size and CPU time, I implemented a set of iptables rules designed to: Limit the number of connections to UDP 53 Match the strings; ripe.net and isc.org (discovered using Wireshark)
First of all if you're using ISC BIND for dual reasons, caching name server plus authoritative name server, then you should split those up: providing yourself and your LAN with BIND as caching name server requires it to have recursion set, your publicly accessible authoritative name server does not.

Please read links / advice posted in: http://www.linuxquestions.org/questi...on-4175417284/, http://www.linuxquestions.org/questi...es-4175450008/ and maybe http://www.openlogic.com/wazi/bid/18...-DNS-with-BIND. You may find you have already implemented all of that but it never hurts to ensure you have. After checking / implementing that Netfilter rate limiting is still useful but you should not need IP blocking anymore (if you do use ipset instead) and having Netfilter do string matches is computationally way expensive (use a Snort rule instead). If you have ensured your NS can't be used for amplification attacks anymore you probably want to make BIND log less.
 
1 members found this post helpful.
Old 02-18-2013, 03:43 PM   #3
accessthecloud
LQ Newbie
 
Registered: Apr 2011
Location: Glasgow, Scotland
Distribution: CentOS 5.5, Plesk 9-10
Posts: 9

Original Poster
Rep: Reputation: 0
Thanks for the reply.

0) & 1) Bug report? : The Plesk Administrator's Guide explains! Link
Quote:
After your install Parallels Plesk Panel, the built-in DNS server defaults to serving recursive queries only from your own server and from other servers located in your network. This is the optimal setting. If your upgraded from earlier versions of Parallels Plesk Panel, your DNS server defaults to serving recursive queries from any host.
I run an authoritative name server for my domains (and my customers). When I query my name server for one of my own domains, I get the right answer. Querying for anything else replies with Query refused. After the upgrade and before I realised it, I was running an open DNS ! My server was obviously found and abused. What annoys me is that they are continuing with the isc and ripe requests, and these are dropped at the firewall.

While I do believe that the rate limiting I added to my firewall is helpful, I am concerned about any performance hit. The string matching is, as you stated; expensive but without this with the rate limiting on, named is using around 10% of 3/4 cores constantly.
Quote:
If you have ensured your NS can't be used for amplification attacks anymore you probably want to make BIND log less.
Definitely cannot be used for amplification attacks. Would altering the logging change the CPU usage?

Snort looks useful and ipset is exactly what I need. The links you provided are also very good for reference. Thanks

I will update what I do in due course.

FYI; statistics of iptables after 36 hours
Code:
num   pkts bytes target     prot opt in     out     source               destination
1      32M 2086M            udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: SET name: DNSLF side: source
2      32M 2080M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: UPDATE seconds: 1 hit_count: 6 name: DNSLF side: source
3    98105 6292K            udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: SET name: DNSHF side: source
4     4918  315K DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 state NEW recent: UPDATE seconds: 7 hit_count: 20 name: DNSHF side: source
5    4819K  318M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

1     2957  183K DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|076c796e6c656f6e03636f6d00|" ALGO name bm TO 65535 udp dpt:53
2    2052K  135M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|0472697065036e657400|" ALGO name bm TO 65535 udp dpt:53
3    2373K  152M DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "|00000000000103697363036f726700|" ALGO name bm TO 65535 udp dpt:53
4    4070K  337M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0
 
  


Reply

Tags
dns, hacking, iptables


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
forward DNS requests Gil@LQ Linux - Networking 3 09-02-2012 07:51 AM
can a local DNS Server be used to handle dns requests going out to the internet baronobeefdip Linux - Server 1 07-03-2012 03:19 AM
stopping dns forwarding requests in BIND shreeram.vk Linux - Server 3 07-10-2008 06:40 AM
DNS, Bind, same ip for all requests NightSoul Linux - Networking 6 03-28-2008 11:39 AM
foward dns requests ? black1 Linux - Networking 1 03-06-2006 03:52 AM


All times are GMT -5. The time now is 07:02 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration