Query large amount of domains

AndrewNZ · 08-21-2014, 01:55 AM

Hi ,

I have a list of domains in a spreadsheet and I want to find out the MX records for all of them. How can I write something in bash to check this?

Thank you.

jessetaylor84 · 08-21-2014, 02:33 AM

To look up the MX record, you can use:

Code:

nslookup -query=mx www.example.com

You will somehow need to create a list of domains from the spreadsheet (how you do so depends on the spreadsheet format). Then you can loop through each of the domains in the list and look it up with nslookup.

If you need guidance on how to use loops in bash, check out this guide.

allend · 08-21-2014, 02:36 AM

Code:

dig +short @8.8.8.8 optusnet.com.au MX

returns

Quote:

50 extmail.optusnet.com.au.

so wrapping the command in a loop

Code:

while read aline; do
  ...
done < domains.txt

should be getting close to what you want.

AndrewNZ · 08-21-2014, 03:01 AM

so where its ... put dig something?

allend · 08-21-2014, 03:09 AM

Yes

Code:

  echo $aline # Echo the line read from domains.txt
  dig +short @8.8.8.8 $aline MX # Do the MX record lookup

AndrewNZ · 08-21-2014, 05:38 PM

When I write the following I just get ">" on a line after?

while read aline; do dig +short $aline mx done < domains.csv

allend · 08-21-2014, 07:17 PM

At the command line you need an additional semicolon to separate the statements.

Code:

while read aline; do dig +short $aline MX; done < domains.csv

The type of record to query needs to specified as MX. Note the capitalisation.

Hopefully your domains.csv file contains the domains listed one per line and you have run the file through a utility such as fromdos to convert the CRLF line endings to LF only line endings.

chris2k · 02-08-2015, 02:18 PM

i would use the for command like descriped here:

Code:

for i in `cat domain.csv`; 
do 
echo $i
dig $i mx +short 
done

I would echo the domain too, because for example google has more mx-records for one domain.

Or if you want to have a semicolon-seperated output:

Code:

IFS='
'
for i in `cat domain.csv`
do
 for j in `dig $i mx +short`
 do
  echo $i";"`echo $j | sed -e "s/^[0-9]* //g"`
 done
done

for example, in your domain.csv is yahoo.com and google.com - this is the output:

Quote:

google.com;alt1.aspmx.l.google.com.
google.com;alt4.aspmx.l.google.com.
google.com;alt2.aspmx.l.google.com.
google.com;alt3.aspmx.l.google.com.
google.com;aspmx.l.google.com.
yahoo.com;mta6.am0.yahoodns.net.
yahoo.com;mta7.am0.yahoodns.net.
yahoo.com;mta5.am0.yahoodns.net.