LinuxQuestions.org - Retrieving FQDN from squid access.log

I am trying to figure out a way to retrieve FQDN from access.log i.e. for http(s)://www.cnn.com/whever/complex/line/this/might/be I would like output to be:

www.cnn.com

Here is what my access.log contain:

Code:

root@cachepilot:/var/log/squid# tail /var/log/squid/access.log

1257884141.586    119 10.182.16.205 TCP_MISS/200 2423 GET http://t0.gstatic.com/images?q=tbn:B3386mi19hBkMM:http://www.artistryofiron.com/Images/Cutouts/lizzard.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.618    176 10.182.16.205 TCP_MISS/200 5745 GET http://t1.gstatic.com/images?q=tbn:DLRVN_v-bM3PsM:http://lounginlizzard.com/store/images/llhammockchair.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.736    117 10.182.16.205 TCP_MISS/200 4045 GET http://t2.gstatic.com/images?q=tbn:UHSmX_XQMcMzOM:http://1.bp.blogspot.com/_ks1ZRojRD8k/SJJTVhimFVI/AAAAAAAAAiE/bhpsac_DHpY/s400/alien_lizzard.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.736      0 10.182.16.205 TCP_NEGATIVE_HIT/204 370 GET http://clients1.google.com/generate_204 - NONE/- text/html

1257884141.756    169 10.182.16.205 TCP_MISS/200 3743 GET http://t1.gstatic.com/images?q=tbn:pMfVy6iLu7pjPM:http://www.buchinger.or.at/pic/animals/20050101-Lizzard-Nakkuru.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.765    192 10.182.16.205 TCP_MISS/200 4775 GET http://t1.gstatic.com/images?q=tbn:zXJFTlypl8JCVM:http://bp1.blogger.com/_RloTrSWsm7A/RipCKwIS3_I/AAAAAAAAAGc/pXo0bLfKV-Q/s400/caiman_lizzard_pantanal-FBAL.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.826    269 10.182.16.205 TCP_MISS/200 6589 GET http://t3.gstatic.com/images?q=tbn:87-PS67EHxfMJM:http://nmsouthernskies.com/PhotoHiRes/Lizzard.JPG - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

1257884141.978    152 10.182.16.205 TCP_MISS/204 452 GET http://images.google.com/csi?v=3&s=images&action=&e=17259,21329,21517,21766,22107,22712&ei=Acr5St7BNIKV8Abl9pTSDA&rt=prt.422,xjs.516,ol.1703 - DEFAULT_PARENT/wwwproxy.k12.de.us text/html

1257884144.617    64 10.182.16.156 TCP_MISS/304 452 GET http://www.etymonline.com/style.css - DEFAULT_PARENT/wwwproxy.k12.de.us text/css

1257884144.618    59 10.182.16.156 TCP_MISS/304 457 GET http://www.etymonline.com/graphics/header.jpg - DEFAULT_PARENT/wwwproxy.k12.de.us image/jpeg

Here is my current script that I work on so far -- it is hair-pulling experience:

Code:

tail /var/log/squid/access.log | 

awk '{print $1,$3,$7}' | 

while read line; do

echo $line

time=$(echo "$line" | cut -d ' ' -f 1)

ip=$(echo "$line" | cut -d ' ' -f 2)

url=$(echo "$line" | cut -d ' ' -f 3)

fqdn=$(echo "$url" | cut -d '/' -f 2)

echo "Time: $time $ip => $fqdn"

done

The output is as follows:

Code:

root@cachepilot:/var/log/squid# tail /var/log/squid/access.log |

> awk '{print $1,$3,$7}' |

> while read line; do

> echo $line

> time=$(echo "$line" | cut -d ' ' -f 1)

> ip=$(echo "$line" | cut -d ' ' -f 2)

> url=$(echo "$line" | cut -d ' ' -f 3)

> fqdn=$(echo "$url" | cut -d '/' -f 2)

> echo "Time: $time $ip => $fqdn"

> done

1257884402.710 10.182.16.156 http://www.etymonline.com/style.css

Time: 1257884402.710 10.182.16.156 =>

1257884402.750 10.182.16.153 http://www.schoolnotes.com/cgi-bin/notesupdate-new.pl

Time: 1257884402.750 10.182.16.153 =>

1257884403.135 10.182.16.153 http://clk.atdmt.com/go/175199563/direct;vt.1;wi.160;hi.600;ai.131775080;ct.d;ea.364/01/

Time: 1257884403.135 10.182.16.153 =>

1257884404.009 10.182.16.97 http://64.12.161.103/aim/fetchEvents?aimsid=088.1218851095.1557230312:cchowe&seqNum=10&rnd=1257884404.436330&timeout=20000&r=84&k=ke1KS-K6BdPDaRnC&f=json&a=%252FwQAAAAAAABBSgVh6p%252BakWe8%252FHCb%252F3YAGJT5VgU26pjC3sle9NGn0XL11zN9gHdCK32tHIf1OMCIJL%252B7B6cabYQnU0BHtC2HFXsNP5h2%252BsbgdAcT0lc2q5o%252FoVloSsXKXsJD1S5YUOHl5KamubP1a6QUEdbGRfrxr6nrkw%253D%253D&dojo.preventCache=1257884379785&c=dojo.io.script.jsonp_dojoIoScript85._jsonpCallback

Time: 1257884404.009 10.182.16.97 =>

I am trying to output the domain name after "=>" i.e. on the last line it should be :

Code:

Time: 1257884404.009 10.182.16.97 => 64.12.161.103

Any suggestions?