LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 09-02-2014, 08:23 PM   #16
newbie14
Member
 
Registered: Sep 2011
Posts: 646

Original Poster
Rep: Reputation: Disabled

Dear Unspawn,
Below is the results. Thus only the first one with the .html is found in those gz files. But there is none for the .exe. How to device the exact lines of .html from this files so that I can put here?

Quote:
# zgrep -m1 -c '.html' /var/log/httpd/access*.[0-9]*.gz|awk -F':' '{if($2 !~ 0) print $1}'
/var/log/httpd/access_log.10.gz
/var/log/httpd/access_log.11.gz
/var/log/httpd/access_log.12.gz
/var/log/httpd/access_log.13.gz
/var/log/httpd/access_log.14.gz
/var/log/httpd/access_log.15.gz
/var/log/httpd/access_log.16.gz
/var/log/httpd/access_log.17.gz
/var/log/httpd/access_log.18.gz
/var/log/httpd/access_log.19.gz
/var/log/httpd/access_log.20.gz
/var/log/httpd/access_log.21.gz
/var/log/httpd/access_log.22.gz
/var/log/httpd/access_log.23.gz
/var/log/httpd/access_log.24.gz
/var/log/httpd/access_log.25.gz
/var/log/httpd/access_log.26.gz
/var/log/httpd/access_log.27.gz
/var/log/httpd/access_log.28.gz
/var/log/httpd/access_log.29.gz
/var/log/httpd/access_log.2.gz
/var/log/httpd/access_log.30.gz
/var/log/httpd/access_log.31.gz
/var/log/httpd/access_log.32.gz
/var/log/httpd/access_log.33.gz
/var/log/httpd/access_log.34.gz
/var/log/httpd/access_log.35.gz
/var/log/httpd/access_log.36.gz
/var/log/httpd/access_log.37.gz
/var/log/httpd/access_log.38.gz
/var/log/httpd/access_log.39.gz
/var/log/httpd/access_log.3.gz
/var/log/httpd/access_log.40.gz
/var/log/httpd/access_log.41.gz
/var/log/httpd/access_log.42.gz
/var/log/httpd/access_log.43.gz
/var/log/httpd/access_log.44.gz
/var/log/httpd/access_log.45.gz
/var/log/httpd/access_log.46.gz
/var/log/httpd/access_log.47.gz
/var/log/httpd/access_log.48.gz
/var/log/httpd/access_log.49.gz
/var/log/httpd/access_log.4.gz
/var/log/httpd/access_log.50.gz
/var/log/httpd/access_log.51.gz
/var/log/httpd/access_log.52.gz
/var/log/httpd/access_log.53.gz
/var/log/httpd/access_log.5.gz
/var/log/httpd/access_log.6.gz
/var/log/httpd/access_log.7.gz
/var/log/httpd/access_log.8.gz
/var/log/httpd/access_log.9.gz
# zgrep -he "(POST|GET)..*\.exe.HTTP\/" /var/log/httpd/access*.[0-9]*.gz
#
 
Old 09-02-2014, 08:53 PM   #17
newbie14
Member
 
Registered: Sep 2011
Posts: 646

Original Poster
Rep: Reputation: Disabled
Dear Unspawn,
Ok I manually extracted some results from some of the log files. I notice most of it are robots.

Quote:
"GET / HTTP/1.1" 200 30 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
66.249.73.11 - - [06/Oct/2013:04:58:40 +0800] "GET /robots.txt HTTP/1.1" 404 208 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

46.229.164.102 - - [06/Oct/2013:19:55:18 +0800] "GET /myweb/login.php HTTP/1.1" 200 3340 "-" "Mozilla/5.0 (compatible; SemrushBot/0.97; +http://www.semrush.com/bot.html)"

66.249.74.32 - - [06/Oct/2013:22:28:52 +0800] "GET /robots.txt HTTP/1.1" 404 208 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.32 - - [06/Oct/2013:22:28:53 +0800] "GET / HTTP/1.1" 200 30 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

66.249.74.32 - - [07/Oct/2013:04:20:16 +0800] "GET /robots.txt HTTP/1.1" 404 208 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

198.27.82.183 - - [07/Oct/2013:18:21:00 +0800] "GET /robots.txt HTTP/1.1" 404 208 "-" "Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)"

31.193.197.37 - - [12/Oct/2013:00:05:32 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"

31.193.197.37 - - [12/Oct/2013:00:05:32 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:32 +0800] "GET / HTTP/1.1" 200 30 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:33 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:34 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:34 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:35 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:35 +0800] "GET /robots.txt HTTP/1.1" 404 208 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:36 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:36 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:37 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:38 +0800] "OPTIONS / HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
31.193.197.37 - - [12/Oct/2013:00:05:38 +0800] "\x16\x03\x01\x01\x8b\x01" 501 218 "-" "-"
31.193.197.37 - - [12/Oct/2013:00:05:38 +0800] "\x16\x03\x01\x01\x8b\x01" 501 218 "-" "-"

75.98.9.254 - - [28/Aug/2013:11:37:03 +0800] "GET / HTTP/1.1" 200 30 "-" "Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)"
Except for these two which I saw looking for some stats on my domain link.

Quote:
37.9.53.51 - - [29/Sep/2013:09:47:44 +0800] "GET /webalizer/usage_201309.html HTTP/1.0" 404 225 "http://mydomain/webalizer/usage_201309.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90)"
37.9.53.51 - - [10/Oct/2013:13:46:16 +0800] "GET /stats/usage_201309.html HTTP/1.0" 404 221 "http://mydomain/stats/usage_201309.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90)"
 
Old 09-04-2014, 12:43 AM   #18
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
I don't see anything worth pursuing here. Anything else to add?
 
1 members found this post helpful.
Old 09-04-2014, 04:04 AM   #19
newbie14
Member
 
Registered: Sep 2011
Posts: 646

Original Poster
Rep: Reputation: Disabled
Dear Unspawn,
Do you want me to go through all the files manually and extract .html logs? Can you suggest how to totally block these robots.

I have this page. I think made a mistake by putting in my folder should I put in /var/www/html or any other location?

Quote:
User-agent: *
Disallow: /
 
Old 09-06-2014, 03:27 AM   #20
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by newbie14 View Post
Do you want me to go through all the files manually and extract .html logs?
I don't know what you mean by ".html logs" but if you mean Apaches access_log then if you're satisfied by what your own answer provided then by all means go do something else. If you don't understand or are curious as to what was going then you investigate further.


Quote:
Originally Posted by newbie14 View Post
should I put in /var/www/html or any other location?
Note spiders look for "/robots.txt" so, yes, in whatever place your web sites docroot (aka its "/") resides. That may be /var/www/html/, yes.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Is there any way run windows executable files in Linux ? prasanthsp Linux - Software 3 03-13-2012 08:57 AM
How to delete protected Windows executable files via Linux? keithrennie Linux - General 2 10-29-2009 07:24 PM
windows executable binary files not working on RHEL5 fzanella Linux - Hardware 1 07-28-2009 02:36 PM
executable files in linux and windows... GEJOE DANIEL Programming 7 01-12-2006 01:59 PM
all files in mounted windows partition are executable zovres Linux - Newbie 4 07-17-2003 06:44 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 08:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration