LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-07-2017, 01:42 PM   #1
bob hope
LQ Newbie
 
Registered: Dec 2017
Posts: 6

Rep: Reputation: Disabled
Cool Sorting failed url attempts


Hi, I need to answer -
How many times was the test.html page accessed successfully
(status code 200) by a python user agent? and also
Which bad URLs were attempted by a python agent?

I am running a vm ubuntu latest version. I am accessing the apache2/error.log file and don't know the correct command (even after extensive research).

I found, how many people generally accessed test.html but don't know specific commands. I am very new to Linux so any help would be gratefully received.

Thanks.
 
Old 12-07-2017, 02:09 PM   #2
!j*
LQ Newbie
 
Registered: Nov 2017
Posts: 11

Rep: Reputation: Disabled
Specifics IN, Specifics OUT!

Hello, and "Welcome to LinuxQuestions.org", as they say. I'm new here also.

Looks like you read the requirements on "which distro", but I'm pretty sure
that they're gonna require more info on your web searches:
What exact search keywords you used, and a couple specific samples of what you found.

So, I'd recommend trying this: search the www for an example line of your two questions, and copy/paste a sample web-log line for each. They will correct you in case you get it wrong, so no harm in showing exactly and fully whatever you tried.

Also, if you need help with grep/sort/uniq, I can find the 'man' pages on-line (I search just: man grep and up it pops, at the top!), and help you there, especially if you show the desired output.

Again, provide more on what work you did (so they won't get mad). Best wishes always, for academic success!

Last edited by !j*; 12-07-2017 at 02:36 PM. Reason: Add a clearer & more precise Title!
 
Old 12-07-2017, 02:39 PM   #3
bob hope
LQ Newbie
 
Registered: Dec 2017
Posts: 6

Original Poster
Rep: Reputation: Disabled
Hi thanks very for your help. It's much more friendly on here than stack. I'm told that I must use the access.log and error.log only to get my results. How would I even now what denotes a python user access? Also if I could post the output of error and access log to a txt file I could search as I went along. Is that possible? Thanks again.
 
Old 12-07-2017, 02:45 PM   #4
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,710
Blog Entries: 3

Rep: Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184
It's possible to do all that. It's easy enough perl. Awk might work for some use-cases, too.

Post a line or two here of what you are looking for. Apache's logs are pretty standard but there are some variations and the devil is in the details.
 
Old 12-07-2017, 03:24 PM   #5
bob hope
LQ Newbie
 
Registered: Dec 2017
Posts: 6

Original Poster
Rep: Reputation: Disabled
Hi thanks. I'm using apache2/access.log and error.log

I used history |.... To find the general access number of the html file. But how I go from there to using a command that includes history | python users access is beyond me.

Sorry, what other code is best to post here? It's via val/log/apache2/access.log and error.log respectively. Cheers.
 
Old 12-07-2017, 10:44 PM   #6
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,710
Blog Entries: 3

Rep: Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184Reputation: 1184
Just show two or three lines from the log file in question which illustrate exactly what you are looking for. Then paste them here in a comment between [code] [/code] tags for readability.
 
Old 12-08-2017, 01:43 AM   #7
rhubarbdog
Member
 
Registered: Apr 2015
Location: Yorkshire, England
Distribution: Linux Mint
Posts: 119

Rep: Reputation: Disabled
Start with
Code:
 grep test.html error.log access.log
you're going to endup using awk, but this will show you what information is being reported about test.html
 
Old 12-08-2017, 03:15 AM   #8
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 3,291

Rep: Reputation: 587Reputation: 587Reputation: 587Reputation: 587Reputation: 587Reputation: 587
See if this works: https://goaccess.io/
 
Old 12-08-2017, 09:48 AM   #9
bob hope
LQ Newbie
 
Registered: Dec 2017
Posts: 6

Original Poster
Rep: Reputation: Disabled
Code:
[Wed Dec 06 11:01:51.340922 2017] [core:error] [pid 22778] [client 129.12.21.67:43904] AH00126: Invalid URI in request GET /././././././../../../../../windows/win.ini HTTP/1.1
[Wed Dec 06 11:01:51.370044 2017] [core:error] [pid 36187] [client 129.12.21.67:43930] AH00126: Invalid URI in request GET /././././././../../../../../winnt/win.ini HTTP/1.1
[Wed Dec 06 11:01:51.406300 2017] [core:error] [pid 109186] [client 129.12.21.67:43970] AH00126: Invalid URI in request GET .\\.\\.\\.\\.\\.\\.\\.\\.\\.\\/windows/win.ini HTTP/1.1
[Wed Dec 06 11:01:51.429850 2017] [core:error] [pid 23358] [client 129.12.21.67:44000] AH00126: Invalid URI in request GET .\\.\\.\\.\\.\\.\\.\\.\\.\\.\\/winnt/win.ini HTTP/1.1
[Wed Dec 06 11:01:51.690182 2017] [core:error] [pid 23533] [client 129.12.21.67:44316] AH00126: Invalid URI in request GET /../../../../../../../../../../../../etc/passwd HTTP/1.1
[Wed Dec 06 11:01:51.710068 2017] [core:error] [pid 23532] [client 129.12.21.67:44344] AH00126: Invalid URI in request GET /./../../../../../../../../../../../etc/passwd HTTP/1.1
[Wed Dec 06 11:01:51.734505 2017] [core:error] [pid 109186] [client 129.12.21.67:44374] AH00126: Invalid URI in request GET //../../../../../../../../../../../../etc/passwd HTTP/1.1
[Wed Dec 06 11:01:51.776560 2017] [core:error] [pid 36185] [client 129.12.21.67:44434] AH00126: Invalid URI in request GET /%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd HTTP/1.1
[Wed Dec 06 11:01:51.849691 2017] [core:error] [pid 36181] [client 129.12.21.67:44514] AH00126: Invalid URI in request GET /././././././../../../../../etc/passwd HTTP/1.1
[Wed Dec 06 11:01:51.996024 2017] [core:error] [pid 23358] [client 129.12.21.67:34196] AH00126: Invalid URI in request GET /../../../../../../../../../../../../windows/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.035684 2017] [core:error] [pid 36179] [client 129.12.21.67:34238] AH00126: Invalid URI in request GET /../../../../../../../../../../../../winnt/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.089146 2017] [core:error] [pid 36182] [client 129.12.21.67:34278] AH00126: Invalid URI in request GET ../../../../../../../../../../../../windows/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.131010 2017] [core:error] [pid 109188] [client 129.12.21.67:34316] AH00126: Invalid URI in request GET ../../../../../../../../../../../../winnt/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.168549 2017] [core:error] [pid 23358] [client 129.12.21.67:34356] AH00126: Invalid URI in request GET ..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\windows\\win.ini HTTP/1.1
[Wed Dec 06 11:01:52.212185 2017] [core:error] [pid 24411] [client 129.12.21.67:34402] AH00126: Invalid URI in request GET ..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\winnt\\win.ini HTTP/1.1
[Wed Dec 06 11:01:52.677372 2017] [core:error] [pid 23465] [client 129.12.21.67:34830] AH00126: Invalid URI in request GET /%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/windows/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.713074 2017] [core:error] [pid 36181] [client 129.12.21.67:34864] AH00126: Invalid URI in request GET /%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/winnt/win.ini HTTP/1.1
[Wed Dec 06 11:01:52.857580 2017] [core:error] [pid 36183] [client 129.12.21.67:34986] AH00126: Invalid URI in request GET %5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5c%2e%2e%5cwinnt%5cwin.ini HTTP/1.1
 
Old 12-08-2017, 09:52 AM   #10
bob hope
LQ Newbie
 
Registered: Dec 2017
Posts: 6

Original Poster
Rep: Reputation: Disabled
I am taking where it says Invalid, the next word is URL although it looks like URI. Or this that something to do with invalid user found? If so this is my next question find invalid user attempts.

I found using the history | command it outputs a list an attempts via python users is on there. So that answers one question! Thanks for the help so far everyone. Appreciate it.
 
Old 12-08-2017, 01:54 PM   #11
!!!
Member
 
Registered: Jan 2017
Posts: 633

Rep: Reputation: 245Reputation: 245Reputation: 245
Thanks for the logs (in code tags, great!). It does help clarify!
Are you ready to study awk yet? (tons of examples on the www; post some attempts)
Maybe add some examples to that subsequent Thread on invalid user.
https://www.linuxquestions.org/quest...ch-4175619193/
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
failed to attempts harish-mn Linux - Security 3 10-22-2016 04:59 PM
SU failed attempts capturing Manivasagam Linux - Server 1 05-22-2012 08:47 AM
failed login attempts smilemukul Linux - Newbie 7 12-16-2010 01:46 PM
Configure Failed logins to lock accounts after 5 failed attempts mccartjd Linux - Newbie 5 05-05-2008 09:02 AM
all attempts failed btb103 Linux - General 1 10-23-2001 06:31 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration