LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-24-2005, 05:48 AM   #1
jonhewer
Member
 
Registered: Apr 2005
Location: London, UK
Distribution: Debian Lenny, Debian Etch, Ubuntu Hardy
Posts: 71

Rep: Reputation: 15
text processing, maintaining a log


hi

my shell script is maintaining a log file, with log entries consisting of a varying number of lines, but always seperared from one another with a blank line

i only really need to keep, say, the last 20 entires in the log - does anyone know a command i can use which will do this, presumably using the fact that log entires are seperated by blank lines

hope that makes sense?

cheers
 
Old 08-24-2005, 06:07 AM   #2
Simon Bridge
LQ Guru
 
Registered: Oct 2003
Location: Waiheke NZ
Distribution: Ubuntu
Posts: 9,211

Rep: Reputation: 198Reputation: 198
Here's a list for you to read the man pages of:

cat, cut, dd, diff, fold, grep, nl, patch, sed, sort, split, tac, tail, tr, wc

grep and sed will probably help the most.

You want to count the blank lines = N, then copy out everything from N-10 onwards. Or you could reverse the line order, delete everything after the tenth blank line, the re-reverese the file.
 
Old 08-24-2005, 06:10 AM   #3
jonhewer
Member
 
Registered: Apr 2005
Location: London, UK
Distribution: Debian Lenny, Debian Etch, Ubuntu Hardy
Posts: 71

Original Poster
Rep: Reputation: 15
i'll have a read, thanks
 
Old 08-24-2005, 07:40 AM   #4
jonhewer
Member
 
Registered: Apr 2005
Location: London, UK
Distribution: Debian Lenny, Debian Etch, Ubuntu Hardy
Posts: 71

Original Poster
Rep: Reputation: 15
erm....any more hints? not sure how i can grab the 10th last blank line onwards ....
 
Old 08-24-2005, 08:20 AM   #5
bathory
LQ Guru
 
Registered: Jun 2004
Location: Piraeus
Distribution: Slackware
Posts: 13,159
Blog Entries: 1

Rep: Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021Reputation: 2021
To view the 20 last entries of your logfile:
Code:
tail -n 20 logfile
 
Old 08-24-2005, 08:34 AM   #6
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 66
Code:
tac logfile | awk '/^$/ {N++}; N<11 {print $0}' | tac
Yves.
 
Old 08-24-2005, 08:43 AM   #7
geeman2.0
Member
 
Registered: Feb 2005
Location: Ontario, Canada
Distribution: Gentoo, Slackware
Posts: 345

Rep: Reputation: 30
You could periodically do something like this:
Code:
tail log.txt > tmp.txt
cat tmp.txt > log.txt
 
Old 08-24-2005, 09:19 AM   #8
jonhewer
Member
 
Registered: Apr 2005
Location: London, UK
Distribution: Debian Lenny, Debian Etch, Ubuntu Hardy
Posts: 71

Original Poster
Rep: Reputation: 15
thanks very much YinYeti, that's perfect
 
Old 08-24-2005, 10:16 PM   #9
Simon Bridge
LQ Guru
 
Registered: Oct 2003
Location: Waiheke NZ
Distribution: Ubuntu
Posts: 9,211

Rep: Reputation: 198Reputation: 198
bathory and geeman2.0: nice try - would be fine if there was one line per entry, but there are multiple lines per entry... each multiple-line entry sep'd by a blank line. Now you see why tail wouldn't do it.

theYinYeti: well done - I missed out awk didn't I
 
Old 08-25-2005, 02:24 AM   #10
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 66
Thanks SimonBridge

Looking back on it, I see that the tip I could have given is the necessity of counting.
Unless you manage some sort of 10-blocks long FIFO, you have to decide when you start counting (and displaying), and when you stop. This can only be done from the tail end, hence the tac.
Next, the counter had to be placed in a variable, because a simple line count did not apply here. And variables are unknown to common text processing tools, except to awk, hence the awk.
Now the output only had to be put in the proper order again, hence the final tac.

Only logic, you see.

Yves.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Processing squid's access.log into a mysql DB eantoranz Programming 3 05-12-2013 03:38 PM
input text file processing the gnu way? zero79 Programming 3 03-04-2005 07:41 PM
text editingor processing exodist Linux - General 2 11-29-2003 10:24 PM
Log all console text cli_man Linux - General 2 10-17-2003 05:02 AM
text processing Gantrep Linux - General 4 02-17-2003 10:37 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:25 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration