LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-16-2015, 03:36 PM   #1
MBA Whore
Member
 
Registered: May 2006
Location: Kansas City, MO
Distribution: Various: pclos, Debian, Ubuntu, etc . . .
Posts: 649

Rep: Reputation: 30
Prevent my logs from growing so large


I have a Kubuntu install that works fine but for some reason, it grows the logs into hundreds of GBs.

Is there a method to prevent those logs from growing so large? I researched "log rotate" but I couldn't understand it.

I would like something established such that after a certain size, the logs are automatically deleted so I don't have to do it manually each time.

How would I do so?

Here are the logs:

#1: /var/log/syslog
#2: /var/log/kern.log

I already determined I don't need them, so I would like to auto-delete them once they reach a certain size.

Thank you.
 
Old 03-16-2015, 05:15 PM   #2
Pearlseattle
Senior Member
 
Registered: Aug 2007
Location: Zurich, Switzerland
Distribution: Gentoo
Posts: 1,001

Rep: Reputation: 142Reputation: 142
Sorry, but you definitely don't want to delete those logs by default.
If you've got those logs growing that large ("hundreds of GBs", hehe...., that's just too funny...) then you definitely want to/must look at the root cause of the problem.

I'm sure that once you fix the issue/s your PC will feed you with additional CPU/IO throuput (which will not be busy discarding the hundreds of GBs of output that you're planning to delete).
 
Old 03-16-2015, 06:06 PM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by MBA Whore View Post
Is there a method to prevent those logs from growing so large?
What Pearlseattle said, really.


Quote:
Originally Posted by MBA Whore View Post
I researched "log rotate" but I couldn't understand it. I would like something established such that after a certain size, the logs are automatically deleted so I don't have to do it manually each time.
Then you have to "research" some more as what you describe pretty much is a default case for 'logrotate'. See contents of /etc/logrotate.d/ and examples in 'man logrotate'.
 
Old 03-17-2015, 04:54 PM   #4
MBA Whore
Member
 
Registered: May 2006
Location: Kansas City, MO
Distribution: Various: pclos, Debian, Ubuntu, etc . . .
Posts: 649

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by Pearlseattle View Post
Sorry, but you definitely don't want to delete those logs by default.
If you've got those logs growing that large ("hundreds of GBs", hehe...., that's just too funny...) then you definitely want to/must look at the root cause of the problem.

I'm sure that once you fix the issue/s your PC will feed you with additional CPU/IO throuput (which will not be busy discarding the hundreds of GBs of output that you're planning to delete).

Forgive my ignorance, but would I really need to retain those logs if this particular desktop is a dedicated "number cruncher"? What I mean is: I donate its cpu and gpu cycles to various online distributed computing projects. That is all I do with this computer.

I have a different computer for "my" needs.

With that in mind, is it ok to auto delete them by default (i.e., upon a certain size, etc?)

Thank you again for your time!
 
Old 03-17-2015, 04:55 PM   #5
MBA Whore
Member
 
Registered: May 2006
Location: Kansas City, MO
Distribution: Various: pclos, Debian, Ubuntu, etc . . .
Posts: 649

Original Poster
Rep: Reputation: 30
On a related note - I can't open the logs because they are so large (the computer just freezes and runs out of ram).

Is there some way I can look into those logs to see exactly which program is causing the trouble?
 
Old 03-17-2015, 06:14 PM   #6
joe_2000
Senior Member
 
Registered: Jul 2012
Location: Aachen, Germany
Distribution: Void, Debian
Posts: 1,019

Rep: Reputation: 308Reputation: 308Reputation: 308Reputation: 308
Quote:
Originally Posted by MBA Whore View Post
On a related note - I can't open the logs because they are so large (the computer just freezes and runs out of ram).

Is there some way I can look into those logs to see exactly which program is causing the trouble?
Again, logrotate is your friend. It'll break the logs into little pieces, easy to open...
 
Old 03-18-2015, 03:19 AM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by MBA Whore View Post
On a related note - I can't open the logs because they are so large (the computer just freezes and runs out of ram).
Is there some way I can look into those logs to see exactly which program is causing the trouble?
You could use 'split' (see 'man split') and then open one chunk at a time or 'head -1000 /path/to/logfile|less;' or let Logwatch generate a report as it's good in tallying errors.
 
Old 03-18-2015, 05:33 AM   #8
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,780

Rep: Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567Reputation: 7567
I would suggest you to use another tool to open those files, do not use an editor (more, less, or most can be used, they will not eat up your ram)
 
Old 03-18-2015, 09:45 AM   #9
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,950

Rep: Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513
How about gzip (or similar), followed by recreating an empty file for more logs.

This is basically what logrotate does.
 
Old 03-18-2015, 03:07 PM   #10
Pearlseattle
Senior Member
 
Registered: Aug 2007
Location: Zurich, Switzerland
Distribution: Gentoo
Posts: 1,001

Rep: Reputation: 142Reputation: 142
Quote:
Originally Posted by MBA Whore View Post
Forgive my ignorance, but would I really need to retain those logs if this particular desktop is a dedicated "number cruncher"? What I mean is: I donate its cpu and gpu cycles to various online distributed computing projects. That is all I do with this computer.

I have a different computer for "my" needs.

With that in mind, is it ok to auto delete them by default (i.e., upon a certain size, etc?)

Thank you again for your time!
Yes, you definitely need to keep those logs and not only that - you'll have to fix this issue and then even still check them from time to time, especially in your case where you'll use your PC as a "number cruncher for distributed computing projects"!

There is more than one main reason:
  1. First of all if your logs are that big (even just more than a few MBs) your PC is probably really busy by writing just those logs.
    Writing that much does not come for free and therefore the CPU&Disk&RAM resources that are used to write those logs will be for absolutely nothing - you'll be spending money on electricity for absolutely nothing instead of helping those projects.
  2. Then there is the issue that if you're doing what you're doing then your PC is permanently connected to the Internet with at least 1 open port for incoming external connections.
    As you plan to not check at all what's going on with it, your lonely PC will definitely be hijacked by some cracker who will then slave it and use it to e.g. access your other PCs/router from inside your LAN and/or join the club of buggers that try to get into my own server or do other nasty things.
There are other reasons for you to make this work even if we forget security and efficiency (e.g. you might have the same problem in the future, maybe your CPU/GPU projects are computing only hot air, maybe you'll run later into some different problem and to diagnose it you'll need to have to look at these files, etc...).


Summed up:
the files syslog and kern.log are extremely important.
If the PC writes a lot of stuff into them your PC is basically shouting "help! help!" and if you don't check & fix the problem your PC will die <gloomy music in the background>.
 
Old 03-18-2015, 05:24 PM   #11
Pearlseattle
Senior Member
 
Registered: Aug 2007
Location: Zurich, Switzerland
Distribution: Gentoo
Posts: 1,001

Rep: Reputation: 142Reputation: 142
Duplicate post with this one.
I don't think that it's fair to have 2 such posts running in parallel.
 
Old 03-18-2015, 05:49 PM   #12
Pearlseattle
Senior Member
 
Registered: Aug 2007
Location: Zurich, Switzerland
Distribution: Gentoo
Posts: 1,001

Rep: Reputation: 142Reputation: 142
@MBA Whore
I read your other threads and I'm not thrilled.
Are you aware of the effort of the persons that write in these forums?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Iptables prevent downloading very large files sanjibgupta Linux - Security 1 11-27-2014 01:01 PM
[SOLVED] Filter pam_rhosts_auth messages to prevent the logs filling up goodgame Linux - Security 2 03-09-2010 03:25 AM
bacula - backup growing to large neocontrol Linux - Software 6 10-14-2008 03:53 PM
mgetty.log.9600 growing to large, how to manage? dgr Linux - Newbie 3 01-16-2008 02:25 PM
Snort database growing large; how to use mysql to delete? logicalfuzz Linux - Software 1 01-12-2006 06:14 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:37 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration