[SOLVED] Can I delete first few lines from a running log file?

james000 · 09-29-2023, 07:18 PM

Hello,
This is RedHat 7 and I have a big log file of 12gb. Logs are constantly writing to it, so I can't rename file. This filesystem have only 4 gb space left, so I can't use logrotate also, for rotating 12gb file.
One solution I can think is, delete first half of lines (approx 32,231,354) and that would make this file size to 6gb.
Can someone suggest, how can I truncate older lines from this log file?

Code:

[root@linux-7 ~]# cat /var/log/drtmessages | wc -l
64462709
[root@linux-7 ~]#

Thanks

frankbell · 09-29-2023, 08:16 PM

I think you'll find this article quite helpful.

Do you have logrotate installed? If not, I'd look at the first.

james000 · 09-29-2023, 08:30 PM

Quote:

Originally Posted by frankbell

I think you'll find this article quite helpful.

Do you have logrotate installed? If not, I'd look at the first.

logrotate will not work, because file size is 12gb and free space in this file-system is only 4gb. Not enough space to complete operation.
I don't want to truncate whole file, but only older half.

sundialsvcs · 09-29-2023, 08:42 PM

Can you USB-attach an external disk drive to this system, to give you the space that you need? Or install another internal one, then use LVM to add it to the storage pool to give you the additional space that you need?

frankbell · 09-29-2023, 08:44 PM

Off the top of my head, I don't see any reason why you couldn't edit log file in the way you suggest, but I have no personal knowledge or experience doing so, so I hope that someone who knows more than I do joins this thread.

Sorry I can't help more.

hazel · 09-30-2023, 12:06 AM

Get yourself a SystemRescue image on CD or usb stick. Boot with it, mount your normal root partition and do your edit. Then, when you reboot your main system, install and configure logrotate.

MadeInGermany · 09-30-2023, 01:56 AM

You cannot delete the initial part of a text file without creating another file, needing extra space.
Even compressing it would temporarily need space for both uncompressed and compressed files.
A truncation from the end is possible, but not desired here.

I would try another filesystem.
Say a

Code:

df /opt /var/log

shows that /opt is a different file system and has 7G avail. Then do

Code:

sed '1,32231354 d' /var/log/drtmessages > /opt/drtmessages.tmp

Directly copying this back is not recommended: concurrent writes can cause data corruption.
Better truncate the busy file:

Code:

: > /var/log/drtmessages

and check after a pause that its size is reduced.
Then move the tmp file back, but to a new name:

Code:

mv /opt/drtmessages.tmp /var/log/drtmessages.old

And finally configure logrotate!

jayjwa · 10-01-2023, 12:24 AM

I'd just whack the whole thing. It can't be that important if you're not tending it properly. From there on out, tend it properly with logrotate, etc.

Code:

cat /dev/null > /var/log/drtmessages

boughtonp · 10-01-2023, 08:12 AM

Quote:

Originally Posted by james000

This filesystem have only 4 gb space left, so I can't use logrotate also, for rotating 12gb file.

You can use logrotate, since it can be configured to compress the old version, and can reduce log files by 80-90%, so - depending on what's in the log - could easily produce a file under 2gb.

Also logrotate defaults to same directory, but can be configured to put files elsewhere - i.e. a different disk, or perhaps to RAM - again, assuming the contents of the logs are actually needed.

Though...

Quote:

Originally Posted by james000

This is RedHat 7

Red Hat 7 is over twenty years old. You presumably mean RHEL 7, and if so why aren't you making use of the support contract you're paying for?

(Also, given RHEL 7 is EOL in under 9 months time, there should be a plan/process in place for migrating away from it...)