Hi,
I have a script which records an IP address. Every couple of hours Cron starts the job which first appends the IP and then a time stamp to a file simple text file.

A what point, number of lines, MB, etc, would it be more efficient to maybe archive this file and start a new one?

Who can tell?

You haven't described what you need the address and time stamp data for.

Why keep a file at all? How many historical address and time stamp records do you require?

If I was looking for a trend or something I'd process inline at each event and only retain data I absolutely required.

That said, a text file of a few lines takes very little space, even if you make it a few hundred lines, maybe a few kilobytes? Max IP is 15 characters. Timestamp say is no more than 30? 45 characters, times 1000 entries, is 45,000? So 45 kbytes of data?

Every couple of hours? So that's a few months of data?

1 members found this post helpful.

Is it a log of some sort? if so, how about logrotate?

1 members found this post helpful.

Thanks for the reply.

It was historical ...
Some time ago my ISP kept dropping my line and, more often than not, assigning a different IP when it came back. I bought a decent router and although that was better it did not cure the problem. Plus the ISP said the problem was caused by me not using their router.
When i complained they insisted that their router showed there was nothing wrong with my connection. It was a while before i noticed that the IP was changing but they said i didn't have any proof. They also insisted i would have to pay the full amount for what remained of the 12 month contract and that a few seconds loss of service wasn't significant enough to warrant a claim.

FYI:
The ISP in question INSIST i was never with them during this period
- even though i have bank records showing that i was sending them regular payments.
And YES, i have issued an SAR request - same "no data" reply.

First entry dated: Mar 2016
Number of lines: 45232
Size: 791.2 Kb
File has been cleared manually down in the past.

Log, effectively YES. But a "log" i've created myself.

You can rotate any file. Does not have to be a system log, or even .log

https://www.linuxquestions.org/quest...-system-36026/

Anyways it doesn't appear that this file, if described as in the first post, could get very large even after a year or more. Rotate or just delete will probably be fine.

1 members found this post helpful.

So for a year of running, take the number of days in a year, converted to hours and divided by the interval.
365 * 24 / 2 = 4380

I wouldn't worry about it. If you're still doing this in 5 years, it might hit 1Mb

1 members found this post helpful.

Is this a home connection, or business connection? Quite often, ISPs will change IP addresses on home connections to prevent people from running business services. As a home user, you shouldn't even notice it.

Assuming one line is 100 bytes, your script generates 1200 bytes a day, half a megabyte per year, five megabytes per ten years. The data generated in a decade is less than a single JPEG file from your $100 camera. Even if you multiply the line size by ten, who cares? Archive it whenever you change your computer.

In general terms, here are a few factors to consider:

How much space is left in your filesystem.
The filesystem type and its configuration. Small files fit in a single block. XFS seems to be efficient for large files (it was created by Silicon Graphics with the purpose of efficient access to media files), but perhaps ext4 has caught up.
How often you look at old data. If the answer is "practically never", you can rotate often. If you often look at years-old data, keep years-old data in that file.
Is the data valuable enough to require a backup. Perhaps that is the only factor you need to consider.

1 members found this post helpful.

I've never seen that command before, TY.
I've just thought of about a million other uses for it.
Many, many thanks
... well, maybe not quite a million, but still lots more things to play with
(i'm old, single and don't play games, but i do like to play).

What can i say?
Your reply clearly goes above and beyond.
Many, many thanks Sefyir

Home (as in old and long since retired).
However, in the past i've always been a Linux whore, so dl'ing my latest fancy has peed me off in the past (until i ignored direct dl's and always torrented.
I've also continued to use DC (orig as an easy chat when we had a SETI team) and constant resets are a pain, plus they are quite embarasing when you're the only one with a rubbish connection.

Thanks for the reply though.

Thanks berndbausch.

TBH i do tend to rarely check the file. As for backups, i'm an old mainframe guy so backups are in my blood. I've probably got a better backup system, with the exception of Faraday cage backup on a remote fallback site, than most large companies.

TBH, everyone's reply answered my query ...
Many thanks everyone.