LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-06-2017, 09:24 AM   #1
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,633

Rep: Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200
How would you extract older entries from this listing?


I am working on a script to clean stale files from a local repository.
Given a listing like:
Code:
/var/cache/packages20170115/slackware/ap/zsh-5.3.1-i586-1.txz	zsh	20170115
/var/cache/packages20170327/slackware/l/zlib-1.2.11-i586-1.txz	zlib	20170327
/var/cache/packages20170115/slackware/l/zlib-1.2.10-i586-1.txz	zlib	20170115
/var/cache/packages20170211/slackware/n/zd1211-firmware-1.5-fw-1.txz	zd1211-firmware	20170211
/var/cache/packages20170115/slackware/n/ytalk-3.3.0-i586-3.txz	ytalk	20170115
/var/cache/packages20170722/slackware/n/yptools-2.14-i586-8.txz	yptools	20170722
/var/cache/packages20170115/slackware/a/xz-5.2.3-i586-1.txz	xz	20170115
/var/cache/packages20170412/slackware/xap/xv-3.10a-i586-8.txz	xv	20170412
/var/cache/packages20170715/slackware/x/xterm-330-i586-1.txz	xterm	20170715
/var/cache/packages20170613/slackware/x/xterm-328-i586-1.txz	xterm	20170613
/var/cache/packages20170115/slackware/x/xterm-327-i586-1.txz	xterm	20170115
/var/cache/packages20161120/slackware/x/xterm-326-i586-1.txz	xterm	20161120
/var/cache/packages20170709/slackware/xap/xscreensaver-5.37-i586-1.txz	xscreensaver	20170709
/var/cache/packages20161102/slackware/xap/xscreensaver-5.36-i586-1.txz	xscreensaver	20161102
/var/cache/packages20161120/slackware/x/xproto-7.0.31-noarch-1.txz	xproto	20161120
/var/cache/packages20170818/slackware/x/xorg-server-xvfb-1.19.3-i586-2.txz	xorg-server-xvfb	20170818
/var/cache/packages20170320/slackware/x/xorg-server-xvfb-1.19.3-i586-1.txz	xorg-server-xvfb	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xvfb-1.19.2-i586-1.txz	xorg-server-xvfb	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xvfb-1.19.1-i586-2.txz	xorg-server-xvfb	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xvfb-1.19.1-i586-1.txz	xorg-server-xvfb	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xvfb-1.19.0-i586-1.txz	xorg-server-xvfb	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xvfb-1.18.4-i586-1.txz	xorg-server-xvfb	20160724
/var/cache/packages20170818/slackware/x/xorg-server-xnest-1.19.3-i586-2.txz	xorg-server-xnest	20170818
/var/cache/packages20170320/slackware/x/xorg-server-xnest-1.19.3-i586-1.txz	xorg-server-xnest	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xnest-1.19.2-i586-1.txz	xorg-server-xnest	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xnest-1.19.1-i586-2.txz	xorg-server-xnest	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xnest-1.19.1-i586-1.txz	xorg-server-xnest	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xnest-1.19.0-i586-1.txz	xorg-server-xnest	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xnest-1.18.4-i586-1.txz	xorg-server-xnest	20160724
/var/cache/packages20170818/slackware/x/xorg-server-xephyr-1.19.3-i586-2.txz	xorg-server-xephyr	20170818
/var/cache/packages20170320/slackware/x/xorg-server-xephyr-1.19.3-i586-1.txz	xorg-server-xephyr	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xephyr-1.19.2-i586-1.txz	xorg-server-xephyr	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xephyr-1.19.1-i586-2.txz	xorg-server-xephyr	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xephyr-1.19.1-i586-1.txz	xorg-server-xephyr	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xephyr-1.19.0-i586-1.txz	xorg-server-xephyr	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xephyr-1.18.4-i586-1.txz	xorg-server-xephyr	20160724
/var/cache/packages20170818/slackware/x/xorg-server-1.19.3-i586-2.txz	xorg-server	20170818
/var/cache/packages20170320/slackware/x/xorg-server-1.19.3-i586-1.txz	xorg-server	20170320
/var/cache/packages20170311/slackware/x/xorg-server-1.19.2-i586-1.txz	xorg-server	20170311
/var/cache/packages20170123/slackware/x/xorg-server-1.19.1-i586-2.txz	xorg-server	20170123
/var/cache/packages20170115/slackware/x/xorg-server-1.19.1-i586-1.txz	xorg-server	20170115
/var/cache/packages20161120/slackware/x/xorg-server-1.19.0-i586-1.txz	xorg-server	20161120
/var/cache/packages20160724/slackware/x/xorg-server-1.18.4-i586-1.txz	xorg-server	20160724
/var/cache/packages20170802/slackware/xap/xlockmore-5.54-i586-2.txz	xlockmore	20170802
/var/cache/packages20170722/slackware/xap/xlockmore-5.54-i586-1.txz	xlockmore	20170722
/var/cache/packages20170412/slackware/xap/xlockmore-5.51-i586-2.txz	xlockmore	20170412
/var/cache/packages20170115/slackware/xap/xlockmore-5.50-i586-1.txz	xlockmore	20170115
/var/cache/packages20170613/slackware/x/xkeyboard-config-2.21-noarch-1.txz	xkeyboard-config	20170613
/var/cache/packages20160724/slackware/x/xkeyboard-config-2.18-noarch-1.txz	xkeyboard-config	20160724
/var/cache/packages20170514/slackware/x/xkbcomp-1.4.0-i586-1.txz	xkbcomp	20170514
/var/cache/packages20170115/slackware/xap/xine-ui-0.99.9-i686-2.txz	xine-ui	20170115
/var/cache/packages20170802/slackware/xap/xine-lib-1.2.8-i586-3.txz	xine-lib	20170802
/var/cache/packages20170722/slackware/xap/xine-lib-1.2.8-i586-2.txz	xine-lib	20170722
/var/cache/packages20170412/slackware/xap/xine-lib-1.2.8-i586-1.txz	xine-lib	20170412
/var/cache/packages20170211/slackware/xap/xine-lib-1.2.6-i586-10.txz	xine-lib	20170211
/var/cache/packages20170115/slackware/xap/xine-lib-1.2.6-i586-9.txz	xine-lib	20170115
/var/cache/packages20170320/slackware/xfce/xfwm4-4.12.4-i586-1.txz	xfwm4	20170320
/var/cache/packages20170802/slackware/a/xfsprogs-4.12.0-i586-1.txz	xfsprogs	20170802
/var/cache/packages20170626/slackware/a/xfsprogs-4.11.0-i586-1.txz	xfsprogs	20170626
/var/cache/packages20170429/slackware/a/xfsprogs-4.10.0-i586-1.txz	xfsprogs	20170429
how would you create a list of older versions of files that can be deleted?
The first field is the actual filename. (Needs to be expanded to include checksum and gpg files, with .md5 and .asc extensions respectively). The second field is the base package name.
The third field is the creation date in ISO format.

Last edited by allend; 09-06-2017 at 09:25 AM.
 
Old 09-06-2017, 09:35 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,490

Rep: Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532
I would use a sort. In perl/python/other languages you can easily construct your own comparison operator which will make it quite easy. But probably a string based sort is enough
 
Old 09-06-2017, 09:37 AM   #3
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,316
Blog Entries: 13

Rep: Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372
What do you do now?

Not trying to be too flip with my answer. My base advice for any script is "Whatever you can type in a command line, you can code into a bash script."

Hence my points here are:
  1. You didn't fully describe your qualifications as to what constitutes a stale file
  2. You did not show any actions which you currently take to remedy this manually
  3. If you currently do have a manual remediation process which you follow, then it would be good to know what you do, for some/all of the other points.
 
Old 09-06-2017, 09:37 AM   #4
hazel
LQ Guru
 
Registered: Mar 2016
Location: Harrow, UK
Distribution: LFS, AntiX, Slackware
Posts: 5,494
Blog Entries: 16

Rep: Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256Reputation: 3256
find will do it. It will find files based on all sorts of criteria, and creation date is one of them. For example, you can find all files created more than one month ago and either list them for deletion or delete them on the fly, using find's -delete option. Read the find man page for more detail.
 
Old 09-06-2017, 09:48 AM   #5
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,633

Original Poster
Rep: Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200
My bad
What I am asking is how would you extract a listing like:
Code:
/var/cache/packages20170115/slackware/l/zlib-1.2.10-i586-1.txz	zlib	20170115
/var/cache/packages20170613/slackware/x/xterm-328-i586-1.txz	xterm	20170613
/var/cache/packages20170115/slackware/x/xterm-327-i586-1.txz	xterm	20170115
/var/cache/packages20161120/slackware/x/xterm-326-i586-1.txz	xterm	20161120
/var/cache/packages20161102/slackware/xap/xscreensaver-5.36-i586-1.txz	xscreensaver	20161102
/var/cache/packages20170320/slackware/x/xorg-server-xvfb-1.19.3-i586-1.txz	xorg-server-xvfb	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xvfb-1.19.2-i586-1.txz	xorg-server-xvfb	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xvfb-1.19.1-i586-2.txz	xorg-server-xvfb	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xvfb-1.19.1-i586-1.txz	xorg-server-xvfb	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xvfb-1.19.0-i586-1.txz	xorg-server-xvfb	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xvfb-1.18.4-i586-1.txz	xorg-server-xvfb	20160724
/var/cache/packages20170320/slackware/x/xorg-server-xnest-1.19.3-i586-1.txz	xorg-server-xnest	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xnest-1.19.2-i586-1.txz	xorg-server-xnest	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xnest-1.19.1-i586-2.txz	xorg-server-xnest	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xnest-1.19.1-i586-1.txz	xorg-server-xnest	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xnest-1.19.0-i586-1.txz	xorg-server-xnest	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xnest-1.18.4-i586-1.txz	xorg-server-xnest	20160724
/var/cache/packages20170320/slackware/x/xorg-server-xephyr-1.19.3-i586-1.txz	xorg-server-xephyr	20170320
/var/cache/packages20170311/slackware/x/xorg-server-xephyr-1.19.2-i586-1.txz	xorg-server-xephyr	20170311
/var/cache/packages20170123/slackware/x/xorg-server-xephyr-1.19.1-i586-2.txz	xorg-server-xephyr	20170123
/var/cache/packages20170115/slackware/x/xorg-server-xephyr-1.19.1-i586-1.txz	xorg-server-xephyr	20170115
/var/cache/packages20161120/slackware/x/xorg-server-xephyr-1.19.0-i586-1.txz	xorg-server-xephyr	20161120
/var/cache/packages20160724/slackware/x/xorg-server-xephyr-1.18.4-i586-1.txz	xorg-server-xephyr	20160724
/var/cache/packages20170320/slackware/x/xorg-server-1.19.3-i586-1.txz	xorg-server	20170320
/var/cache/packages20170311/slackware/x/xorg-server-1.19.2-i586-1.txz	xorg-server	20170311
/var/cache/packages20170123/slackware/x/xorg-server-1.19.1-i586-2.txz	xorg-server	20170123
/var/cache/packages20170115/slackware/x/xorg-server-1.19.1-i586-1.txz	xorg-server	20170115
/var/cache/packages20161120/slackware/x/xorg-server-1.19.0-i586-1.txz	xorg-server	20161120
/var/cache/packages20160724/slackware/x/xorg-server-1.18.4-i586-1.txz	xorg-server	20160724
/var/cache/packages20170722/slackware/xap/xlockmore-5.54-i586-1.txz	xlockmore	20170722
/var/cache/packages20170412/slackware/xap/xlockmore-5.51-i586-2.txz	xlockmore	20170412
/var/cache/packages20170115/slackware/xap/xlockmore-5.50-i586-1.txz	xlockmore	20170115
/var/cache/packages20160724/slackware/x/xkeyboard-config-2.18-noarch-1.txz	xkeyboard-config	20160724
/var/cache/packages20170722/slackware/xap/xine-lib-1.2.8-i586-2.txz	xine-lib	20170722
/var/cache/packages20170412/slackware/xap/xine-lib-1.2.8-i586-1.txz	xine-lib	20170412
/var/cache/packages20170211/slackware/xap/xine-lib-1.2.6-i586-10.txz	xine-lib	20170211
/var/cache/packages20170115/slackware/xap/xine-lib-1.2.6-i586-9.txz	xine-lib	20170115
/var/cache/packages20170626/slackware/a/xfsprogs-4.11.0-i586-1.txz	xfsprogs	20170626
/var/cache/packages20170429/slackware/a/xfsprogs-4.10.0-i586-1.txz	xfsprogs	20170429
from the listing in post #1?
 
Old 09-06-2017, 09:55 AM   #6
Sefyir
Member
 
Registered: Mar 2015
Distribution: Linux Mint
Posts: 633

Rep: Reputation: 316Reputation: 316Reputation: 316Reputation: 316
What defines stale?

Just use apt-get's autoclean?

Quote:
autoclean (and the auto-clean alias since 1.1)
Like clean, autoclean clears out the local repository of retrieved
package files. The difference is that it only removes package files
that can no longer be downloaded, and are largely usele
ss. This
allows a cache to be maintained over a long period without it
growing out of control. The configuration option
APT::Clean-Installed will prevent installed packages from being
erased if it is set to off.

Last edited by Sefyir; 09-06-2017 at 09:56 AM.
 
1 members found this post helpful.
Old 09-06-2017, 09:58 AM   #7
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,316
Blog Entries: 13

Rep: Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372
Still sorry, but is there a better way to describe this other than asking us to compare two moderately long lists from different posts? The very plain and simple up/down/up on the browser while totally ineffective, tells me that the first entries are different, entirely. I fear that if I take the time to load them up into something like emacs and use ediff to compare I may find 20-30 lines which are just ... different.
 
Old 09-06-2017, 10:10 AM   #8
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,633

Original Poster
Rep: Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200
Quote:
Hence my points here are:

You didn't fully describe your qualifications as to what constitutes a stale file
You did not show any actions which you currently take to remedy this manually
If you currently do have a manual remediation process which you follow, then it would be good to know what you do, for some/all of the other points.
1. File created before the last date
2. Manual action is what I want to remove.
3. I want to eliminate the manual cut and paste process
 
Old 09-06-2017, 01:33 PM   #9
norobro
Member
 
Registered: Feb 2006
Distribution: Debian Sid
Posts: 792

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
Perhaps I'm missing something but awk will produce the second listing from the first:
Code:
awk 'dups[$2]++' input_file
You will need to sort if the first list is not ordered by base package name then by date descending.
 
1 members found this post helpful.
Old 09-06-2017, 09:08 PM   #10
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,633

Original Poster
Rep: Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200Reputation: 2200
Thanks! Just what I needed.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LISTING entries of mysqldatabase wackysiya Linux - Newbie 2 06-22-2014 04:09 PM
sftp ls command, short listing vs long listing slufoot80 Linux - General 3 02-05-2013 10:40 AM
How can I extract the .rpm filename from long list of entries using bash shell script Glenn D. Linux - Software 2 07-14-2012 05:42 AM
how to sort output at latest entries without disturbing the previous entries record nabmufti Programming 4 02-11-2008 11:36 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:47 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration