LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-30-2015, 04:20 AM   #1
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Rep: Reputation: 15
Command required Linux .


Dear all ,

We are using Centos as OS at specific location high no of files getting download in night we need to stop back dated or old files .

To identified old date or time stamp i am using below script .

find /home/tplinux/tpadminhost/reqHist_ejxml -type f -mtime +30 -print | xargs -I {} mv {} /home/tplinux/tpadminhost/reqHist_ejxml/reqold

but above is not use full as old files coming with current time step so i need to move files with below logic.

File name as below .

file 1) hid_160_9_176_8.ejmedia.2014062700.1435434441.1435434441.xml

need to above red color as validation and then move those files only .

again explaining .

On daily base high not of files coming where time stamp is current data but files coming from source which has old data as mark in above red , we need to identified and print and then if correct output then move to other location need your input on above .

Few more sample files .

with 2015 and 2014 both date (2015062900 & 2014062900)

hid_160_9_176_8.ejerror.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejheader.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejheader.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejheader.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejitem.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejitem.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejitem.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejitemizer.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejitemizer.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejitemizer.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejitempromo.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejitempromo.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejitempromo.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejlogging.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejlogging.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejlogging.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejmedia.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejmedia.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejmedia.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejmngrcmd.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejmngrcmd.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejmngrcmd.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejpromo.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejpromo.2014062800.1435521161.1435521161.xml
hid_160_9_176_8.ejpromo.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejstorerecall.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejstorerecall.2014062800.1435521161.1435521161.xml
hid_160_9_176_8.ejstorerecall.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejsubtotal.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejsubtotal.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejsubtotal.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejsupint.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejsupint.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejsupint.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejtrailer.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejtrailer.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejtrailer.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejuserstruct1.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejuserstruct1.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejuserstruct1.2015062900.1435606532.1435606532.xml
hid_160_9_176_8.ejuserstruct2.2015062700.1435434441.1435434441.xml
hid_160_9_176_8.ejuserstruct2.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejuserstruct2.2015062900.1435606532.1435606532.xml

Regards
dhiren shah
 
Old 06-30-2015, 05:07 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,836

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
here you can see how can you move files with awk: http://awk.info/?doc/tip/moving.html
and you need something like this to decide:
Code:
awk -F. ' { # $3 is the date here, you can use it to construct the destination dir 
 ....
} '
 
Old 06-30-2015, 06:20 AM   #3
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
not understand .
below command i know .


ls -rtl | awk '{print $9}'

output below .

hid_10_130_54_8.ejlogging.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejitempromo.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejitemizer.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejitem.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejheader.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejerror.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejdataent.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejcoupon.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejbank.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ej.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejuserstruct2.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejuserstruct1.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejtrailer.2015062900.1435608045.1435608045.xml
hid_10_130_54_8.ejsupint.2015062900.1435608045.1435608045.xml

but how to identified second . 2015062900 or 2014052000 ?

how can find old date files from above command .
 
Old 06-30-2015, 06:22 AM   #4
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,836

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
that's what I told you:
ls -1tr | awk -F. '{ print $3 }'
probably works
 
Old 06-30-2015, 06:43 AM   #5
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
Thanks Pan it is working....


Need one more understanding as we need to move files which date is more than x days .

above command output coming .

below .

2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900
2015062900


we need to move 2014 files so how to right .

today's date - 20150629 > 10 then files need to move or some other logic , please guide on same as well .
 
Old 06-30-2015, 06:46 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,836

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
that was the link I gave you. You just need to implement your conditions. You can do something like this:
if ( $3 > "some date" ) { print ("mv s, %s", $0, "some dir") | "sh" }
(not tested)
 
Old 06-30-2015, 07:10 AM   #7
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
can you please help me for sequance of command .

path:

/home/tplinux/tpadminhost/reqHist_ejxml/reqold

Command :

Step -1

ls -1tr | awk -F. '{ print $3 }'

then what next command .

if ( $3 > "some date" ) { print ("mv s, s", $0, "some dir") | "sh" }

Or help me with sequence of command or script as i am not getting .
 
Old 06-30-2015, 07:17 AM   #8
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Posts: 952

Rep: Reputation: 217Reputation: 217Reputation: 217
Dhiren,

pan64 has given you a good clue for inner comparison BUT you have to read carefully AND ALSO put effort to write full code for 2 comparisons ($3 within $9)

As I understand, ninth field (as per post #3) gives the file name from which you have to compare on third field which has actual creation date.

So if the date to be selected is 20140627, then I might go
ls -rtl | awk '{printf("s^%s\n", $9, $0}' | awk -F. '{if($3=="2014062700") print $0}' | awk -F^ '{printf("mv %s dest/dir/path\n", $2}' >> mover.sh
Explanation:
First awk prints ninth field of original file (as you have done) PLUS writes a carat character ^ PLUS original file name.
Second awk uses selects only those matching required date.
Third awk removes column 1 (which contains selection information only).

Now mover.sh should have a series of mv commands which you can run with sh command. Also till you are sure of the correct output you need not run mover.sh

I have to test it out but I think it will work.

You also think and try and test.

OK

Last edited by AnanthaP; 06-30-2015 at 07:34 AM.
 
Old 06-30-2015, 08:01 AM   #9
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
Dear All ,

Thanks for quick update but seem i am not able to explain my issue properly .

Below command : ls -rtl | awk '{printf("s\n", $9, $0)}' | awk -F. '{if($3=="2015062900") print $0}'

then move is not correct exception .

my issue is that i am receiving old files which not specific date it will be any date which is name format will be like below

i need to identified old files with name of the files and move to them from huge list of files .

as sample below .

hid_160_9_176_8.ejmedia.2014062700.1435434441.1435434441.xml

hid_160_9_176_8.ejerror.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejheader.2015062700.1435434441.1435434441.xml

above three files first file hid_160_9_176_8.ejmedia then Date 2014062700 which is old date which will be any date not fix those files need to identified with today date - 10 days and move to other location that is the exception of the out put .

hope i am able to communicate my issue properly still gap let me know your understanding .

Please remember back date will be change it should be 2014 or 2013 or any month and any date those files need to move base on file name .
 
Old 06-30-2015, 09:37 AM   #10
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,634

Rep: Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965
Quote:
Originally Posted by dhirendrs View Post
Dear All ,
Thanks for quick update but seem i am not able to explain my issue properly .

Below command : ls -rtl | awk '{printf("s\n", $9, $0)}' | awk -F. '{if($3=="2015062900") print $0}'

then move is not correct exception . my issue is that i am receiving old files which not specific date it will be any date which is name format will be like below i need to identified old files with name of the files and move to them from huge list of files . as sample below .

hid_160_9_176_8.ejmedia.2014062700.1435434441.1435434441.xml
hid_160_9_176_8.ejerror.2015062800.1435521161.1435521161.xml
hid_160_9_176_8.ejheader.2015062700.1435434441.1435434441.xml

above three files first file hid_160_9_176_8.ejmedia then Date 2014062700 which is old date which will be any date not fix those files need to identified with today date - 10 days and move to other location that is the exception of the out put .

hope i am able to communicate my issue properly still gap let me know your understanding .
Please remember back date will be change it should be 2014 or 2013 or any month and any date those files need to move base on file name .
We understand your issue fine...but what I can't understand is why you cannot take the examples and information you've been handed, and actually try something of your own, rather than asking us to fix it for you.

Can you show us what YOU have done/tried to get this working?
 
Old 06-30-2015, 09:44 AM   #11
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,862
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
@OP:
Off: I think you should make some effort on your own
On: Here is a script that might give you some ideas
Code:
#!/bin/sh

LIMIT00=$(date -d '111 days ago' +'Y%m%d00')

for i in hid*; do
    printf '%s\n' "$i" | (
	IFS=. read UNUSED UNUSED DATUM00 UNUSED
	if [ "$DATUM00" -lt "$LIMIT00" ]; then
	    printf 'Wow %s *is* old\n' "$i"
	else
	    printf 'Wow %s is not too old\n' "$i"
	fi
    )
done
 
1 members found this post helpful.
Old 07-02-2015, 01:23 AM   #12
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
Dear Thanks for guild line.

but i have tested above script not giving current out put .

Wow hid_160_8_128_2.ejmngrcmd.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejmngrcmd.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejpromo.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejpromo.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejpromo.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejstorerecall.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejstorerecall.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejstorerecall.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejsubtotal.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejsubtotal.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejsubtotal.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejsupint.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejsupint.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejsupint.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejtrailer.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejtrailer.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejtrailer.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejuserstruct1.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejuserstruct1.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejuserstruct1.2015070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejuserstruct2.2014070100.1435779263.1435779263.xml is not too old
Wow hid_160_8_128_2.ejuserstruct2.2015062900.1435606413.1435606413.xml is not too old
Wow hid_160_8_128_2.ejuserstruct2.2015063000.1435692730.1435692730.xml is not too old
Wow hid_160_8_128_2.ejuserstruct2.2015070100.1435779263.1435779263.xml is not too old

Above mark in red is old files 2014070100 but out put showing is not too old .
 
Old 07-02-2015, 01:51 AM   #13
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,836

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
as it was already mentioned:
1. Here is a script that might give you some ideas
2. actually try something of your own, rather than asking us to fix it for you

and now add set -xv at the beginning of your script and you will see what's happening (that will not solve anything, just print a lot about how it works)
 
2 members found this post helpful.
Old 07-02-2015, 02:26 AM   #14
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,862
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
Other than that, you can add printf's to debug eg:

Code:
	IFS=. read UNUSED UNUSED DATUM00 UNUSED
        printf 'Debug: compare filepart="%s" with limit00="%s"\n' "$DATUM00" "$LIMIT00"
	if [ "$DATUM00" -lt "$LIMIT00" ]; then
 
Old 07-03-2015, 07:28 AM   #15
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
Thanks for your valuable input ,

I have debug with sh -xv abc.sh and found below .

i have update script as per below .

Debug .

#!/bin/sh

LIMIT00=2014010100

for i in hid*; do
printf 's\n' "$i" | (
IFS=. read UNUSED UNUSED DATUM00 UNUSED

echo $LIMIT00
echo $DATUM00

if [ "$DATUM00" -lt "$LIMIT00" ]; then
printf 'Wow %s *is* old\n' "$i"
else
printf 'Wow %s is not too old\n' "$i"
fi
)
done




----------------------------------

Out put showing wrong .

2014010100
2015063000
Wow hid_160_8_128_2.ejuserstruct1.2015063000.1435692730.1435692730.xml is not too old
2014010100
2015070100
Wow hid_160_8_128_2.ejuserstruct1.2015070100.1435779263.1435779263.xml is not too old
2014010100
2015070200
Wow hid_160_8_128_2.ejuserstruct1.2015070200.1435865525.1435865525.xml is not too old
2014010100
2014070100
Wow hid_160_8_128_2.ejuserstruct2.2014070100.1435779263.1435779263.xml is not too old
2014010100
2015063000
Wow hid_160_8_128_2.ejuserstruct2.2015063000.1435692730.1435692730.xml is not too old
2014010100
2015070100
Wow hid_160_8_128_2.ejuserstruct2.2015070100.1435779263.1435779263.xml is not too old
2014010100
2015070200
Wow hid_160_8_128_2.ejuserstruct2.2015070200.1435865525.1435865525.xml is not too old



2014010100 < 2014070100

As per script it should be out put " Wow %s *is* old\n'" but it is giving wrong so please help on above .
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Required help in cut command LittleMaster Linux - Server 9 11-23-2012 01:13 AM
Command/Script required to send an email alert in case command will not respond rajaniyer123 Linux - General 1 05-19-2012 01:12 PM
help required with the tar command bsaheb Linux - Newbie 3 11-13-2009 08:47 AM
Help required for sed command. pinga123 Linux - Newbie 1 11-06-2009 12:45 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:10 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration