LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-23-2008, 02:10 PM   #1
visitnag
Member
 
Registered: Mar 2008
Posts: 147

Rep: Reputation: 15
awk program..using index()


Dear all,

I have a file of having 200,000 lines. I wanted to do some extraction from this file (suppose policy no, name, age, sex etc.,) using awk, but it is taking lot of time. When I have done the same thing using cobol it gave quick result. Is indexing is possible in Awk? Kindly lemmi know and tell me how to use indexing in awk with a sample program. Thank you alot.

bye
 
Old 08-23-2008, 02:49 PM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
awk uses flat files, there's no index. What does your
script look like, how long is your "lot of time"?

Maybe we can help you find a better way of searching
if we knew the approximate data format and the code
you're using with it.


Cheers,
Tink
 
Old 08-24-2008, 12:20 PM   #3
visitnag
Member
 
Registered: Mar 2008
Posts: 147

Original Poster
Rep: Reputation: 15
hello tink
Thank you for your reply. can you throw some idea how to find and delete big (waste) files in RHEL4. I used the command ls -ltRS which lists out all the files in a directory and subdirectories recursivly and size order. But the problem is how to delete the big files with a single shell script. Is there any command to find the files along with their path?
 
Old 08-24-2008, 12:23 PM   #4
visitnag
Member
 
Registered: Mar 2008
Posts: 147

Original Poster
Rep: Reputation: 15
find files and delete...

hi all
can anybody throw some idea how to find and delete big (waste) files in RHEL4. I used the command ls -ltRS which lists out all the files in a directory and subdirectories recursivly and size order. But the problem is how to delete the big files with a single shell script. Is there any command to find the files along with their path? Ofcourse i will select the waste files one by one and delete them with a single stroke.
 
Old 08-24-2008, 12:39 PM   #5
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
If you have KDE installed, try using the disk space view in konqueror. This will scan all sub-directories of some directory (e.g. your HOME directory) and show you where the space is going.

A similar tool under Gnome is the Gnome Disk Analyser.

See here for more details on both of them.

The core utils commandline tool to use for getting this sort of information is du. You need to experiment with it to understand how it works though. The -s option is useful. See the manual page for more information on that tool.
 
Old 08-24-2008, 12:44 PM   #6
allez
Member
 
Registered: Jul 2008
Location: Russia/Siberia/Krasnoyarsk
Distribution: SuSE, CentOS, FreeBSD
Posts: 77

Rep: Reputation: 21
Quote:
But the problem is how to delete the big files with a single shell script. Is there any command to find the files along with their path?
You should read man find.

This example finds files with size over 1 gigabyte and deletes them.
Code:
find /where/to/search -type f -size +1G -exec rm -f {} \;

Last edited by allez; 08-24-2008 at 12:46 PM.
 
Old 08-24-2008, 01:36 PM   #7
b0uncer
LQ Guru
 
Registered: Aug 2003
Distribution: CentOS, OS X
Posts: 5,131

Rep: Reputation: Disabled
Or if you already can list all the files using ls or some other means (provided that the output of that command is "suitable"), just use xargs to perform rm on each of the files (note, this is pseudo-code):

Code:
command-that-provides-filelist-to-be-deleted |xargs rm
That way they should get deleted too. Though I'd probably use find (as suggested above by allez), but it's your choice Maybe you should add the interactive switch to rm too, so that it asks on each file if it should be removed, or then you should maybe run the command without rm first (to see what's on the to-be-removed list), because it's nasty if you accidentally delete some important files..well, you have backups, don't you?
 
Old 08-24-2008, 02:06 PM   #8
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
find / -size +10M -type f -exec rm {} \;
Modify the size part to your liking.

Beware of potential ill effects, e.g. deleting
a large log-file while a process is still writing
to it. The file will "disappear", but the space
it occupies won't be freed (and it will in fact
keep growing as it's a log) until the process
is stopped.



Cheers,
Tink
 
Old 08-24-2008, 03:15 PM   #9
rjlee
Senior Member
 
Registered: Jul 2004
Distribution: Ubuntu 7.04
Posts: 1,990

Rep: Reputation: 69
Quote:
Originally Posted by b0uncer View Post
Or if you already can list all the files using ls or some other means (provided that the output of that command is "suitable"), just use xargs to perform rm on each of the files (note, this is pseudo-code):

[code]command-that-provides-filelist-to-be-deleted |xargs rm[/code
You would be better to use:
Code:
xargs -0rm
that's the number 0, not the letter O.

You would then have to separate the files with the ASCII-null character rather than whitespace, but you can then delete files with spaces in the name.
 
Old 08-24-2008, 04:54 PM   #10
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Took the liberty to merge the two related threads ...

Please keep your topics in one space rather than diverting
efforts of people into several threads. That's as bad as
double-posting ....
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
awk program. a little query! indiancosmonaut Programming 7 07-05-2007 10:48 AM
How add input value to some varieble in awk program this is probably funny for some of you but I do not know what to do. I have sarajevo Programming 2 05-25-2007 08:54 AM
Counting number of occurance in awk program sarajevo Programming 2 11-01-2006 11:31 AM
How to run a shell command containing awk, and grep within a C program Linh Programming 3 06-05-2003 08:05 PM
Running a shell command containing awk and grep within a C program Linh Programming 1 06-05-2003 07:51 PM


All times are GMT -5. The time now is 10:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration