display one line from multiple files

christinek · 12-09-2011, 01:42 PM

I have 500 files that I have to look through but only interested in 20th line. Is there any way for me to view the 20th line of all of the 500 files at the same time?

I've been using
head -n 20 filename | tail -n 1
but then I have to do this 500 times...

Thank you for the help!

kbscores · 12-09-2011, 01:45 PM

If they are all in same directory you could do:

Code:

for i in `ls -1 /directory`
do
head -n 20 $i| tail -n 1
done

druuna · 12-09-2011, 01:49 PM

Hi,

Are all the files in the same directory?

If so:

Code:

for THISFILE in `ls`; do sed -n '21q;20p' $THISFILE; done

If you also want/need the file to be printed:

Code:

for THISFILE in `ls`; do echo -n "$THISFILE : " ; sed -n '21q;20p' $THISFILE; done

Hope this helps.

colucix · 12-09-2011, 02:36 PM

Using awk:

Code:

awk 'FNR==20' *

To print the filename use an explicit print statement. Moreover, if the files are huge in size, you may want to speed up the whole process by adding nextfile:

Code:

awk 'FNR==20{print FILENAME ":", $0; nextfile}' *

Telengard · 12-09-2011, 02:42 PM

I started with nine files.

Code:

test$ ls
file_1  file_2  file_3  file_4  file_5  file_6  file_7  file_8  file_9
test$

Each file is formatted as shown.

Code:

test$ cat file_1
file_1 line_1
file_1 line_2
file_1 line_3
file_1 line_4
file_1 line_5
file_1 line_6
file_1 line_7
file_1 line_8
file_1 line_9
file_1 line_10
file_1 line_11
file_1 line_12
file_1 line_13
file_1 line_14
file_1 line_15
file_1 line_16
file_1 line_17
file_1 line_18
file_1 line_19
file_1 line_20
(... continues ...)
file_1 line_50

Then I tested my script to print line 20 from each file.

Code:

test$ awk 'FNR == 20' file_*
file_1 line_20
file_2 line_20
file_3 line_20
file_4 line_20
file_5 line_20
file_6 line_20
file_7 line_20
file_8 line_20
file_9 line_20
test$

You say you're working with 500 or more files, so my pattern file_* would expand into quite a long list. I can imagine such a long command line having negative consequences. With that in mind, it may be wise to execute the script on one file at a time. This is easily automated with the find command.

Code:

test$ find -maxdepth 1 -name 'file_*' -exec awk 'NR == 20 {print FILENAME ":" $0}' {} \;
./file_9:file_9 line_20
./file_5:file_5 line_20
./file_4:file_4 line_20
./file_1:file_1 line_20
./file_2:file_2 line_20
./file_6:file_6 line_20
./file_7:file_7 line_20
./file_8:file_8 line_20
./file_3:file_3 line_20
test$

If you would prefer the list be sorted by filename, then simply pipe the output of find into sort.

Code:

test$ find -maxdepth 1 -name 'file_*' -exec awk 'NR == 20 {print FILENAME ":" $0}' {} \; | sort
./file_1:file_1 line_20
./file_2:file_2 line_20
./file_3:file_3 line_20
./file_4:file_4 line_20
./file_5:file_5 line_20
./file_6:file_6 line_20
./file_7:file_7 line_20
./file_8:file_8 line_20
./file_9:file_9 line_20
test$

If you also want to print line 20 of files from subdirectories then specify how many directories deep in the -maxdepth clause. If you want to recurse infinitely deep into all subdirectories then the -maxdepth clause may be omitted.

Code:

test$ find -name 'file_*' -exec awk 'NR == 20 {print FILENAME ":" $0}' {} \; | sort
./file_1:file_1 line_20
./file_2:file_2 line_20
./file_3:file_3 line_20
./file_4:file_4 line_20
./file_5:file_5 line_20
./file_6:file_6 line_20
./file_7:file_7 line_20
./file_8:file_8 line_20
./file_9:file_9 line_20
./subdir/file_10:file_10 line_20
test$

References

HTH

EDIT
colucix made a good point about accelerating awk by using nextfile to ignore all lines after line 20. If I incorporate that into my last example I get this.

Code:

test$ find -name 'file_*' -exec awk 'NR == 20 {print FILENAME ":" $0; nextfile}' {} \; | sort
./file_1:file_1 line_20
./file_2:file_2 line_20
./file_3:file_3 line_20
./file_4:file_4 line_20
./file_5:file_5 line_20
./file_6:file_6 line_20
./file_7:file_7 line_20
./file_8:file_8 line_20
./file_9:file_9 line_20
./subdir/file_10:file_10 line_20
test$

colucix · 12-09-2011, 03:07 PM

Quote:

Originally Posted by Telengard

You say you're working with 500 or more files, so my pattern file_* would expand into quite a long list. I can imagine such a long command line having negative consequences.

Good point! It may result in the notorious and feared "Arguments list too long" error.

Juako · 12-09-2011, 03:23 PM

The proverbial sed-based one:

Code:

find <base_dir> -type f -exec sed -n '20{p;q}' {} \;