Fetch x lines from file

jiml8 · 07-18-2008, 09:23 AM

Quote:

Originally Posted by ghostdog74

its a mysql database dump. i doubt each line will be 3MB each. I am also curious, if mysql dump is a binary file, then how is OP going to get those lines.

OP specified it was a text file.

Does tail scan the file in place? Or does it load it into memory? A file that big...you can't load into memory.

ghostdog74 · 07-18-2008, 10:22 AM

well, with a little experiment, we can find out. Its a 300Mb++ file.

Code:

# ls -l file
-rw-r--r-- 1 root root 361380637 Jul 18 10:10 file
# wc -l file
15515859 file
# time more +28271 file > /dev/null

real    0m20.881s
user    0m20.385s
sys     0m0.312s
# time sed '1,28271 d' file > /dev/null

real    1m3.363s
user    1m2.032s
sys     0m0.436s
# time awk 'NR > 28271' file> /dev/null

real    0m12.692s
user    0m12.209s
sys     0m0.400s
# time tail +28271 file > /dev/null

real    0m0.321s
user    0m0.112s
sys     0m0.208s

jiml8 · 07-18-2008, 01:13 PM

tail is definitely looking at it in place I would say.