Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have a simple task, and I thought of giving a try at dd.
I want to skip first 10 blocks of file, and then read 10 blocks.
dd manual says that , skip=BLOCKS
skip BLOCKS ibs-sized blocks at start of input
but since I am using bs [which is same as fs block size for now] it overrides ibs-size
so ideally it means skip bs in number of blocks
Just tried with this config --
created a file which has 10 at first line, 11 at second and likewise, 99 at end.
Code:
linuxbox# dd if=num bs=16 count=2 ---[1]
10
11
12
13
14
15
16
17
18
19
20
2+0 records in
2+0 records out
32 bytes (32 B) copied, 8.5812e-05 s, 373 kB/s
linuxbox# dd if=num bs=16 count=1 skip=1 ---[2] -- this looks ok
5
16
17
18
19
20
1+0 records in
1+0 records out
16 bytes (16 B) copied, 6.891e-05 s, 232 kB/s
linuxbox#dd if=num bs=16 count=1 skip=2 ---[3] bad
21
22
23
24
25
1+0 records in
1+0 records out
16 bytes (16 B) copied, 6.797e-05 s, 235 kB/s
linuxbox#
with [3] I am not sure at the first line there is no character displayed? or is it the newline that is displayed, since the start of offset, is at that point say for 12 : 1, 2, newline?
I am not sure dd is helping me with this problem, how do I read data in files at particular offset/ w.o reading the whole file? ie. skipping blocks.
Be very careful with dd. It doesn't protect you from your own mistakes in any way, and you can easily hose a filesystem with it if you use the wrong output.
Assuming the standard blocksize=512, how about something simple like this:
Code:
head -c 20b file | tail -c 10b
If the block size is different, you'll have to calculate the exact byte size to use, naturally.
Each line of text in your file contains 3 bytes. When you read blocks of 16 bytes and skip the first two blocks, that works out to 10 complete lines (30 bytes) and the two digits of the next line. The next byte is the newline character at the end of that line, and that will be the first byte in the output. You seemed to understand what was happening when you skipped 16 bytes (1 block), so why would the result of skipping 32 bytes be such a mystery?
This wont be exact. [Now I really feel, that I should have posted the question with normal test file, and not lines of data in file].
my requirement is to read data from an offset with count. so I want something that skips the initial addresses, and read the data from given offset.
Head /tail will read these 20lines, but outputs the 10 which are required, but I dont want to read the extra thing. just the count after an offset.
rknichols,
I was not sure of the fact that newline was encountered correctly.
Thanks, your answer confirms it, and I could possibly have a solution to my problem now, but I need to confirm that, whether dd suppresses the o/p [read whole file but o/p only the required set, asked]. if it just skips the initial range that would be it.
Is your offset a byte count, or is it a line count? You won't get anywhere until you get that straight. If you are dealing with a known byte count, then dd can do the job just fine. If you need to count lines which can be variable length, then there is no choice except to read the file from the beginning and count the newline characters. Text processing tools like head, tail, sed, etc. can do that for you, but dd cannot.
Whenever possible, dd will do a skip by using a seek() call to skip over the initial data without reading it. If dd is reading from a non-seekable stream (such as a pipe), then it has no choice but to read and discard that data. Unless you are skipping over quite a bit of data, there is really little difference. The common file system blocksize is 4 kilobytes, so if the seek is less than that the OS needs to read the whole block anyway.
Yeah I have an idea of full block read in case of the request is less than that width. Thanks you again for the valuable info provided.
I have a 64k fs bs, and the dd looks like working fetching the required data when the requests are block aligned.
Also I am trying it over nfs , but in this case the rsize I have is only 32k, that creates couple of requests for block read. But I think I am getting the desired o/p now. Thanks.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.