LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-13-2012, 10:35 AM   #1
prasadic
LQ Newbie
 
Registered: Apr 2008
Posts: 5

Rep: Reputation: 0
Post dd skip problem


I have a simple task, and I thought of giving a try at dd.
I want to skip first 10 blocks of file, and then read 10 blocks.

dd manual says that , skip=BLOCKS
skip BLOCKS ibs-sized blocks at start of input

but since I am using bs [which is same as fs block size for now] it overrides ibs-size

so ideally it means skip bs in number of blocks

Just tried with this config --

created a file which has 10 at first line, 11 at second and likewise, 99 at end.
Code:
linuxbox# dd if=num bs=16 count=2    ---[1] 
10
11
12
13
14
15
16
17
18
19
20
2+0 records in
2+0 records out
32 bytes (32 B) copied, 8.5812e-05 s, 373 kB/s
linuxbox# dd if=num bs=16 count=1 skip=1     ---[2] -- this looks ok
5
16
17
18
19
20
1+0 records in
1+0 records out
16 bytes (16 B) copied, 6.891e-05 s, 232 kB/s
linuxbox#dd if=num bs=16 count=1 skip=2      ---[3] bad

21
22
23
24
25
1+0 records in
1+0 records out
16 bytes (16 B) copied, 6.797e-05 s, 235 kB/s
linuxbox#
with [3] I am not sure at the first line there is no character displayed? or is it the newline that is displayed, since the start of offset, is at that point say for 12 : 1, 2, newline?

I am not sure dd is helping me with this problem, how do I read data in files at particular offset/ w.o reading the whole file? ie. skipping blocks.
 
Old 05-13-2012, 11:17 AM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
I'm not on a Linux box, so I can't test.....

the "skip" refers to bytes in the file, and not to lines of data.....try doing a hexdump to see what kind of stuff comes before the actual data.
 
Old 05-13-2012, 12:20 PM   #3
prasadic
LQ Newbie
 
Registered: Apr 2008
Posts: 5

Original Poster
Rep: Reputation: 0
OK I will try with hexdump.
When I refer to lines it in turn actually refers to bytes. Just to make it simpler I used lines of data
 
Old 05-13-2012, 12:37 PM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
Be very careful with dd. It doesn't protect you from your own mistakes in any way, and you can easily hose a filesystem with it if you use the wrong output.

Assuming the standard blocksize=512, how about something simple like this:

Code:
head -c 20b file | tail -c 10b
If the block size is different, you'll have to calculate the exact byte size to use, naturally.
 
Old 05-13-2012, 12:50 PM   #5
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,776

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
Each line of text in your file contains 3 bytes. When you read blocks of 16 bytes and skip the first two blocks, that works out to 10 complete lines (30 bytes) and the two digits of the next line. The next byte is the newline character at the end of that line, and that will be the first byte in the output. You seemed to understand what was happening when you skipped 16 bytes (1 block), so why would the result of skipping 32 bytes be such a mystery?
 
Old 05-13-2012, 12:55 PM   #6
prasadic
LQ Newbie
 
Registered: Apr 2008
Posts: 5

Original Poster
Rep: Reputation: 0
This wont be exact. [Now I really feel, that I should have posted the question with normal test file, and not lines of data in file].
my requirement is to read data from an offset with count. so I want something that skips the initial addresses, and read the data from given offset.
Head /tail will read these 20lines, but outputs the 10 which are required, but I dont want to read the extra thing. just the count after an offset.
 
Old 05-13-2012, 01:00 PM   #7
prasadic
LQ Newbie
 
Registered: Apr 2008
Posts: 5

Original Poster
Rep: Reputation: 0
rknichols,
I was not sure of the fact that newline was encountered correctly.
Thanks, your answer confirms it, and I could possibly have a solution to my problem now, but I need to confirm that, whether dd suppresses the o/p [read whole file but o/p only the required set, asked]. if it just skips the initial range that would be it.
 
Old 05-13-2012, 01:03 PM   #8
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,776

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
Is your offset a byte count, or is it a line count? You won't get anywhere until you get that straight. If you are dealing with a known byte count, then dd can do the job just fine. If you need to count lines which can be variable length, then there is no choice except to read the file from the beginning and count the newline characters. Text processing tools like head, tail, sed, etc. can do that for you, but dd cannot.
 
Old 05-13-2012, 01:10 PM   #9
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,776

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
Whenever possible, dd will do a skip by using a seek() call to skip over the initial data without reading it. If dd is reading from a non-seekable stream (such as a pipe), then it has no choice but to read and discard that data. Unless you are skipping over quite a bit of data, there is really little difference. The common file system blocksize is 4 kilobytes, so if the seek is less than that the OS needs to read the whole block anyway.
 
Old 05-13-2012, 02:07 PM   #10
prasadic
LQ Newbie
 
Registered: Apr 2008
Posts: 5

Original Poster
Rep: Reputation: 0
Yeah I have an idea of full block read in case of the request is less than that width. Thanks you again for the valuable info provided.
I have a 64k fs bs, and the dd looks like working fetching the required data when the requests are block aligned.
Also I am trying it over nfs , but in this case the rsize I have is only 32k, that creates couple of requests for block read. But I think I am getting the desired o/p now. Thanks.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] problem adding ssh keys to skip password prompt vikas027 Linux - Software 27 09-26-2008 03:24 PM
Install problem cd wont skip ? greatwhitepoet Linux - Software 2 08-21-2005 05:37 AM
how do i skip e2fsck? murrowman789 Slackware 8 07-20-2005 05:25 PM
Transcode DVD soundtrack rip skip problem fr8_liner Linux - Software 0 11-08-2004 05:32 PM
did i skip ahead? American Psycho Linux From Scratch 0 04-21-2004 12:28 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration