Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
It should do exactly what you're trying to do. The difference could be in the units of bs. K means 1024 bytes, whereas kB means exactly 1000 bytes. Take a look at the man page of dd:
Quote:
BLOCKS and BYTES may be followed by the following multiplicative suffixes: c =1, w =2, b =512, kB=1000, K =1024, MB=1000*1000, M =1024*1024, xM =M GB =1000*1000*1000, G =1024*1024*1024, and so on for T, P, E, Z, Y.
It should do exactly what you're trying to do. The difference could be in the units of bs. K means 1024 bytes, whereas kB means exactly 1000 bytes. Take a look at the man page of dd:
thanks for your reply.
There is a thing i don't understand. Every time i execute "tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=QWER",
the size of QWER is not constant. sometimes it is 7xKB, sometimes it is 8xxKB, 7xxKB...
There is a thing i don't understand. Every time i execute "tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=QWER",
the size of QWER is not constant. sometimes it is 7xKB, sometimes it is 8xxKB, 7xxKB...
Actually your command should generate a file of exactly 1048576 bytes. Where do you see 7xKB or 8xxKB. Please, show us your command and the long listing of the newly created file, e.g.
Made some headway. I understand what, but not why.....
Code:
gazl@slack:/tmp$ dd if=/dev/zero bs=1M count=1 of=/tmp/out1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.000897899 s, 1.2 GB/s
gazl@slack:/tmp$ cat /dev/zero | dd bs=1M count=1 of=/tmp/out2
0+1 records in
0+1 records out
98304 bytes (98 kB) copied, 0.00011617 s, 846 MB/s
gazl@slack:/tmp$ ls -l out*
-rw-r--r-- 1 gazl users 1048576 Nov 15 14:24 out1
-rw-r--r-- 1 gazl users 98304 Nov 15 14:24 out2
But count=2 gives us a clue
Code:
gazl@slack:/tmp$ cat /dev/zero | dd bs=1M count=2 of=/tmp/out3
dd: warning: partial read (98304 bytes); suggest iflag=fullblock
0+2 records in
0+2 records out
196608 bytes (197 kB) copied, 0.000294146 s, 668 MB/s
Why we're not seeing that warning when count=1 I don't know but from experimentation it seems that when using piped input dd needs iflag=fullblock for any blocksize over 98304
Code:
gazl@slack:/tmp$ cat /dev/zero | dd iflag=fullblock bs=1M count=1 of=/tmp/out4
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00135746 s, 772 MB/s
gazl@slack:/tmp$ ls -l out*
-rw-r--r-- 1 gazl users 1048576 Nov 15 14:24 out1
-rw-r--r-- 1 gazl users 98304 Nov 15 14:24 out2
-rw-r--r-- 1 gazl users 196608 Nov 15 14:25 out3
-rw-r--r-- 1 gazl users 1048576 Nov 15 14:27 out4
gazl@slack:/tmp$
Something feels broken here, but I really don't know what is going on (my system is heavily messed around with so it's quite possible I broke something myself)
Thanks GazL. I cannot reproduce this behaviour on my current system (CentOS 6 2.6.32-71.29.1.el6.i686) since it gives me always the same and correct result. Anyway, the fullblock workaround is fine: maybe it prevents some latency in the I/O stream.
I'd certainly consider fullblock nothing more than a workaround as it shouldn't be necessary and it suggests to me that something is broken.
I see this with both coreutils 8.11 and 8.12, with glibc-2.13 and kernel 3.1.1. Slackware64 13.37 (heavily messed with)
Problem is my system is a bit of a plaything: I have updated the kernel, headers, glibc and coreutils locally so the scope for having broken something myself is quite wide and it's going to make identifying the cause somewhat difficult.
The "count=n" operand doesn't mean "nfull blocks". When 'dd' performs a read from a pipe, the amount of data returned by the read cannot exceed the amount of data was buffered in the pipe at that moment. If you haven't told dd to do something different (e.g., with the "fullblock" iflag), it's going to treat whatever it's got as a block. You'll get similar results from communications lines, tape drives with variable block size, ..., any device providing a data stream where all of the data you requested might not be immediately available.
That explanation makes sense, but why does colucix older dd work? Is this a change of behaviour in more recent versions of dd?
Also, if what you say is the case then the 'partial read' warning ought to be displayed for any value of count=n on receipt of an incomplete block, which wasn't happening on a count=1.That has to be considered a bug
Also, if what you say is the case then the 'partial read' warning ought to be displayed for any value of count=n on receipt of an incomplete block, which wasn't happening on a count=1.That has to be considered a bug
The problem is that a short read is normal if there is no more data to come, and dd cannot tell whether or not that is the case without going back and trying another read, which "count=1" (in the absence of "iflag=fullblock") has specifically told it not to do.
FWIW, the amount of data returned by that single read from the pipe appears to depend on the glibc version. On a system with glibc-2.11 I consistently see 8192 bytes. On systems with glibc-2.12 (Scientific Linux 6) and glibc-2.14 (Fedora 16) I see a variable amount. What's odd is that SL-6 should be the same as CentOS 6, which colucix reports as consistently yielding the full 1-megabyte block.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.