why `tr "\000" "\125" < /dev/zero | dd bs=1K count=1 of=01data` creates wrong size?
I tried a command "tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=01data", then the size of 01data is not 1MB...
Code:
tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=01data Thanks. |
Quote:
|
It should do exactly what you're trying to do. The difference could be in the units of bs. K means 1024 bytes, whereas kB means exactly 1000 bytes. Take a look at the man page of dd:
Quote:
|
Quote:
There is a thing i don't understand. Every time i execute "tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=QWER", the size of QWER is not constant. sometimes it is 7xKB, sometimes it is 8xxKB, 7xxKB... |
Quote:
Code:
$ tr "\000" "\125" < /dev/zero | dd bs=1M count=1 of=QWER |
I'm seeing this too.
Made some headway. I understand what, but not why..... Code:
gazl@slack:/tmp$ dd if=/dev/zero bs=1M count=1 of=/tmp/out1 Code:
gazl@slack:/tmp$ cat /dev/zero | dd bs=1M count=2 of=/tmp/out3 Code:
gazl@slack:/tmp$ cat /dev/zero | dd iflag=fullblock bs=1M count=1 of=/tmp/out4 |
Thanks GazL. I cannot reproduce this behaviour on my current system (CentOS 6 2.6.32-71.29.1.el6.i686) since it gives me always the same and correct result. Anyway, the fullblock workaround is fine: maybe it prevents some latency in the I/O stream.
|
I'd certainly consider fullblock nothing more than a workaround as it shouldn't be necessary and it suggests to me that something is broken.
I see this with both coreutils 8.11 and 8.12, with glibc-2.13 and kernel 3.1.1. Slackware64 13.37 (heavily messed with) Problem is my system is a bit of a plaything: I have updated the kernel, headers, glibc and coreutils locally so the scope for having broken something myself is quite wide and it's going to make identifying the cause somewhat difficult. |
The "count=n" operand doesn't mean "n full blocks". When 'dd' performs a read from a pipe, the amount of data returned by the read cannot exceed the amount of data was buffered in the pipe at that moment. If you haven't told dd to do something different (e.g., with the "fullblock" iflag), it's going to treat whatever it's got as a block. You'll get similar results from communications lines, tape drives with variable block size, ..., any device providing a data stream where all of the data you requested might not be immediately available.
|
That explanation makes sense, but why does colucix older dd work? Is this a change of behaviour in more recent versions of dd?
Also, if what you say is the case then the 'partial read' warning ought to be displayed for any value of count=n on receipt of an incomplete block, which wasn't happening on a count=1.That has to be considered a bug |
Found this: https://bugzilla.redhat.com/show_bug.cgi?id=668247
... which is talking about ibs= but seems related to this topic and confirms what rknichols was saying above. So, looks like dd used to cope with this stuff, but they changed it at some point and this new behaviour is now considered "working as designed". This looks like one to remember. I can see this catching a lot of people out. On the plus side, it appears I haven't broken my system after all. ;) |
Quote:
FWIW, the amount of data returned by that single read from the pipe appears to depend on the glibc version. On a system with glibc-2.11 I consistently see 8192 bytes. On systems with glibc-2.12 (Scientific Linux 6) and glibc-2.14 (Fedora 16) I see a variable amount. What's odd is that SL-6 should be the same as CentOS 6, which colucix reports as consistently yielding the full 1-megabyte block. |
All times are GMT -5. The time now is 06:22 AM. |