Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
a block device, /dev/sda, will be the raw hw device.
a mount point (entrance to a filesystem, etc) will be on a block device
I thought cp command only worked on regular files (-) and dirs (d), but not special files like block devices, character files, named pipes, etc.
Don't files and dirs have start and end to them, and cp works on that. A block device has no "start" and "end" within a block range that spans 0 to X (however big the device is). How would cp command know where the start and end is of a block device?
Last edited by Linux_Kidd; 02-27-2024 at 05:15 PM.
The debian installation guide uses cp to copy an ISO file to a USB device i.e. /dev/sdx so there should be no reason you can not use it to copy in reverse and it does work.
The debian installation guide uses cp to copy an ISO file to a USB device i.e. /dev/sdx so there should be no reason you can not use it to copy in reverse and it does work.
# cp debian.iso /dev/sdX
# sync
cp there though is reading a file from a filesystem, the iso file has a start and end to it.
When you make block device the source (cp /dev/sdx ./iso.img), how does cp know where the start and end of the block device is?
Doesn't the destination also need a start/end. I am not a kernel person but a block device is basically an abstraction layer between the user and the I/O system calls. I'm guessing the maybe the inode contains that info.
cp does an open() on a file or device, and then read() the data until it gets an EOF. In the case of a device the device driver gives the data and the EOF.
Last edited by MadeInGermany; 02-27-2024 at 07:44 PM.
cp does an open() on a file or device, and then read() the data until it gets an EOF. In the case of a device the device driver gives the data and the EOF.
That's interesting, special file devices have just 1 inode, and the inode always says "size:0", so what then does cp read() actually read if the inode info says "no size" and "no blocks"?
In the device inode the major/minor number pair points to a device driver.
The file access calls like open() read() close() ioctl() are redirected to the device driver.
cp itself does not care about inodes. It has 2 file descriptors, one for reading and one for writing.
Anything which can be read can be copied, including a partition or a full raw disk.
It is not important if the fd is a real file on a filesystem or a virtual file on a virtual filesystem or a remote whatever on a remote real or virtual host, the kernel itself will manage the read and write operation (using the appropriate driver/interface).
From this point of view dd and cp are quite similar, just copy is mainly used for real filesystems and dd is mainly used for other kind of data.
Indeed, cp and dd do much the same thing. It's just that cp has options like recursing into directories, handling file attributes, making links instead of copying, not overwriting existing files, etc., while dd has options suited for copying devices, such as setting I/O block sizes, handling errors, bypassing the kernel's buffer cache, and (especially useful for large transfers) showing the progress of the operation.
Now dd does have the reputation of being dangerous, but that's just because it is typically used to do dangerous things, like overwriting devices. You can say that dd is dangerous because you could easily zero out the wrong device, but really you could do exactly the same thing with cp, it's just that usually you wouldn't be doing that.
When a util opens a file and does a read(), the abtraction layer between kernel and device (a driver) has to handle feeding the read() with data, but the kernel holds a mapping of that file, the file is all over the disk (fragmented), but the process just receives the file data, has no clue the where it came from, or how fragmented it was. At some point in the read() there will be an eof, and data will stop flowing back to the process.
read(file) <-->kernel<-->major,minor<-->mapping<-->disk
so start of file, pointers to next block, ... , last block(eof)
That's fairly straghtforward.
When a device block is opened, does the kernel handle read() differently? It must if we are expecting the read() to start at block 0, and end is last block of the actual device, AND, it must read() the device blocks sequentially.
How does the kernel know how to handle read() of a reg-file file differently than read() block-device file ?
I know it was mentioned that inode makes no diff, but is that true? How would kernel know the diff between reg-file or block-device if kernel does not read the info in the inode? open() must invoke kernel to read the inode data, no?
I need to craft a test to copy a storage block device to a file on another storage device, will use 'cp' and 'ddresecue'. If they both do exactly the same thing then each file should have same MD5 SHA hashes.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.