LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-07-2019, 01:27 AM   #1
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Rep: Reputation: Disabled
Will adding data to a hdd that already has data on it using the “dd” command overwrite the pre-existing data?


Will adding data to a hdd that already has data on it, by using the "dd" command, overwrite that pre-existing data? I do NOT want to overwrite any data. Data is being copied over from 1 hdd to another one, both of which do not have an OS and only have 1 partition each.

DD COMMAND (example from article):
Code:
dd if=/dev/sda1 of=/dev/sdb1
The dd command is for copying over an entire partition, which is what I want to do. This commands seems to be what I need, though I am not sure what to use for "of". The partition name is NOT an 'sdxx', but rather just a name I gave it when encrypting the partition. Lsblk shows "sdx", and then "ABC" for the partition name.

I imagine I would use
Code:
"dd if=/dev/sdXX of=/dev/ABC"
since the partition name is "ABC".

I don't know if the dd command needs to include "bs=#" or not.

source: #4 from - https://tecadmin.net/drive-and-parti...th-dd-command/

Thanks.

Last edited by hddfsck; 09-07-2019 at 02:35 AM. Reason: changed the dd command i would use to copy partition on hdd1 to partition on hdd2.
 
Old 09-07-2019, 02:37 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 20,184

Rep: Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757Reputation: 3757
The question is certainly now more understandable since your edit. However, withholding information is not going to help you get decent answers. Post the entire lsblk - uneditted. Your interpretation of what it shows is not useful.
 
Old 09-07-2019, 02:39 AM   #3
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995
What exactly do you want to achieve? Copy a harddisk partition to another harddisk partition, or to a file, or something different?

By definition, if you copy something from disk A to disk B, you overwrite all or part of whatever there is on disk B.

This command:
Code:
dd if=/dev/sdXX of=/dev/ABC.img
will copy the content of sdXX (presumably a SCSI disk) to a file named /dev/ABC.img. ABC.img is almost certainly not a device file. Since /dev is a RAM-based filesystem in most modern Linux distros, you will very quickly fill /dev and the command will fail.

EDIT: Section #4 on the referenced web page copies an entire disk to another disk. But you seem to have pre-existing data on the destination disk that you don't want to overwrite. It's still not clear to me what you want to achieve.

Last edited by berndbausch; 09-07-2019 at 02:42 AM.
 
Old 09-07-2019, 02:45 AM   #4
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 18,414
Blog Entries: 12

Rep: Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626Reputation: 5626
Quote:
Originally Posted by hddfsck View Post
Will adding data to a hdd that already has data on it, by using the "dd" command, overwrite that pre-existing data?
...
Code:
dd if=/dev/sda1 of=/dev/sdb1
YES, this command WILL overwrite (or rather make unusable) ALL existing data on /dev/sdb1.
You can, however, tell dd to write to a file instead of straight to the hardware.
Example:
Code:
dd if=/dev/sda1 of=/where/sdb1/is/mounted/sda1.bak
bs=<something_larger> is recommended because the default blocksize is very small (historical reasons i guess), and it would take very long to copy.
 
Old 09-07-2019, 03:35 AM   #5
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ondoho View Post
YES, this command WILL overwrite (or rather make unusable) ALL existing data on /dev/sdb1.
You can, however, tell dd to write to a file instead of straight to the hardware.
Example:
Code:
dd if=/dev/sda1 of=/where/sdb1/is/mounted/sda1.bak
bs=<something_larger> is recommended because the default blocksize is very small (historical reasons i guess), and it would take very long to copy.
Thanks! What if...

3 hdd's:

I use the dd command to copy the 1st hdd partition to the 2nd hdd partition which is completely a 'blank' hdd. Could I then copy and paste from the 2nd hdd to the 3rd hdd (that already has data on it that I can't delete) and end up with a bit-by-bit of the original files(hdd1) but now on hdd 3? I don't think this will work - I will end up just copying the files over, but not bit-by-bit.

The purpose of copying the files by copy and paste to the 3rd hdd is to keep the encryption on the 3rd hdd.

Also, to add "bs", should I change the command to "dd if-/dev/sdx" bs=4096 of=/dev/sdb" ?

I can't have it write straight to the hardware because I will lose my encryption, but thanks.

Last edited by hddfsck; 09-07-2019 at 03:38 AM.
 
Old 09-07-2019, 03:42 AM   #6
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
What exactly do you want to achieve? Copy a harddisk partition to another harddisk partition, or to a file, or something different?

By definition, if you copy something from disk A to disk B, you overwrite all or part of whatever there is on disk B.

This command:
Code:
dd if=/dev/sdXX of=/dev/ABC.img
will copy the content of sdXX (presumably a SCSI disk) to a file named /dev/ABC.img. ABC.img is almost certainly not a device file. Since /dev is a RAM-based filesystem in most modern Linux distros, you will very quickly fill /dev and the command will fail.

EDIT: Section #4 on the referenced web page copies an entire disk to another disk. But you seem to have pre-existing data on the destination disk that you don't want to overwrite. It's still not clear to me what you want to achieve.
I want to copy the content of one hdd to another hdd. The destination hdd has data on it that can't be deleted. My goal, which I can not figure out how to do, is to copy the data from source hdd, bit-by-bit, to the hdd that 'already' has data on it, in which this partition is encrypted (only 1 partition on hdd), and do it in a way whereby I do not lose the encryption.

So, keep the encryption on final destination hdd and get it there bit by bit. Thanks.

Last edited by hddfsck; 09-07-2019 at 03:44 AM.
 
Old 09-07-2019, 04:46 AM   #7
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ondoho View Post
YES, this command WILL overwrite (or rather make unusable) ALL existing data on /dev/sdb1.
You can, however, tell dd to write to a file instead of straight to the hardware.
Example:
Code:
dd if=/dev/sda1 of=/where/sdb1/is/mounted/sda1.bak
bs=<something_larger> is recommended because the default blocksize is very small (historical reasons i guess), and it would take very long to copy.
... Using the dd command, If I have the hdd write to a file that I make on the destination hdd, can I keep my encryption on the destination drive?; I need to keep my encryption and write it in dd.

I Know that if i use a cloning device that if the contents being sent over to the destination hdd are not coming from an encrypted hdd, then the destination hdd will lose its encryption, if it originally had it.

Thanks.

Last edited by hddfsck; 09-07-2019 at 04:50 AM.
 
Old 09-07-2019, 04:55 AM   #8
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995
Quote:
Originally Posted by hddfsck View Post

So, keep the encryption on final destination hdd and get it there bit by bit. Thanks.
In this case, don't copy the source disk to /dev/sdX, but to the device file that represents the decrypted partition.

I may be able to provide more detailed instructions if you tell us how the destination partition is encrypted, and how you plan to use the encrypted disk. Will you attach it to a Linux computer or a different OS? Also, do you need to keep the encryption on the destination disk, or is it OK to wipe it out and re-encrypt it with a Linux-compatible method like LUKS?
 
Old 09-07-2019, 05:04 AM   #9
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
In this case, don't copy the source disk to /dev/sdX, but to the device file that represents the decrypted partition.

I may be able to provide more detailed instructions if you tell us how the destination partition is encrypted, and how you plan to use the encrypted disk. Will you attach it to a Linux computer or a different OS? Also, do you need to keep the encryption on the destination disk, or is it OK to wipe it out and re-encrypt it with a Linux-compatible method like LUKS?
Thanks. Sounds like you are saying to create a 'file' on the destination hdd and copy the source hdd files via dd there. The encryption is LUKS and no it is not ok to wipe it out. It will be attached to a linux-OS only. But you also said that the device file would be on the "decrypted partition"; this is not good. Yes, getting the source files to the destination hdd via dd without encryption would be better than not getting it there at all, but, if possible, I was trying to see if I could do both, encryption and dd, to the destination hdd. Thanks.
 
Old 09-07-2019, 05:11 AM   #10
zeebra
Senior Member
 
Registered: Dec 2011
Distribution: Mageia, Slackware
Posts: 1,266
Blog Entries: 14

Rep: Reputation: 400Reputation: 400Reputation: 400Reputation: 400Reputation: 400
I think what he wants to do is to write data from source X to a location on disk destination Y with dd.

Let's say dd if=/dev/sda1 of=/dev/sda2/somefolder BS=Y. Would the data from source X then be readable through the destination folder, and would it be sane to write it to the folder in that way?

It would be possible to test this with some smaller disks, like USB sticks. Say one encrypted one 4gb with something on and write into a folder on another usb disk. Some people might already know the answer, I don't know the answer for sure, I just have a feeling it would work. Disk data is decrypted first, source could be anything.

Last edited by zeebra; 09-07-2019 at 05:12 AM.
 
Old 09-07-2019, 05:35 AM   #11
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995Reputation: 1995
Quote:
Originally Posted by hddfsck View Post
Sounds like you are saying to create a 'file' on the destination hdd and copy the source hdd files via dd there
No.
Quote:
The encryption is LUKS and no it is not ok to wipe it out. It will be attached to a linux-OS only.
Attach the destination disk to the computer and identify its device file, let's say /dev/sdX. Then
Code:
cryptsetup luksOpen /dev/sdX mydestination
You will have to enter the correct passphrase. This will create a device file named /dev/dm-Y (where Y is a number) and a symbolic link /dev/mapper/mydestination that points to /dev/dm-Y. Everything written to /dev/mapper/mydestination is encrypted, then written to /dev/sdX. When you read from /dev/mapper/mydestination, data is first read from /dev/sdX, then decrypted.

Copy the source disk to this device file. Close the decrypted device with
Code:
cryptsetup luksClose /dev/mapper/mydestination
After this, an encrypted copy of your source disk is on /dev/sdX.

Last edited by berndbausch; 09-07-2019 at 07:15 AM. Reason: typo
 
1 members found this post helpful.
Old 09-07-2019, 07:03 AM   #12
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,674

Rep: Reputation: 886Reputation: 886Reputation: 886Reputation: 886Reputation: 886Reputation: 886Reputation: 886
Quote:
Originally Posted by hddfsck View Post
Will adding data to a hdd that already has data on it, by using the "dd" command, overwrite that pre-existing data? I do NOT want to overwrite any data. Data is being copied over from 1 hdd to another one, both of which do not have an OS and only have 1 partition each.

DD COMMAND (example from article):
Code:
dd if=/dev/sda1 of=/dev/sdb1
If /dev/sdb1 (or whatever you're using as the of= partition) is EMPTY: no, there is nothing TO overwrite.
If the partition is not empty: yes, you will overwrite everything that's already there.

dd is essentially a block by block copy, so it doesn't look at the contents of the destination at all, it will make essentially a copy of the source partition to it.

BTW: the destination partition must be at least as big as the source (if=) one, or the copy will fail because of lack of space.
The arguments of dd are
if= input file (or partition, or device)
of= output file (or ...)
 
Old 09-07-2019, 08:57 AM   #13
system001
Member
 
Registered: Nov 2018
Location: Portland Oregon Canada
Distribution: Kubuntu 19.04 - Manjaro 18.0
Posts: 199

Rep: Reputation: 15
after reading you originasl post i'm not even going to bother to read the rest of the post. from what your saying you're simply combining two drives, right? if so and you have the space for the operation on the one drive simply use gpoarted to do it.

install gparted if not installed
launch gparted
look at the space of the drive you want to copy, shrink it's partition down to just 10% over what's being used
look at the drive you want to copy to, shrink it's partition to just over 10% over what's being used
now look at the amount of free space you now have on the drive you want to copy to, do you have enough space to copy the one partition to?
if so then do your copy.
 
Old 09-08-2019, 10:45 AM   #14
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
No.


Attach the destination disk to the computer and identify its device file, let's say /dev/sdX. Then
[code]
cryptsetup luksOpen /dev/sdX mydestination

"This will create a device file named /dev/dm-Y (where Y is a number) and a symbolic link /dev/mapper/mydestination that points to /dev/dm-Y. Everything written to /dev/mapper/mydestination is encrypted, then written to /dev/sdX. When you read from /dev/mapper/mydestination, data is first read from /dev/sdX, then decrypted."

Copy the source disk to this device file. Close the decrypted device with
Code:
cryptsetup luksClose /dev/mapper/mydestination
After this, an encrypted copy of your source disk is on /dev/sdX.
Thanks.
Copy the source disk to this device file.

#1) Which file? (1) /dev/dm-Y or (2) /dev/mapper/mydestination ?

Seems like you are referring to "/dev/dm-Y" since that is "file" and the other is a "symbolic link".

#2) The files are going to be copied "byte-by-byte"? I have deleted files on the source disk and if in trying to get those files back the disk gets corrupted, I need to have an exact replica, byte-by-byte, that I can use to try to get those files back.

Just to be sure: you do know that I (1) already have data on the destination drive and (2) the destination drive IS already encrypted with LUKS? I think you do. Thanks.

Last edited by hddfsck; 09-08-2019 at 12:18 PM. Reason: clarification
 
Old 09-08-2019, 10:47 AM   #15
hddfsck
Member
 
Registered: Aug 2019
Distribution: Debian; Ubuntu
Posts: 122

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ehartman View Post
If /dev/sdb1 (or whatever you're using as the of= partition) is EMPTY: no, there is nothing TO overwrite.
If the partition is not empty: yes, you will overwrite everything that's already there.

dd is essentially a block by block copy, so it doesn't look at the contents of the destination at all, it will make essentially a copy of the source partition to it.

BTW: the destination partition must be at least as big as the source (if=) one, or the copy will fail because of lack of space.
The arguments of dd are
if= input file (or partition, or device)
of= output file (or ...)
Thanks. Sounds like you are saying that the method that I described "if,of" will not work for my particular circumstance.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Adding 2 SAS HDD (RAID -0) to existing 4 HDD (2 RAID-0), then swapping 2 pairs kaza Linux - Hardware 2 01-23-2019 04:23 PM
How to overwrite existing files by using tar command (Remove extra files) Kalibo Linux - Newbie 9 12-26-2013 01:42 AM
Adding kerberos authentication to existing network with LDAP already deployed btmiller Linux - Server 2 01-31-2013 09:39 PM
how to install pre existing wordpress directory on a existing wordpress to run a web ajbardhan Linux - Software 3 04-28-2012 08:01 PM
Adding new Linux firewall/router on network with pre-existing gateway/router grittyminder Linux - Networking 4 08-13-2008 03:17 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration