Page 1 of 3 123 LastLast
Results 1 to 10 of 21

Thread: Transfer all data from hard disk

  1. #1
    Join Date
    Apr 2006
    Beans
    3,016

    Transfer all data from hard disk

    Hi all,

    Please advise what will be the easy way transferring all data from a 4TB HD, ext 4 format, to a brand new 4TB HD, NTFS format. There is no OS on the HD.

    The new 4TB HD will be used for data storage only in another dual boot PC, sharing data between Ubuntu and Win 10. No OS will be installed on the new 4TB HD

    Thanks

    Regards

  2. #2
    Join Date
    Apr 2014
    Beans
    854

    Re: Transfer all data from hard disk

    Cp or rsync. With no special permissions on ntfs its pretty basic. Will take a long time though.

    If over network just scp instead of cp. Start it in a tmux session and let it run.
    Last edited by Tadaen_Sylvermane; November 14th, 2021 at 05:00 AM.

  3. #3
    Join Date
    Aug 2017
    Location
    melbourne, au
    Beans
    Hidden!
    Distro
    Lubuntu Development Release

    Re: Transfer all data from hard disk


  4. #4
    Join Date
    Apr 2006
    Beans
    3,016

    Re: Transfer all data from hard disk

    Quote Originally Posted by Tadaen_Sylvermane View Post
    Cp or rsync. With no special permissions on ntfs its pretty basic. Will take a long time though.

    If over network just scp instead of cp. Start it in a tmux session and let it run.
    Hi Tadaen_Sylvermane,

    Thanks for your advice.

    I have been running cp command on Ubuntu Terminal for long time. As you mentioned that it will take long time to dump the data on an old 4TB SATA HD to a new 4TB SATA HD, even though they are connected in the same PC, say PC-1.

    I have performed following test;

    1) Start Ubuntu 20.04 Terminal on PC-1, which runs Oracle VirtualBox.
    2) $ cp /path/to/old_HD/directory_1/ /path/to/new_HD/
    3) Remove the new_HD from PC-1 and insert it in an USB Hard Drive Docking
    4) On PC-2, a Dual boot PC, Ubuntu 20.04 on SSD_1 and Windows10_SSD-2, start Ubuntu and plugin the USB Hard Drive Docking to it.
    5) Ubuntu detects the new_HD, reading directory_1 and its files without problem. Files can be edited. Even Window 10 VM and Linux OS VMs can read director_1 and its files
    6) On PC-2 again, start Windows 10 and plugin the USB Hard Drive Docking to it. Windows 10 can read directory_1 and its files on the new_HD

    The above test proves that data of old_HD, of ext 4 format, can be copied to the new_HD, of NTFS format, without problem.

    Now my problem is how to dump the data on the 4TB old_HD to the 4TB new_HD at the least time?

    Regards
    Last edited by satimis; November 14th, 2021 at 11:20 AM.

  5. #5
    Join Date
    Mar 2011
    Location
    U.K.
    Beans
    Hidden!
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: Transfer all data from hard disk

    One option you might consider (depending on your priority Windows/Ubuntu) is to install RLinux in Windows to read ext4 files.
    Then you could use an external dual docking bay to clone ext4 to ext4, in background.
    https://www.youtube.com/watch?v=3f-W...stalkSolutions

    I use the Startech dual docking bay which can carry two SSD's. In fact I am using Ubuntu 20.04 now from external SSD.
    So I have three drives: 1 internal HDD plus 2 external SSD's.

  6. #6
    Join Date
    Oct 2005
    Location
    Lab, Slovakia
    Beans
    10,502

    Re: Transfer all data from hard disk

    I would use rsync, since it does proper error detection and correction. In a large set of data, you may get one or two errors with ordinary cp and rsync will prevent that.

  7. #7
    Join Date
    Apr 2006
    Beans
    3,016

    Re: Transfer all data from hard disk

    Quote Originally Posted by dragonfly41 View Post
    One option you might consider (depending on your priority Windows/Ubuntu) is to install RLinux in Windows to read ext4 files.
    Then you could use an external dual docking bay to clone ext4 to ext4, in background.
    https://www.youtube.com/watch?v=3f-W...stalkSolutions

    I use the Startech dual docking bay which can carry two SSD's. In fact I am using Ubuntu 20.04 now from external SSD.
    So I have three drives: 1 internal HDD plus 2 external SSD's.
    Hi dragonfly41,

    Thanks for your advice

    What is RLinux? To install Linux on Windows?

    I has considered Diskinternals, Linux Reader, before

    If You Need to Access Ext4 from Windows
    https://www.diskinternals.com/linux-...-from-windows/

    Then I need to install Diskinternals on all Win 10 VMs. I think data sharing on HD is my direction.

    This spare PC, PC-2, is a triple-boot PC, controlled by BIOS;
    1st boot - Ubuntu 20.04 on SSD-1, running Oracle VirtualBox
    2nd boot - Windows 10 on SSD-2 (bare-metal OS)
    3rd boot - Ubuntu 20.04 SSD-3 (bare-metal OS)

    All data are stored on a Storage HD, ext 4 format and without OS installed.

    I need those bare-metal OS for testing, in case VirtualBox unable to forward some hardware to VMs.

    I only have a single-bay docking device.

    Now I'm reading some documents regarding "xcopy" of Command Prompt (Windows 10 application). It seems having more options able to compress data in transferring. If I can sort out my problem here I prefer to go Linux way.

    Regards

  8. #8
    Join Date
    Apr 2006
    Beans
    3,016

    Re: Transfer all data from hard disk

    Quote Originally Posted by HermanAB View Post
    I would use rsync, since it does proper error detection and correction. In a large set of data, you may get one or two errors with ordinary cp and rsync will prevent that.
    Hi HermanAB

    Thanks for your advice

    Just found;
    How do I transfer a large number of files using rsync?
    https://support.cpanel.net/hc/en-us/...s-using-rsync-

    I suppose I should follow;
    When you want to "push files" to a server from the current server you are logged into, the following command can be used........

    One question I have to consider;
    In case of failure, the data on the old HDD won't be damaged? That is my top priority consideration

    Thanks

    Regards

  9. #9
    Join Date
    Apr 2014
    Beans
    854

    Re: Transfer all data from hard disk

    I think one thing to consider is you can only push that data so fast. Even with a very powerful cpu and stuff it will max at the i/o speeds of the drives themselves. You could throw a quad socket with 12 cores on each at it and it will still cap at 100MBs or so depending on the drives (spinning drives). Also if it's a bunch of smaller files instead of a few big ones that alone will slow it down. There really is no high speed method that I'm aware of. 4tb is a lot of data so it will take time no matter how you slice it.

    In case of failure, the data on the old HDD won't be damaged? That is my top priority consideration


    The copy process doesn't do any writing to the source generally so unless the source drive itself crashes and burns as far as it's concerned it's just data being read. Nothing more.

    Doesn't help with the ext4 > ntfs transition but this is why raid exists. Keep redundant copies so you aren't waiting on a major downtime. I know in my case with snapraid for media it's all fine and good but when I lose a drive whatever is on that drive is MIA until the snapraid recovers it to the new drive. Proper raid eliminates this problem and the downtime associated with it.

    *EDIT* A further option if you have a server running is just keep it on the server. Then you can keep a "single" copy so to speak and any machine can access it. You can run a samba share and nfs export at the same time with zero problems. At that point format is irrelevant.
    Last edited by Tadaen_Sylvermane; November 14th, 2021 at 04:39 PM.

  10. #10
    Join Date
    Apr 2006
    Beans
    3,016

    Re: Transfer all data from hard disk

    Quote Originally Posted by Tadaen_Sylvermane View Post
    I think one thing to consider is you can only push that data so fast. Even with a very powerful cpu and stuff it will max at the i/o speeds of the drives themselves. You could throw a quad socket with 12 cores on each at it and it will still cap at 100MBs or so depending on the drives (spinning drives). Also if it's a bunch of smaller files instead of a few big ones that alone will slow it down. There really is no high speed method that I'm aware of. 4tb is a lot of data so it will take time no matter how you slice it.
    Hi Tadaen_Sylvermane,

    That is a very good advice.

    I'm considering creating 4 folders and dividing the data into 4 parts. Each folder will contain 1 parts of the data. Then transfer 4 folder one after one.

    Any comment? Thanks

    The copy process doesn't do any writing to the source generally so unless the source drive itself crashes and burns as far as it's concerned it's just data being read. Nothing more.
    Noted and thanks.

    Doesn't help with the ext4 > ntfs transition but this is why raid exists. Keep redundant copies so you aren't waiting on a major downtime. I know in my case with snapraid for media it's all fine and good but when I lose a drive whatever is on that drive is MIA until the snapraid recovers it to the new drive. Proper raid eliminates this problem and the downtime associated with it.

    *EDIT* A further option if you have a server running is just keep it on the server. Then you can keep a "single" copy so to speak and any machine can access it. You can run a samba share and nfs export at the same time with zero problems. At that point format is irrelevant.[/COLOR]
    No I don't run server here.

    Regards

Page 1 of 3 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •