What would be the best way to migrate 4 of 22TB NTFS formatted drives’ data to TrueNas scale’s ZFS? Ideally migrate all 4 drives at the same time, regular file copying could be slow due to many small files, but i don’t see a way out since ZFS and NTFS is not compatible with block level clone and i don’t want to use iscsi so i can get full feature from ZFS taking care of my file system and share via smb or nfs later. As far as I know import NTFS drive as readonly is only available on core, and since Truenas 22 NTFS mount has been discontinued and there is no apt install. I don’t really want to use iscsi, but NFS / SMB. Any help is appreciated.
Honestly, it’s typically better to just bite the bullet and transfer via SMB. It’s the safest option in terms of least astonishment while transferring data.
4 Likes
Thanks for the suggestion, any advise on the method of the copy? I am thinking to use something like cwrsync, but not entirely sure yet. I know I will have problem with default explorer as many files has way longer names than 256 characters limit for explorer.
Also, Since this is a one time thing to migrate data. Do you think it is a good idea for me to just install older version like TrueNas Scale 22.xxx or Truenas Core which still has the NTFS capability to migrate the data, and after that’s done, I will import the migrated ZFS pool to 24 when it release in 2 days? Thanks!
Once you’ve got TrueNAS set up and all the datasets set up, you can boot into a Linux live USB and do the copy there. Assuming you’ve got both old and new drives attached to the same physical computer, that is.
Boot into generic Linux, mount the ZFS pool, mount the NTFS drives, and rsync between them.
1 Like
I wonder if you could do the same thing with a Sandbox? Ie, hostmount the dataset and the disk device into the sandbox… and then install whatever packages are needed to NTFS mount the disk…
Not a good idea. You can’t do it while Truenas is running. This is a once only step, just boot into a maintenance operating system.
Thanks for the suggestion. This does seem to be a viable method I will try that first, but the way I interpreted ericloewe’s response which it seems to be suggesting TrueNas scale(which is based on debian linux) had issues with NTFS driver etc, and could cause problems or what not which is why iX deprecated the feature in the first place. Even through I personally likes r sync NTFS to ZFS on the same system like what you suggested, with the added safety of my drives are on another system and are offline to windows, so no hooks and no applications actively using it; but still, idk if that would be safest route to take since it’s probably impossible for me to track over 50TB+ of data’s integrity after transfer. I can only assume native Linux has good support and robust data transfer via rsync cross file system(NTFS to ZFS). Regardless, I will give a try first on everyone’s suggestion and see which one is going to workout for me.
1 Like
Before doing anything too drastic, can you do a backup of your disks (if single) using tools such as Acronis True Image and have the image files stored on ZFS over SMB? At least, you can revert if something bad happens later.
Then you can experiment with moving files using rsync or other means.
I had issues in the past when I tried to import a NTFS formatted drive into a dataset under FreeNAS. Some folders and files couldn’t be imported, possibly due to long filenames or character set.
Good suggestion. I indeed have imaged all my live data to spare drives almost like raid1 config at the moment; and yes, I really worry about silently losing data and not know about it and thinking everything went fine. Worst case I will just go with ericloewe’s suggestion and slowly migrate via SMB, at least when something goes wrong I would know, probably with teracopy or some third party software.
Hello, I’ve tried mounting both NTFS and ZFS on a freshly installed ubuntu. I am not particularly familiar with the procedure. It seems like I won’t be able to access my ZFS dataset in normal user and I have to elevated to root by running “sudo su”, once that’s done, I tried to “rsync -avhP /mnt/ntfs/ /ZFSpool/dataset/” and I could only create folders but non of the actual files were transmitted with error "mkstemp xxxxxx/file/ failed: operation not permitted(1) ; i was about to workaround the problem by not using the -a for rsync but remove -p -g -o, ending up like “rsync -rltvhP xxxxx xxxxxxxxx”
while this worked and copied all folder and files over, I am really not familiar with linux / windows permissions, I wonder if this would cause any issue down the line?
what would be the recommended way to do this? Thanks…
Also, just to clarify, my ZFS pool is empty, so I don’t mind create a ZFS pool in older TrueNas like 22 or core where NTFS support is still there, transfer data, upgrade later, but same permission issue still persist. So any input would be great.
Can’t you just install windows on a spare drive on another computer/don’t you still own a Windows computer?
I’m not familiar with mounting NTFS under Linux but I’d assume copying from Windows would be the easiest method.
I would follow @ericloewe advice and just mount a SMB share on a Windows machine and transfer via SMB. You can copy the files with something like total commander, which would give you a verify option.
I do have windows machines and the HDDs are currently attached to the Windows machine. So moving with SMB has some small downside for me, first it’s gonna be capped at 6Gbps for my current network(not mentioning my current network with opnsense is kinda wack), and I have multiple disks to move, supposedly directly attached storage will be safer and faster.
To be completely honest, for the past data migrations I always done block level clone or imaging, never file level migration; I know I had some issues with my data where some files are corrupted and I never ran Chkdsk because I DO NOT want windows to delete ghost files that are corrupted so (when I came across which file is corrupted even though the file is dead, at least I can identify them and try to get it from somewhere either from backup or internet); on top of that I have lots files with path / name longer than 256 which just drives windows insane; and some edge case weird files with weird ownership and permission sets; So I will most likely run into lots file copy issue during the process.
Lastly, another concern is that, unless I open up a fresh Windows VM without any hooks and apps that access the HDDs that I intend to migrate, current live Windows machine has lots apps and hooks on to the data. So ideally I would like to “offline” those disks, which is why I wanted to just connect to the TrueNas server directly, one to avoid network, another is to avoid hooks.
But yes, I would still favor data integrity / safety for this one time migration over speed or convenience, just trying to weight in my options to find the optimal method before pulling the trigger.