For the sake of this post, say you are downloading a bunch of Linux ISOs. Incoming downloads go to a dedicated download dataset (tank/downloads). Once complete, you copy your ISOs into their respective distro datasets: tank/ubuntu, tank/arch, tank/mint, etc.
Basic concerns:
Once a download is finished, I will seed an ISO until it gets to around a 1:1 ratio. Afterwards. I will delete the file in the downloads dataset but keep the file in the separate distro dataset. This is because I don’t know how to seed from a different dataset. It feels like I am introducing a lot of fragmentation on the NAS, which is entirely HDD.
This setup is obviously not ideal because in order to continue seeding, I would effectively have all files duplicated the downloads and distro-specific datasets. How can I seed content once it gets placed where I want it to live?
One idea I had was simply to download and seed the ISOs outside the NAS and pull files to the NAS once they are finished. This addresses fragmentation concerns, but not duplication.
The other idea was to research the arrstack. I don’t know anything about it. Would this solve both problems?
How do you have things set up to effectively torrent?
My reasoning is that SSDs will better handle the fragmented way ISO’s get downloaded and then when the download is complete, an ordered file can be placed on the HDD where it can be archived.
To revisit this thread - I got two 1.92TB SSDs and want to incorporate them into my pool. I am not sure the best way to do that, however, since my pool is a six 16TB-disk Raidz2 setup.
sorry for the very basic question, but am I able to add 2 smaller mirrored SSDs as a separate vdev to this larger pool? I should probably spend time with the docs.
Longer Answer: Sort of yes, but highly not recommended
With a 6 x 16TB disk RAID-Z2 vDev, you have 2 disks of redundancy. Adding 2 smaller SSDs as a Mirror vDev gets you 1 disk of redundancy for that vDev. That means loss of both the smaller SSDs means loss of the entire pool, including all the data on the RAID-Z2 vDev.
Further, in general it is not recommended to mix data vDev types, RAID-Zx & Mirror. The one exception is a Special Allocation Device, which does not seem suitable for you. And, you still don’t have 3 SSDs to match the recommended 2 disk redundancy of RAID-Z2.
So in reality, no.
ZFS has some great features. However, at times it is not the most flexible file system and RAID manager.
I provide an archival service for a lot of data utilizing the bittorrent peer-to-peer sharing protocol.
If I’m understanding your goal, here’s something I used to do:
qbittorrent-nox (superior, best client) container, Settings → Downloads, enable “Keep incomplete downloads in”, set /mnt/flash/downloads as the path (sata ssd raidz)
Create a qbittorrent category called “BSD,” set the save path as /mnt/spinners/deprecated/BSD
Add a TrueNAS Core ISO torrent to the client, choosing “BSD” as the category.
What will happen is:
qBittorrent will download the flash pool
Once the download is complete, it’s automatically moved to the rust pool at the defined deprecated ISO storage path
My goal was to reduce fragmentation. Was not sure if behind the scenes qbittorent does the copy in a sequential way, but I make assumptions sometimes.
Eventually stopped doing this, because I scripted/automated some archival/download tasks to distribute torrents across multiple clients and having the client move data around in the background just complicated things – or How I Learned to Stop Worrying and Love the Fragmentation.
Really, disk fragmentation hasn’t noticeably impacted my seeding. My sad 200mbps uplink stays saturated. When I relocate my server and have 10gbit available, I’ll maybe revisit…
