Here is a discussion thread for the “ZFS AnyRaid” feature. This is from thread:
Here are the links from that thread:
https://www.reddit.com/r/zfs/comments/1ktm9zv/comment/mty9x4j/
For the other thread’s comparison to BTRFS, it is somewhat accurate. BTRFS in theory allows 3 disks to be in a 2 way Mirror, which forces the 2 copies to be on different disks. I don’t remember off hand about dissimilar sized disks, but I would guess that BTRFS supports that.
You cannot just arbitrarily mix different sized drives though?
The other two drives in an “AnyRaid-mirror” vdev must at least be equal in sum to the largest drive, if I understand correctly?
This would not be possibe?
- “AnyRaid” two-way mirror vdev
–12TB sda
– 6 TB sdb
– 4 TB sdc
There’s no way that even with the 64GB “chunks” system you could fill up the entire 12TB drive, and still offer full pool capacity and a 1-drive loss redundancy protection.
If you lose the 12TB drive, how can the 10 TB of the combined remaining drives hold at minimum one copy of your data, and still able to resilver the missing 12TB drive to regain full two-copy redundancy?
The only way this could be possible is if the 12TB drive is limited to 10 TB of writable space. Do I have this wrong?
Shh! 
Just wave around “ZFS magic”…
HexOS is Häxan OS after all: By its own marketing, it does the black magic act of having users paying for a cloud interface for the privilege of hosting their own data locally…
Sounds reasonable, but I suspect that the size calculation algorithm is going to be quite convoluted, especially when dealing with the later addition of further arbitrarily sized drives. Which should make for interesting testing, edge cases, and… potential bugs. 
1 Like
They did mention that moving the 64GB chunks around to “expand” the storage was possible. So if in an example you have the below, but basically full:
- AnyRaid” two-way mirror vdev
–12TB sda
– 6 TB sdb
– 4 TB sdc
You then later add a 10TB disk, it could “move” some of the 64GB chunks to the new disk to allow new 2 way Mirror 64GB chunks to be written on 2 different disks.
Without the ability to move 64GB chunks around, adding anything over 2TBs would be worthless. (Because this vDev is full in the example… so both 6TB & 4TB disks have no free space.)
It would be very nice if this AnyRAID-Mirror, (and RAID-Zx), would incorporate built in partial de-fragmenting. I say partial because this would seem to be at the 64GB chunk level, not the file level.
Ive manually done this sort of thing in the past by partitioning drives into 1TB slices.
You can essentially “move” a slice by resilvering it wherever you want.
Yes, its very difficult to manage manually, but fairly easy for an algorithm.
Considering HexOS is essentially a user friendly API management platform of TrueNAS, this new AnyRAID feature will be implemented and supported within TrueNAS first.
The viable sequence will be:
ZFS first
then TrueNAS
then HexOS
It’s useful for systems that grow organically with one drive at a time. This removes the constraints for getting specific drive sizes.
1 Like
This looks like a really compelling feature, especially once anyraid-raidz is implemented. The two closest offerings would be:
Unraid’s Array feature, which lacks real self healing (if you run into a checksum error on a data disk you are essentially hosed, you can try to remove+replace causing it to rebuild from parity but you have only a prayer that whatever checksum error you ran into isn’t also lurking in parity). it also has really terrible write performance, due to how it has to recalculate parity for any write to any disk (so no parallelism) however read is unaffected. The “cache” pool concept is neat, new files allocate on SSD and move to the Array (or another pool) during a scheduled mover program (only works on files not currently open/locked).
there’s MergerFS (or Stablebit Drivepool on Windows) + SnapRAID. this one is a little different as it’s not realtime parity so it doesn’t come with the massive write penalty that Unraid’s implementation does. it also surprisingly does have the ability to scrub and self-heal as it does checksum files and parity separately. However being scheduled and completely file-based comes with a whole host of other limitations and puts it squarely in the category of “best for cold storage” where files do not change often. it does have the neat ability to undelete a file though.
It should also be mentioned that the above solutions on Linux rely on FUSE (to pool the disks into a single mount) which can be rather unperformant with lots of files in-play.
ZFS anyraid once implemented appears to have the benefits of both of these (somewhat higher performance, self-healing, real-time) while still being “flexible”. though I do have questions and thoughts:
for allocation to be possible over time the space has to be able to be re-balanced periodically and intelligently, as described above by others not every combination of disks or existing files will allow meaningful use of space on the drives leading to waste. for Mirrors this is simple, as you only need two places to put a file, but for a raidz stripe I can imagine this getting incredibly complex.
how wide is AnyRaid RAIDZ going to stripe, do you take a hit on data/parity ratio when there’s not enough free space for a full stripe or does the allocation fail? is it going to be like Unraid/Snapraid solutions where parity isn’t distributed but lives on the largest disk? I’d like to know more about the details of the RAIDZ implementation however this is a long way off.
top-level and leaf VDEV removal? hopefully this is possible, as I understand it this VDEV type at it’s core leverages indirect blocks so I don’t see why it wouldn’t be (for reference, mirror and stripe vdev removal creates indirect blocks for the referenced data).
1 Like