Block Cloning...How much are you saving?

TrueNAS General
1

Cringe Alert: I Am a Data Hoarder

One of my primary TrueNAS servers’ roles is to be a backend for my Plex storage. As part of that, I have many stages and processes surrounding new data ingestion.

The Problem: Wasting Space with Multiple Copies

TLDR: I end up with multiple copies of large video files, which wastes a ton of space.

Without going into too much detail, my videos go through the following processes:

  1. Remuxing source into an mkv for portability and for HDR content.
  2. Rencoding with Handbrake into 1080P H.264 4.1 Rec709 w/dual audio and/or 2160P H.264 Rec709 5.1 w/dual audio in an mp4 for compatibility/streaming
  3. Creation of multiple versions of new metadata files (cover art, subtitles) by TMM and Plex
  4. Then the reencoded MP4 file is re-written with ffmpeg to include updated metadata tags from TMM’s .nfo files and subtitles from the .srt files
  5. Restoration Work Some videos (old films, home movies) get special processing with Topaz AI… these deviate a bit from process but basically they are exported in ProRes and then reencoded or imported in PowerDirector for color work.

Because things HAVE gone wrong…Multiple file copies exist of each of these multiple versions of things.

Alot of this is automated but alot of it is still manual…again because things have gone wrong xD. The end result of this insanity is that I end up wasting a whole ton of space. Up to 4x waste in some cases. That was until blockcloning tho.

I was upgrading/growing my pools consistently every year or so for the past decade. I haven’t in a couple of years now because I had invested more $ into the problem the last go around. Still, I usually peruse for black Friday deals on shuckables and I’m not going to bother this time.

root@rawht[~]# zpool get all ice | grep bclone
ice   bcloneused                     10.0T                          -
ice   bclonesaved                    25.0T                          -
ice   bcloneratio                    3.49x                          -

I’m saving 25 Terabytes!

I’m genuinely curious if others have other somewhat niche usecases that they’ve seen huge savings with. I’m also equally interested in usescases which haven’t seen savings on.

PLEASE Share your results don’t forget to replace POOLNAME with the name of your pool.

sudo zpool get all POOLNAME | grep bclone
2

First there was the “Show me your tiny RAM, mere peasants”, and now it’s “Show me how little space you save with block-cloning, so that I may laugh at you with my 25 TiB savings”.

This forum is really hurting my ego. :cry:

BCLONE_SAVED  BCLONE_USED  BCLONE_RATIO
12K           12K          2.00x

Unfortunately, most of my data originates from clients (which are rsync’d to the TrueNAS server), or it’s pulled in from an internet download. This completely bypasses the possibility of block-cloning.

1 Like
3

i have made 2 sons with those pools

bcloneused                     63.9K   
bclonesaved                    63.9K        
bcloneratio                    2.00x

bcloneused                     0                            
bclonesaved                    0                             
bcloneratio                    1.00x

Btw, i don’t have redundant data, on opposite i try to avoid those situation as possible… having really not so much data to manage for the moment Is pretty viable for me

1 Like
4

Lol I swear I was not bragging. I’m genuinely curious :frowning:

5

big time savings :wink:

# zpool get all tank | grep bclone
tank  bcloneused                     47.8M                          -
tank  bclonesaved                    47.8M                          -
tank  bcloneratio                    2.00x                          -

But early days I guess :wink:

1 Like