Should I alter my Photos dataset recordsize? Other settings?

TrueNAS General
1

Just finishing setting up TrueNAS… Setting up my Photos dataset before I copy the data over, wondering if I should adjust the recordsize.. or any other parameter to improve performance? I just ran a quick check on the existing dataset, and getting this over ~175k files:

  • <1MB: 67638

  • <10MB: 5580

  • <100MB: 98753

  • <1GB: 1644

  • >=1GB: 7

So it’s a LOT of 50MB+ camera raw files, and a lot of sidecar XMP files that are just a few hundred kilobytes. I’m trying to understand recordsize and it SOUNDS like it would be beneficial to alter given the 50MB+ files, but then there’s also a ton of really small 100-500KB files… and that throws me for a loop.

Any other settings or adjustments worth considering for the dataset? I think compression would be just about useless as I don’t believe these files really even could be compressed?

2

I’d guess 128k or 256k will be your sweet spot but it may be worth testing. Make some test datasets, one with recordsize=128k, one with 256k, one with 512k, one with 1M, copy over a representative sample of your dataset into each dataset, and test your workflow out on each dataset.

I’ve personally found that Lightroom chugs like an elderly dog with my photos on a network drive no matter the settings or network speed. If you’re also using Lightroom, you may not notice much of a difference between recordsize values.

3

Anywhere from 1M to 4M will yield the best performance, especially if you’re not doing “in-place modification”, which very few non-DB software actually does.

Going higher than 8M can reduce performance gains from losing parallel processing of multiple blocks at a time.

A 100KB file in a dataset with a recordsize of 4M will consume 100K.

2 Likes
4

What does “testing” it actually entail? An I testing to compare file transfer speed or something? That feels inconsistent in the best of times :frowning:

I hope I don’t experience very sluggish performance with Lightroom… :crossed_fingers:t2:

5

Hmm very different answer to the one above you haha

I have to figure out what “testing” it actually entails - how I determine if 128k or 4M is actually better for my data

6

Not so different once you realise that recordsize is a maximum, not a minimum (i.e. files smaller that recordsize will use smaller records).
Raw pictures are quite responsive to loseless compression; LZ4 or some low level zstd can only be beneficial.

7

I would say because your workload is dominated by large files I’d go with a 1MB recordsize. As @etorix says anything smaller will be saved using smaller records anyhow.

Based on that you may say why not go larger like 4MB, well there is a tradeoff. If you want to read a small portion of a large file ZFS needs to read an entire 4MB block in this instance even if it only needs a few hundred KB. So it’s a balancing act hence why @jro suggested testing. Testing being put some data in some datasets and do some things and compare.

8

Between 1 and 4M, with lz4 on. The small files will be smooshed together if compression is on. I’m assuming this is WORM data (Written once, read many).

1 Like