TrueNAS Scale Setup on used Gigabyte EPYC MJ11-EC1 Board - Backup NAS

TrueNAS General
,
1

TrueNAS Scale Setup on used Gigabyte EPYC MJ11-EC1 Board - Backup NAS

While waiting for Black Friday, in order to buy my 4 x 16TB SG IW-PRO disks for my Home (Primary) NAS.
I decided to "get the needed parts for a Backup NAS, using my old 4TB disks from the Home NAS.

I got the Mobo (x2) here:

I got the 2 x 32GB ECC Ram of ebay.
The Mobo seems picky chose Ram with care

I got these “cable” parts from “Aliexpress”

1 x 4 Sata ports Cable (Mobo special plug) - Mini Slim Line SAS 4.0 SFF-8654 4i 38pin
https://www.aliexpress.com/item/1005004567410190.html

2 x Sata cable - For Mobo SATA’s
https://www.aliexpress.com/item/1005005468899117.html?

1 x Sata Power cable x4 (PSU has 6 SATA Power)
https://www.aliexpress.com/item/1005007423555168.html

Today I have finished building the Backup NAS, consisting of the below parts.

Backup NAS - (Node 304 / Gigabyte MJ11-EC1)
TrueNAS Dragonfish-24.04.2.5
Fractal Design Node 304
Seasonic G12 GM - 550W PSU
Gigabyte MJ11-EC1 Mini-ITX Mainboard
AMD EPYC 3151 4x2,7 Ghz
64GB ECC Ram - 2 x Samsung 32GB 2Rx4 PC4-2666V DDR4 RAM M393A4K40BB2-CTD7Q
Patriot P300 256GB NVME Boot Disk
5x4TB WD-RED, 1x4TB WD-Purple - RAIDZ2

Well …
Currently the WD-Purple disk is in the summerhouse, so there’s an 8TB Tosh-N300 in right now.
But i’ll swap the Tosh with the WD-Purple asap.

I just finished today

epyc-nas-01
epyc-nas-011642×811 152 KB

Edit:
I used this 3-D printed IO Shield

epyc-backplate
epyc-backplate673×548 469 KB

Config Decisions:
I decided NOT to enable HyperThreading on the CPU for now.
I decided NOT to enable Secure boot.
I did the “Naughty stuff” and split up the NVME Boot disk into 64GB Boot , and 150GB system/Application pool.

Frontpanel - Layout "snipped from the MJ11-EC0 manual) - Worked

MJ11-EC0-FrontPanel-Header
MJ11-EC0-FrontPanel-Header1078×854 81.9 KB

The Seasonic PSU is nice, nearly silent.
It came with a 8x12v CPU Plug - Ftting directly in the MJ11 Mobo 12v power adapter.
Beware the PSU is deep - For a Node304 chassis.
If you have a mobo with PCie slots, and want to use a long adapter (longer than the mini-itx board. You might run into trouble with the psu power cables & pluggable power connectors (no room for them).

5 Likes
2

Isn’t the below dm-0 & dm-1 Linux LVM’s ?
Why is TrueNAS using LVM on the Boot/System/App disk ??

/dev/disk/by-uuid:
total 0
lrwxrwxrwx 1 root root 10 Dec 15 16:21 15406560877212412735 -> ../../sde2
lrwxrwxrwx 1 root root 10 Dec 15 16:21 2d8d7a03-b557-4804-85fa-ee9f728b40d4 -> ../../dm-0
lrwxrwxrwx 1 root root 15 Dec 15 16:20 7668598265710082891 -> ../../nvme0n1p3
lrwxrwxrwx 1 root root 15 Dec 15 16:20 9675080067192537736 -> ../../nvme0n1p5
lrwxrwxrwx 1 root root 15 Dec 15 16:20 F34E-B873 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 10 Dec 15 16:21 cfa01f83-c5a1-442a-8a8a-0595c116c20d -> ../../dm-1
nas-02:~$
3

When i come back from the summerhouse (xmas)
I’ll bring back the 4TB WD Purple, that is going to replace the 8TB Tosh-N300 in the 6x4TB RAIDZ2 Pool.

Since the NAS isn’t in “prod yet” i could just erase & redo the pool.
But I was thinking i could use this as a “disk replacement / resilvering” experience too.

Can anyone give me some hints to do that ?

Step :
1: Power off the NAS
2: Replace 8TB N300 with the 4TB WD Purple (CMR)
3: Power on the NAS
4: ???

I’d expect it to be “fast” …
I haven’t even made a dataset on the NAS yet …

4

First off, love the build.

However, you can not replace a bigger drive with a smaller one, that would be impossible.

5

That is indeed peculiar, AFAIK TrueNAS doesn’t use LVM for anything anymore.
If I were to guess. it’s related to what you did here:

6

I haven’t …
The 8TB was added to an existing 4TB disks pool, and is “just seen as” 4TB

1 Like
7

The LVM indicators are not “visible” on anything but a :

ls -lR /dev/disk/

Section

/dev/disk/by-uuid:

So it could be an artifact ???

Maybe you can give it a shot too, on a “Non Naughty” Bootdisk, and see if you have it too.

8

I do in fact also have dm-X links, but in my case I know they are related to residual swap partitions since my SCALE install is ancient.

I verified this by cross-referencing the UUIDs with lsblk -f, they all match the md124-md127 swap partitions that are, afaik, no longer in use.

9

Ahh …
I selected yes to swap , during install of DF too.

That’s probably it.

1 Like
10

Does anyone have a hint wrt. the replacement of a disk in a 6-disk RAIDZ2, as asked above ?

11

If you have a spare SATA port and the disk to be replaced is not totally failed:
Power off.
Install the new drive.
Power on.
In TrueNAS GUI, replace the failing drive (Storage > Pool > Status).
When done, TrueNAS will automatically offline the old drive.
Power off and remove the drive.

1 Like
12

Thank you for helping me again :+1:

This is mostly a learning excersize, on replace/resilver… I hope.

My RAIDZ2 pool right now consists of 5 x 4TB WD-Red (CMR), and in order to get it going for test i added a brand new Tosh N300 8TB as the 6’th drive (only 4TB is seen/used on the N300).

Because i knew i had a new WD Purple 4TB (CMR) in the summerhouse, i had already taken the decision to use that one instead of the 8TB N300, once i got there. So the pool is still empty/unused, I haven’t even made a single dataset on it yet. If i loose it, no harm is done.

Instead of destroying & recreate the pool, I thought i might get a “replace drive” learning experience, by removing the 8T and install the 4TB instead.

As it’s a RAIDZ2 , and i’m just replacing one drive. I suppose i can just swap the drives ?.

I wrote these steps in an above post … Hoping someone would help me out with step 4 and on …

Step :
1: Power off the NAS
2: Replace 8TB N300 with the 4TB WD Purple (CMR)
3: Power on the NAS
4: ???

So this would be step 4

In TrueNAS GUI, replace the failing drive (Storage > Pool > Status).

Edit:
What would be the reasoning for adding the disk to a spare SATA plug, if i have just one failed drive in a RAIDZ2 ?

13

If the old drive is failing but has not completely failed (i.e. it is still readable, but possibly unreliable), keeping it while resilvering maintains full redundancy.

1 Like
14

Awesome build! I have the same mobo, is your CPU fan annoying as well?

15

I have noticed it at one time, but I haven’t used it much yet …
I have some fans on the way, that was recommended on STH.
https://www.aliexpress.com/item/32761590665.html

I already use one for my Linux server setup (N100) - These are very silent.
But only ought one, the first time.

16

I have successfully replaced the 8TB-Tosh with a 4TB-WD-Purple
That was a head-scrather (for a newbie) …

1:
I replaced the Tosh w. the WD-Purple , and the system booted with a : Pool degraded message , and a “found new disk”

2:
I tried to ad the disk to the Rust-Pool1 , but i failed miserably every time.

3:
Then i fumbled around , and found some menu under (I think)
Storage → Topology → Rust-Pool1 → Manage devices
There i could "Click on the “Offline Disk” , and chose : Replace
I then selected the “Newly discovered” 3.64TB disk (WD-Purple)
And then it briefly looked like this:

replace-disk
replace-disk1919×866 80.9 KB

Now the WD-Pur is seen as sdc

tosh-replaced-w-wd-pur
tosh-replaced-w-wd-pur1919×862 116 KB

And the pools now looks like this:

pool-status
pool-status1651×430 97.4 KB

When i had the box open , i also replaced the “Stock EPYC Fan” with the one mentioned above.

Now all i have to do is :
1:
To switch the NAS IP address to “static”

2:
Review the Secutiry settings on BMC & Truenas
I’m quite sure there is a BMC update that fixes a nasty "ssh bug.
Unfortunately the solution was to disable the ssh adminuser access :face_with_symbols_over_mouth:

3:
Maybe crossflash Bios with the M0 version.
Might not …

But besides the above : I now have a working backup NAS

17

I noticed the WD-Purple changed device from sdc to sde, after a reboot.
Is the /dev/sd? assignment always that random ?

I suppose it doesn’t matter as TN prob uses UUID or SN for recognition.

18

Yes to both.