TrueNAS Installation issue and some second thoughts

TrueNAS General
, ,
1

I have a Dell R320 with a couple of small, fast 900GB drives and a couple of 8TB drives, all of which I wish to shadow/mirror/RAID 1 (I think i have the number right). I have installed TrueNAS to a 250GB SSD from a USB bootable and that seems to have worked OK. Now, here’s the issue.

When I power on the machine, it runs through the self tests and gets to the numeric menu on the serial port (VGA). However, the Samsung display is, for some unknown reason, not displaying the first 6-12 columns. Not a problem as I know what the options are for the menu from screenshots on these very pages. However, it does not display the full IP address that I assume I need to attach to via a GUI (in this case, from an Alienware laptop with network cable plugged into the blade) and I need to be able to use the TrueNAS GUI interface to do the network set ups and the disk configurations, apparently…

From taking one of the Network options from the text menu, I suspect that the address is 169.254.0.2, so, if I plug my Alienware into the correct Blade NIC I should be able to reach http://169.254.0.2/ right?

Well, apparently not. I think the most likely address this is for is the iDrac port, but there are two other NICs at the back. Done the swapping the plug thing and none of them can be reached or pinged on that address from the laptop. The laptop doesn’t have a VGA port, so I am using the external monitor to connect to the R320.

Does anyone have a clue how to perform the configuration stage from here? To perhaps confirm the IP address and/or tips for how to open the GUI (I was thinking the Chrome browser would do it if it can see the network). Ooooh… Maybe I need to set the laptop NIC address to be in the same network as the blade NIC, thinking about it… I wonder if it is using the wifi address I have given it by default, even though I switch off the wifi in order to find the blade…

The other thing is, though, the blade has hardware RAID built in, so what is the need to run TrueNAS on this machine? Is it better than hardware RAID?

I do, however, want to run a cloud utility, probably the secure VPN to access it as cloud storage and possibly run other applications via VMs (do I need one for each role or can one VM perform many functions)? That may be the only reason for running TrueNAS - the apps/VMs - but I want to make sure I am on the right track.

Any help would be gratefully received, folks.

Cheers,
Alan.

2

Hi AlanL,

You’re likely seeing the 169.254.x.x IP because TrueNAS couldn’t obtain a DHCP lease, meaning it’s not yet on your local network. To access the GUI, either assign your laptop a static IP in the same range (e.g., 169.254.0.10) and subnet 255.255.0.0, or use the VGA console menu (Option 1) to manually set a static IP for TrueNAS that’s within your local network. This should let you reach the GUI via your browser.

As for TrueNAS vs hardware RAID—TrueNAS is designed to use ZFS, which offers better data integrity, snapshots, and flexibility than traditional RAID. It’s best to disable the hardware RAID controller and use each disk in JBOD mode to allow TrueNAS full control with ZFS. If your goal includes running apps or VMs, TrueNAS SCALE is a solid platform for that, supporting Docker and Kubernetes for added versatility.

Regards

3

Thanks for your reply. Hmmm, yeah, I did try setting a static IP on the blade in the home network range, but still could not ping it, no matter what port the cable was plugged into. I will have to revisit this and try to pin down the NIC hw address to the static IP I assign.

So there is good data reason for using TrueNAS as opposed to hardware NAS. That’s good to know. I am not sure why the previous owner decided on the two 900GB drives - way overkill for cacheing and not much use for storage. Maybe I use them somewhere else…

4

Hi Alan,

You’re definitely on the right track with your setup. For the IP issue, a 169.254.x.x address means your R320 isn’t getting an IP from DHCP. Try assigning a static IP on your Alienware in the 169.254.x.x range and connect directly via Ethernet to test GUI access. If that fails, hook up a temporary VGA-compatible monitor to get the full TrueNAS console view. As for RAID, TrueNAS prefers handling storage itself via ZFS, so it’s best to disable the hardware RAID and pass drives through individually (JBOD mode). And yes, one VM can run multiple services, but separating them by role improves stability and ease of management.

Regards

5

Thank you for that! I love the idea of one VM for a given service to ease management. Complexity massively multiplies possibility for error! That makes a lot of sense.

2 Likes
6

Well I finally managed to use the GUI. Try as I might, I could not get to see the iDRAC or NICs from the laptop no matter what addresses I used, so I bit the bullet and decided to hang the server on the network. Not trivial: I had to find my 16 port Ethernet switch in order to connect the three ports on the server. I also ended up moving my modem to make things easier when I have it all running and want to put it in a permanent place.

Anyway, to the GUI. Great, I could log in, see the memory, CPUs, but… No disks? Tried the old IT Crowd move - nope! The PERC was not visible. No idea why! Anyway, this evening I tried reseating it and it popped back into visibility! It might have moved slightly when I was putting the PERC battery connector back on, I guess, but I swear it was reporting that the battery was now OK so therefore it must have been able to see the controller… Meh!

Well, great, now I can see disks. It looks like things aren’t straight forward, though: I need to create pools of VDEVs(?) by the look of things, and it only shows me total sizes, not individual drives AFAICT. Currently, there should be two 8TB drives visible and two 900GB 2.5" fast SAS things (although for the life of me I cannot think what they are useful for - way too big for cache duties and not really fast enough; way to small for useful storage… Answers on a postcard, please…)

I am not playing with it tonight - I will look further into it tomorrow, I think. Brain hurts!

7

OK, I thought I knew why I was only seeing the two small drives: they are SAS, the big ones are SATA and the previous owner had them configured in one big virtual drive, which TrueNAS likely only saw the combined small drives as usable.

So I got rid of the virtual drives - great - the blade sees four drives! TrueNAS now sees nothing but the boot drive. Dammit!

A bit more reading… Apparently the PERC (6) controller is not a good option for TrueNAS. Not sure if I have the 6 variant, it displays as a 710 mini (no P that I could find).

Good news: you can change the controller (flash it) to IT mode and TrueNAS will like it!
Bad news: this is not trivial and I could brick the controller.

In other news, I have a T110 server, too, and a bunch of 3TB drives. Maybe I try putting FreeNAS onto that rather than the blade… Although I have not investigated its hardware yet - I may still have to flash it…

An option is to use the blade as a server with hardware RAID to serve disks with, but then what OS to use? I don’t have a Windows server licence and I am not sure it would allow me to use it as an alternative to OneDrive or Google Drive or any paid-for cloud services: I want to run my own.

The T110 I could then put FreeNAS on and, with at least one mirrored set up configured, it is not a huge disaster if it all goes badly wrong.

I thought it was too easy and I did wonder why the blade was selling at a fairly cheap price. It seems others have come across the PERC problem for FreeNAS, or is this the cynic in me?

Does anyone have direct experience of getting a PERC-based system and TrueNAS to play nice?

8

Replace “it’s best” with “it’s mandatory”, at least if this is not a playground system.

Also, disabling RAID mode on a RAID adapter is not enough. One needs an HBA with IT mode firmware.

2 Likes
9

Indeed - another reason this is not trivial, but at least that comes with the Fodeesha flash which I have been looking at.

I did wonder if making the blade set up one virtual disk per drive might work, so TrueNAS sees all four separately. It might not be ideal but is it workable (I don’t need superfast, but I do like data security)?

Oh well, I am out of time for this weekend.

10

If you mean to configure the RAID adapter such that it present the 4 drives individually to the OS, that is a bad idea.

Define “workable”. This is the one discussion that comes up time and again. Someone does this and things look fine. So they rush off and tell the world that this kind of setup “works”.

Nobody doubts that this is the case. But “works” in the context of a NAS means not to loose data under almost all circumstances. And the latter means to avoid any and all possible risks. Is it possible that the RAID adapter setup works even when a disk fails? Yes, of course. Does it present a higher risk than an HBA flashed to IT mode. Also yes.

The problem is that statistics, and risk is largely about statistics, is a counter-intuitive topic. It often evades common sense and gut feeling. That is why the standing recommendation is what it is.

Good luck!

11

Yup, that is what I was thinking. Yes, I know TrueNAS likes to see bare drives to be able to do anything with them, including monitoring, faster operations, etc. But if that works in the sense of allowing TrueNAS to configure mirrors for me, maintains data security in the event of one going off line and allows me to swap in another drive for recovery, then that is all I really need. What is it that makes this a bad idea? Do we have specifics (tried, tested, failed) or is this just “it’s not recommended so we daren’t do it”?

In essence, the virtual drive from PERC would be like (not the same as) an HBA, so it will be different, possibly unpredictable. If it allows TrueNAS to shadow (does my age show in this?) those drives and let me run some cloud, share and streaming type apps in VMs then that is all I need.

I suspect the previous owner got nervous at having to flash the controller and gave up on the idea. Is the risk of bricking the controller and losing the lot greater than the risk of operating TrueNAS slightly out of the box? That’s a lot of unknowns! On the plus side, I have some disks spare, I can do a bit of testing before trying to flash the controller.

Another thought… So the PERC needs flashing for TrueNAS, but is there a disk controller available that fits in the Dell blade that TrueNAS likes (without having to potentially brick it)?

12

If this thing has some pcie slots open, get an LSI SAS HBA model 9211-8i, flashed to IT mode, off ebay or Amazon, install it and go. Make sure it comes with the breakout cables, you’ll need em, although if you have a mishmash of SAS and SATA drives I don’t know how that looks on the card/cable side…but since the card is natively mini-sas it may not be super complicated, maybe one x4 sata cable and one x4 sas cable; sellers can probably piece it together. Shouldn’t be very expensive either. Toss that perc in the bin.

That card model, the 8i suggests, only supports 8 drives because it’s a 4x2 breakout system with 2 connectors onboard. You can go upscale and get a 16 disk controller but it doesn’t sound like you’d need that here. Those cards are well known in the *BSD/Linux world and the drivers are bulletproof. They’ll either work perfect or not work at all.

Also some corrections and considerations. Freenas is not a thing, but it was probably 10 years ago. Been Truenas for a long time. :grinning:

Secondly, the concept of vdevs and pools. Pools consist of vdevs, and vdevs consist of drives. It’s a hierarchy. You can mix different types of vdevs within pools. With your setup I’d probably have 2 pools. One for the little drives and one with the large drives. So there would be 1 vdev in each pool, consisting of mirrored drives. But you could make both those vdevs into a single pool, keeping in mind that your pool could be slower as it must wait for the slowest drives.

Third and lastly, let’s talk VM’s. Half the genius of Truenas these days is Docker, which has an app catalog with no less than 200 apps in it now. You’ll want to look for stuff you want to run in the app catalog first. It’s simple and easy. This precludes your perceived need to run apps in a VM, but that is still an option. Not everything is in the app catalog or has a Docker equivalent.

1 Like
13

I’m pretty sure I followed this (at least as one of many) when I went through almost the same as you a fair time ago now.

The issue with two controllers fighting for control is, if one does something the other doesn’t like while it is doing something, it is your data that gets toasted. The controllers couldn’t care less that your data isn’t there any more but you will. Trust me on this one, I lived it.

This page also helped, his YT page is very good too.

Two 900GB SAS drives would be perfect for a full blown UNIX install operating as RAID1. Database transactions would be in that domain.

The R310 should/might have an iDRAC which you can load in a browser from an admin station and view the machine as if you had a monitor, keyboard and mouse connected locally. This needs to be setup but you can then cold boot remotely.

Good luck with it.
Cheers,

1 Like
14

Passing virtual drives does NOT maintain data security because the controller can reorder writes and lie to ZFS as to what is queued for write and what is actually committed to permanent storage at any given time. This behaviour is known to result in pool corruption and data loss.

2 Likes
15

Thank you, that makes a lot of sense. I understand more now. There is, indeed, an iDRAC port with a bizarre IP address, but accessing it via the T110 server and it looks like it might be ready to install TrueNAS right away, so I am going to use the system I built on the Dell and put that SSD into it, along with the two 8GB drives. If I can get that up and running then I can spend a bit more time on getting the Dell running.

16

Thanks for this - it makes things a lot clearer. There may well be a PCI slot open and I was wondering what to do with that, but I will look at that later.

Not sure why FreeNAS got into half the post and TrueNAS in the rest, but that’s my brain for you! They don’t make it easy, though, with FreeNAS, TrueNAS Core and TrueNAS Scale and that’s before we get into versions. :smile:

Yeah, I was thinking of two lots of mirrored disks, but I will use larger than 900GB for the other pool. I am going to put the 8TB ones in the other server which I think will run up with less effort; then I can work more on the Dell.

And yes, the VMs thing is useful. Docker - is this an environment in which to run lots of apps (Jellyfin, “MyCloud” (insert correct name that I don’t know here), audio streamers, etc or - like VMs - would I put one app into one instance of Docker? This is still very relevant to the T110 server, of course!

17

The fact that you very probably will lose data.

When there is a layer of abstraction like a RAID controller between the ZFS implementation in the kernel and the disk drives, experience has proven that all the fine data safety mechanisms in ZFS are very likely to fail in hard to debug circumstances.

People have lost their entire pools over and over - just don’t use a RAID card.

1 Like
18

I’ll oversimplify it. Docker is a thing that manages images, and these are write-once, run anywhere images, regardless of platform. Since Docker is built in, you hit the app catalog and get a list of images that have been tuned somewhat for TrueNas. Usually you can click to get one, save it and you’re off to the races. Other times, you have to feed it some configuration before it’s ready to go. From then on Truenas will manage those images and make their services available to you; letting you know when updates are available, updating them with your permission and even rolling them back if they fail an upgrade. There are hundreds, so you can probably find whatever you want to set up in the app catalog. You mentioned cloud hosting; nextcloud is a popular app in the catalog. Audio streamers and video stuff, take your pick. While you’re tinkering and researching, I think you can even look at the app catalog outside of TrueNas now over at TrueNAS Apps - Docker Compose and Containerized Linux Apps; you’ll probably want to look at the community apps.

1 Like
19

Well, the good news is that I got the T110 up and running with an SSD disk for TrueNAS, 2x8TB drives and 2x3TB drives. Further, the 8TB drives are now mirrored, which is what I wanted.

Alas, one of the 3TB drives is not available to TrueNAS:

image
image1063×741 56.7 KB

image
image1465×179 12.4 KB

It looks like there is a RAID partition from the previous owner, but my Un*x is rusty - how do I remove those partitions on sdb1 and sdb2 and make the rive available to TrueNAS?

Then there is the issue of how to connect to the TrueNAS system from Windows - I suspect there’s a step I am missing here. TRUENAS is visible from windows, but I can’t connect to it, presumably because I need to send it authentication. But that’s for after I get the disks up and running…

20

You can always nuke the partition table with zeroes; something like
sudo dd if=/dev/zero of=/dev/sdb bs=1M count=40
Breathe slowly, deeply, and triple check your command before hitting Enter…
But first, look whether there is a “Force” tick box when manually adding a drive to a vdev, or a “Wipe Disk” button, somewhere.

Have you defined a SMB user and his password? You don’t connect as ‘admin’ to the share.