Current version: TrueNAS SCALE 25.04.2.6 Fangtooth
Upgrade target: TrueNAS SCALE 25.10.4 Goldeye
First attempt failed with:
Checksum mismatch for rootfs.squashfs
Second attempt after Download Updates + Apply Pending Update failed with:
unsquashfs failed with exit code 1
xz uncompress failed with error code 9
failed to read/uncompress file /tmp/…/usr/bin/mstreg
After reboot:
freenas-boot ONLINE
tank1 ONLINE
ssd_apps ONLINE
No known data errors
Apps still working
I’m guessing here based on what little you have said, but this could be hardware related; Either bad RAM or a failing boot drive.
Check and post the smart details of the boot drive with sudo smartctl -a /dev/sdX Replacing the X with whatever letter your boot drive currently has.
It would also be good if you run memtest to verify stability, at least overnight.
If that doesn’t lead you to the cause then you’re really going to need to post full hardware details. Without that it’s difficult to offer constructive input.
Thanks for reply! See the smart-test result below.
Boot drive SMART check completed.
Boot drive:
Crucial MX500 500 GB
Model: CT500MX500SSD1
Firmware: M3CR043
Power-on hours: 11487
SMART overall-health: PASSED
Short self-test:
Completed without error
Extended self-test:
Completed without error
Important SMART attributes:
Reallocated NAND blocks: 0
Reported uncorrectable: 0
Current pending ECC: 0
Offline uncorrectable: 0
UDMA CRC errors: 0
Program fail count: 0
Erase fail count: 0
Write error rate: 0
Temperature: 26 C
Percent lifetime used: about 3 %
There is a warning:
“ATA error count 0 inconsistent with error log pointer”
but ATA Error Count is 0 and the log entry appears to be from power-on lifetime 0 hours.
Boot drive does not appear to show obvious SMART problems. I will plan an overnight memtest next.
if everything comes back clean, you could always backup the config file & manually install the new version.
When you have no real idea, you blame “errors” in memory or disk.
Real memory or disk errors show up constantly/randomly, not only during one specific operation.
Previous TN releases had a bug in the installer used in the GUI.
Beyond ensuring that you have enough free space in the boot drive, you may need to resort to do a cold ISO install and reapply your saved config.
Boot pool has plenty of free space:
freenas-boot 464G size, 30.5G allocated, 433G free, 6% capacity, ONLINE.
Boot drive SMART:
Crucial MX500 500 GB
SMART PASSED
Short self-test completed without error
Extended self-test completed without error
No reallocated NAND blocks
No reported uncorrectable errors
No pending ECC errors
No offline uncorrectable errors
No UDMA CRC errors
So the boot drive and boot pool do not currently show obvious issues.
I agree that to blame error on memory RAM is not logical. The system is othervise stabie. No errors at all. Some few checksum-errors on disks have occured.
I verified the manual update file locally.
Official TrueNAS-SCALE-25.10.4.update.sha256:
a89d4c6c8c40e1ba1456e505b5995484e68d6f4b9d82a23252d6e217631a9301
My downloaded TrueNAS-SCALE-25.10.4.update:
a89d4c6c8c40e1ba1456e505b5995484e68d6f4b9d82a23252d6e217631a9301
So the downloaded .update file matches the official checksum.
Manual update still fails with:
Checksum mismatch for ‘rootfs.squashfs’
Automatic update and pending update also failed earlier with checksum/unsquashfs/xz errors.
Boot drive:
Crucial MX500 500 GB
SMART PASSED
Short self-test completed without error
Extended self-test completed without error
No reallocated/pending/uncorrectable/CRC errors
Boot pool:
freenas-boot ONLINE
464G size, 30.5G allocated, 433G free, 6% capacity
After reboot:
freenas-boot ONLINE
tank1 ONLINE
ssd_apps ONLINE
No known data errors
Apps are running normally.
This makes me suspect an installer/update-process issue rather than a bad download or boot-drive space problem.
maybe is another dead end… but have you tried to update to a minor version, like the 25.10.0 before try the update to the latest?
Any data that passes through failing zones of memory can get silently corrupted, that includes any data the system processes as it prepares a new boot environment. If that happens the data will be corrupt every time it’s read, because it was corrupt as before it was committed to disk. This will as far as I know not show up as ZFS errors. This is how bad memory can kill ZFS pools.
Given how little effort is required to set up and test the RAM it’s a no-brainer to run a test overnight. But I have no horse in this particular race, you spend your troubleshooting time however you like.
Ok, I will do that test later this week. Just to rule out a RAM fault.
It was RAM failure!! 6 errors on first pass.
Changed RAM and runed a new test:
Result: FAIL
According to Chatgpt I cannot run a Truenas-server even on only one failure.
I shall try to change from 1600 to 1333 Mhz (if the Motherboard allows that) and run a new test.
I concur with the chatbot output. Any memory error is so severe that you should not be running the server with that present.
@winnielinnie Here’s another for your list…
Slow down! I was about to add another to the list, this time of an ECC RAM stick failing, and now this happens! 
For anyone out there who has RAM in their NAS servers, there’s one way to never fail a memtest: Don’t run memtests.
I am pointing at you and laughing. Again. At this point I think you’re trolling since you have this overzealous response anytime someone suggests a memtest.
Please have a look here and finally drop this crusade against memtests.
It’s likely that the failing RAM is behind your update issues, but it’s not a sure thing. Sometimes errors have multiple layers.
In your case you now know that you have (had?) RAM issues, likely for some time. That leaves you with the delicate issue of dealing with the fallout; you don’t really know what data has been corrupted by the faulty RAM.
When you finally get working RAM in the system that passes extended testing, I recommend that you get a fresh TrueNAS config saved and that you then reinstall TrueNAS. You simply don’t know what havoc the RAM has caused in the OS. (careful not to overwrite your data drives during install!)
The other issue is with the data-pools. That’s more precarious since the width of corruption there can be more difficult to get a grasp on, much less address. Unfortunately, if you have backups and those backups are based of the server with bad RAM, you may find that all copies of some data may exhibit corruption.
There’s also the possibility that you may have future “exciting” file system errors sitting around waiting for you to trip on when just the right read or write happens.
I changed RAM Mhz from 1600 to 1333. And after Memtest runed 4 pass I got PASSED.
Now I have updated Truenas from Fangtooth to Goldeye 25.10.4 with success. And also migrated the apps-pool from Tank1 (harddisks) to ssd_apps (SSD-disks) with the new feature in Goldeye with success. From now on I think its very important to test RAM before using it, specially if you have bought used ones.
How dare you. Those with “deep digital systems knowledge” should chastise you for spreading this blasphemy!
This is possible, especially without ECC RAM that can catch bit flips before allowing new data to be written to disk.
I show an illustration of “hidden” corruption that not even ZFS or scrubs will detect. According to ZFS, the JPEG with garbled pixels or random lines is “healthy” because it matches the stored checksum on the disk.
At least you’ve found the issue, and have a temporary fix with downclocking. This being DDR3, it should hopeflly be possible to replace it at acceptable cost.
Fortunately, it seems the problem was the speed set on the RAM in the BIOS.
