Thanks for the hint @NickF1227 - zfs send/receive does the trick
Edit: I noticed the zvol had a Provisioning Type: Sparse after using send/receive to restore it. I was able to set it back to Provisioning Type: Thick using zfs set refreservation=auto pool/zvol
Lol its funny that you mention that. This is an example of where TrueNAS GUI defaults differ from default ZFS. Sort of a guard rail because sparse can be dangerous.
This box should take care of that problem for UI supported replications (Basically non â.â folders). There are of course other ramifications (mostly positive) of this, but its checked by default when you use the wizard.
But putting my sysadmin hat on, my Sensei always taught me to âuse sparse volumes for all of my storage volumes for everythingâ and âdo the damn math yourselfâ and âalso donât let anyone else f* with the SANâ. This can be non-trivial for all vendors.
So I actually personally prefer the ZFS behavior because itâs what I am used to after being on the other side⌠but I understand why the TrueNAS default is different and its better because of it.
I imported my non-Instance VMs via the UI and they are sparse, but the source ones were sparse as well.
root@m50[~]# zfs list | grep .ix-virt
fire/.ix-virt 640G 3.04T 140K legacy
fire/.ix-virt/buckets 140K 3.04T 140K legacy
fire/.ix-virt/containers 140K 3.04T 140K legacy
fire/.ix-virt/custom 640G 3.04T 140K legacy
fire/.ix-virt/custom/default_bryce-64k 11.6G 3.04T 5.63G -
fire/.ix-virt/custom/default_casey-64K 47.0G 3.04T 36.9G -
fire/.ix-virt/custom/default_casey-d-16K 77.3G 3.04T 77.3G -
fire/.ix-virt/custom/default_dreadnought-64K 15.6G 3.04T 9.88G -
fire/.ix-virt/custom/default_michael-64k 16.8G 3.04T 13.3G -
fire/.ix-virt/custom/default_orion-64k 5.32G 3.04T 3.81G -
Yes,
The link is:
In essence though, youâd be wanting to do the exact opposite, which is fine, it means you can replicate, clone or move out.
And you want to then update the volmode and primary/secondary caches.
volmode gets changed to default, primarycache and secondarycache get changed to all
Its the volmode which will âactivateâ the zvol again after it gets moved/copied out of incus
I am so glad I use Proxmox for all my virtualization needs. I am dumbfounded that IX Systems would include Incus without more complete testing first
Sure, more mature virtualuzation product, BUT!
Proxmox is NOT an appliance!
Say whatever you must, but when you need to configure BASIC stuff like IOMMU, or an UPS, in the command lineâŚ
TN will get to be an excellent virtualization platform within a year!
Proxmox has zero plans for being able to restore your ENTIRE configuration with ONE file!
I rate IT people by how fast/easy they can restore any service!
I am not sure of the benefit of TrueNAS being an appliance when their latest major release was a beta product at best.
It hasnât in fifteen years, what makes you think it will in the next?
Edit: OK, it hasnât had virtualization for fifteen yearsâbut it has for at least ten. And I donât doubt some improvements will be made within the next year (that bar is pretty low with 25.04.1). But it wonât be in the same universe with ESXi, xcp-ng, or Proxmox.
Do you disagree with our Software Status page?
Where do you classify yourself?
Tbh, if people actually followed the Software Status page there wouldnât be any problems.
If you have production VMs you should simply stay on 24.10. If you donât and it breaks itâs your own fault for not waiting.
tbh the Software Status page is nothing more than iXâ âget out of jail freeâ card. They can push a new major release, aggressively promote it, belittle those who are hesitant to immediately upgrade (âwhereâs the fun in that?â), and when itâs a dogâs breakfast, âoh, we didnât recommend anyone use that release, just look at our software status page.â
Thereâs just no excuse for the sixth major release of software like TrueNAS to have new major features that are marked as âexperimental.â
In my opinion, This is FREE open source software. If you want rock solid then put up the dollars and buy it. I for one like being on the cutting edge. Experimental is what open source is all about.
Thatâs a cop-out. Thereâs plenty of free, open-source software thatâs at least as solid as anything on the commercial market. FreeNAS was, by and large, this way. TrueNAS CORE was mostly this way. SCALE (or now CE) has been a dumpster fire in this regard, largely because iX talks out of both sides of their collective mouths when it comes to who should upgrade and when.
Thatâs what nightlies are for.
Nonsense.
Agreed, and to add, thereâs plenty of commercial software that rests heavily on open-source.
Indeed, and it follows from the directly above.
Actually IMO this is the opposite of what should happen.
iX are now saying that running old-style KVM VMs alongside Incus is technically almost as simple as re-enabling the old functionality. If it is that easy, then why didnât iX deliver the experimental Incus technology alongside the old virtualisation technology in the first place.
Indeed, why donât iX have a strategy to deliver almost all technologies in this parallel fashion?
When I questioned whether it was technically possible to deliver Docker alongside Kubernetes, iX eventually admitted that it would have been possible to do this. This would probably NOT have been quite as simple as running them alongside - they might e.g. have needed to run Kubernetes in a Docker container. BUT iX said it would have been possible, but they chose NOT to do so.
This is a repeating theme - they could deliver a better migration path for new technologies, but they choose not to.
But perhaps what has happened with Incus should be the model for delivering all new technologies.
iX tries to implement all new technologies in parallel with old technologies that they will eventually replace (though there will be exceptions where this is technically impossible - e.g. you canât run kernels in parallel, or services which need to use the same standard ports).
If you do this, so long as the new technologies donât impact other services, it doesnât matter if the new technologies are âexperimentalâ because you can use as many following minor and major releases as you need to round out fully the technology.
Eventually the technology will be fully rounded out and mature, and you can then advise all users to migrate, ideally by providing user triggered migration scripts to do the heavy lifting.
Finally - or perhaps at the same time as point 3. - you can announce that the old technologies will be removed in the next major release, allowing a reasonable amount of time (say 6 months) for users to action the migrations from old technology to new technology.
Because this moves away from sudden-death migrations, I believe that it could actually result in faster adoption of new releases (which seems to be a metric that iX like to crow about). Certainly I delayed my migration to EE for as long as I possibly could in order to reduce the risks of my app migrations failing - and I doubt that I was alone in this - had they delivered Kubernetes and Docker infrastructure in parallel I would have moved to EE much much earlier.
I think you will find that Open Source is mostly about security, cooperation, synergy and longevity:
(Indeed I can give several examples - good and bad - from the open source world to support the above points.)
Then you are in the minority - home-labbers who just want to tinker may feel this way, but anyone who uses their TrueNAS system for real-life production (even home) wants stability not bleeding edge.
As I have pointed out above, introducing âexperimentalâ technologies does NOT have to be bleeding edge for existing users. Introducing them in parallel would allow existing technologies to remain rock solid whilst new technologies start bleeding edge and then mature into rock solid replacement technologies.
This has NOTHING to do with whether you pay or not - it is a business / technical strategy as to how you deliver new technologies that decides this, not $$$$.
I think that there are two or three issues here that need unpacking:
Chopping and changing containerisation technologies - iXâs track record of selecting technologies has not been great.
As I understand it they originally chose Kubernetes because of Gluster, only for Kubernetes to drop it. I am personally doubtful that there was a user requirement for Gluster - homelab users might enjoy playing with it, but non-enterprise production users probably donât need it, and if Enterprise customers want it then they would most likely stand up their own bespoke Kubernetes environments to run it and not run it underneath TrueNAS. I would be interested to see the iX research that shows there was a user need for Gluster rather than it beinga technology that the iX folks fancied playing with themselves.
With the benefit of hindsight, I think that we can all probably agree that this turned out to be a bad choice, and that it would have been better for everyone if they had gone with Docker in the first place.
It remains to be seen whether choosing Incus is a good long-term decision. I am unclear whether the âexperimentalâ nature of Incus in 25.04 is due to incomplete implementation by iX or poor technical choices by iX or shortcomings in Incus or a combination of these - and if it is âexperimentalâ due to shortcomings in Incus, then (again with hindsight) iX were at best premature in switching to it (insufficient research before taking this decision?) or at worst it may turn out that Incus will never be perfect and never a long term solution.
And not having been a TrueNAS Core user at the time, I cannot remember the details, but wasnât there a technology they switched to that was so bad that an entire release with it was scrapped?
The most disappointing thing about the choice of Incus is that iX do not seem to have learned anything from their mistakes with Kubernetes and before - they have simply repeated the same strategic mistakes yet again.
Parallel delivery - since they announced the return of the EE virtualisation in parallel with Incus in 25.04.2, it has become abundantly clear that they could easily have delivered the new âexperimentalâ Incus technology alongside the existing virtualisation, and that (yet again) they chose not to do so.
The most disappointing thing for me is that iX are again repeating the mistakes of the past - under pressure they eventually admitted that they could have implemented Docker in parallel with Kubernetes and that they chose not to.
âExperimentalâ as a strategy - IMO delivering âexperimentalâ functionality as a strategy is actually a good thing, so long as A) it is delivered in parallel with existing technologies; and B) it doesnât impact the reliability of all the other existing services.
In the case of Incus, I believe that iXâs intent was to deliver a rocks solid, fully formed virtualisation solution based on Incus, but time-scales and perhaps Incus functionality meant that they didnât achieve it - and consequently the âexperimentalâ tag appears to me to be a post-release normalisation of a poor quality delivery by iX and NOT a strategy.
So I would encourage iX to learn from this and to adopt âexperimentalâ delivery of new technologies in parallel with existing technologies as a deliberate and positive strategy - and I believe that this will benefit everyone: Users will get a better migration path and be able to stay current without as much angst; and iX will be able to take as much time as they need to deliver mature functionality, and see better migratuon stats for new versions than ever before.
Yes, I absolutely disagree with the Software Status page - but only because it does NOT distinguish between users who:
IMO due to the âexperimentalâ nature of Incus and the current lack of parallelism there needs to be a separate Classification grid for each of the above.
As and when 25.04.2 is delivered which can run existing VMs, then we can probably simplify back to a single grid.
Definitely âConservativeâ - my NAS is only for home use by myself and my wife - for backups, some shared files and media streaming, but nevertheless it is a PRODUCTION server that I will not put at risk of new, immature technologies.
I only migrated from DF to EE because of the automated migration deadline - and that deadline was a consequence of EE Docker functionality also being âexperimentalâ because it was incomplete.
Yes, so the recommendation is 24.10.2.2⌠. donât use LXCs yet.
Is that not appropriate advice?
Weâve gotten well off the original subject of the specific changes to virtualization.
While Kris and I discussed a bit about changes in development and Enterprise focus on the podcast, I recommend we start up a separate thread on that front if we want to continue that discussion.
Weâll be clarifying a few more things on todayâs episode as well.
Thanks.