That’s why I use Oracle VirtualBox for all my mission critical VMs.
3 Likes
Suggestion for iX staff: you should read up on the prototyping model that Firaxis used in developing Civilization 4, by creating a pre-Alpha group of people hand selected from the community to test and feedback in the years prior to game release. There’s an hour long interview somewhere on the internet from GDC 2006.
You are all people of culture so I’ll take it as read that you know Civ4 in is the best designed version of civ.
That’s probably the sort of model that works best for a community technical advisory council.
2 Likes
As if it is something bad. Joel Spolsky wrote about it almost 25 years ago (take a look at section 5).
1 Like
They do offer nightly versions, but the community testing could certainly be made more structured.
Of course, this would also require reliable testers - I have often found that there is a lot of communicated demand for this but then when it’s finally offered few testers actually show up at the roll call.
You really don’t think much of users if you think that we are all so self-centred as to only be able to comment about our own specific needs (though of course that is what you will get more of because that is what we know).
Some of us have both enough technical knowledge and real-life experience and empathy to be able to predict what other users might want.
But despite these insights being available in the user community, there appears AFAICS to be literally zero interest from anyone in the TrueNAS staff in tapping into it.
The only “interest” (and the air-quotes seem fully justified) is a cynical ploy by marketing with a fake advisory council.
If we were talking about features like robust AD connectivity instead of what we have today were RID sometimes fails and somehow ID’s get auto generated into the TDB range and all heck breaks loose…
You might have more than 20% of people interest in quality improvements like that… there is also the fact that AD backend is harder than ever to implement because of deprecation of the GUI for that in Windows server it would be REALLY nice if there was an automated backend to auto populate NFS IDs in AD based on the SIDs in AD so AD could be the authoritative source of IDs in all cases.
Indeed. And to add, that quote from iX above really calls for some self-reflection…
-
users ”not wanting change” is in my experience not true in general, but if it were in this case, why could that be…? Perhaps a review of past changes could provide some clues (yes I’m making a point here and not speaking hypothetically)
-
users generally preferring bug fixes, perhaps says something about the overall quality of the product as it stands right now? Perhaps should be taken seriously and not brushed aside as background noise which only gets in the way of pushing new features.
4 Likes
The vocal minority of a few upset that a threadbare, minimal, weak VM hypervisior function got dropped and replaced with a powerful platform that would take everyone into the future is testament to that myopia!
I mean, it’s sad that VMware dropped the ability to run OS/2 as a guest OS – but at some point, technology leaves the past behind.
And, some people get mad; that’s not new either, e.g.:
1997 User big mad that Apple discontinued OpenDoc
But, leadership requires sometimes the realizing that futility of “pleasing everyone” and that “sometimes you have to make changes that disappoint some so that you may better serve others” and “the fear you should have is not disappointing a few but not delighting the many”.
Now that I’ve had a chance to experiment with Fangtooth and Incus, I’m pretty impressed. Harnessing Incus provides a way for TrueNAS to leapfrog into offering some really important functionality that makes TrueNAS virtualization – not a minimal functionality – but something actually competitive with leading players.
E.g., Incus has:
- A well-thought out API for automation (like PowerCLI)
- Existing tools for multiplatform remote management (like managing VMs in Workstation Pro on Windows, Mac, or Linux)
- STATEFUL snapshoting (like pretty much every other hypervisor)
- Host to host copy-move (sometimes even live migration) of VMs/Instances (like VMotion)
- Toolsets to migrate from other hypervisor platforms (like ESXi) to Incus
… just to mention the few I experimented with. There’s probably even more like clustering functionality.
It’s getting to the point where – with Docker Compose and Incus functionality built on top of a solid storage solution – I’d recommend TrueNAS pretty enthusiastically as an all around solution to lots of clients.
Great work. Don’t let the chronic complainers bother you.
Focus on Learning, Not Punishment
Celebrate Effort and Progress
Embrace Failure as a Prerequisite for Innovation
Encourage Risk-Taking (Calculated Risks)
Celebrate Experiments (Regardless of Outcome)
… even if not everybody in the community might do so.
2 Likes
Yeah so about that…
3 Likes
- Also, FWIW, Incus is dropped in the upcoming release.
Well, clients can be different. Although, as it was stated up in the discussion, 0% of iX paid enterprise customers use TrueNAS for VMs. Nuff said.
As someone who worked in Enterprise IT for c. 30 years, I can say with reasonable certainty that Enterprises wanting:
-
Virtualisation in their data centres will use a dedicated virtualisation platform and not use TrueNAS for virtualisation.
-
Virtualisation in their branch offices will usually try hard to centralise it to a data centre - but in the rare situation of wanting both storage and multiple virtual servers in branch offices they would IMO be more likely to choose something other than TrueNAS which would have better remote management capabilities.
As a Community of free TrueNAS users, we really do need to remember that pretty much all of the container functionality (Docker, virtualisation, LXCs) are developed at their cost for our benefit without it generating any significant extra revenue for them. (And since TrueNAS is aiming for FIPS (??) security accreditation, all these container functions may add considerably to the costs of making the NAS functions FIPS compliant.)
That doesn’t mean that their developments couldn’t be done better, but in the end they are done for our benefit and not theirs.
1 Like
We open source users would probably be surprised at how few resources iX allocates to the development and testing of the non-enterprise features of Truenas CE. Their frequent requests for Github and community help in these areas result from the need to allocate 80-90% of iX resources to the enterprise side in order “to keep the lights on.” The aggressive rollout of new non-enterprise features seems necessary to keep the user base engaged and growing at the fast rate experienced so far but there is the risk that too many changes and ignoring ease of use will cause growth to slow. Truenas CE is on an upward spiral of growth driven by aggressive new features but the growth is threatened to be unwound by a difficult migration path forward. A drop off in the rate of user base growth/adoption would be the only reason iX would depart from its current plans.
1 Like
More than impressive, actually amazing.
For example, my oldest TrueNAS lab box is a very old CPU:
Intel(R) Core™ i7 CPU 950 @ 3.07GHz
Cores: 4 cores
Threads: 8 threads
… which has some, very feeble virtualization features. The old TrueNAS virtualization would complain it could use only one CPU:
With Incus running VMs, I can setup and even run multiple VMs:
(Like, it can run the pfSense and Win10 VMs similtaneously! Ubuntu 16, however, seems too much for it - it’s running, but hung here.)
But, rather than have to manage through the TrueNAS web interface, I can peek into some of the Incus features, and see some awesome possibilities, like remote management of VMs from a Windows machine.
Install some stuff on Windows
PS C:\Users\JEverett> winget install LinuxContainers.Incus virt-viewer --accept-package-agreements
(1/2) Found Incus [LinuxContainers.Incus] Version 6.14.0
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://github.com/lxc/incus/releases/download/v6.14.0/bin.windows.incus.x86_64.exe
██████████████████████████████ 24.3 MB / 24.3 MB
Successfully verified installer hash
Starting package install...
Path environment variable modified; restart your shell to use the new value.
Command line alias added: "incus"
Successfully installed
(2/2) Found VirtViewer [RedHat.VirtViewer] Version 11.0.256
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://releases.pagure.org/virt-viewer/virt-viewer-x64-11.0-1.0.msi
██████████████████████████████ 80.3 MB / 80.3 MB
Successfully verified installer hash
Starting package install...
Successfully installed
Open new shell to get added programs/paths, try out some commands
PS C:\Users\JEverett> incus remote list
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| NAME | URL | PROTOCOL | AUTH TYPE | PUBLIC | STATIC | GLOBAL |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| images | https://images.linuxcontainers.org | simplestreams | none | YES | NO | NO |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| local (current) | unix:// | incus | file access | NO | YES | NO |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
SSH/Shel to TrueNAS to setup remote Incus management client
PS C:\Users\JEverett> ssh tn01
Linux TrueNAS01 6.12.15-production+truenas #1 SMP PREEMPT_DYNAMIC Mon May 26 13:44:31 UTC 2025 x86_64
TrueNAS (c) 2009-2025, iXsystems, Inc. dba TrueNAS
All rights reserved.
TrueNAS code is released under the LGPLv3 and GPLv3 licenses with some
source files copyrighted by (c) iXsystems, Inc. All other components
are released under their own respective licenses.
For more information, documentation, help or support, go here:
http://truenas.com
Warning: the supported mechanisms for making configuration changes
are the TrueNAS WebUI, CLI, and API exclusively. ALL OTHERS ARE
NOT SUPPORTED AND WILL RESULT IN UNDEFINED BEHAVIOR AND MAY
RESULT IN SYSTEM FAILURE.
Welcome to TrueNAS
Last login: Wed Jul 30 11:06:26 2025
root@TrueNAS01[~]# incus config set core.https_address 192.168.51.11:8443
root@TrueNAS01[~]# incus config show
config:
core.https_address: 192.168.51.11:8443
root@TrueNAS01[~]# incus config trust list
+----------------------+--------+-------------+--------------+----------------------+
| NAME | TYPE | DESCRIPTION | FINGERPRINT | EXPIRY DATE |
+----------------------+--------+-------------+--------------+----------------------+
| SYL-VM-JELAB-UBU24.1 | client | | 23d305cbffaa | 2035/07/16 18:31 EDT |
+----------------------+--------+-------------+--------------+----------------------+
root@TrueNAS01[~]# incus config trust add win11-desktop
Client win11-desktop certificate add token:
eyJjbGllbnRfbmFtZSI6IndpbjExLWRlc2t0b3AiLCJmaW5nZXJwcmludCI6ImEyOWE5NjYzNDQ2OTI3NmE1YmVhZmY1Y2YxNGM0ZDE4ZGRkYjgzOGE4OWRjOGRkZjZjYTYxNTgxZWFhYTM5MWUiLCJhZGRyZXNzZXMiOlsiMTkyLjE2OC41MS4xMTo4NDQzIl0sInNlY3JldCI6IjEwYTQ3NDk3YTlhZGE0ZWRlODU0NzIwMGEyNDhkODZiNTM4ZjVjMjE5NjM2OGRjZDg2Mzk5M2ZiOGU3MDk3ZGIiLCJleHBpcmVzX2F0IjoiMDAwMS0wMS0wMVQwMDowMDowMFoifQ==
root@TrueNAS01[~]# incus config trust list
+----------------------+--------+-------------+--------------+----------------------+
| NAME | TYPE | DESCRIPTION | FINGERPRINT | EXPIRY DATE |
+----------------------+--------+-------------+--------------+----------------------+
| SYL-VM-JELAB-UBU24.1 | client | | 23d305cbffaa | 2035/07/16 18:31 EDT |
+----------------------+--------+-------------+--------------+----------------------+
| win11-desktop | client | | 50601c21520d | 2035/07/28 12:45 EDT |
+----------------------+--------+-------------+--------------+----------------------+
Back to Windows to connect with this trust
PS C:\Users\JEverett> incus remote add TrueNAS01 192.168.51.11
Generating a client certificate. This may take a minute...
Certificate fingerprint: a29a96634469276a5beaff5cf14c4d18dddb838a89dc8ddf6ca61581eaaa391e
ok (y/n/[fingerprint])? y
Trust token for TrueNAS01: eyJjbGllbnRfbmFtZSI6IndpbjExLWRlc2t0b3AiLCJmaW5nZXJwcmludCI6ImEyOWE5NjYzNDQ2OTI3NmE1YmVhZmY1Y2YxNGM0ZDE4ZGRkYjgzOGE4OWRjOGRkZjZjYTYxNTgxZWFhYTM5MWUiLCJhZGRyZXNzZXMiOlsiMTkyLjE2OC41MS4xMTo4NDQzIl0sInNlY3JldCI6IjEwYTQ3NDk3YTlhZGE0ZWRlODU0NzIwMGEyNDhkODZiNTM4ZjVjMjE5NjM2OGRjZDg2Mzk5M2ZiOGU3MDk3ZGIiLCJleHBpcmVzX2F0IjoiMDAwMS0wMS0wMVQwMDowMDowMFoifQ==
Client certificate now trusted by server: TrueNAS01
Test out Incus commands from Windows client, set remote context
PS C:\Users\JEverett> incus remote list
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| NAME | URL | PROTOCOL | AUTH TYPE | PUBLIC | STATIC | GLOBAL |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| TrueNAS01 | https://192.168.51.11:8443 | incus | tls | NO | NO | NO |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| images | https://images.linuxcontainers.org | simplestreams | none | YES | NO | NO |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
| local (current) | unix:// | incus | file access | NO | YES | NO |
+-----------------+------------------------------------+---------------+-------------+--------+--------+--------+
PS C:\Users\JEverett> incus remote switch TrueNAS01
PS C:\Users\JEverett> incus remote list
+---------------------+------------------------------------+---------------+-------------+--------+--------+--------+
| NAME | URL | PROTOCOL | AUTH TYPE | PUBLIC | STATIC | GLOBAL |
+---------------------+------------------------------------+---------------+-------------+--------+--------+--------+
| TrueNAS01 (current) | https://192.168.51.11:8443 | incus | tls | NO | NO | NO |
+---------------------+------------------------------------+---------------+-------------+--------+--------+--------+
| images | https://images.linuxcontainers.org | simplestreams | none | YES | NO | NO |
+---------------------+------------------------------------+---------------+-------------+--------+--------+--------+
| local | unix:// | incus | file access | NO | YES | NO |
+---------------------+------------------------------------+---------------+-------------+--------+--------+--------+
Manage my VMs remotely
PS C:\Users\JEverett> incus list
+---------------+---------+------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------+---------+------+------+-----------------+-----------+
| ElastiFlowVA | STOPPED | | | CONTAINER | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Ubuntu16-INC | RUNNING | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Win10-test | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| pfSense-INCUS | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
PS C:\Users\JEverett> incus stop -f Ubuntu16-INC
PS C:\Users\JEverett> incus list
+---------------+---------+------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------+---------+------+------+-----------------+-----------+
| ElastiFlowVA | STOPPED | | | CONTAINER | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Ubuntu16-INC | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Win10-test | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| pfSense-INCUS | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
PS C:\Users\JEverett> incus start Win10-test
PS C:\Users\JEverett> incus list
+---------------+---------+------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------+---------+------+------+-----------------+-----------+
| ElastiFlowVA | STOPPED | | | CONTAINER | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Ubuntu16-INC | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| Win10-test | RUNNING | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
| pfSense-INCUS | STOPPED | | | VIRTUAL-MACHINE | 0 |
+---------------+---------+------+------+-----------------+-----------+
PS C:\Users\JEverett> incus console Win10-test --type=vga
Bam, I’m running a Windows 10 VM on TrueNAS – with UEFI boot, Secure Boot, and TPM on a machine that doesn’t even support that on bare hardware – and with a remote console on my Windows host:
And, it’s easy. And, there’s Linux and MacOS client equivalents. I’ve run TrueNAS, except on some old lab boxes, under ESXi – never thought to use the the TrueNAS virtualization functions. This is a game changer. And that’s just one feature of many waiting to be tapped.
3 Likes
Incus was just short lived experiment and it will be permanently removed from Truenas.
VMs and system containers will be managed by libvirt which is the same backend that was used for VMs in 24.10 and previous.
I am also Incus fan and I also hoped for Incus integration but this is what it is.
Yeah so about that…
2 Likes
…which is frankly a completely idiotic decision, and means that most of the 25.04 development cycle is just thrown out the window. “We half-assed it to meet our arbitrary and self-imposed six-month release cycle, it was a hot mess, so let’s throw it out the window and go back to what we were using before.” In what universe does this make sense?
Does iX have any plans at all?
7 Likes
Well, I already pissed of kris with my “tight lipped” comment, so I will gladly leave this to other big guys here. 
I am just glad I finally know how it really is and that its permanent removal and not just temporary. Even though I really looked forward to Incus. 
I guess I just wonder what about Incus is bad or why they think its not suitable for Truenas. 
1 Like
I initially read the last word there, missing the s, and with a slightly different letter at the end. Maybe I’ve been watching too much Nuxanor on YT.
But in any event, though I can’t fathom why, it seems iX are on a mission to reduce trust levels among their users to an absolute minimum.
I guess in a universe where you’ve already released “FreeNAS Voldemort,” anything can make sense.
2 Likes