Migrating from TrueCharts

[Protopia]

@Captain_Morgan

@Protopia

You should provide a dropdown to allow a user to define what type of user they are from an upgrade perspective which would limit upgrades offered to those appropriate to that type of user;

Not so sure…we’d have to automate it and users can wrongly classify themselves. Has anyone else managed to do this?
We’d really prefer users read the release notes 1st.

What an unbelievable response:

• The dropdown could have an info tooltip that explains what “Mission-critical”, “Conservative”, “General”, “Early Adopter” and “Tester” mean.
• Asking if anyone else has done this is a deflection. iX can write OpenZFS enhancements, and can create a great UI to docker, but they are suggesting that they can only code a dropdown and a query to match the value of this dropdown to a tag on a release if someone else has demonstrated it is possible first?
• Yes - of course everyone would prefer that every user upgrading has read the release notes first. But if you are non-technical, and non-skeptical, and have upgraded successfully umpteen times in the past without bothering to read the release notes every time and have grown to trust TrueNAS upgrades as being trouble-free, then if you are not aware that this upgrade has significantly greater risk than previous ones (because the announcements do not say anything about this upgrade being different), it is I think understandable that they may go ahead with the upgrade without reading the Release Notes and then get burned. And even more so when iX enthusiastically encourages you to upgrade despite a single and IMO somewhat obscure web page that states the software is only suitable for testing and not for production use even by Early Adopters (much less non-technical users who won’t know how to fix things that break and would almost certainly classify themselves as “Cautious” if asked).

I may be wrong about this, but based on the ongoing rah-rah encouragement to upgrade to a version that they state themselves to be suitable only for “testing” and the ongoing dismissal of the risks and apparent disregard for the risk of broken apps, definitely gives the appearance is that iX just want to boost their upgrade numbers without caring about (non-technical) users getting their systems partially-broken by the upgrade.

@Captain_Morgan

@Protopia

You should be much more explicit in your announcements about the types of users who should upgrade and not just give a bland link to the Software Status Page;

No, we can’t do this. We really don’t know the quality of a particular version for 1 - 8 weeks after its released and we have feedback. The Software status page lets us do a quality assessment asynchronously.

Rubbish!!! In each and every announcement - blog, T3 podcast, forum posts, you can state what your current view of the quality is and say e.g. “we would like you to upgrade but please be aware that as of today we think our software is only suitable for Testers and not suitable for Early Adopters, General, Cautious or Mission-Critical users.” But you choose instead to continue to encourage everyone to upgrade even though you know full well that the software is not yet stable enbough for non-technical users to upgrade to and that their systems may end-up partially broken.

@Captain_Morgan

@Protopia

You should stop making the only measure of the success of a release the total number of people who have installed it, and start breaking this down into successful and unsuccessful upgrades and by new installs, upgrades from betas/release candidates, and normal upgrades (and if you have the above suggested dropdown of user type) by this user type;

We don’t. We see the number of upgrades as evidence of demand. We monitor bug reports and forum requests very carefully. After that we review changes to the software status page.

Rubbish!! This is completely back to front. Demand is driven by you enthusiastically encouraging all users to upgrade despite knowing that the software is only suitable for “Testers”.

@Captain_Morgan

@Protopia

If you are going to do anything that risks stuff not working afterwards or that requires users to take preparatory actions, then you should do a pre-upgrade sanity check and warn the user if any part of the upgrade is going to fail and break something;

Difficult to do… warnings go away after software quality has improved, but changing the software will worsen quality. It would also slow development. We instead invest in good “rollback” solutions, release notes and the software status page.

We have finite resources and have previously had lots of negative feedback on warnings being annoying.

Rubbish!!! You have stated here that you can improve the migration process without needing to update the TrueNAS code by changing the individual app migration scripts downloaded during the migration. A pre-upgrade check that does a trial migration and gives warnings only when some apps don’t successfully migrate (failed iX app migration, TrueCharts Apps, users’ bespoke apps) would not be at all annoying - if all is good, a user doesn’t get a warning, and if they get a warning they will be able to choose to abort the upgrade for the moment or to continue knowing they will have fixes to do afterwards. You would also get feedback on broken app migrations BEFORE users end up with broken apps rather than only after that happens.

@Captain_Morgan

If we provide the software status page on the update page and perhaps a link to the release notes, would this be sufficient? We don’t want static information in the UI.

No - it is a start, but you need to stop enthusiastically encouraging all users to upgrade when you know full well that the software is not yet suitable for anyone other than “Testers” and you need to start being up-front and explicit in all EE upgrade-related communications about who the upgrade is suitable for and the need to review the Release Notes and do the pre-requisite preparatory actions.

@Captain_Morgan

@Protopia

It is my view that software should only be considered ready for a full release when it is suitable for use by Early Adopters, and you should delay full releases until this is the case. According to the Software Status Page you linked to 24.10.0 is NOT even suitable for Early Adopters who should remain on 24.04.2.3, and since you state it is only suitable for “Testers” that clearly suggests to me that this software is still in a test state and not ready for release. (I would only ever expect Betas and RCs to be in the Testers row.)

I would consider that the “Conservative” view.

Rubbish!!! iX is quite literally stating that currently this version is suitable only for “Testers”. That doesn’t mean that other users can’t make an informed decision about whether to upgrade their systems anyway, and the community has a reasonable number of technically-competent users who would be prepared to do this. And indeed, some of these users are the ones who installed the RCs and gave feedback.

However, in retrospect, whilst I still believe that in general my statement above is still the case, but for this release whose update is significantly more complex then providing that you give appropriate warnings I think that once you have had all the feedback you can from RCs you do probably need to produce a .0 version to get more of the technical users to upgrade and give you further feedback.

@Captain_Morgan

Next week, we will review that Status page and adjust. By that time, there will be about 20,000 system weeks of runtime and we’ll have a pretty good idea of quality relative to Dragonfish.

And will this review only take into account the stability of the O/S or will it take into account all the users who reported unexpectedly broken apps (regardless of whether they are iX, TrueCharts or bespoke apps)?

@Captain_Morgan

We also have a very solid QA process… but it doesn’t account for creative App sets ups and the enormous range of hardware choices. Only systems testing by the Community catches this.

I fully believe this statement to be true - in both respects.

Your QA cannot possibly account for every combination of hardware. And even though Scale → Scale upgrades start from having a system that is already working on specific hardware, things can still break unpredictably (e.g. in Linux e.g. SAS9300-8i Detected in BIOS, but Disks Not Showing in TrueNAS GUI 4).

@Captain_Morgan

@Protopia

You should be more open to listening to advice and warnings given during the release development e.g. that many users will upgrade too early and without planning and that better warnings are needed;

We are open… its why we started the software status page because of the very same feedback.

Rubbish!! You asked for my suggestions, but this response shows that you are NOT listening.

1. The Software Status page is only of use if it is widely publicised.
2. Ignoring what this page actually says in favour of rah-rah marketing in all other communications simply drowns out what this page says.