Slow boot evey morning

Hello all

Just recently i am experience a (IMHO) a very slow boot of my truenas scale server.

Truenas is installed on a bare metal r720 with dual cpus 32 cores and 128 gb of ram and truenas boots for an ssd.

On my installation i have jellyfin rsync and tailscal as apps installed.

My server start as a wol process from home assistant (on different hardware on same network) at 07:00.

I will take upto 6 minutes to complete the boot

Is this normal? If so i must rethink my expectations if not how can i fix this?

than you in advance for taking the time to help me

Watch the console while it boots and see where and what is, possibly, causing issues. Is the BIOS taking up time or just the OS?

You will have to take a look at logs.

in addition you can try from a root shell:

systemd-analyze time
systemd-analyze blame

@heis2201 @SmallBarky

Thank you all for your prompt reply

I am attaching here the results

ruenas% systemd-analyze time

Startup finished in 29.637s (kernel) + 1min 36.144s (userspace) = 2min 5.782s

truenas.target reached after 1min 36.144s in userspace.

And

Truenas% systemd-analyze blame

50.525s middlewared.service

16.180s ix-vendor.service

15.490s ix-zfs.service

11.021s docker.service

10.390s ix-wait-on-disks.service

9.246s ix-etc.service

5.916s ix-netif.service

2.112s nfs-idmapd.service

1.774s ix-reporting.service

880ms ix-postinit.service

737ms containerd.service

716ms console-setup.service

560ms ix-syncdisks.service

522ms incus-startup.service

521ms smartmontools.service

497ms user@3000.service

491ms syslog-ng.service

348ms nfs-server.service

343ms proc-fs-nfsd.mount

301ms ix-preinit.service

294ms systemd-modules-load.service

248ms systemd-udev-trigger.service

I truly do not see here how i can reconclie thisnwith my perception. I can see that the process here is circa 2.75 minutes

Once again thank you all

2 min 6 sec isn’t bad, right? I would stop bothering, makes no difference to your day if the machine is up 20 seconds earlier…

Be aware that some of these services are executed in parallel, so the result can be misleading.

To figure out the bottleneck, run

systemd-analyze critical-chain

Watch the console at boot. It is most likely the server itself that takes some time, especially RAM checks. But 6 min on server HW seems normal to me. Used to turn them on and get a cup of tea or coffee before returning.

1 Like

6 minutes boot time is not unexpected depending on motherboard and number of hdds you have installed. NVMe boot drive can cut it down to around 3 minutes. BIOS and hdds checks are usually the longest parts of the boot process so doesn’t seem abnormal to me.

2 Likes

Thank you for your response . I performed the comand here above and i get:P

─ix-vendor.service @1min 21.967s +16.061s
└─ix-postinit.service @1min 21.075s +842ms
└─multi-user.target @1min 21.014s
└─containerd.service @1min 20.250s +761ms
└─local-fs.target @1min 24.710s
└─mnt-.ix\x2dapps-app_mounts-jellyfin-transcodes.mount @1min 29.762s
└─mnt-.ix\x2dapps-app_mounts-jellyfin.mount @1min 29.310s
└─mnt-.ix\x2dapps-app_mounts.mount @1min 28.898s
└─mnt-.ix\x2dapps.mount @1min 28.760s
└─local-fs-pre.target @1.150s
└─systemd-tmpfiles-setup-dev.service @1.126s +17ms
└─systemd-tmpfiles-setup-dev-early.service @1.030s +43ms
└─kmod-static-nodes.service @761ms +132ms
└─systemd-journald.socket @677ms
└─system.slice @481ms
└─-.slice @481ms

@Glitch01 @SmallBarky @ProfessionalAmateur @heis2201

Thank you all for your patience and help.

I just found strange that with a 32 core and 128gb machine the process is so slow.

Does any one know how i could add an mvme on a dell 720?

Once again thank you for taking the time to respond and for your help

I doubt an nvme will help. Most of the time with server grade hardware things like initializing the ipmi and ecc memory training happen before the OS boot process and won’t benefit from an nvme

2 Likes

@LarsR

Thnk you for your input

How exactly are you assessing when the boot is complete?

Console messages, ping to NAS, access apps, access GUI?

How long is the BIOS startup?

With older servers with IPMI, the BIOS startup can take 1-3 minutes, plus say 3 min for TN boot.

@truenas-fan thank you the time you are taking to help me.

I notice that the server is up when explorer network neiborhood sees the server, when other applications (depending on the server s content) remove the notification of a fault (server not available ).

I do not use IPMI

If you notice in my previous message the apps (jellyfin takes one minute to load). I hope this is necessary to happen when Jellyfin updates. Other processes show +ms/s. I do not know what this means. If this is significant, do i need do do something?

I concluded that that i need to reset my expectations

My curious nature begs me to ask: Why are you booting on a daily basis? This practice can reduce the life of spinning discs. (motor spin up and parking / un-parking of the read/write heads) If you prematurely fail just one disc because of frequent power cycling, the cost of the disc negates any savings on the electric bill. Just a thought.