ElectricEel-24.10.1 Issue

ruffhi · January 4, 2025, 3:43pm

That looks like a pretty nice bit of coding. JSON formatting is a good solution for the smtp info too . I am going to lift your code and use it - thanks.

I must say that I am really impressed with the argument parser ( argparse — Parser for command-line options, arguments and subcommands). It looks very powerful and comes with pre defined help processes and other options.

oxyde · January 4, 2025, 4:20pm

I’m a beginner using python, but for work despite we don’t use It i have had to learn a bit for achieve some stuff that with our infrastructure was kinda hard… JSON can be readed for almost everything (tsql too, what I have to use much), but honestly i’m not so happy on this, for store this kind of data we should point to something “not human readable” at least. But first wanna understand if this can be the right way or not before put more effort!

Yep, kinda usefull and easy go use

ruffhi · January 4, 2025, 9:34pm

I now have some of my bash scripts using the python email code. They run from CLI, they run from CRON. I will port the balance of my scripts over to this code and test them for a few nights.

Note that I am still on the prior version of EE - TrueNAS-SCALE-24.10.0.2

oxyde · January 5, 2025, 9:54am

I have updated to .10 and i’m merging my snipplets to the 3.0.7 version of the multi report: i’m receving the emails (nor manual and from cron) like before, and the config backup too.
I think that using this approach will not require all the stuff for separate core/scale<.10 and >.10 so i’m not using the beta version. But is just my assumption, in case i will test on Core too.
I’m not handle all the attacchments atm, for my need the config is enough, but in case shouldn’t be too much tricky

ruffhi · January 5, 2025, 5:09pm

And I have just updated my TrueNAS-SCALE-24.10.0.2 to the latest version and it is running / sending emails quite happily with the python code.

ruffhi · January 5, 2025, 5:45pm

For those of you dying to know what happened in the 5th test between Australia and India … it was a very low scoring affair (no team scored more than 200) and Australia end up winning by 6 wickets after chasing down 162 in the fourth innings. Australia win the series 3-1 and regain the Border / Gavaskar Trophy for the first time in 10 years.

This also confirms that Australia will make the Test Final (played at Lords in England later this year) against South Africa.

oxyde · January 5, 2025, 7:01pm

if you need to manage attachments i have modified the MR to handle like that

b_attach=() declare an array variable empty

append_attachments() {
    local filepath="$1"
    if [[ -n "$filepath" ]]; then
		b_attach+=("$filepath")
    fi
}

call this function everytime you need to append a new file path,

and then when you call the py pass like that --attachment_files "${b_attach[@]}", the py will manage (refering to my version).

I have made a better merge of those snipplet into the MR and give a copy to joe for a better review… hope we can achieve this soon!

joeschmuck · January 10, 2025, 5:17am

Multi-Report Beta 5

Attached is the current Beta. I believe it works fairly well right now. It is not ideal, I do not like the Chart not being in the message body as it was before however @oxyde is working on a possible solution, a small Python script which can accompany Multi-Report, and this script that others could use for other personal use that needs to send an email.

I still have to update the -config section to account for the new email address stuff, as well as the Help Info and a Change Log, however this “should” be fully functional otherwise. Maybe I will update the Quick Start Guide as well.

Please give it a test, provide me some feedback. And while you are at it, if you have any comments/changes to the “default” values I have in the multi_report_config.txt file (I never seem to get feedback on those values), such as drive temp alarm setpoints, should a feature be enabled/disabled, etc… then please let me know. I’ve been wanting to ask the community about the default values but didn’t want to create a thread for it, so I’m piggy-backing on this thread.

If I don’t hear any negative comments in 2 weeks, I’m pushing this out the door. I will provide another beta update here if @oxyde has a solution that I can incorporate which will generate the email properly to include the Chart in the email body. The Subject Line still works as before and I personally use that to trigger me to look harder if there is an error message.

Please note that you will need to edit lines 10 and 11 to make the email work if you are using SCALE 24.10.1. Anything earlier will use sendmail to send like normal.

And this would not have been possible without the assistance of @oxyde and @ruffhi, thank you both.

multi_report_v3.1_2025_01_10.txt (519.5 KB)

Vollans · January 10, 2025, 7:45am

Stupid question, I know. But would the default admin account work, or do I need to create something special with special settings?

ruffhi · January 10, 2025, 10:13am

Bah! This would not have been possible without joeschmuck.

The community owes you. Let me know if you are even in Boston and we will have a beer / wine / tea / coffee / other.

Jeff_Goldrich · January 10, 2025, 11:31am

New beta works fine on ElectricEel-24.10.0.2. I’ll set up a spare server tomorrow and test it on 24.10.1. Don’t want to upgrade main server at this point. Rolling back from 24.10.1 to 24.10.0.2 messed up SMB -ACL permissions causing several apps to mal-funtion. Was no fun troubleshooting. Don’t want to have to roll back again.

joeschmuck · January 10, 2025, 3:17pm

I wouldn’t upgrade to 24.10.1 unless there was a real reason to, if you desire Multi-Report or other scripts which use sendmail to function normally.

Not a stupid question. It needs to be a privileged account. It can be root if you like, may will use that, you can use truenas_admin if you set up that account, or any other privileged account you create.

I did get one response this morning about no email received and I think I tracked it down to some obscure ASCII character issue. Just waiting on confirmation of the fix and I will post an update, but I doubt this will affect most people, and I could be wrong

Vollans · January 10, 2025, 8:14pm

That’s why I’m asking. I’ve tried both root and admin accounts, and so far no email. If I turn on standard output on the cron task I do get an email from cron showing it’s running, so email per se works, just not from the script.

joeschmuck · January 10, 2025, 8:31pm

I knew they would not be the only one.

@oxyde has given me a small python script, working out the formatting now. I “hope” to have a fully working script in the next few hours, so look tomorrow if the python script works as we hope.

ruffhi · January 10, 2025, 9:51pm

I think this is the whole reason they killed SendEmail. They didn’t want a script sending an email from the system. The python script uses smtp and (for me) gmail authorization to send the email. It is still a script we run sending an email … but it is not the TrueNAS system authorizing the sending of the email.

Theo · January 10, 2025, 10:59pm

I installed this and ran it on my 24.10.1 Scale with email configured with Google Auth. I ran it as an local_admin user directly from cron and it ran fine and sent me the attached report which is properly formatted after I download it from email.

G-Prime · January 11, 2025, 4:43am

So after tinkering with the API a bit, looks like the reason we’re having a problem is that the HTML body of the email is too big. The API actually rejects any mail.send API request over 64KB unless you have an attachment, but the HTML/text still has to be 64KB or less.

Maybe the play here is to put the summary into the body of the email as HTML, and then the detailed data as attachments. On my machine, the single file of HTML is over 200KB. It also looks like there’s potentially a ton of room to reduce character count by using preformatted text for the table instead of HTML and CSS. Each cell of the tables is over 100 bytes, and that adds up extremely quickly.

Another alternative might be to put in a feature request asking for the API size limit to be relaxed, but I would anticipate a hard pushback from the devs on that.

oxyde · January 11, 2025, 5:47am

As Joe already said, i have made a new totally version that retrieve the mail config from truenas, switch to the right “provider”, and send email/attachments accordingly.
Meanwhile some test from Joe for the mains functionality, i will still work to improve “here and there”.
So as was happening before

M.R. want need any additional configuration to work for email
Is a compatible solution nor for who set SMTP and oauth in TN
no risk to “expose” sensitive data into unnecessary config file
no payload limitations (in my test i used 5mb attachment without issue)
Should work nor with Core and Scale

Also that script can be invoked stand alone, for every purpose (at the same identical way of the example of last day).

Jeff_Goldrich · January 11, 2025, 6:40am

I don’t get any attachments or charts with Beta5 on 24.10.1 - Must be doing something wrong.

Received Email

TrueNAS does not support sendmail function, using curl.
Multi-Report v3.1Beta5 dtd:2025-01-10 (TrueNAS Scale 24.10.1)
Checking for Updates
Current Version 3.1Beta5 – GitHub Version 3.0.7
No Update Required
Running Drive Self-test Script: /mnt/ASrock/scripts/drive_selftest.sh

Joe’s SMART Drive Self-test Script - Run: 11 January 09:03:48 AM
Script Version: 1.0 (1 January 2025), TrueNAS Scale 24.10.1

Logging Enabled
Using External Configuration File

Short Test Mode:(2) “Short SMART Test All Drives”, Running Week Option, No Sorting

Long Test Mode:(1) “Long SMART Test on 1 Drive(s) Per Day”, Running Week Option, Sorting by: Serial

Sorting Drive Test Order …

Authorized Test Days: Mon, Tue, Wed, Thu, Fri, Sat, Sun
Total Drives Recognized: 12

DANGER Will Robinson, you have 12 drives and 7 authorized days to test.
ROBOT will make adjustments, please wait… Geez that was fast!
Maybe you need more drives, Waka Waka! ROBOT made a funny.
Recalculation Complete, the new value is 2 drive(s) spread across the 7 authorized days.

Drive(s) previously scheduled to be tested: sdg, sdf, sdd, sdk, sdi, sde, sda, nvme0, sdj, sdh
Drive(s) scheduled for testing today: sdc sdb

Drive(s): “sdb sdc” were removed from the Short testing list
It/They are already scheduled for the Long test, no need to duplicate efforts

RUNNING SHORT TEST: sda sdd sde sdf sdg sdh sdi sdj sdk nvme0
Drive: sda in Short Test
Drive: sdd in Short Test
Drive: sde in Short Test
Drive: sdf in Short Test
Drive: sdg in Short Test
Drive: sdh in Short Test
Drive: sdi in Short Test
Drive: sdj in Short Test
Drive: sdk in Short Test
Drive: nvme0 in Short Test

RUNNING LONG TEST:
Drive: sdc failed to start Long Test
DARN!!! - “sdc” is still running a test: (90% remaining)
Skipping drive “sdc”
Drive: sdb failed to start Long Test
DRATS!!! - “sdb” is still running a test: (90% remaining)
Skipping drive “sdb”

ALL TESTS COMPLETED OR RUNNING IN BACKGROUND

09:04:27.08: Main Program End, Wrapping things up…
Elapsed Time: Program 0 minutes and 38.17 seconds elapsed.

Your 130 Second Delay, Please Standby…
Saving Log (file name: /mnt/ASrock/scripts/DS_Logs/drive_test_11.txt)
Exiting Joe’s SMART Drive Self-test Script
Collecting Report Data…

oxyde · January 11, 2025, 1:09pm

I have published the code of the script here, so everyone can test/comment/suggest/improve.
Everytime the script is launched, will create a log file in a folder mrlog placed where you put the script file, only the 15 last log will be preserved (everything older will be automatically deleted). If you face problem feel free to share (no sensitive data is inside, just the steps).