Multi-Report

Cellobita · January 18, 2025, 4:20pm

I’m using SendGrid for mail relaying - for my modest needs it has been both free and very reliable. The Authenticated SMTP user for their service is indeed apikey, and they generate a very long password to go with it:

Server: smtp.sendgrid.net
Port: 587
Security: TLS (STARTTLS)
Authenticated SMTP
Username: apikey
Password: SG.V-EnrLRITLa4ax3SAhFzwQ.-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

I suppose I could change this to another service like Gmail if there’s no other way, but it would be a nuisance.

P.S.: can you programmatically retrieve this “From Email:” from the TN Email options? I have it set correctly there on all my servers (TN itself won’t send messages if this is not the case). If this is not possible, can you parse this info from the multi_report_config.txt file, where it is also set correctly?

joeschmuck · January 18, 2025, 4:34pm

Thanks @oxyde for jumping in here. I’m certain you know more about what is happening to the email.

@GrimmReaperNL and @Cellobita
If @oxyde generates a new sendemail.py script, if you run multi_report.sh -update_sendemail the email program should update. Of course if you just delete the mr_sendemail.py file, the next time Multi-Report is run, it will automatically install the current version available.

And one last option, you can turn on the automatic updates for sendemail which will take care of this all the time an update comes out.

As for the “SENDEMAIL.PY UPDATE AVAILABLE ->” message, if this was the first run, then that is “normal” unfortunately. I need to address this as in the first run it will be automatically installed if it does not exist. If this message happens in the future, it means there was an update and you can choose to install it or not. The update messages will be fixed in time when I have either a critical update or if in a week or two I can push an update, but they should not impede operation.

Rationale of my mind: I do not care for “Automatic Updates”. I would always rather have some control over what is being installed on my system, and it helps with troubleshooting a problem because you would know that the only thing you did was allow an update and then things broke. I hope you can understand my old way of thinking. But I did allow for the end use the option to make it all fully automatic, but that isn’t the default setup.

EDIT: Also, if you (anyone) thinks the default values for Multi-Report (any of the settings) should be changed, please post what you would like changed. If it makes sense then I will change it.

oxyde · January 18, 2025, 4:45pm

No need, i will fix it. Stay tuned

GrimmReaperNL · January 18, 2025, 4:48pm

I’ve run v3.1 twice so far (once at midnight, once at the following 10am), and it’s listed the update message both times.

I’ll try to update as you’ve mentioned.

Edit: Updated and ran the report. Update available message is gone now

MrTVirus · January 18, 2025, 4:50pm

Interesting, I am using SendGrid as well and struggling to send mail using the latest report.sh from @dak180 . I wonder if its the same problem as I don’t receive the report(s), but I do receive the successful test messages from TrueNAS

Cellobita · January 18, 2025, 4:57pm

I’m not familiar with that script. I do send some reports daily from my TN boxes (mostly the results of some homemade rclone backup scripts), and a simple

cat $summaryfile | /usr/bin/mail -s "${subject}" "suporte@tenaya.com.br"

suffices (there is no need here for fancy html formatting, or attaching files - it is just a plain text file with the result, piped to the /usr/bin/mail program). The ${subject} variable is calculated from the rclone exit code, and lets me know at a glance if the script run was successful.

P.S.: this works even in 24.10.1

oxyde · January 18, 2025, 5:28pm

@Cellobita @GrimmReaperNL pushed new version with both fixes.
Test and let me know if is enough! Be sure to have the latest 0.07 version

@joeschmuck don’t know if is me, but i couldn’t update the script automatically (auto upgrade true):

Checking for Updates
   1. Multi-Report - Installed: 3.1, GitHub:  3.1
   2. Drive_Selftest - Installed: 1.02, GitHub: 1.02
   3. sendemail.py - Installed: 0.06, Github: 0.07
UPDATE AVAILABLE
Running Drive Self-test Script: /mnt/SSDPool/Script/MultiReport/drive_selftest.sh

(have check and the file was the 0.06)
Despite deleting the file working as expected

oxyde · January 18, 2025, 5:30pm

Probably yes, afaik he probably start using my script too

GrimmReaperNL · January 18, 2025, 5:48pm

Manually updated and ran the report. Email now shows as coming from ‘me’. Which I think happens when there’s no name, only an email address. Which I guess works for me.

@joeschmuck ‘s script allows you to configure a ‘from’ though. That’s not being used. Guess that’s more coming from joe’s script instead of yours @oxyde .
Or maybe that’s just truenas’ oauth’s doing.

Vitosi4ek · January 18, 2025, 6:07pm

Just updated to 3.0.7 and 24.10.1, and surprisingly it works just fine. Though IMO the fact that drive_selftest is set to off by default should be mentioned front and center in the documentation, since I was kind of confused at first when I ran the updated script and it told me that “no tests pending” or whatever.

Also, I’m not sure if the 130-second delay after the self-tests is quite enough. Of my 10 drives, 2 couldn’t complete the test in time and the email popped up an error saying too much time has passed since the last test. They did complete the test successfully after around 7 minutes, so I set my value to 600 seconds just to be safe, but I’m curious how it is for other people.

Nevertheless, enormous thanks for the work you put into making it work. I would’ve just never updated past 24.10.0.2 if it wasn’t for that.

Cellobita · January 18, 2025, 6:23pm

0.07 confirmed working, and the “from” address is correctly preserved!

Fine work, thank you very much!

joeschmuck · January 18, 2025, 6:49pm

While I’m glad to hear that, it still bothers me that you had the issue in the first place. No one else reported this issue so I suspect there is some minor differences in the multi_report_config.txt file. Please send me a -dump emailextra or just send me the config file, that is all I really need. I will try to emulate your setup to see if I can replicate the issue.

joeschmuck · January 18, 2025, 6:52pm

I apparently have something going on there to figure out, unfortunately.

MrTVirus · January 18, 2025, 6:55pm

Thanks to everyone’s hard work I finally got a report to generate and send via email.

joeschmuck · January 18, 2025, 7:00pm

Glad to hear that it worked.

I understand. I can change the default to running Daily Short tests and Weekly Long tests, however in beta testing, I received feedback that if someone has SMART testing scheduled via TrueNAS GUI, the drives get tested more than they should, so I just turned them off.

EDIT I just updated the Multi-Report User Guide on Github to add that note, in big bold letters.

Okay, but that is odd that a drive (HDD/SSD/NVMe) takes over 2 minutes for a Short test. You did the right thing to set the value to 600 seconds. And the entire purpose of this timer is to complete all the Short tests before the actual report is generated, so the Chart shows the completion/failure of the SMART test. If a drive is having a Long test, there is not delay for those, Long tests can take hours to over 24 hours depending on the drive.

It was a group effort. And as you can see, @oxyde will have his hands full for the next few days if any more small problems come up, different email services, oh yes, fun times!

GrimmReaperNL · January 18, 2025, 7:04pm

On its way good sir.

Quidnam · January 18, 2025, 7:24pm

I seem to be having the same issue with “File name too long” errors being generated and sent in a “CronTask Run” email following the report email, with references to temprunfile.sh

I just sent an “emailextra” report. When I executed this from the command line, I did see similar “File name too long” errors being generated (but without reference to temprunfile.sh) as shown below.

The cron command executed for the script is /mnt/elephant/scripts/multi_report.sh

Here’s a portion of the command output:

Saving Log (file name: /mnt/elephant/scripts/DS_Logs/drive_test_18.txt)
Exiting Joe's SMART Drive Self-test Script
Collecting Report Data...
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>Drive WD-WX22D8328JTL actual Power On Hours = 8289<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>Drive WD-WX22D8328JTL actual Power On Hours = 8289<br>Drive WD-WCC7K6PKX8SH actual Power On Hours = 8854<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>Drive WD-WX22D8328JTL actual Power On Hours = 8289<br>Drive WD-WCC7K6PKX8SH actual Power On Hours = 8854<br>Drive WD-WX52D93K4Y8N actual Power On Hours = 4226<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>Drive WD-WX22D8328JTL actual Power On Hours = 8289<br>Drive WD-WCC7K6PKX8SH actual Power On Hours = 8854<br>Drive WD-WX52D93K4Y8N actual Power On Hours = 4226<br>Drive WD-WX52D934XLPL actual Power On Hours = 2116<br>: File name too long
/mnt/elephant/scripts/multi_report.sh: line 4434: Drive WD-WCC7K0UX1VXU actual Power On Hours = 58748<br>Drive WD-WCC4E6LXPR26 actual Power On Hours = 84071<br>Drive WD-WX12D839PUD5 actual Power On Hours = 8289<br>Drive WD-WX12D932TN7H actual Power On Hours = 8289<br>Drive WD-WX22D83EJ68D actual Power On Hours = 8289<br>Drive WD-WX12D83EYJEH actual Power On Hours = 8289<br>Drive WD-WX12D8332RY8 actual Power On Hours = 4248<br>Drive WD-WX12D8332PPE actual Power On Hours = 8085<br>Drive WD-WX12D83J62TN actual Power On Hours = 8289<br>Drive WD-WX12D8332ESL actual Power On Hours = 8289<br>Drive WD-WX22D8328YTR actual Power On Hours = 8289<br>Drive WD-WX22D8328JTL actual Power On Hours = 8289<br>Drive WD-WCC7K6PKX8SH actual Power On Hours = 8854<br>Drive WD-WX52D93K4Y8N actual Power On Hours = 4226<br>Drive WD-WX52D934XLPL actual Power On Hours = 2116<br>Drive WD-WX22D83NW2VP actual Power On Hours = 1215<br>: File name too long
Changing nvme0 power state from 0 to 4.
Checking to see if we remained at power state 4
Power state is 4

joeschmuck · January 18, 2025, 7:52pm

Got it. You are the second person to report this specific problem, which is kind of good from a troubleshooting perspective. I have asked @MikeO3 to try a few things. If they result in a solution, I will let you know how to “get past it”. If I’m able to isolate the actual problem, I will send you both test versions to try out and see if the problem was fixed, before I push it to everyone.

But time for a very late lunch.

Davvo · January 18, 2025, 9:37pm

Adding my 2 cents here. Seeing the issues here I tried running the script ahead of time (I run it weekly on wednesday).

I get two mails, one after the other, instead of the usual one.
The first is the unexpected behaviour, and is as follows:

TrueNAS truenas.local: CronTask Run
./scripts/multi_report.sh: line 4434: Drive REDACTED actual Power On Hours = 21520 Drive REDACTED actual Power On Hours = 21520 Drive REDACTED actual Power On Hours = 13005 Drive REDACTED actual Power On Hours = 13006 Drive REDACTED actual Power On Hours = 11279 Drive REDACTED actual Power On Hours = 21702 : File name too long

System specs as in my signature.
Do you want me to -dump emailextra?

joeschmuck · January 18, 2025, 9:42pm

Yes, please. Hopefully there will be something in common with the other two folks.