ElectricEel-24.10.1 Issue

joeschmuck · December 31, 2024, 7:24pm

What command are you using to send the data?

TimoJ · December 31, 2024, 7:43pm

I’m using this script. Logfile is made with dak180’s freenas-report version. This script has 800 lines limit so that I get below 64k size.

#!/bin/bash
html_data=$(head -n 800 /tmp/logfile.html | jq -Rs .)
subject=Report
Email=email@gmail.com
midclt call -job mail.send "{
    \"subject\": \"${subject}\",
    \"html\": $html_data,
    \"to\": [\"${Email}\"]
	}"

awalkerix · December 31, 2024, 9:15pm

Any ideas how to change this 64kB limit?

This is a generic security setting for our websocket server. It’s not user-configurable and probably will never be user-configurable for security-related reasons that is beyond scope of discussion here.

joeschmuck · December 31, 2024, 9:54pm

Attach it as a file, not as HTML.

TimoJ · December 31, 2024, 10:12pm

I have tried attaching it as file, with ChatGPT’s help, of course… But no matter what I tried, I always get this error: mail_message.subject: attribute required

awalkerix · December 31, 2024, 10:14pm

That means you haven’t set the mail subject. It’s required. If you need to check arguments you’re passing to a middleware job you can run the command midclt call core.get_jobs | jq

TimoJ · December 31, 2024, 10:27pm

This is the script I’m trying to use, not sure what is wrong
edit. got html as an attachment working using parts of @joeschmuck beta script.

joeschmuck · December 31, 2024, 10:51pm

Here is what I have come up with. Feel free to examine it, beat me about it, it is not finished but close. I have to verify I trapped all the attachments, I feel like I am missing possibly two of them, which gives me the opportunity to streamline the attachments.

I did make it to be backward compatible should sendmail come back.

This is only the new Multi-Report script section. There is a separate script to actual schedule and test drives which is sitting on github and it can be run alone.

Things to know:

Lines 9 and 10 must have the user account and password for the curl command to work. It would be nice if we didn’t need that.
Line 14 tells the script if sendmail is supported or not. This is actually important to people running CORE for example where the send.mail API doesn’t exist or work, not sure, didn’t investigate, and don’t care. or TrueNAS 24.10.0.2 it can be either value, but as you guessed it, 24.10.1 required a “No” value.
All the fun stuff really starts at line 2036 where it use to be a single line, now it is many many lines. You will see that I have a lot of comments to myself, I do that to remind me what the hell I was thinking two days ago. I like to clean that us before the script is final.
I did add the PORT value in there, hopefully it works, it works fine for me.
And I need to run this on CORE again, but I don’t expect any issues there.

What I would like to look into is:

How to test if sendmail works within the script. Unfortunately if you type the command, it works. Maybe echo $? will report it. I don’t feel like rebooting the machine again to verify it tonight.
If there was a way to make a user account that would run curl, and basically curl only and we could set that as the typical account name/password. Not sure if that is a smart security thing, probably not.

Inputs welcome on the email stuff, the rest of it, we know I’m no programmer.

Cheers!
multi_report_v3.1_2024_12_31_BETA.txt (516.7 KB)

Jeff_Goldrich · January 1, 2025, 12:22am

Absolutely… If you ever get lost in the Philippines, give me a shout out.

Jeff_Goldrich · January 1, 2025, 12:45am

Hey Mark… just tried to run the new beta on ElectricEel-24.10.0.2 with no luck
truenas_sendmail_support=“No”
see attachments.

GrimmReaperNL · January 1, 2025, 1:24am

Hi Joe,

Tried the script. Did the following steps:

Downloaded file.
Added my password in the file.
Renamed file ‘multi_report.sh’.
Uploaded file to nas as truenas_admin.
Sudo su’d.
Ran script with -config.
Gave script an email address, told it to find current error etc.
Config was saved (with ‘root’ as owner?).
Ran script.
Script errored:

./multi_report.sh: line 770: /tmp/memory_free.txt: Permission denied
./multi_report.sh: line 788: /tmp/memory_free.txt: Permission denied

Script downloaded drive-selftest script.
Script (tried to) run test script.
Script errored:

/mnt/TrueNAS/scripts/drive_selftest.sh: line 1: 404:: command not found

I didn’t get an email.

I checked the drive_selftest file. It says “404: not found”.

I might have done something wrong though…

joeschmuck · January 1, 2025, 2:02am

If I didn’t make easy enough, I failed.

I did say Beta, Ha Ha

My server is powered down right now.

I have been running the script as ‘root’. Yes, a big no no and that could be the issue.

@GrimmReaperNL I will send you an email with my bazillion questions and then post whatever the solution was here in the forums.

@Jeff_Goldrich try to run the script with `truenas_sendmail_support=“Yes” and hopefully it will run fine like Multi-Report version 3.07, it should on 24.10.0.2.

ruffhi · January 1, 2025, 2:58am

Joe - thanks for leading the charge on this email issue. My approach to reporting on my NAS is different from yours (I do lots of small, focused scripts rather than one huge one) but I have been lifting code and ideas (or is it ideas and code) from your script.

I also create small libraries / function collections that every script uses. I have 3 at the moment - one for drive names and info (model, serial, type, pool, partuuid) and another that just managers the emailing stuff.

I will be lifting the new email code into a new library and testing it over the next few days.

joeschmuck · January 1, 2025, 5:13pm

I’m glad I could be of some help. For me it is all about sharing information and learning.

ruffhi · January 1, 2025, 5:55pm

This seems to be working for me. That said, my email subjects are longer than I thought they would be.

Here is the email subject I am getting with my HDD Info script …

The email subject that is being passed to the ‘midclt call -job mail.send’ command is …

… so it looks like it is adding the TrueNAS ScalePROD.local information. I am ok with that as it is a) the name of my server and b) information I want to see in the header). I will remove the [SCALEPROD] addition from my code.

oxyde · January 1, 2025, 6:30pm

i had finally time to take a look on the beta version.
I’m still on 24.10.02, testing with truenas_sendmail_support="Yes" work as expected (if you need, for Core users, i can test on the backup system), i just point because i see this comment

sendmail -t -oi < "$logfile_header"		# I have no idea if the format will be good or not.

So far so good, but despite using truenas_sendmail_support="No" like 24.10.1 users script fail on the function create_Email ()

 error 7585 - - - - -\n<br>\n<br>\nEnd of data section\n",
                "to": ["**@gmail.com"]
                }"
./multi_report_v3.1_2024_12_31_BETA.sh: line 1975: 929486 Killed                  ( midclt call -job mail.send "{
                \"subject\": \"${subject}\",
                \"html\": $html_data,
                \"to\": [\"${Email}\"]
                }" ) > /dev/null 2<&1

i see the condition to enter there is the non existence of the file attachment.

So for test this other condition (with an attachment) i just set my statistical send on Wed, but the script complain about a file drive_selftest.sh… so i change the config backup day to Wed instead and launch again, but same result. I later realize that
https://raw.githubusercontent.com/JoeSchmuck/Multi-Report/refs/heads/main/drive_selftest_v1_2024_12_11.txt
is returning 404

This is the file need?

github.com

JoeSchmuck/Multi-Report/blob/main/drive_selftest_v1_2024_12_27.txt

#!/bin/bash

LANG="en_US.UTF-8"
if [[ $TERM == "dumb" ]]; then          # Set a terminal as the script may generate an error message if using 'dumb'.
    export TERM=unknown
fi
# Version 1.0

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

### FILE NAME SHOULD BE 'drive_selftest.sh' TO INTEGRATE WITH MULTI-REPORT
###
### MULTIPLE DRIVE SMART SELF-TEST SCRIPT (TO EASE YOUR CONFIGURATION NEEDS)
###
### WHAT DOES THIS SCRIPT DO?
###
### THIS SCRIPT WAS ORIGINALLY DESIGNED FOR PEOPLE WHO HAVE A LOT OF DRIVES AND
### DO NOT WANT TO SCHEDULE EACH DRIVE INDIVIDUALLY FOR SMART TESTING.
### ADDITIONALLY THIS CAN BE USED WITH MULTI-REPORT 3.1 OR LATER TO RUN SMART TESTS
### AND UTILIZE THE MULTI-REPORT CONFIGURATION FILE IF DESIRED.

This file has been truncated. show original

ruffhi · January 1, 2025, 6:49pm

Hmmm … I updated my emailing library script. My script runs from the CLI but will not run from a cron job. Further, the cron job is not giving me any of the standard (or error) output. Swapping back to my old email script and I do get the standard (or error) output emailed to me.

I will compare my two email scripts and see what change is borking the standard (or error) output from a cron job.

oxyde · January 1, 2025, 6:58pm

Slapped the file i linked above for the self test, and seems working.
Find the email issue, at least for me:

changed this long - with -- (thanks visual studio code xD), email received with all attachments.
Will try to understand why with no attachment is failing for me, and if is possible to add che chart.html as body of the email instead of an attachment

joeschmuck · January 1, 2025, 7:36pm

Same here for 24.10.1. More security “features” I suspect. I use to use ‘root’, but that fails now. This stuff is really killing me. I’m not a real computer IT guy, permissions is something sort of new to me for a home computer. User aname and password, that is all I need. So I have setup a new admin account, it works fine using SSH, but still not in CRON.

joeschmuck · January 1, 2025, 7:39pm

Yes. I updated it but I never reflected that in Multi-Report. It is there now.