ElectricEel-24.10.1 Issue

ruffhi · January 1, 2025, 7:40pm

My SSH is root. The cron job user is root. And I am running 24.10.0.2. My old code runs. The new code doesn’t run.

joeschmuck · January 1, 2025, 7:49pm

I make a lot of notes to myself to help me remember what I was thinking.

I also replaced the attached script above with a corrected version for the drive selftest script.

That is a good one for sure.

And user permissions is the real killer for running a CRON job in 24.10.1. Time for me to read more about permissions and CRON. Maybe the TrueNAS Search AI can help.

joeschmuck · January 1, 2025, 7:53pm

Interesting, I will need to roll back to 24.10.0.2 to test the CRON job part. I hadn’t done that yet. But, did you change line 14 to truenas_sendmail_support="Yes" ? If not, then I hope that fixes it.

oxyde · January 1, 2025, 7:55pm

atm i’m really struggling on add the chart.html as body of the email, seems really that the data is too big to send via curl. The call is killed but neither adding --verbose say anything else… im totally a linux newbie xD

ruffhi · January 1, 2025, 8:02pm

I am not running your script. I am running my own script and lifting parts from yours.

Here is my ‘send email’ ‘function’ …

Email_Send()
{
printf "%s\n" "</pre></body></html>
--${boundary}--" >> ${FileToEmail}
cat $FileToEmail > zemailData/tempemail.txt

html_data=$(cat zemailData/tempemail.txt | jq -Rs .)		# THIS WORKS ONLY FOR NO ATTACHMENTS, SO FAR.

# echo $EmailSubject
#echo "Must be SCALE - No Attachments"
#(
#	midclt call -job mail.send "{
#		\"subject\": \"${EmailSubject}\",
#		\"html\": $html_data,
#		\"to\": [\"${EmailAddress}\"]
#		}"
#) > /dev/null 2<&1

sendmail -t < ${FileToEmail}
#mail -s "${EmailSubject}" < ${FileToEmail} ${EmailAddress}
}

This version is a mixture of my old code and the new code. You can see two ‘send email’ commands (old using sendmail and new using midctl).

The above code runs and generates an email of echo output to the root user email address.

If I run with the new send email code NOT COMMENTED OUT (ie #s removed) …

# echo $EmailSubject
#echo "Must be SCALE - No Attachments"
(
	midclt call -job mail.send "{
		\"subject\": \"${EmailSubject}\",
		\"html\": $html_data,
		\"to\": [\"${EmailAddress}\"]
		}"
) > /dev/null 2<&1

… then I get a mixture when running from cron . No output from the echos and no email from the midclt statement. I do get the email from the sendmail -t < ${FileToEmail} statement.

joeschmuck · January 1, 2025, 8:09pm

You have been very helpful so Linux may not be your thing, but I think you have it.

As for the curl issue, I don’t think it is length but I could be wrong. We could make a cut down version and just use that file vice the one it creates, for testing purposes.

Right now the issue is permissions for 24.10.1 CRON. But I need to run out of the house and to a friends house for dinner.

oxyde · January 1, 2025, 8:15pm

At least something i think is a good news, i’m talking about user/pass needed.
In my system, i created an api key and modified my curl param as

curl -k --location-trusted -v -i -H "Authorization: Bearer $curl_apikey" \

This should be enough for not let users put into multi alert config file theyr credentials… but off course they need to create and provide a valid api key.

joeschmuck · January 1, 2025, 8:16pm

@ruffhi
Your problem could be the --${boundary}--" >> ${FileToEmail} line. Comment it out and see what happens. I had to remove all of my boundaries as they were only used to insert attachments and you have to send attachments differently now. But if you figure a way to send attachments inside one file, I want to hear about it.

Let me know how that goes.

ruffhi · January 1, 2025, 8:38pm

That is a fair comment. I doubled up my email file … emailfile.txt (for sendmail) and emailfile2.txt for the new code. I get two emails when running from cli.

AND I get two emails when running from cron

Seems the html file needs to be 100% html and nothing else.

ruffhi · January 1, 2025, 9:13pm

I am finding that the new send command is very touchy about html structure when sending using cron. Not so touchy when sending using cli.

Edit: And it doesn’t like big html files. I have a cut down table that worked with 6 rows but not with 7 rows (all rows are identical).

Runs from cli, not from cron.

oxyde · January 1, 2025, 9:39pm

Instead of get better result, i can’t neither run commands that until yesterday were working…
basic example (just removing curl from now, but having same exactly behavior):

#!/bin/bash
subject='test'
Email='*'
html_file="*"
html_data=$(cat "$html_file" | jq -Rs .)  
echo "$html_data" | wc -c
midclt call -job mail.send "{
    \"subject\": \"$subject\",
    \"html\": $html_data,
    \"to\": [\"$Email\"]
    }"

this command (that just yesterday works reading from “large” ~80kb file) now is just failing for every file larger than 4kb
./testinvio.sh: line 42: 1514820 Killed
generating 1mb of error file for a 4kb payload .
I used a basic text file, no html involved… i’m really confused

btw @joeschmuck this command (with really really small file) for me work sending html as body:

html_data=$(cat "$html_file" | jq -Rs .)  

curl -k --location-trusted -v -i -H "Authorization: Bearer $curl_apikey" \
    --form-string "data={\"method\": \"mail.send\", \"params\": [{\"subject\": \"${subject}\", \"to\": [\"${Email}\"], \"html\": ${html_data}, \"attachments\": true}]}" \
    -F "file=@/mnt/SSDPool/Script/InvioMail/attachment.json" \
    http://127.0.0.1:${port_address}/_upload/

oxyde · January 1, 2025, 9:58pm

@joeschmuck don’t know why i’m having different behaviour just running those commands as sniplet… but putting this into the beta multi report works!!!

	port_address=$(midclt call system.general.config | jq | grep ui_port | sed "s/[^0-9.]*//g")
	curl_apikey="my_super_secret_apikey_slapped_here"
	#echo "$port_address"
#(	curl -k --location-trusted -v -i -u $curl_account:$curl_password --form-string 'data={"method": "mail.send", "params": [{"subject": "'"${subject}"'", "to": ["'"${Email}"'"], "html": "'"${html_data}"'", "attachments": true}]}' -F "file=@/tmp/attachment.json" http://127.0.0.1:${port_address}/_upload/) > /dev/null 2<&1
	(
		curl -k --location-trusted -v -i -H "Authorization: Bearer $curl_apikey" \
			--form-string "data={\"method\": \"mail.send\", \"params\": [{\"subject\": \"${subject}\", \"to\": [\"${Email}\"], \"html\": ${html_data}, \"attachments\": true}]}" \
			-F "file=@/tmp/attachment.json" \
			http://127.0.0.1:${port_address}/_upload/
	) > /dev/null 2<&1

near lines 2096 (prob bit different i have made some change here and there to debug).
tell me if is working for you too

joeschmuck · January 2, 2025, 12:31am

@oxyde Not certain I will have the time to do that tonight, definitely tomorrow. I need to create my API Key.

Question (if you know the answer): How important is keeping the API key secret? Can we give it out and no one could use it anyway because it must match the user account on the local TrueNAS? Just curious. I am always aware of classifications and when I see something that may be impacted, I ask as I want everyone to feel protected.

Stux · January 2, 2025, 1:51am

An API key is essentially root access to your system.

Don’t share it. And it only works on yours.

You should be able to revoke/delete an api key after creating it.

It’s essentially an alternative to the user/password combination.

oxyde · January 2, 2025, 7:54am

Exactly as @Stux said.
But IMHO Is better if we use It for mainly a reason: despite atm an api key can just be created i see on documentation that in next version you can also use a context user

Also this will probably mean that this part will be more implemented in future (like adding other usefull features like command or method restriction).

All Is just my personal assumption… But i still think Is a better choice than put user/password on our config file

joeschmuck · January 2, 2025, 11:07pm

Well I have no idea what this means, maybe I need to reinstall 24.10.1?

But here is goes, someone can check my work.

I created a very small script for testing, there is no reason to run Multi-Report over an over when a small script can aid in troubleshooting.

Edit the script to add your email address to it.
Run the attached script from the CLI, verify you get a normal email.
Examine the log for mail.send (you must use the drop down on the right). Notice the length of the email html file and the last few lines.
There will not be a CRON job related to this since you used the Shell/SSH.
Now setup a CRON Job to run the script and then force it to run. (I used root and command: cd /mnt/mypath/ && ./test_cron.txt, use whatever makes it work for you.)
Did you get a normal email? Sometimes I got a normal email followed by an abnormal email. Sometimes I just got the abnormal email.
Examine the log for the CRON, I can see that the message was repeating itself many times and I believe the buffer filled up when the full script was running and it crashed the script. Why did it do that? I have no idea.

Also, you may find out that your GUI, it may not respond normally. When running the full script I would get locked out of the GUI periodically. Using this smaller script version I have not been locked out.

Now you want to see something special, run the full script and check that CRON and send.mail Jobs, a lot of repeating. Or maybe my installation is fried.

test_cron.txt (557 Bytes)

NugentS · January 2, 2025, 11:52pm

Results as follows

Run from ssh as root - no email. Resuts in session were

Status: (none)
Total Progress: [########################################] 100.00%
True

Run multiple times

But I did get an email from @joeschmuck 's msn address for some reason - once to my gmail address which TrueNAS is relaying through, as a reply to CRON Test. Note that the Test never got to my actual email address

joeschmuck · January 3, 2025, 12:56am

My mistake. I have fixed that. That is what I get for rushing to dinner.

hervon · January 3, 2025, 1:48am

Test_cron works perfectly with 24.10.1 using a gmail address. Thanks.

awalkerix · January 3, 2025, 1:49am

Starting in 25.04 (next major release) API keys will be per-user-account (inheriting same privileges as the associated user). Otherwise you’re right in saying it’s password-equivalent.