Really need some help here. I don’t know where to look next. I am reposting here what I created as a bug report a week ago. Here is a link if anyone has access: Log in with Atlassian account
Any more info requred please let me know. My network topology is two PFSense firewalls in High Availability mode. The LAN switch is a Cisco 3750 with 10gb fibre to both firewalls Both servers have two physical network connections to the switch - those ports are configured to VLANs 10 (network VLAN) and 25 (Server VLAN) and share them to internall apps and VMs via bridges. They also share a dedicated network card for inter-server communication
FIRST
Both servers cannot seem to sync the Truenas or Truecharts application catalogues anymore. I have checked the system time and it looks fine. BIOS time is set to UTC time and the server Local time is reporting correctly. The system also automatically resets the BIOS time to UTC time upon every reboot even if I change it manually to something else.
SECOND
Windows 10 VMs on 24.04.1 are having multiple (maybe related) issues. Both VMs are running the latest stable version of the windows VirtIO drivers (0.1.240). One system formerly had a VPN installed on it (I removed it for testing) and the other has only been used for testing and has NEVER had a VPN installed on it.
Download speeds from the internet struggle to hit 1Mbps. SMB fileshare transfers from VMs to SMB shares fail every time but VMs to either server seem unaffected. I am attaching some screenshots from 2 VMs showing bandwidth testing using Ookla. No other devices or PCs on site are showing any issues with internet speeds.
File transfer uploads to the server SMB shares fail but downloads to the VM work fine.
I was able to get one VM working after rolling back the Truenas OS to a previous version. Clones of this one working VM also work fine but any new VMs created have the same networking issue - downloads struggle to reach 1MBPS but uploads are uncapped.
As an update to this - I just rebuilt my test server with a freshly downloaded copy of 24.04.1.1 and reloaded my configuration file and the issue still remains - I keep getting errors trying to sync application catalogues and newly created VM’s still have thier download speeds throttled.
Both app catalogues on both servers cannot sync. The only network change within the last couple months I made was to add some rules to reduce “Bufferbloat” which I currently have disabled while I figure out what is going on. I also set my pi-hole servers to stop blocking traffic while the updates try and run.
As for rollbacks - I rolled back to 24.04.0 and then to 24.04-RC.1. The first rollback fixed the network issues I had with one of the 2 Windows VMs I was running but not both. If I try and create a new VM now, the network is throttled as I described.
Finally had some time to do some testing today after that update a few days ago. Networking on newly created windows VMs are working fine again - uploads and download speeds look great.
Still cannot sync the application catalogue with iX systems. BIOS time is set to UTC time and local time is reporting correctly. Not sure where else to look but would like to have this fixed before I start messing with jailmaker. Any advice would be appreciated!
