That’s why I use an external ssd. I imagine that controls where on the flash writes occur better than a flash drive (where sector 0 is likely the same physical place each time).
When I mount an iso on a vps, before installing I’ll run a sha256sum after booting into it.
dd if=/dev/sr0 bs=2048 count=9999999999999 | sha256sum
Compare this to the sha256sum ran on the iso itself. Still, I suppose this only accounts for the mounted iso being unaltered. With ventoy, can’t really verify memory contents. It’s not out of the realm of possibility to inject something that runs at a lower layer.
PXE booting using fully open source tools sounds like the right way to go.