Complete Guide to TrueNAS CORE Virtualisation Under Proxmox
Complete Guide: Step 5 - Setup TrueNAS Networking
Follow this guide to install your network interfaces. I am just going to cover the high level network topology here.
In my setup, vnet0 is the management interface. This is a dedicated network not accessible from the other networks. vnet1 is a dedicated link to a second server in my rack, while vnet2 (with the VLANs) serve data to all my 4 networks (I don’t serve network files over the management network).
With a typical NAS setup, we either:
Expose the network share from a single IP, and open up firewall rules to allow computers from other subnets to access the network share. This is known as Layer 3 (L3). While this provides better security you are at the mercy of your router (the device responsible for inter-network communications, AKA routing). Forwarding data between two different subnets can consume a fair amount of CPU. I am using a software router (Unifi UDM-Pro) where CPU cycles are consumed when performing inter-vlan routing. More CPU cycles are needed if you intend to firewall your inter-vlan traffic.
At time of writing most consumer routers you can get in the market today are software routers. Hardware routers are still pretty expensive.
Expose different network IP on all the networks you want to serve files on. This is known as Layer 2 (L2) and it means a slightly faster transfer speeds because your packets typically goes straight from point to point if both computers are connected to the same switch, and without changing the packet header. While this provides better speed there is a big risk with security, as your NAS IP is open to any machines on the subnet.
Previously there is no other way to easily mix the best of both worlds - until you virtualise TrueNAS inside Proxmox. This gives you a hybrid L2/L3 setup where all your clients connect to your NAS on Layer 2 (i.e. same subnet), and Proxmox firewall provides the layer 3 network protection. This is a fairly good compromise.
Yes you can use a powerful Intel i7 or a AMD flagship CPU as your software firewall (using something like pfSense or Untangle). However there is still something that cannot be easily overcome - jumbo frames!
Using a hybrid L2/L3 Proxmox/TrueNAS solution means it’s less complex to mix Jumbo frame subnets with others. Your router do not have to worry about fragmenting the packets as it does inter-vlan routing. You do not need to worry about DF bit in your packets.
Everything is simplified with the hybrid setup, you put all the devices you want to communicate with Jumbo frames on it’s own network, and it will inter-communicate with jumbo frames on L2. Internet traffic will still behave as normal.
Firewall management with Proxmox is also a lot simpler to administer, as compared to UDM-Pro’s horrible firewall configuration. I have used Untangle and pfSense before moving to UDM-Pro. The two software routers are far superior to UDM-Pro, unfortunately my OCD simply refuse to accept the crippled firewall-less Unifi network controller.
Right now I have managed to find a solution I am happy with.
Founder and lead architect of Snakeoil OS - the ultimate audiophile operating system for music playback. My primary focus is in applying technology without losing the human touch.
I was writing a similar guide for the teens that come to the IT workshop in our charity for children association and I found your guide. You did fantastic work. This is very helpful. 🙏🙏🙏🙏
Ryzen 5650 Pro CPU since it has ECC support and is an APU so no IPMI needed.
AsRock Riptide X570 mobo since it supports ECC, lots of PCIE slots, and has excellent IOMMU grouping (and costs $120 new atm)
NEMIX ECC ram DDR4 3200 4x32GB ~$350. 1 stick was bad and it only cost me time as warranty was fully honored. Note memtext x86 actually didnt detect it unless i tested 1 stick at a time though journalctl reported the bad stick accurately post mortem.