I've finally got a Netmaker network up and running but I'm facing some strange behavior in connectivity between peers.
My Setup is the following:
* 2 real servers in different data centers running Proxmox VE. Both servers are running netclient.
* each server provides 3 VMs which shall be able to talk to each other to form a HA k3s cluster. The VMs run behind NAT in subnet 192.168.0.0
* 1 Netmaker Server Node running publicly on EC2
With a basic mesh setup only the 2 bare-metal host servers can reach every peer. Peers can only reach other peers within the same data center. (Yes, I activated UDP hole punching).
So I thought I make both host servers a relay node each for the VMs running on them.
This ended up looking like the first picture.
Thing is... that now only the 2 Relay Servers were able to reach each other and that's it.
Now the fun part... If I only make one of the 2 servers a relay server like on the second picture, everything works like a charm. And it doesn't matter if aio1 or aio2 is the relay for their VMs.
It's kinda mutually exclusive. Both as relay don't work. None as relay doesn't work but a single one works. It's a bit odd.
Has someone of you faced something similar or am I missing something obvious here?