https://netmaker.org logo
Powered by
Setup
b

big-sugar-32241

04/12/2023, 6:30 AM
Hi guys -- came across Netmaker when researching about MeshVPN options out there. As far as I understand, there are 2 components -- Netmaker Server and Netclients. What is the suggested way of setting up the Netmaker Server: - in an EC2 instance in the cloud? - inside a K8s cluster? - what if I want to setup multiple Netmaker Servers for redundancy? My use case is that I have a fleet of M1 Mac devices out in the wild spread throughout the globe and all of these will become clients in a Mesh Network setup Thus, the server (I presume) has to sit outside of this fleet (see the options above) Let me know if this is the correct approach towards using Netmaker. Also, is 
netclient
available as a single binary?
b

bored-island-21407

04/12/2023, 11:17 AM
Netmaker server should be installed on a machine with a static public ip. A vps from any of the cloud providers is ideal. Netclient is a single binary. There is also a service file to control starting and stopping of the daemon. The service file is embedded within the binary and will will deployed to correct location by running./netclient install
b

big-sugar-32241

04/12/2023, 11:20 AM
I see, so, is downloading of 
netclient_darwin_arm64
from https://github.com/gravitl/netclient/releases/tag/v0.18.5 is enough? Because it's a binary I ask since there is also this file called 
Netclient-M1.pkg
b

bored-island-21407

04/12/2023, 11:21 AM
After downloading the binary, run ./netclient install
b

big-sugar-32241

04/12/2023, 11:23 AM
Also, does the node/client have to have 
wireguard
installed before?
b

bored-island-21407

04/12/2023, 11:23 AM
joining a network will not result in anything showing with wq show unless the netclient daemon is running
netclient does not depend on wireguard-tools
b

big-sugar-32241

04/12/2023, 11:24 AM
I see
so even if I don't have 
wireguard-tools
installed, I can still run 
sudo wg
after 
./netclient install
right?
b

bored-island-21407

04/12/2023, 11:26 AM
wg is part of wireguard-tools. Not needed for netclient to operate but wg show won't work without it
Fyi, there is also a homebrew package for netclient
b

big-sugar-32241

04/12/2023, 11:29 AM
If I have one server and one client and I join the client then 
sudo wg
on server has two entries while 
sudo wg
on client has just 1 entry is this expected?
actually wait
now it's fixed
2 entries in both
thanks
b

bored-island-21407

04/12/2023, 11:29 AM
If the daemon on the client is not running, yes expected behavior
b

big-sugar-32241

04/12/2023, 11:30 AM
basically, earlier, I did 
netclient join -t TOKEN
before 
./netclient install
and that was the source of all misery
thanks a lot @bored-island-21407
one quick last question perhaps -- where can I see the daemon running?
I am on an M1 mac
and my server is a EC2 in the cloud
b

bored-island-21407

04/12/2023, 11:31 AM
The join command updates config files, the setup of the wireguard interface is handled by daemon
b

big-sugar-32241

04/12/2023, 11:32 AM
I see, and wireguard wasn't setup because of not running 
./netclient install
b

bored-island-21407

04/12/2023, 11:32 AM
Look at your running services or use pgrep -a netclient
b

big-sugar-32241

04/12/2023, 11:33 AM
also, if I have just one server in the EC2, how many nodes can it handle in the mesh network
what if I want to setup multiple servers?
b

bored-island-21407

04/12/2023, 11:34 AM
stress testing of server is ongoing.
You can have multiple servers if you like. A client can join networks on multiple servers
b

big-sugar-32241

04/12/2023, 11:35 AM
is there any guide around setting up multiple servers
how they communicate with each other etc.
this is for redundancy of course
b

bored-island-21407

04/12/2023, 11:36 AM
Look in docs for HA (high availability)
There is a link to docs in UI
b

big-sugar-32241

04/12/2023, 11:37 AM
For HA -- we have options of Kubernetes and multiple EC2s right?
b

bored-island-21407

04/12/2023, 11:38 AM
I think it is k8, not 100% sure
b

big-sugar-32241

04/12/2023, 11:38 AM
ok, will check, thanks a lot
what is the difference between
register -t TOKEN
and 
join -t TOKEN
?
b

bored-island-21407

04/12/2023, 11:41 AM
join is an alias of register (used to be different)
b

big-sugar-32241

04/12/2023, 11:42 AM
one last question
when I do
./netclient_darwin_arm64 install
I get 
dragonfruit@Dragonfruits-Mac-mini Downloads % sudo ./netclient_darwin_arm64 install
open ./build/appicon.png: no such file or directory[netclient_darwin_arm64] 2023-04-12 17:10:58 setting OS 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting version 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting netclient hostid 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting name 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting macAddress 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting wireguard keys 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting wireguard interface 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting listenport 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting proxyListenPort 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting MTU 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting traffic keys 
[netclient_darwin_arm64] 2023-04-12 17:11:00 open /usr/local/bin/netclient: no such file or directory 
[netclient_darwin_arm64] 2023-04-12 17:11:00 error installing daemon open /usr/local/bin/netclient: no such file or directory
this is on a fresh new device
b

bored-island-21407

04/12/2023, 11:43 AM
You should change the name of binary to netclient
b

big-sugar-32241

04/12/2023, 11:43 AM
/usr/local/bin/netclient: no such file or directory
basically
I still get the same
even after renaming
what made it work earlier for me is that I also installed via 
Netclient-M1.pkg
but I was trying without it
b

bored-island-21407

04/12/2023, 11:45 AM
change netclient_darwin_amd64 to netclient and rerun ./netclient install
b

big-sugar-32241

04/12/2023, 11:46 AM
yes I did that, seems like 
bin
folder didn't exist for me, running again
thanks, worked
@bored-island-21407 I joined a new node again and while it appears in 
sudo wg
for both server and 1st node the 
ping
to it doesn't work and I get 
Request timed out
but all 3 nodes (1 server + 2 client) appear as Healthy in UI
10.101.0.1 is server 10.101.0.2 is 1st client 10.101.0.3 is 2nd client
b

bored-island-21407

04/12/2023, 11:57 AM
When you run wg show is a handshake showing? Rx and tx show positive numbers
is there a firewall on node3?
b

big-sugar-32241

04/12/2023, 11:59 AM
so 
ping 10.101.0.3
from the machine with 
10.101.0.2
doesnt work but it works when I do it from server aka 
10.101.0.1
so yeah, handshake is not present in 
10.101.0.3
entry when doing 
sudo wg
from client with 
10.101.0.2
b

bored-island-21407

04/12/2023, 11:59 AM
Is node3 behind NAT?
b

big-sugar-32241

04/12/2023, 11:59 AM
I am not sure, I have asked, btw, this node3 doesn't have 
wireguard-tools
installed
and ping to node3 from server (node1) works but ping to node3 from node2 doesn't work
b

bored-island-21407

04/12/2023, 12:01 PM
node2 and node3 are on different lans or the same?
b

big-sugar-32241

04/12/2023, 12:01 PM
different wifi different cities
b

bored-island-21407

04/12/2023, 12:01 PM
setup server as relay and relay node3
b

big-sugar-32241

04/12/2023, 12:02 PM
why is this needed?
also, is there a doc on how to do it?
b

bored-island-21407

04/12/2023, 12:03 PM
node2 and node3 are probably both behind NAT, depending upon the type of NAT they sometimes cannot communicate directly
b

big-sugar-32241

04/12/2023, 12:04 PM
I have UDP Hole punch enabled btw, but I guess that doesn't help?
and wouldn't relay be slower?
b

bored-island-21407

04/12/2023, 12:04 PM
On the host tab, select the server host where you can set it to be a relay
a relay adds an extra hop so it will be slower, how much depends on routes between all the nodes
b

big-sugar-32241

04/12/2023, 12:06 PM
so I go to hosts, click on server (node1) and turn on 
is relay
?
maybe I have to turn on 
is relay
for 
node3
and not 
server (node1)
right?
b

bored-island-21407

04/12/2023, 12:08 PM
server should be relay and node3 should be relayed
b

big-sugar-32241

04/12/2023, 12:08 PM
ok, I did that, still same issue cannot ping node3 from node2
Request Time out and No entry for 
latest handshake
when doing 
sudo wg
on node2
b

bored-island-21407

04/12/2023, 12:09 PM
It sometimes takes a little while for relay routing to settle down
b

big-sugar-32241

04/12/2023, 12:10 PM
is the same problem experienced with other tools such as Zerotier, Tailscale etc.?
PS -- I haven't tried them
and am new to this domain
b

bored-island-21407

04/12/2023, 12:11 PM
Yes, and they all have different methods for getting around the problem
b

big-sugar-32241

04/12/2023, 12:11 PM
I see, and I guess I came across the benchmarks article on medium do those benchmarks incorporate relaying?
b

bored-island-21407

04/12/2023, 12:12 PM
Which benchmarks?
b

big-sugar-32241

04/12/2023, 12:12 PM
https://medium.com/netmaker/battle-of-the-vpns-which-one-is-fastest-speed-test-21ddc9cd50db
b

bored-island-21407

04/12/2023, 12:13 PM
I am not sure whether relays were included in those tests
b

big-sugar-32241

04/12/2023, 12:14 PM
I see, still waiting for handshake to appear btw Should node2 be relayed as well? I ask since it could be a similar issue of node3 not being able to access node2
b

bored-island-21407

04/12/2023, 12:14 PM
If you want to understand the problem better, read up on STUN and TURN
You should only have to relay one of the nodes
b

big-sugar-32241

04/12/2023, 12:14 PM
and if relaying is turned on, does it use relay all the time OR priority is still given to UDP Hole Punch / other ways
b

bored-island-21407

04/12/2023, 12:16 PM
When relaying , all traffic to,from the relayed node goes through the relay
b

big-sugar-32241

04/12/2023, 12:17 PM
also, still waiting for handshake, any ETA available? and hmm, this relay stuff has to be manually turned on after looking into issue? can it not be automatically figured out?
I ask since if there are 100s of nodes
then manual inspection would be cumbersome
b

bored-island-21407

04/12/2023, 12:17 PM
It is being worked on
it is complicated to get correct. Especially preventing unneeded relays
b

big-sugar-32241

04/12/2023, 12:20 PM
ok, I am still waiting for relay stuff to come up
btw @bored-island-21407 sorry for tagging again node1 can ping both node2 and node3 node2 cannot ping node3 but can ping node1 node3 cannot ping both node1 and node2
b

bored-island-21407

04/12/2023, 12:28 PM
there must be a firewall that prevents node3 from communicating
it is strange that node1 can ping node3 but node3 cannot ping node1
b

big-sugar-32241

04/12/2023, 12:29 PM
this was after turning relay on
I have disabled, lets try again
seems like the latest handshake was 22 mins ago for node3
and now node1 cannot ping node3 too
b

bored-island-21407

04/12/2023, 12:31 PM
can you delete node3 and then rejoin?
b

big-sugar-32241

04/12/2023, 12:31 PM
ok
I just made it rejoin, seems like handshake entry for node3 is present in server (node1) but not in node2
but then, the handshake entry present for node3 in server (node1) doesn't update
I mean, neither node1 or node2 can contact node3
relay is disabled btw
b

bored-island-21407

04/12/2023, 12:41 PM
for the relay to work; the relay and relayed need to be able to communicate
so in your case node1 and node3 need to communicate or relay won't do anything
b

big-sugar-32241

04/12/2023, 12:42 PM
if node2 pings node3, request timed out if node1 pings node3, the terminal has not output and hangs aka something like this 
ubuntu@ip-172-31-94-115:~$ ping 10.101.0.3
PING 10.101.0.3 (10.101.0.3) 56(84) bytes of data.
b

bored-island-21407

04/12/2023, 12:43 PM
what is output of ping -c 4 10.101.0.3
b

big-sugar-32241

04/12/2023, 12:44 PM
PING 10.101.0.3 (10.101.0.3) 56(84) bytes of data.

--- 10.101.0.3 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3052ms
this is from server (node 1)
b

bored-island-21407

04/12/2023, 12:45 PM
can you ping the publicip of node1 from the server?
b

big-sugar-32241

04/12/2023, 12:45 PM
this is from node2 
PING 10.101.0.3 (10.101.0.3): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

--- 10.101.0.3 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
> publicip of node1 from the server? server is node1 only you mean something else I guess?
b

bored-island-21407

04/12/2023, 12:46 PM
node3
typo
b

big-sugar-32241

04/12/2023, 12:47 PM
so its 171.76.80.43 according to google search of whats my IP
so it's the same
PING 171.76.80.43 (171.76.80.43) 56(84) bytes of data.

--- 171.76.80.43 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3059ms
b

bored-island-21407

04/12/2023, 12:48 PM
there has to be a firewall in front of node3
b

big-sugar-32241

04/12/2023, 12:49 PM
I see, and is relay able to solve this?
not really, right?
b

bored-island-21407

04/12/2023, 12:49 PM
depends on firewall settings
if it is blocking everything, then there is not much you can do
b

big-sugar-32241

04/12/2023, 12:50 PM
is the same issue faced by Zerotier et al i.e. other tools as well? aka no workaround in such a case?
b

bored-island-21407

04/12/2023, 12:51 PM
wg has to establish a handshake; until that is done, not much any tool can do about it
b

big-sugar-32241

04/12/2023, 12:52 PM
but handshake for node3 was present in 
sudo wg
output of 
node1
, but not in output of 
node2
b

bored-island-21407

04/12/2023, 12:53 PM
that is not unusual if node3 and node2 are both behind NAT.... node3 can see node1 because node1 is a vps that does not have NAT
b

big-sugar-32241

04/12/2023, 12:54 PM
but yeah, handshake for node3 doesn't update, I mean, the timestamp doesnt update
b

bored-island-21407

04/12/2023, 12:54 PM
the persistent keepalive is 20 seconds, correct?
b

big-sugar-32241

04/12/2023, 12:54 PM
yes
interface: netmaker
  public key: wKLx81Cv6Fx5WvWl8xtV+2jG55zpw+zNCgG1nOpUORE=
  private key: (hidden)
  listening port: 51821

peer: sopYNdvjx5ZTQy9feW36oEkaLwR3ZCHKaSdycAMSCRs=
  endpoint: 182.69.183.89:7543
  allowed ips: 10.101.0.2/32
  latest handshake: 20 seconds ago
  transfer: 26.16 KiB received, 19.18 KiB sent
  persistent keepalive: every 20 seconds

peer: K1/sc53TDIp786+jzAY0wl0evSQCbYTPj3ab3n2dAgk=
  endpoint: 127.0.0.1:53751
  allowed ips: 10.101.0.3/32
  latest handshake: 17 minutes, 40 seconds ago
  transfer: 212 B received, 29.89 KiB sent
  persistent keepalive: every 20 seconds
this is output from node1
b

bored-island-21407

04/12/2023, 12:56 PM
what is output from node3
b

big-sugar-32241

04/12/2023, 12:56 PM
node3 doesn't have 
wireguard-tools
installed
b

bored-island-21407

04/12/2023, 12:57 PM
ok
b

big-sugar-32241

04/12/2023, 12:57 PM
and doesn't have 
brew
also installed + some other person has control over it so I have to ping him to run commands etc 😅
#netmakerJoin Discord
Powered by