https://netmaker.org logo
Powered by
Title
r

refined-doctor-2016

09/15/2022, 10:39 AM
I still cant get to ingress client to stop being "NATed" behind the ingress server (also acting as the netmaker server-docker). RCE=on and postup/down has masquerade command removed as advised but I Ext-Clients are still NAT behind the servers IP. Inside the netmaker docker I see: iptables -t nat -S -P PREROUTING ACCEPT -P INPUT ACCEPT -P OUTPUT ACCEPT -P POSTROUTING ACCEPT -A PREROUTING -p tcp -m tcp --dport 53 -j DNAT --to-destination 172.21.0.3:53 -A PREROUTING -p udp -m udp --dport 53 -j DNAT --to-destination 172.21.0.3:53 -A POSTROUTING -j MASQUERADE
So there is still some masquerading going on but its not defined per "network"
b

bored-island-21407

09/15/2022, 10:41 AM
edit the postup/postdown fields in your ingress gateway node
r

refined-doctor-2016

09/15/2022, 11:16 AM
I did edit them out. I removed on both the "masq" commands and not only restarted docker but the full VM... :S
Maybe Ive missed something stupid :S
my current postup on the netamker server (and ingress gate) is: iptables -A FORWARD -i nm-netmain -j ACCEPT ; iptables -A FORWARD -o nm-netmain -j ACCEPT
@bored-island-21407 any ideas?
b

bored-island-21407

09/15/2022, 12:34 PM
what is in /etc/netclient/config in container
r

refined-doctor-2016

09/15/2022, 12:43 PM
[Interface] PrivateKey = +38= ListenPort = 51821 Address = 10.200.200.254 PostUp = iptables -A FORWARD -i nm-netmain -j ACCEPT PostUp = iptables -A FORWARD -o nm-netmain -j ACCEPT PostDown = iptables -D FORWARD -i nm-netmain -j ACCEPT PostDown = iptables -D FORWARD -o nm-netmain -j ACCEPT MTU = 1280 [Peer] PublicKey = = AllowedIps = 10.200.200.100/32, 10.10.0.0/24, 10.10.10.0/24 PersistentKeepalive = 20 [Peer] PublicKey = = AllowedIps = 10.200.200.120/32 PersistentKeepalive = 20 [Peer] PublicKey = += AllowedIps = 10.200.200.30/32, 10.30.0.0/24 PersistentKeepalive = 20 [Peer] PublicKey = = AllowedIps = 10.200.200.101/32 PersistentKeepalive = 20 [Peer] PublicKey = = AllowedIps = 10.200.200.7/32 [Peer] PublicKey = += AllowedIps = 10.200.200.6/32
less /etc/netclient/config/nm-netmain.conf
b

bored-island-21407

09/15/2022, 12:46 PM
Your third peer has strange allowedips
r

refined-doctor-2016

09/15/2022, 12:46 PM
"10.200.200.30/32, 10.30.0.0/24"?
b

bored-island-21407

09/15/2022, 12:47 PM
Is it an egress gateway
r

refined-doctor-2016

09/15/2022, 12:47 PM
yes. 10.30.0.0/24 and 10.10.10.0 are egress
peer 1 and 3
works great
b

bored-island-21407

09/15/2022, 12:49 PM
what happens if you run wg-quick down and wg-quick up in the container
r

refined-doctor-2016

09/15/2022, 12:49 PM
.7 and .6 are EXT-cli with ingress on main server 10.200.200.254
wg-quick down nm-netmain wg-quick: `/etc/wireguard/nm-netmain.conf' does not exist
b

bored-island-21407

09/15/2022, 12:50 PM
have you tried putting the ingress on a different node
You need the full path to the conf file
r

refined-doctor-2016

09/15/2022, 12:51 PM
no I havent.
bash-5.1# wg-quick down /etc/netclient/config/nm-netmain.conf Warning: `/etc/netclient/config/nm-netmain.conf' is world accessible [#] ip link delete dev nm-netmain [#] iptables -D FORWARD -i nm-netmain -j ACCEPT [#] iptables -D FORWARD -o nm-netmain -j ACCEPT bash-5.1# wg-quick up /etc/netclient/config/nm-netmain.conf Warning: `/etc/netclient/config/nm-netmain.conf' is world accessible [#] ip link add nm-netmain type wireguard [#] wg setconf nm-netmain /dev/fd/63 [#] ip -4 address add 10.200.200.254 dev nm-netmain [#] ip link set mtu 1280 up dev nm-netmain [#] ip -4 route add 10.200.200.7/32 dev nm-netmain [#] ip -4 route add 10.200.200.6/32 dev nm-netmain [#] ip -4 route add 10.200.200.30/32 dev nm-netmain [#] ip -4 route add 10.200.200.120/32 dev nm-netmain [#] ip -4 route add 10.200.200.101/32 dev nm-netmain [#] ip -4 route add 10.200.200.100/32 dev nm-netmain [#] ip -4 route add 10.30.0.0/24 dev nm-netmain [#] ip -4 route add 10.10.10.0/24 dev nm-netmain [#] ip -4 route add 10.10.0.0/24 dev nm-netmain [#] iptables -A FORWARD -i nm-netmain -j ACCEPT [#] iptables -A FORWARD -o nm-netmain -j ACCEPT bash-5.1#
b

bored-island-21407

09/15/2022, 12:53 PM
That looks good
r

refined-doctor-2016

09/15/2022, 12:53 PM
still after down and up. firewall on 10.10.0.0 sees the ingrees traffic as "10.200.200.254" .. not a .6
.254 seems to NAT ext clients :S no?
b

bored-island-21407

09/15/2022, 12:54 PM
Probably due to docker networking
r

refined-doctor-2016

09/15/2022, 12:55 PM
maybe run --host?
its a linode. only running netmaker
b

bored-island-21407

09/15/2022, 12:57 PM
in order to run in host mode, you have to bind mount some other directories but is doable
other option is to run netclient on host and use that node as the gateway
r

refined-doctor-2016

09/15/2022, 12:58 PM
aha!!! is it ok to run "another" netclient on the VM outside docker?
b

bored-island-21407

09/15/2022, 12:58 PM
yep
r

refined-doctor-2016

09/15/2022, 12:58 PM
cool! I can try it.
Ill have to hardcode other 51XXX ports to it, I assume
b

bored-island-21407

09/15/2022, 12:59 PM
yes, you will need to use something outside of 51821-51830 as those are mapped to netmaker container
r

refined-doctor-2016

09/15/2022, 1:06 PM
wow weird
[netclient] 2022-09-15 13:05:34 joining netmain at api.nm.176-58-97-100.nip.io:443 [netclient] 2022-09-15 13:05:34 error installing: error creating node 500 Internal Server Error {"Code":500,"Message":"Key: 'Node.LocalAddress' Error:Field validation for 'LocalAddress' failed on the 'ip' tag"} [netclient] 2022-09-15 13:05:34 error running command: wg-quick down /etc/netclient/config/.conf [netclient] 2022-09-15 13:05:34 wg-quick: `/etc/netclient/config/.conf' does not exist
is it because: VERSION: v0.15.2
on netclient?
b

bored-island-21407

09/15/2022, 1:07 PM
no
not sure why you are getting that error but you can set the LocalAddress by using the --localaddress flag in the join command
r

refined-doctor-2016

09/15/2022, 1:12 PM
ok --localaddress with the server public address worked. maybe it was becuase both server and netclient are on the same vm
@bored-island-21407 starting netclient on the same vm with the netmaker server kill network access on the VM and I think everything goes south..
nm dashboard is dead too
root@ubuntulondon:~# systemctl start netclient.service root@ubuntulondon:~# client_loop: send disconnect: Broken pipe
haha
b

bored-island-21407

09/15/2022, 1:29 PM
run the daemon from the command line 
netclient daemon -vvvv
...... any errors
r

refined-doctor-2016

09/15/2022, 1:38 PM
the new netclinet seems ok via daemon:
[netclient] 2022-09-15 13:37:37 [mqpublish.go-46] checkin(): checkin with server(s) for all networks [netclient] 2022-09-15 13:37:37 [mqpublish.go-145] Hello(): checkin for netmain complete [netclient] 2022-09-15 13:37:53 [mqhandlers.go-204] UpdatePeers(): server/client version mismatch server: v0.15.1 client: v0.15.2 [netclient] 2022-09-15 13:37:54 [mqhandlers.go-252] UpdatePeers(): network: netmain received peer update for node londonMainGate netmain
from the `vm I can ping out. but others cant ping in
maybe this is a place we use "is local"?
b

bored-island-21407

09/15/2022, 1:39 PM
you should run the same version of server and client
r

refined-doctor-2016

09/15/2022, 1:39 PM
my endpoint IP is the same with LocalAddress
b

bored-island-21407

09/15/2022, 1:40 PM
your LocalAddress should be one of your private ip bound to your main interface device
r

refined-doctor-2016

09/15/2022, 1:42 PM
sorry now Im confused
we are working on the same VM as the netmake docker server.
b

bored-island-21407

09/15/2022, 1:43 PM
what is the output of 
ip a
r

refined-doctor-2016

09/15/2022, 1:44 PM
root@ubuntulondon:~# ip a |grep inet inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host inet 176.58.97.123/24 brd 176.58.97.255 scope global eth0 inet6 2a01:7e00::f03c:93ff:fef9:4ed1/64 scope global dynamic mngtmpaddr noprefixroute inet6 fe80::f03c:93ff:fef9:4ed1/64 scope link inet 172.21.0.1/16 brd 172.21.255.255 scope global br-ba5355db619b inet6 fe80::42:2aff:fe71:460c/64 scope link inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 inet6 fe80::7031:55ff:fece:69cd/64 scope link inet6 fe80::4476:47ff:feb9:9b6d/64 scope link inet6 fe80::c0d7:e4ff:fe3f:89b7/64 scope link inet6 fe80::7014:ddff:fecd:f943/64 scope link inet6 fe80::687d:7ff:fe6a:a44c/64 scope link inet 10.200.200.250/32 scope global nm-netmain
I used 176.58.97.123 as localaddress
b

bored-island-21407

09/15/2022, 2:01 PM
that is a public ip
r

refined-doctor-2016

09/15/2022, 2:01 PM
true
and the ip of the netmaker server
:S
b

bored-island-21407

09/15/2022, 2:02 PM
add a private ip, such as 192.168.12.24 to eth0 and then use that as your LocalAddress
r

refined-doctor-2016

09/15/2022, 2:03 PM
hm ok! Ill try it
@bored-island-21407 has anyone tested running a second netclinet outside the docker where the netmaker is?
ping worked nice with the new "private" local address
b

bored-island-21407

09/15/2022, 2:15 PM
have done it in the past; not recently
r

refined-doctor-2016

09/15/2022, 2:16 PM
when I set ingress... everything is dead
maybe some networking loop
b

bored-island-21407

09/15/2022, 2:16 PM
possibly...
have you tried putting the ingress on another node (one that is publicly accessible, i.e. it has a static public ip)
r

refined-doctor-2016

09/15/2022, 2:17 PM
it tries to find the public address
[netclient] 2022-09-15 14:16:18 [mqpublish.go-62] checkin(): error encountered checking public ip addresses: public address not found [netclient] 2022-09-15 14:16:18 [mqpublish.go-76] checkin(): network: netmain local Address has changed from [176.58.97.123] to 176.58.97.123
it tries to override the localaddress and it gets confused with the main server (inside the docker)
ok I think Ill try to run ingress on another box.
#clientJoin Discord
Powered by