A few questions and suggestions about wag

[this-username-has-been-taken]

Hello!

First of all I'd like to thank the author for such a great tool! There are not many options for Wireguard authentication exist and this one combines just the right features! I hope the project will keep going!

I got myself a bit familiar with the app and would like to ask a few question about running it into production mode and make a few suggestions.

Questions

1. I am trying to run wag without NAT and so far had no luck. I have two LAN's defined on router: 172.17.17.0/24 - main LAN and 172.17.18.0/24 - for wag clients. The wag host itself is located in the main LAN, but wireguard IP is set to 172.17.18.0/24 (second LAN IP range). I can establish connection to wg, but have no access to 172.17.17.0/24 (DNS, Keycloak, any other services). I have added these hosts to allow group, but that didn't work. I also put another host in the 172.17.18.0/24 subnet and I can access both nets from it: thus it is not a firewall issue... I appreciate any suggestion for further investigation and fixing it.
   If I use NAT, everything works fine.
2. Is is somehow possible to adjust PersistentKeepalive parameter in a client's config? It is always set to 10 and I didn't find any setting for that.
3. I found out that not all options are loaded (or at least picked by the service) from the config file after service restart. E.x. if I change NAT option - it value will be picked, but if I change DownloadConfigFileName - it won't be picked by the service. Could you please tell me what options are ignored in the config file after initial setup?
4. I was unable to find devices.db file anywhere. I believe the data structure has been changed since ver.8. Is this file still present?
5. What is recommended backup strategy for the service? I'd like to backup settings, users db and their wireguard configs.
6. I am using Let's Encrypt certs and have to renew them every now and then. Should I restart the service after renewing a certificate to apply it or it will be parsed without restart?
7. I was unable to overwrite a client's address due to error: it might be a bug.
   
   The log says:

Nov 09 11:56:19 mar-vpn-01 wag[448]: 2024/11/09 11:56:19 http: panic serving @: runtime error: index out of range [0] with length 0
Nov 09 11:56:19 mar-vpn-01 wag[448]: goroutine 9686 [running]:
Nov 09 11:56:19 mar-vpn-01 wag[448]: net/http.(*conn).serve.func1()
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:1947 +0xbe
Nov 09 11:56:19 mar-vpn-01 wag[448]: panic({0x144fa60?, 0xc000210b88?})
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/runtime/panic.go:785 +0x132
Nov 09 11:56:19 mar-vpn-01 wag[448]: github.com/NHAS/wag/internal/data.AddRegistrationToken({0xc000c0c880, 0x40}, {0xc00093dcf4, 0x7}, {0xc00093dcd6, 0xd}, {0x0, 0x0, 0x0}, 0x1)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /home/nhas/go/src/github.com/NHAS/wag/internal/data/registration.go:144 +0x465
Nov 09 11:56:19 mar-vpn-01 wag[448]: github.com/NHAS/wag/internal/data.GenerateToken({0xc00093dcf4, 0x7}, {0xc00093dcd6, 0xd}, {0x0, 0x0, 0x0}, 0x1)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /home/nhas/go/src/github.com/NHAS/wag/internal/data/registration.go:118 +0xb2
Nov 09 11:56:19 mar-vpn-01 wag[448]: github.com/NHAS/wag/pkg/control/server.newRegistration({0x1aa5830, 0xc00049f340}, 0xc000f27180)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /home/nhas/go/src/github.com/NHAS/wag/pkg/control/server/registrations.go:103 +0x4fe
Nov 09 11:56:19 mar-vpn-01 wag[448]: github.com/NHAS/wag/pkg/httputils.(*HTTPUtilMux).Post.(*HTTPUtilMux).AllowedMethods.func1({0x1aa5830, 0xc00049f340}, 0xc000f27180)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /home/nhas/go/src/github.com/NHAS/wag/pkg/httputils/util.go:25 +0x194
Nov 09 11:56:19 mar-vpn-01 wag[448]: net/http.HandlerFunc.ServeHTTP(0xc000145c00?, {0x1aa5830?, 0xc00049f340?}, 0x9185c7?)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:2220 +0x29
Nov 09 11:56:19 mar-vpn-01 wag[448]: net/http.(*ServeMux).ServeHTTP(0xc00080eb20?, {0x1aa5830, 0xc00049f340}, 0xc000f27180)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:2747 +0x1ca
Nov 09 11:56:19 mar-vpn-01 wag[448]: net/http.serverHandler.ServeHTTP({0x1aa1460?}, {0x1aa5830?, 0xc00049f340?}, 0x6?)
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:3210 +0x8e
Nov 09 11:56:19 mar-vpn-01 wag[448]: net/http.(*conn).serve(0xc00103b680, {0x1aaa6d8, 0xc00075ed20})
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:2092 +0x5d0
Nov 09 11:56:19 mar-vpn-01 wag[448]: created by net/http.(*Server).Serve in goroutine 296
Nov 09 11:56:19 mar-vpn-01 wag[448]:         /usr/lib/go/src/net/http/server.go:3360 +0x485
Nov 09 11:56:19 mar-vpn-01 wag[448]: 2024/11/09 11:56:19 unable to create new registration token:  Post "http://unix/registration/create": EOF

8. It looks like the DNS servers are not displayed correctly if there are more than one defined on the Settings page. It is also impossible to save them (workaround: put a new line between the servers and save):
   

Suggestions

1. When downloading a client's config a new line is always appended on top of the file. It is not necessary and can be committed.
2. Add possibility to put a FQDN name in the DNS field in the Settings. If a FQDN name is defined in the DNS section of a wireguard config it will be put as DNS suffix into the connection (adapter settings).
3. It looks like that reload command (see https://github.com/NHAS/wag?tab=readme-ov-file#management) is not implemented. I suggest to remove it form the docs.

[NHAS]

Hey all of these are great suggestions. Thanks for your feedback.

I've been a tad busy working on other things right at the moment but I'll answer some questions that you've posed.

1. Have you setup a reverse route from your hosts? Otherwise they just see traffic coming from the wag subnet and might not know where to send their packets to return it.
2. Nope
3. Yeah sorry about this, in an effort to move all the configuration file options into the web UI its created a bit of an inconsistent state for what can be set in UI and what can be set in the config.
   The general rule of thumb, is that anything that would require a full service start can only be defined in the config, otherwise the admin UI should allow you to set it.
4. device.db is a legacy hangover from when wag used sqlite as its primary datastore, this changed in version v8.0.0 with the move to ETCD you can just ignore it.
5. The easiest way of doing this is adding a witness node to wag, so you have another wag service running an etcd node, but not actually starting the vpn, then all the settings, wireguard peers and rules will be automatically synced to the witness on change.
   Due to the way etcd works you may want to set up two witnesses, as etcd requires 3 devices for quorum (i.e if one of the two nodes goes down wag will stop working)
6. You will have to restart the service, this is something I've been meaning to add to wag for quite some time. Its just a bit difficult to update the cert for the internal network as relying on acme http-01 is often easiest and this would require DNS proof
7. This is definitely a bug
8. Also a bug

[NHAS]

Feel free to raise all your suggestions and questions 7 and 8 as issues :)

[this-username-has-been-taken]

Thank you for the reply!

Regarding the first question: running WAG in non-masquerading (non-NAT) mode. I have figured it out. A small manual that might be useful for OPNSense users:

1. Install WAG on a host inside your LAN. Because Wireguard is a Layer 3 protocol - there is no need in creating a new (separate) network on your router for WAG. But make sure that Wireguard network has different address that your LAN otherwise you will get routing mess. Remember the wireguard network address (usually x.x.x.0/24).
2. On your WAG server enable net.ipv4.ip_forward=1 and net.ipv4.conf.all.proxy_arp=1 options. Make its IP static and remember it.
3. Create a Port forward rule: from WAN to the WAG Wireguard port.
4. Create a Pass rule in the WAN firewall section to allow input traffic on the forwarded port.
5. Create a Pass rule in the LAN firewall section to allow connections from the WAG Wireguard network (the one you chose in step 1).
6. Go to Firewall -> Settings -> Advanced and enable "Static route filtering" setting. This is required to resolve asymmetric routing in the OPNSense.
7. Go to System -> Gateways -> Configuration and add a new gateway in the LAN interface with the priority less than WAN gateway and address pointing to the Wireguard server (from step 2).
8. Go to System -> Routes and create a new route to WAG Wireguard network address via freshly created Gateway.
9. Go to Firewall -> NAT -> Outbound. Set "Hybrid outbound NAT rule generation" mode and add a new rule: Interface = WAN; Source = Wireguard network.
10. Set up WAG with NAT option set to false.
11. That's it. You should be good to go. I also recommend to create a firewall alias for the WAG Wireguard network: it is much easier to manage it that way.

I will report bugs and suggestions as issues.

For now I got one more question.
I am using a Keycloak to manage user access thus I have to put the Keycloak server IP into the Allowed section of WAG ACLs.
But the server that runs Keycloak also runs a few other services that I'd like users to access, but only after authorization.
I haven't figured how to set ACLs to allow unauthorized users to access only Keycloak's IP and 443/tcp port before authorization (otherwise they won't be able to authorize) and to allow them full access to the server after successful authorization.

What I tried is: putting the whole network (172.20.20.0/24) under the Mfa section and the Keycloak service (172.20.20.20 443/tcp) under the Allowed section. This will provide access to Keycloak, but won't provide access to other services located on 172.20.20.20 after authentication.
How to resolve this situation?

[this-username-has-been-taken]

Hmm. The same situation happens with access to DNS servers.
WAG puts them in the Allow section by default with port definition: 53/tcp. If I need to access the DNS server by another port (e.x. for administrative tasks) after authorization - there will be no access even if whole subnet with DNS server is mentioned in the Mfa section.

[this-username-has-been-taken]

Answering my own question regarding not being able to access either Keycloak or any other service except Keycloak: I found a solution.
To resolve this ACL rule I put all the ports + icmp protocol explicitly in the Mfa section except for the Keycloak port and protocol (they are kept in the Allow section).

E.x.
If you have a Keycloak listening on 10.10.10.11:8443 and would like to have access only to the Keycloak service before authorization and to the whole server after you should put:
10.10.10.11/32 1-8442/any 8443/udp 8444-65534/any icmp into the Mfa section and
10.10.10.11/32 8443/tcp into the Allow section.
I.e. explicitly define all the port ranges and protocols.

The same principal works with DNS. If you want to access all ports and protocols on the DNS server after authorization, put {DNS IP}/32 1-52/any 54-65535/any icmp into the Mfa section. You don't have to put anything in the Allow section here because WAG will do it for you automatically.

[krambrock]

Answering my own question regarding not being able to access either Keycloak or any other service except Keycloak: I found a solution. To resolve this ACL rule I put all the ports + icmp protocol explicitly in the Mfa section except for the Keycloak port and protocol (they are kept in the Allow section).

E.x. If you have a Keycloak listening on 10.10.10.11:8443 and would like to have access only to the Keycloak service before authorization and to the whole server after you should put: 10.10.10.11/32 1-8442/any 8443/udp 8444-65534/any icmp into the Mfa section and 10.10.10.11/32 8443/tcp into the Allow section. I.e. explicitly define all the port ranges and protocols.

The same principal works with DNS. If you want to access all ports and protocols on the DNS server after authorization, put {DNS IP}/32 1-52/any 54-65535/any icmp into the Mfa section. You don't have to put anything in the Allow section here because WAG will do it for you automatically.

I had kind of the same story with the dns-server and filed it in a (rejcted) but: #90. I'll also put my code for the config.json here, it might help explaining it for some people:

I found out how to solve it (in case anybody has the same problem). Having this:

"Wireguard": {
    "DevName": "wg0",
    "ListenPort": 5555,
    "PrivateKey": "*** MYKEY ***",
    "Address": "10.1.2.1/24",
    "MTU": 1420,
    "DNS" : ["192.168.2.1", "192.168.2.254"]
}

You need this ACLs:

"Acls": {
	"Policies": {
		"*": {
			"Mfa": [
				"192.168.2.0/24",
				"192.168.2.1/32 1-52/any 54-100000/any icmp",
				"192.168.2.254/32 1-52/any 54-100000/any icmp"
			]
		}
	}
}

Leading to this diagnostics:

{
    "user": {
        "Policies": [
            "10.1.2.1/32 policy [public(20) any/any]",
            "192.168.2.1/32 policy [mfa(8) 1-52/any mfa(8) 54-34464/any mfa(16) any/icmp public(20) 53/any]",
            "192.168.2.254/32 policy [mfa(8) 1-52/any mfa(8) 54-34464/any mfa(16) any/icmp public(20) 53/any]",
            "192.168.2.0/24 policy [mfa(16) any/any]"
        ],
        "Devices": [
            {
                "LastPacketTimestamp": 0,
                "Expiry": 0,
                "IP": "10.1.2.2",
                "Authorized": false
            }
        ],
        "AccountLocked": 0
    }
}

It's a bit suprising to me and maybe not most elegant but it works. The dns-server is accessable via dns only without mfa and is fully accessable with mfa.

[NHAS]

Thanks for writing these solutions out!

Wag works by using the rule that is the most specific to determine what policies it should apply. So if you have a subnet like /24 then a rule that is defined per host /32 is more specific so it uses those policies.

The order of preference for policies goes like this deny -> mfa -> allow, so if you have multiple conflicting policies for a single port (or range) they will be evaluated like that.

Sorry for not responding, I have been unwell!

[NHAS]

I've added your suggestions into the unstable branch

[this-username-has-been-taken]

First of all I hope you are feeling better!

Thank you for the reply :)
I have put WAG into service. So far everything is great and the users like Wireguard + authentication combo.
The only enhancement request I got from users was to implement redirect to WAG login page if needed, but AFAIK that is not possible due to Wireguard protocol limitations.

[NHAS]

It's not, not possible. It's just very difficult to implement and won't work for tls