Migrating from pfSense to OPNsense

A few weeks ago, Netgate announced that they were making changes to the support subscriptions for some versions of pfSense (https://www.netgate.com/blog/addressing-changes-to-pfsense-plus-homelab).  I’d already been considering switching to another software defined firewall/router OS for some time, this was the kick in the ass I needed.  As a pfSense CE user, this change doesn’t directly affect me - for now.  But my gut tells me that it’s only a matter of time before it does, and I decided to be proactive about migrating rather than be forced to do so after something bad happens (like Netgate deciding to stop offering pfSense CE, or paywalling features I need).

I’m not going to lie, the migration wasn’t exactly smooth, but it was worthwhile and makes me wish I hadn’t bet on the wrong horse back when I started using pfSense back in ~2015-2016.  Prior to the switch, I was using DD-WRT and I can’t express in words how much of a step up in functionality it was going from custom firmware on a consumer grade router to running a proper software defined firewall/router on dedicated hardware.  So while I appreciate pfSense for the years of service, it was just time to move on to something new.  While pfSense and OPNsense are both FreeBSD based and largely run the same exact services, with OPNsense tending to be on newer versions, the implementation of features is quite different

The Hardware:

When I moved into my new house back in July, out of necessity I ended up using one of my old Mac Pros as my pfSense router, because for whatever reason at my new house, I simply could not get an external IP address on the old pfSense router I’d been using for the past several years (a Lenovo M93P Tiny), so I was forced to go with something that had dual NICs and at the time the Mac Pro was the only thing I had on hand that fit the bill.  Clocking in at 24 cores @3.6Ghz, 128GB of RAM and booting from a 1TB NVMe drive, the MacPro was massively overpowered to be relegated to router duty, but it got me up and running.  The plan was always to migrate from the Mac Pro to some kind of NUC clone with dual NICs.  I eventually settled on this one  https://www.amazon.com/gp/product/B0C339KVH9/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1 a Beelink Mini-PC.  With a quad core, 16GB of RAM and perhaps most importantly, dual 1GBe NICs, it checked the boxes.  I don’t own any gear with 10GBe or even 2.5Gbe, so this was sufficient.

The Migration:

In order to actually pull off the migration from pfSense without causing total chaos in my home network, I had to do some gymnastics.  With the existing pfSense router in place, what I did was install OPNsense on new router box, with the WAN port on the OPNsense router plugged in to the LAN (aka VLAN0) on my existing pfSense home network, and then gave the LAN network on the OPNsense box a different subnet address than my pfSense LAN.  By default, both use 192.168.1.0/24 for the LAN, which needs to be changed on the subordinate router (the OPNsense box) or else you run into routing issues because you LAN and WAN cannot exist in the same subnet.   Any subnet you create on the subordinate router that conflicts with the same subnet on the master router will become unroutable.  In order to get past this on the LAN, I simply changed the LAN subnet to 192.168.2.0/24 temporarily to work around this.

For initial setup, I just ran through the wizard which takes you through the basic configuration of your WAN, LAN and DNS.  What you get with an out of the box install of either pfSense or OPNsense is a pretty bare bones router - basically a blank slate with a single /24 network and DNS domain.

Once I was able to get out to both my pfSense LAN and out to the internet from my workstation plugged into the LAN port on the OPNsense router, I set out to begin copying configs over from pfSense to OPNsense.  One thing to keep in mind, if you create an exact duplicate entry of anything on the subordinate router, eg; a VLAN or a DNS entry, you lose the ability to access the actual thing on the master router from behind NAT, which makes troubleshooting kind of a pain if you do anything wrong.  That being said, for the 'basics' of networking, most things are basically a 1:1 from pfSense to OPNsense

The rest of the migration process was really just a logical check list of services, so the rest of this document will follow that flow.

One thing to note is that configuring SSL is done under System > Trust > Certificates, and then once you’ve added an SSL cert you can set the Web UI to use it under System > Settings > Administration

You probably also want to make sure hardware acceleration and temperature sensors are turned on.  The actual settings for your hardware may vary but my CPU supports QAT.  Just set thermal sensors to whatever brand CPU you have.

Interfaces:

The first thing I did was copy all of my interfaces from pfSense to OPNsense.  I have several VLANs, subnets and DHCP segments that needed to be copied over.  My VLAN/subnet scheme consists of VLAN tags on multiples of 10, with a single /24 subnet on each, eg;

VLAN0(LAN) = 192.168.1.0/24
VLAN10 = 192.168.10.0/24
VLAN20 = 192.168.20.0/24
VLAN30 = 192.168.30.0/24

Pretty straightforward.  Just keep in mind, once you create a conflicting subnet on the subordinate router, that same subnet on the master router becomes unroutable, so you can deal with this in a couple different ways:

  1. Do it anyway and just cross your fingers you got everything right and nothing blows up when you eventually cut over the routers (least amount of work, might cause things to blow up)
  2. you can create differing configurations for everything and then change them back to the original values once you cut over the routers (this will DEFINITELY cause things to blow up, but should be recoverable, but takes a lot of additional work)
  3. you could set up completely different subnets on the new router and go back and re-IP literally everything after you cut over the routers (will DEFINITELY cause EVERYTHING to blow up, and will likely NOT be recoverable for some services, and will take the most amount of work)

I’m lazy… I chose option 1.  So with my VLANs and subnets in place, and everything except the LAN subnet non-routable, I moved on to the next major pain in the ass.

DHCP:

Thankfully, I only use DHCP on "client" subnets so there wasn’t a ton for me to configure.  Infrastructure things like iSCSI, vMotion, or management interfaces I typically just use static IPs because… I’m lazy.  In my case I only have 3 subnets that have DHCP enabled; my LAN (VLAN0), my Openshift network (VLAN10) and my VM network (VLAN20).

I just copied the values from the DHCP server settings on pfSense and applied them to the OPNsense server.  Most of the fields are more or less the same, just keep in mind a couple of things.

First, make sure to re-create all the static mappings, and double check all the advanced DHCP options.  I use PXE boot to install my Openshift bare metal nodes and there are a couple of extra fields you have to configure for that to work.  For PXE boot you should configure the field under ’TFTP server’ on a per VLAN basis.  Just an observation of one of the quirks of BSD networking, a lot of the time if a service has to cross subnets and those subnets are not bridged (which I typically do not do for security reasons) you’re going to incur some pretty big network throughput penalty because the packet has to make multiple hops; server (VLAN1) > switch > router (VLAN1) > router (VLAN2) > switch > server (VLAN2) instead of making a single hop.  Not really sure why BSD handles things that way but that’s been my observation thus far.  So for things like TFTP (aka; PXE boot) you really want to have the TFTP server listening on the interface for the VLAN the client resides on - in this case VLAN10 (my Openshift VLAN).  So make sure that you configure your TFTP server to listen on the VLAN it’s going to be serving (or set it to 0.0.0.0, but that might cause unexpected consequences, like devices reimagine themselves on reboot… just be smart about how you configure it.  I would not ever recommend turning on your PXE server for you LAN).

You need to configure 2 settings for TFTP:

TFTP Hostname - this is often referred to in PXE booting as "next server".  this should be the IP address of the PXE server, which in my case is the IP of the VLAN10  interface on OPNsense (we’ll get into configuring TFTP later on)

Set Bootfile - in pretty much all cases, this should be pxelinux.0.  It’s the PXE bootable binary file that TFTP serves to clients, and is part of the 'syslinux' package on any Linux distro.

TFTP server options on OPNsense

And just for good measure/backwards compatibility, I typically set these options manually under 'Additional Options'.  As per the DHCP spec, these two options are option 66 and 67 respectively.

DHCP options 66 and 67

DNS:

DNS was the first big pain in the ass I ran into, and I may end up changing it up in the future due to some nagging issues with split domain resolution while on VPN for work.  I hear that using dnsmasq instead of unbound might be a potential solution, but unbound is the default DNS server on both pfSense and OPNsense.  It’s what I’ve been using for years, and  it’s served me quite well.

So for basic DNS, setting up A records and CNAMEs is really straightforward.  Just navigate to Services > Unbound DNS > overrides and recreate all your DNS records from pfSense to OPNsense.  I had a few hundred, and not all of them were still valid, so this part is just a tedious manual grind.  Take your time, validate that everything is correct.  Just keep in mind - again, as soon as you configure the DNS override on the subordinate router, it becomes unresolvable on the master router.  The only major difference between pfSense and OPNsense implementation of Unbound here is that the Aliases (CNAMEs) are down at the bottom of the page and do not show up in the list of Host Overrides (A records).

For example, here are the aliases for my Openshift ingress VIP.  There are quite a few.  Take your time and make sure you get everything transferred over.

CNAME (alias) records

But what about things that aren’t A records or CNAMEs?  Well, for things like SRV and TXT records for services like etcd or LDAP/Kerberos, on pfSense there is actually a field that allows you to create custom entries under 'Advanced Settings'.  This field, for whatever reason, was deprecated from OPNsense a while back and now if you want to create custom DNS entries like these you have to do so manually.  Seems kind of like a huge step back in usability, but nobody asked me…  Also, just like pfSense, OPNsense does NOT support wild card A records in the host overrides field, so you’ll need to create those manually (eg; for your Openshift Ingress *.apps.cluster.domain route)

To create a custom DNS entry on OPNsense, you can upload a .conf file to the directory /usr/local/etc/unbound.opnsense.d/, for example, I have a SNO cluster.  Here’s the wildcard record

server:
local-zone: "apps.sno.ocp.cudanet.org" redirect
local-data: "apps.sno.ocp.cudanet.org 86400 IN A 192.168.10.254"

saved as /usr/local/etc/unbound.opnsense.d/sno.conf

Once you upload one or more .conf files to this directory, make sure to restart the unbound service for it to take effect.  Worth noting, OPNsense doesn’t actually read the file from here, it symlinks it to /var/unbound/etc/*.conf.  Do not manually edit files in /var/unbound/etc/ as they will get overwritten when the service restarts the next time, which if I understand correctly, actually happens every time you create a new A record.

I may end up moving to dnsmasq at some point in the future, but for now unbound is working.

Firewall Rules:

Of course, OPNsense’s first job is being a firewall, and second a router.  But you need to configure your interfaces so you have something to apply rules to.  There were no big surprises here, configuration of the firewall is actually pretty much the same between pfSense and OPNsense.  Just take your time, make sure you get everything copied over.  Forgetting to create allow rules on interfaces will create headaches later on.  I don’t do anything fancy with groups or aliases, so this part was pretty straightforward.

NAT:

Honestly, this section could/should be broken out into a few sections, as NATing alone is its own thing.  Outbound NAT should be set to automatic and should just work fine by default if you ran through the setup wizard after you deployed OPNsense.  If you use VPN (which I do) you’ll have to set up hybrid outbound NAT, but that’s going to be later on. The big things to make sure you set up under NAT for now are going to be port forwards (things that are externally accessible outside your network).


So that covers the basics.  If you don’t use any additional features on pfSense, you should be able to swap out your router at this point (and pray that all hell doesn’t break loose when everything loses it’s DHCP lease and DNS cache… which it will…).  If you do use any additional features I can only provide my experience with the ones I use, some of which are installed as additional plugins, others are available out of the box.

Authentication:

I use Active Directory for LDAP.  The setup of LDAP wasn’t exactly intuitive on OPNsense, but then again the setup of LDAP on pfSense wasn’t very intuitive either.

Under System > Access > Servers configure your LDAP server accordingly.  I have a pretty simple AD setup, and these are the values I used

The big thing is to make sure to use the full distinguished name of the bind account.

Then navigate to 'Tester' and make sure binding works

Successful test of LDAP bind
💡
My account exists in the pfadmins group - this is the group I created for pfSense administrators. I wasn’t about to rename the group and change a bunch of stuff, so I just reused the existing group membership. You’ll probably want to create an admins group specifically for OPNsense admins.

Then navigate to Groups and create a group and specify what permissions that group should have.  For an admin, set permissions to GUI All pages.  You can call it whatever you want, I probably should have called this opnadmins, but I was trying to copy what I did on pfSense.  OPNsense doesn’t pull LDAP groups like pfSense does, and as I found out, you need to create external users manually and assign them to groups yourself.

Pasted Graphic 6.png

Then in order to add an LDAP user, go to 'Users' and click the 'Import' button (the cloud with a down arrow icon).  This part wasn’t very intuitive to me…


Pasted Graphic 7.png


Then select the LDAP account you want to add.

Pasted Graphic 9.png

Then add your new user to the appropriate group (in my case, the 'pfadmins' group I created earlier).  Then log out and try logging in with your LDAP credentials.


Pasted Graphic 10.png


Success!




PXE Boot:

PXE (pre loaded execution environment) is more or less synonymous with TFTP (trivial file transfer protocol), but the two are not exactly the same thing.  Successfully PXE booting usually involves another protocol to handle "large" files while TFTP only provides small files required to kickstart the boot process.  I previously used pfSense as my PXE server and I wanted to port the same functionality over to OPNsense.  When it comes to third party software, OPNsense seems to support pretty much all the same things as pfSense, with the added bonus that the software versions available on OPNsense tend to be more updated.

To set up a PXE server on OPNsense, first you have to install the TFTP plugin, navigate to System > Firmware  > Plugins and search for os-tftp and hit the plus button to install it.  Worth noting: you might need to be on the latest version of OPNsense to install plugins, so you should probably check for updates first.  Once installed, you should see a new tab for 'TFTP' under 'Services'.  Under the 'General' tab, check the 'Enable' box and set a listen address.  You can use 0.0.0.0 to listen on all interfaces, but be careful with that.  You don’t want to accidentally re-image you laptop as an Openshift worker node…

Pasted Graphic 11.png

Click save.  However, the service will not start yet because you need to create the /usr/local/tftp directory on your OPNsense server (not very intuitive).  Then you should be able to start the TFTP service.  Then you can test that it’s working.  On the OPNsense server create a file, eg;

echo "Hello world" > /usr/local/tftp/helloworld.txt

and then on your workstation

tftp 192.168.10.1 -c get helloworld.txt
cat helloworld.txt
Hello world

So now that we’ve established that TFTP is working, we need to populate out the TFTP root with the pxelinux bootloader files.  At least on RHEL/Fedora, you can install the package syslinux-tftpboot which will install everything you need into the /tftpboot directory.  You can simply copy the entire contents of this directory over to your OPNsense server’s /usr/local/tftp/ directory, eg;

scp -r /tftpboot/* root@192.168.1.1:/usr/local/tftp/

but at this point, you still don’t have a functioning PXE server.  Sure, you can boot up a DHCP client and it will pull pxelinux.0, but without a config and a payload, it’s not going to do anything useful.

As I mentioned before, I use PXE to image my bare metal Openshift nodes for UPI installs.  Now, I could probably write an entire blog article just about setting that up, but I’ll just give the 'Readers Digest' version here.  Basically, you need to create one or more configs.  By default, any client should boot the /tftpboot/pxelinux.cfg/default config (or in the case of OPNsense, /usr/local/tftp/pxelinux.cfg/default.  So we’ll need to create that file and directory

on the OPNsense server

mkdir -p /usr/local/tftp/pxelinux.cfg

and then you can create one or more configs.  The PXE spec allows you to create per-client configs by MAC address, with the naming convention of 01-mac-address-dividied-by-dashes  (all lowercase) eg; for the MAC address 00:11:22:33:44:55:AA the config name would be 01-11-22-33-44-55-aa

This is super useful so that you can do things like automatically imaging an Openshift node as a worker, master or bootstrap node.  If you do not create a 'default' config, clients without a MAC specific config will fail to PXE boot, which can be a good safety precaution actually.

Not to go too in depth on PXE booting Linux, typically the way it works is the bootloader (pxelinux.0) loads a config file.  The config file will supply a kernel and init RAM disk via TFTP (they’re small).  Larger files like a live root file system would need to be served using some other protocol; FTP, NFS and HTTP/S are common.  In my case, I just wanted all my PXE files to be hosted by my OPNsense server, so the only option there was HTTPS.

For PXE booting RHCOS images, I placed the initramfs and kernel images in /usr/local/tftp/images (the directory does not exist by default so you have to mkdir it) and the live root filesystem and ignition files have to be served by HTTPS.  You can create arbitrary paths on the OPNsense web server under the directory /usr/local/www, eg; /usr/local/www/pxeboot

For the three node types I created individual configs called 'master', 'worker' and 'bootstrap' respectively.  Then I symlinked to the appropriate node config for each of the MAC addresses, eg;

Pasted Graphic 12.png

that way I only have to modify the three configs and the nodes will receive the changes.  Here’s an example of one of my configs to demonstrate how this works and where files should be placed

UI vesamenu.c32
MENU BACKGROUND        bg-ocp.png
MENU COLOR sel         4  #ffffff std
MENU COLOR title       1  #ffffff
TIMEOUT 10
PROMPT 0
MENU TITLE OPENSHIFT 4.x INSTALL BARE METAL PXE MENU
LABEL INSTALL MASTER
KERNEL images/rhcos-live-kernel-x86_64
APPEND initrd=images/rhcos-live-initramfs.x86_64.img coreos.live.rootfs_url=https://pxe.cudanet.org/pxeboot/rhcos-live-rootfs.x86_64.img coreos.inst.install_dev=/dev/nvme0n1 coreos.inst.ignition_url=https://pxe.cudanet.org/pxeboot/master.ign
💡
https://pxe.cudanet.org is covered by my Letsencrypt wildcard cert, which will save you a lot of headaches with PXE clients having a valid SSL cert. Also, pxe.cudanet.org is an alias that resolves to the Openshift VLAN interface IP (192.168.10.1) on my OPNsense router. That way I can have PXE use valid SSL on the specific VLAN for the clients using it, otherwise you end up with extremely slow booting. In this configuration, you can boot the RHCOS installer in under a minute, whereas I was getting timeouts or it would take several minutes (upwards of 10) to boot.

You can test this out by spinning up a VM bridged to the PXE VLAN (VLAN10 in my case) and creating the necessary symlink to one of the configs.  In fact, that’s how I do my bootstrap node.  Bootstrap is a temporary node so running it as a VM makes a lot more sense than having a dedicated physical box.  In the past, I’ve also used one of the worker nodes as a bootstrap node, and then re-imaged it as a worker and joined to the cluster after the fact, which works too, but requires additional configuration on your load balancer afterwards.

Screenshot 2023-11-18 at 10.24.55 AM.png

HAProxy:

Another service I like to run on my router is the load balancer(s) for my Openshift (or other Kubernetes) clusters.  Like with pfSense, HAproxy is available as a plugin on OPNsense.  Just navigate to 'System > Firmware > Plugins' and search for os-haproxy and click the + icon to install.

While HAproxy does the exact same things on OPNsense as it does on pfSense, the implementations are completely different and took a little extra time to wrap my head around.

First off, any ip address and port to be load balanced must first be declared under 'Real Servers' before they can be added to a backend pool which differs from pfSense’s implementation.  While I don’t necessarily think this is a bad thing, it wasn’t entirely clear that you had to create a 'server' before you could add it to a load balancer.  Also, your list of servers will get pretty large, so make sure you use a good naming convention to make it clear what servers you’re adding to which backend.

Pasted Graphic 13.png

Then once you’ve created ALL the servers for you backends, you can create your backends and populate them out with the servers from a drop down list.  Under 'Virtual Services' go to 'Back End Pools' and create a new backend.  Make sure to set the proper protocol (http or tcp) and a balancing algorithm (round robin is usually sufficient)

Pasted Graphic 14.png

Save the backend, and then repeat for all the other backends you need to create (eg; API, machine config, http and https ingress for Openshift).

Pasted Graphic 15.png

Then once you’ve created all your backends, you are going to need to create one or more VIPs (virtual IPs) to assign your frontends to.  Navigate to 'Interfaces > Virtual IPs' (unlike pfSense where VIPs are under the Firewall tab).  Create one or more VIPs.  For OCP, you’ll want to use an IP alias on the VLAN interface for your Openshift VLAN.  Make sure to check the box to 'Deny service binding' so that the OPNsense web UI does not load on this IP.  Then click save.

Pasted Graphic 16.png
💡
Openshift IPI installs require two separate VIPs for ingress and API, but UPI installs don’t. Why bother with the additional complexity if you don’t need to?

Navigate back to 'Services > HAproxy > Settings > Virtual Services > Public Services'.  A 'Public Service' in OPNsense parlance is a frontend.

Create one or more frontends to correspond with your backends.  Specify the IP address of the VIP(s) you just created, as well as the port number.  At least for Openshift, TCP should be the frontend type.  Select the appropriate backend pool for the frontend.  Again, using a good naming convention will help you keep things logical, eg; prod-ocp-api-fe is the frontend to prod-ocp-api-be on my load balancer.

Pasted Graphic 17.png

repeat as necessary to create all the appropriate frontends

Pasted Graphic 18.png

Now, the next part is not absolutely necessary in order for you load balancer to function, but it makes troubleshooting a lot easier, as this is what will populate out the stats page (which we’ll configure  next).

Go to 'Rules and Checks > Health Monitors' and create a health monitor for each of the backends you created earlier.  Setting check type to TCP and specifying the appropriate port should be all that is required for Openshift services.

Pasted Graphic 19.png


save, and repeat for each of the backends.  Now you can enable the HAproxy load balancer.  Navigate to 'Settings > Service'.  Check the 'Enable HAProxy' checkbox and apply.  You may also want to save and test syntax at this point to make sure your HAproxy config is going to be valid.  Assumign all is well, the HAproxy services should be up.


At this point, you can see the status of your load balancer backends and frontends under the 'Statistics' tab, but it’s not exactly easy to understand.  I much prefer the good old HAproxy stats page, which you can configure under 'Settings > Settings > Statistics'.  The standard port for HAproxy stats is usually port 9000, so that’s what I used.  You should specify the IP address that your OPNsense WebUI listens on (should be the LAN IP, 192.168.1.1 by default) and check the 'enabled' and 'enable remote access' boxes.  Apply and save.


Pasted Graphic 20.png


Now you should be able to navigate to http://192.168.1.1:9000/haproxy?stats to view your HAproxy stats page, but it has no refresh interval so you’d have to manually refresh the page to get updated stats.  Not ideal, so let’s configure the refresh interval.  Unfortunately, there is no tunable in the web UI for this, and the feature itself isn’t even documented.  I just figured it out by trial and error


just do


echo "     stats refresh 5s" >> /usr/local/etc/haproxy.conf


and restart the HAproxy service.  the stats page should refresh every 5 seconds now.



Pasted Graphic 21.png

Wireguard VPN:


The implementation of Wireguard on OPNsense is a bit different from pfSense, but with a little trial and error I was able to get it working.  The basic concepts are the same; Wireguard uses SSL certs for authentication between and instance and one or more peers.  Thankfully I took pretty good notes on setting up Wireguard on pfSense, which made the process of setting it up and getting things configured pretty easy by following the OPNsense docs here https://docs.opnsense.org/manual/how-tos/wireguard-client.html


It’s honestly pretty straightforward - just follow the instructions in order on Opnsense’s docs.  I would highly recommend creating an interface for Wireguard - Step 5A in the docs.


  1. install the wireguard plugin . Navigate to 'System > Settings > Firmware > Plugins' and install the plugin 'os-wireguard'
Pasted Graphic 25.png
  1. configure your Wireguard instance.  Navigate to 'VPN > Wireguard > Settings'
    create a new instance and configure it as follows:

    set 'name:' to whatever you want, eg; 'wireguard'
    generate a new publick/private keypair
    listen port should be set to 51820
    set MTU to 1420
    under 'advanced' you can specify a DNS server that clients will use, eg; 192.168.1.1
    Set 'Tunnel Address' to an IP address that is currently not in use.  While it’s not technically a VLAN on my router, I want my wireguard traffic on the 192.168.50.0/24 subnet which would be the equivalent of my VLAN50.


Pasted Graphic 26.png


  1. create an interface for Wireguard


navigate to 'Interfaces > Assignments' and click on 'New Interface' and select the wireguard devices (should be called 'wg1').  Configure the wg1 device as follows

Pasted Graphic 28.png
enabled
locked
no IPv4 or IPv6 address
💡
make sure to restart unbound DNS service in order to enable DNS resolution on the new interface

create one or more peers:

navigate to 'VPN > Wireguard > Settings > Peers' and create a new peer

give it a descriptive name, eg; 'ryans-iphone'

generate a new key pair.  This can be done with 'Wireguard Client Tools'
eg; wg genkey | tee private.key | wg pubkey > public.key, or it can be copied from a client device (see step 10)

You do not necessarily need to create or use a pre-shared key, it’s just an optional additional layer of security.  I may go back and add one later, but for now I just wanted my VPN to work.

'Allowed IPs' is a bit confusing.  This option should be called 'client (or peer) IP address'.  It should be an unused IP address that exists in the same subnet as the Wireguard interface, however you must use /32 as the netmask.  Someone much smarter than me should explain why using /32 is necessary.  I haven’t got a clue, I just know that it’s mandatory for Wireguard to work.


DO NOT specify a n endpoint address.  This part was particularly confusing as well.  without specifying an address, how would the client know what to connect to?  well… you do specify an endpoint… just not in the peer config on the wireguard instance.  You would do that on the client profile on the remote device.

set endpoint port to 51820

set the instance to the wireguard instance you just created

Pasted Graphic 27.png

outbound NAT:

navigate to 'Firewall > NAT > Outbound' and change the mode from 'Automatic outbound NAT rule generation' to 'Hybrid outbound NAT rule generation'

configure a new manual rule as follows:

Interface: WAN
TCP/IP version: IPv4
Protocol: any
source address: wireguard net
source port: any
destination address: any
destination port: any
translation target: interface address
description: Wireguard outbound NAT

save and apply

Pasted Graphic 29.png
  1. Firewall Rules:

    Navigate to 'Firewall > Rules > WAN' and configure a new rule as follows:

    action: pass
    quick: enabled
    Interface: WAN

Direction: IN
TCP/IP version: IPv4
Protocol: UDP

source: any
destination: WAN address
destination port range: (other) 51820

Description: Wireguard Incoming NAT

Pasted Graphic 30.png

save and apply

Navigated to 'Firewall > Rules > Wireguard' and configure a new rule as follow:

action: pass
quick: enabled
interface: wireguard
direction: in
TCP/IP version: IPv4
protocol: any
source: wireguard net
destination: any
description: wireguard allow clients

Pasted Graphic 31.png


save and apply

  1. Normalization rules

    Navigate to 'Firewall > Settings > Normalization' and create a new rule.  configured as follows

    inteface: wireguard
    direction: any
    protocol: any
    source: any
    destination: any
    description: Wireguard MSS Clamping IPv4
    Max MSS: 1380
    TOS/DSCP: do not change
Pasted Graphic 33.png


save and apply.



  1. client configuration:

    I’m just using my iPhone as an example here, the process will remain largely the same for any client.

    On the Wireguard app, create a new profile.

    generate a new key pair.  Copy the public key to the peer config on your Wireguard server and make sure they match.
    set the address to the client IP address you specified on the server, but make sure you use /24 and not /32 like you specified on the peer config on the server (confusing…)
    set listen port to 51820
    MTU should be automatic
    DNS servers: you can leave this blank or specify the IP address of your DNS server
    Public Key: this should be your wireguard server’s public key
    PSK: leave this blank ( I may go back and configure one at some point)
    Endpoint: typically, I set this to a DNS alias that resolves to my WAN IP.  I use DDNS to update my DNS provider (cloudflare) automatically, that way I don’t have to specify an IP address.  Must be in the form of server:port, eg; vpn.cudanet.org:51820
    allowed IPs: 0.0.0.0/0 is full tunneling (sends all traffic).  you can split tunnel by specifying one or more subnets, eg; 192.168.1.0/24,192.168.10.0/24.  Theoretically, you may see some performance increase by split tunneling but the improvement was marginal in my testing.

    save the profile