Late last May Ubiquiti released the v 4.4.41 update for their UniFi Security Gateways and it promptly broke my VPN tunnel. I didn't have time to dig into it at the time so I just rolled back to the previous 4.4.36 release which worked swimmingly. At the end of last month they released the 4.4.44 version with several security fixes so I decided to devote some time to it today.
Architecture
To refresh you, dear reader and perhaps determine if the rest of this is going to help you at all, I will again describe the VPN setup that I have here. I terminate an IPsec tunnel using IKEv2 between several different endpoints and my OpenBSD VPN hub. For site-to-site links I run GRE on top so I can leverage OSPF and BGP for dynamic routing. That means on the USG there is an ipsec-peer and a tun interface running GRE.
Diagnosis
I could talk across the raw IPsec tunnel no problem but GRE encapsulated packets would never get replied to. I did some digging with tcpdump(8) and discovered that the VPN server was replying but they were never showing up on the USG's WAN port. I immediately suspected something had changed in the USG's firewall and while looking at the rule set revealed no changes, I was noticing that with each OSPFv2-hello being sent by my server, the WAN_LOCAL rule to drop invalid states was incrementing. I flushed that rule from the rule set and the GRE traffic began to work again.
The fix
It took a fair amount of faffing about to get to a solution. My flushing of various firewall rules from the cli caused the controller to fail to provision the new rules I was trying to create in the UI. In the end I had to reboot several times to make sure that the IPtables rules were synchronized with what the controller thought they were. I ended up building the rule below in the UniFi controller UI and then implementing it in the USG cli. If you set this rule up before you upgrade from 4.4.36, you shouldn't have to mess around in the cli at all.
Allow GRE in the UniFi controller
Allow GRE in the USG CLI
mernisse@bdr01$ configure
mernisse@bdr01# set firewall name WAN_LOCAL rule 2002 description "Allow GRE on WAN"
[edit]
mernisse@bdr01# set firewall name WAN_LOCAL rule 2002 protocol gre
[edit]
mernisse@bdr01# set firewall name WAN_LOCAL rule 2002 action accept
[edit]
mernisse@bdr01# commit
[edit]
mernisse@bdr01# exit
Warning: configuration changes have not been saved.
mernisse@bdr01$
Conclusion
I'm not sure what changed that suddenly GRE tunnel packets were being marked as having invalid states on the input filter of the WAN interface, but it was an easy fix. Hopefully if you have a similar setup this can help you out.
☕