Remote access VPN users are disconnected on every LB, ACL, port forward or firewall change (xl2tpd restarted on every iptables pass) #14545

problem

Since 4.22.1.0, every remote access VPN (L2TP/IPsec) client on a router is disconnected whenever an unrelated rule
changes on that router: creating or deleting an LB rule, an ACL rule, a port forward, a firewall rule, a static NAT or
a site-to-site VPN. The client has to reconnect each time. Some clients (macOS) don't reconnect on their own.

cloud.log on the router shows xl2tpd being restarted on every one of those passes, although none of the L2TP files
changed except the secrets file, which was rewritten with the same content:

Nothing to commit. The /etc/ipsec.d/l2tp.conf file did not change
Wrote edited file /etc/ipsec.d/ipsec.any.secrets
Nothing to commit. The /etc/xl2tpd/xl2tpd.conf file did not change
Nothing to commit. The /etc/ppp/options.xl2tpd file did not change
Executing: ipsec update
Executing: systemctl restart xl2tpd
Executing: ipsec rereadsecrets

Root cause, line numbers from the 4.22 branch (56306ab), same code in 4.23.0.0 and main:

• systemvm/debian/opt/cloud/bin/configure.py:1277, in configure_l2tpIpsec(): secret.empty() before
  secret.addeq(": PSK ...") and secret.commit().
• systemvm/debian/opt/cloud/bin/cs/CsFile.py:51-53: empty() clears both config (the copy read from disk) and
  new_config. is_changed() (CsFile.py:42-46) compares the two, so after empty() the file always reads as
  changed and commit() always returns True.
• configure.py:1247-1250: since Restart xl2tpd service when the VPN configuration is modified #12422 (the fix for VPN setting changes are not propagated to the Virtual Router #12195), a changed config means systemctl restart xl2tpd
  instead of systemctl start xl2tpd. Restarting xl2tpd kills every pppd session.
• CsRemoteAccessVpn.process() runs from IpTablesExecutor.process() (configure.py:1615-1616), which runs for every
  data bag with process_iptables: True (configure.py:1673-1690), not only for VPN changes.

Before #12422 the same pass ran systemctl start xl2tpd, a no-op on a running service, so the always-changed secrets
file did no harm.

Impact: on 4.22.1.0 and later, any router with a remote access VPN drops all its VPN users on any rule change in the
network or VPC. With CKS, the cloud controller manager's LB and ACL reconciles trigger it too.

versions

4.22.1.0, 4.22.1.1 and 4.23.0.0 (any version containing #12422). Seen on 4.23.0.0 with KVM, VPC with redundant
routers, advanced networking. Not specific to the hypervisor.

The steps to reproduce the bug

1. Create a VPC, enable remote access VPN on its source NAT IP and add a VPN user.
2. Connect an L2TP/IPsec client and keep a ping running to a VM in a tier.
3. Create (or delete) an LB rule, or add an ACL rule, in that VPC.
4. The client's session drops. cloud.log on the router shows "Wrote edited file /etc/ipsec.d/ipsec.any.secrets"
   followed by systemctl restart xl2tpd.

What to do about it?

Use secret.repopulate() instead of secret.empty() in configure_l2tpIpsec(). repopulate() (CsFile.py:55-56)
clears only new_config and keeps the copy loaded from disk, so an unchanged PSK compares equal and xl2tpd is only
restarted when something really changed, which is what #12422 intended. A changed PSK still restarts it. PR to follow.