Tag Archives: failover

CLI, Web

Configure Zimbra live replication

Leave a comment

I’ve recently configured live active replication from my Zimbra e-mail server to a backup server. This is really slick – in the event of primary server failure, I can bring up my secondary in a matter of minutes with no data loss. I used the Zimbra live sync scripts on Gitlab to accomplish this.

These are my notes on things I needed to do in addition to the readme to get things to work properly on my Zimbra 8.8.15 Open Source Edition installs on CentOS 7 boxes.

Install atd (at package):
sudo yum install atd

Make sure the backup server has the same firewall rules as the primary: https://wiki.zimbra.com/wiki/Ports

On the backup server, configure DNS for the mail server to resolve to the Backup server’s IP address. hostname: mail.server.dns -> mirror mail server.

Disable DNS forwarding for primary mail server domain if configured (to ensure mail goes to backup server in the event of switchover.)

Clone over prod mail server, spin up and change network settings:

  • keep hostname (important)
  • change IP, DNS, hosts to use new IP address/network

/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/hosts
/etc/resolv.conf

Ensure proper VLAN settings in backup VM (may be different than primary)

Systemd service:
add Environment=PATH=/opt/zimbra/bin:/opt/zimbra/common/lib/jvm/java/bin:/opt/zimbra/common/bin:/opt/zimbra/common/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin
WorkingDirectory=/opt/zimbra

Remove start argument from ExecStart: ExecStart=/opt/zimbra/live_sync/live_syncd

This is the complete systemd unit for live sync:

[Unit]
Description=Zimbra live sync - to be run on the mirror server
After=network.target

[Service]
ExecStart=/opt/zimbra/live_sync/live_syncd
ExecStop=/opt/zimbra/live_sync/live_syncd kill
User=zimbra
Environment=PATH=/opt/zimbra/bin:/opt/zimbra/common/lib/jvm/java/bin:/opt/zimbra/common/bin:/opt/zimbra/common/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin
WorkingDirectory=/opt/zimbra

[Install]
WantedBy=multi-user.target

Time limit

It looks like there’s a time limit for how long Zimbra keeps redo logs. It means you will get a lost mail situation if you try to bring your primary server back up after it’s been offline for too long (more than a few weeks.) If you’ve been failed over to your secondary mail server for more than two weeks, you’ll want to do the reverse procedure – clone the backup to the primary, edit IP addresses, then run the zimbra live sync. Log into the restored server to ensure mails from greater than 2 weeks ago are all there.

Networking

Wireguard one-way traffic on USG Pro 4 after dual WAN setup

Leave a comment

I have a site-to-site VPN between my Ubiquiti USG Pro-4 and an OpenWRT device over wireguard . It’s worked great until I got a secondary WAN connection as a failover connection since my primary cable connection has been flaky lately.

When you introduce dual-WAN on Ubiquiti devices you have to manually configure everything since the GUI assumes only one WAN connection. I configured my manual DNAT (port forwards) for each interface successfully but struggled to figure out why suddenly my Wireguard VPN between my two sites only went one way (remote side could ping all hosts on local side, but not visa-versa.)

After some troubleshooting I realized the firewall itself could ping the remote subnet just fine, it just wasn’t allowing local hosts to do so. I couldn’t find anything in firewall logs. Eventually I came across this very helpful page from hackad.nu that helped me to solve my problem.

The solution was to add a Firewall Modify rule specifically for the eth0 interface (where all my LAN traffic is routed through) to allow the source address of the subnets I want to traverse the VPN, then apply that modifier to the LAN_IN firewall rule for that interface. I had to do it for any VLANs I wanted to be able to use the Wireguard tunnel as well (vifs of eth0, VLAN 50 in my case)

Here is the relevant config.gateway.json sections, namely “firewall” and “interfaces”:

{
    "firewall": {
        "modify": {
            "Wireguard": {
                "rule": {
                    "10": {
                        "action": "modify",
                        "description": "Allow Wireguard traffic",
                        "modify": {
                            "table": "10"
                        },
                        "source": {
                            "address": "10.1.0.0/16"
                        }
                    }
                }
            }
        },
        "interfaces": {
            "ethernet": {
                "eth0": {
                    "firewall": {
                        "in": {
                            "ipv6-name": "LANv6_IN",
                            "modify": "Wireguard",
                            "name": "LAN_IN"
                        }
                    },
                    "vif": {
                        "50": {
                            "firewall": {
                                "in": {
                                    "ipv6-name": "LANv6_IN",
                                    "modify": "Wireguard",
                                    "name": "LAN_IN"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

This did the trick! Wireguard is working both directions again, this time with my dual WAN connections.