Tag Archives: Zimbra

CLI, Web

Zimbra commercial SSL renewal procedure

Leave a comment

My quick notes on what I have to do every year to upgrade my Zimbra mail certificate with a new Namecheap SSL certificate:

  1. Request CSR
    • /opt/zimbra/bin/zmcertmgr createcsr comm -new -subject "/C=COUNTRY/ST=STATE/L=LOCATION/O=ORG/OU=OU/CN=CN.EXAMPLE.ORG" -subjectAltNames CN.EXAMPLE.ORG
    • cat /opt/zimbra/ssl/zimbra/commercial/commercial.csr
  2. Upload CSR, verify domain, receive cert bundle
  3. Copy CRT & CA Bundle files to /tmp/cert
  4. Change permissions of files to allow zimbra user to use them:
    sudo chown zimbra /tmp/cert
    sudo chown zimbra /tmp/cert/*
  5. Verify it works against private key
    zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key /tmp/cert/ISSUED_CRT.crt /tmp/cert/CA_BUNDLE.ca-bundle
  6. Import new key
    zmcertmgr deploycrt comm /tmp/cert/ISSUED_CRT.crt /tmp/cert/CA_BUNDLE.ca-bundle
  7. Restart zimbra
    zmcontrol restart
CLI, Web

Configure Zimbra live replication

Leave a comment

I’ve recently configured live active replication from my Zimbra e-mail server to a backup server. This is really slick – in the event of primary server failure, I can bring up my secondary in a matter of minutes with no data loss. I used the Zimbra live sync scripts on Gitlab to accomplish this.

These are my notes on things I needed to do in addition to the readme to get things to work properly on my Zimbra 8.8.15 Open Source Edition installs on CentOS 7 boxes.

Install atd (at package):
sudo yum install atd

Make sure the backup server has the same firewall rules as the primary: https://wiki.zimbra.com/wiki/Ports

On the backup server, configure DNS for the mail server to resolve to the Backup server’s IP address. hostname: mail.server.dns -> mirror mail server.

Disable DNS forwarding for primary mail server domain if configured (to ensure mail goes to backup server in the event of switchover.)

Clone over prod mail server, spin up and change network settings:

  • keep hostname (important)
  • change IP, DNS, hosts to use new IP address/network

/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/hosts
/etc/resolv.conf

Ensure proper VLAN settings in backup VM (may be different than primary)

Systemd service:
add Environment=PATH=/opt/zimbra/bin:/opt/zimbra/common/lib/jvm/java/bin:/opt/zimbra/common/bin:/opt/zimbra/common/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin
WorkingDirectory=/opt/zimbra

Remove start argument from ExecStart: ExecStart=/opt/zimbra/live_sync/live_syncd

This is the complete systemd unit for live sync:

[Unit]
Description=Zimbra live sync - to be run on the mirror server
After=network.target

[Service]
ExecStart=/opt/zimbra/live_sync/live_syncd
ExecStop=/opt/zimbra/live_sync/live_syncd kill
User=zimbra
Environment=PATH=/opt/zimbra/bin:/opt/zimbra/common/lib/jvm/java/bin:/opt/zimbra/common/bin:/opt/zimbra/common/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin
WorkingDirectory=/opt/zimbra

[Install]
WantedBy=multi-user.target

Time limit

It looks like there’s a time limit for how long Zimbra keeps redo logs. It means you will get a lost mail situation if you try to bring your primary server back up after it’s been offline for too long (more than a few weeks.) If you’ve been failed over to your secondary mail server for more than two weeks, you’ll want to do the reverse procedure – clone the backup to the primary, edit IP addresses, then run the zimbra live sync. Log into the restored server to ensure mails from greater than 2 weeks ago are all there.

CLI, Web

Fix cron output not being sent via e-mail

Leave a comment

I had an issue where I had cron jobs that output data to stdout, yet mail of the output was never delivered. Everything showed fine in cron.log :

Aug  3 21:21:01 mail CROND[10426]: (nicholas) CMD (echo “test”)
Aug  3 21:21:01 mail CROND[10424]: (nicholas) CMDOUT (test)

yet no e-mail was sent. I finally found out how to fix this in a roundabout way. I came across this article on cpanel.net on how to silence cron e-mails. I then thought I’d try the reverse of a suggestion and add MAILTO= variable at the top of my cron file. It worked! Example crontab:

MAILTO=”youremail@address.com”
0 * * * * /home/nicholas/queue-check.sh

This came about due to my Zimbra box not sending system e-mails. In addition to the above, I had to configure zimbra as a sendmail alternative per this Zimbra wiki post: https://wiki.zimbra.com/wiki/How_to_%22fix%22_system%27s_sendmail_to_use_that_of_zimbra

CLI, Web

Updating Zimbra to latest version

Leave a comment

Recently a remote code execution bug came to light with Zimbra. It prompted me to update to the latest patch. I had some e-mail deliverability issues afterward. Here are my patch notes:

  • Download the latest version from https://www.zimbra.com/try/zimbra-collaboration-open-source/
    • Follow instructions as listed here: https://wiki.zimbra.com/wiki/Zimbra_Releases/8.8.15/P32#Redhat
  • Untar downloaded file, cd into directory and run ./install.sh as root
  • Re-install latest patches (I had frustrating 500 errors until I discovered this was the fix)
    • sudo yum reinstall zimbra-patch
  • Re-do any customization you’ve done to zimbra core
    • https://techblog.jeppson.org/2022/08/configure-zimbra-to-use-anymxrelay/
    • In my case, it was adding these lines to the smtp-amavis section:
    -o smtp_tls_security_level=none
    -o smtp_tls_wrappermode=no
  • Restart Zimbra services
    • sudo -u zimbra zmcontrol restart
CLI, Web

Configure Zimbra to use AnyMXRelay

1 Comment

It turns out if you want to configure Zimbra to use an external SMTP relay service it can be a bit of a headache if that service doesn’t use port 25 or 587 to receive encrypted relay mail. Such is the case with AnyMXRelay. I decided to use AnyMXRelay to relay my mail since my Linode box keeps getting put on weird shadow blocklists despite mxtoolbox saying everything was fine.

It took some digging but I finally found this article on Zimbra’s wiki outlining what needs to happen. There are a few manual settings that need to be put in place on the command line in order to get this to work – namely, smtp_tls_wrappermode and smtp_tls_security_level.

In addition to the steps taken in this how-to for sending mail through a relay, you must also make these changes:

postconf -e smtp_tls_wrappermode=yes   # No Zimbra setting for smtp_tls_wrappermode yet
zmprov ms `zmhostname` zimbraMtaSmtpTlsSecurityLevel encrypt
zmprov ms `zmhostname` zimbraMtaSmtpTlsCAfile /opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt
zmprov ms `zmhostname` zimbraMtaSmtpSaslSecurityOptions noanonymous
zmprov ms `zmhostname` zimbraMtaSmtpSaslAuthEnable yes
zmprov ms `zmhostname` zimbraMtaSmtpSaslPasswordMaps lmdb:/opt/zimbra/conf/relay_password

Zimbra 8.5+ periodically applies settings automatically, so once you’ve made these changes, watch /var/log/zimbra.log for these lines

zmconfigd[25662]: Fetching All configs
zmconfigd[25662]: All configs fetched in 0.07 seconds
...
zmconfigd[25662]: All restarts completed in 1.80 sec

Once you see them, you can send some test mail. Tail /var/log/zimbra.log to see if it worked or to see any error messages.

If you get these error messages:

HANGUP after 0.08 from [IP]:56518 in tests before SMTP handshake
#or#
status=deferred (Cannot start TLS: handshake failure)

It means you must also add two configuration lines to the amavis configuration file in /opt/zimbra/common/conf/master.cf.in 

-o smtp_tls_security_level=none
-o smtp_tls_wrappermode=no

So the complete section looks like this:

smtp-amavis unix -      -       n       -       %%zimbraAmavisMaxServers%%   smtp
    -o smtp_tls_security_level=none
    -o smtp_tls_wrappermode=no
    -o smtp_data_done_timeout=1200 
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

Once you made the changes, save the file and restart all zimbra services with zmcontrol restart

The above disables TLS security for the antivirus piece. This could cause security issues if you Zimbra configuration is distributed to multiple hosts. In my case, this is an all-in-one server, so it does not matter.

Once I made the above changes, mail flowed through my external SMTP server successfully!

CLI, Web

Self host postfix SMTP relay for Zimbra Mail Server

Leave a comment

My notes for spinning up a small Debian linode server to act as an SMTP relay for my home network (note you will have to engage with linode support to enable mail ports for new accounts.)

Relay server configuration

Install postfix

sudo apt install postfix

Modify main.cf

sudo vim /etc/postfix/main.cf

Under TLS parameters, add TLS security to enable secure transfer of mail 

smtp_tls_security_level = may
I decided not to open up postfix to the internet but instead my relay has a wireguard tunnel and postfix is allowed to relay only from that VPN subnet.

Add your subnets and relay restrictions further down:

mynetworks = 127.0.0.0/8 <YOUR_SERVER_SUBNET>
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated check_relay_domains
relay_domains = <MY_DOMAIN_NAME>
myhostname = <RELAYSERVER_HOSTNAME>
inet_interfaces = 127.0.0.1, <IP_OF_WIREGUARD_VPN_INTERFACE>

Zimbra configuration

In Zimmbra admin panel, edit your mail server

Configure / Servers / your_mail_server

MTA section

Add the DNS name and port of the relay system next to “Relay MTA for external deliverability”

If it won’t let you save, saying ::1 is required, you can add ::1 to MTA Trusted networks, however, on my Zimbra server this broke postfix. The symptoms were e-mails hanging and not sending. To fix, log into the Zimbra mail server and run as the zimbra user:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks ‘127.0.0.1 192.168.0.0/16’ (list of networks you had before but excluding ::1)

Then, issue postfix reload

That was it. A simple postfix SMTP relay which only accepts mail from my internal VPN (it doesn’t listen on the external interface at all.)

Troubleshooting

Realyed mail shows red unlock icon in Gmail (mail getting sent unencrypted)

Per postfix documentation I needed to enable secure transfer of mail by adding 

smtp_tls_security_level = may

to main.cf

Mail does not send after adding ::1 to MTA Trusted Networks

Remove it via the CLI and reload postfix:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks '127.0.0.1 192.168.0.0/16' (list of networks you had before but excluding ::1)
postfix reload
Web

Zimbra expired ldap certificate fix

Leave a comment

I started getting SSL errors with my Zimbra mail server despite having a valid SSL certificate everywhere I knew where to check. When I tried to use zmcontrol status I got this error:

Unable to start TLS: SSL connect attempt failed error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed when connecting to ldap master.

Eventually I found this blog post explaining the problem – it’s with the LDAP component in Zimbra. You have to switch it from ldap to ldaps. Why did this change? I do not know.

ZIMBRA_HOSTNAME=_your_mail_server_dns_hostname_

sudo -u zimbra bash
zmlocalconfig -e ldap_master_url=ldaps://$ZIMBRA_HOSTNAME:636
zmlocalconfig -e ldap_url=ldaps://$ZIMBRA_HOSTNAME:636
zmlocalconfig -e ldap_starttls_supported=0
zmlocalconfig -e ldap_port=636
zmcontrol stop
zmcontrol start

This did the trick. The errors went away.