Category Archives: Web

Fix cron output not being sent via e-mail

I had an issue where I had cron jobs that output data to stdout, yet mail of the output was never delivered. Everything showed fine in cron.log :

Aug  3 21:21:01 mail CROND[10426]: (nicholas) CMD (echo “test”)
Aug  3 21:21:01 mail CROND[10424]: (nicholas) CMDOUT (test)

yet no e-mail was sent. I finally found out how to fix this in a roundabout way. I came across this article on cpanel.net on how to silence cron e-mails. I then thought I’d try the reverse of a suggestion and add MAILTO= variable at the top of my cron file. It worked! Example crontab:

MAILTO=”youremail@address.com”
0 * * * * /home/nicholas/queue-check.sh

This came about due to my Zimbra box not sending system e-mails. In addition to the above, I had to configure zimbra as a sendmail alternative per this Zimbra wiki post: https://wiki.zimbra.com/wiki/How_to_%22fix%22_system%27s_sendmail_to_use_that_of_zimbra

Updating Zimbra to latest version

Recently a remote code execution bug came to light with Zimbra. It prompted me to update to the latest patch. I had some e-mail deliverability issues afterward. Here are my patch notes:

  • Download the latest version from https://www.zimbra.com/try/zimbra-collaboration-open-source/
    • Follow instructions as listed here: https://wiki.zimbra.com/wiki/Zimbra_Releases/8.8.15/P32#Redhat
  • Untar downloaded file, cd into directory and run ./install.sh as root
  • Re-install latest patches (I had frustrating 500 errors until I discovered this was the fix)
    • sudo yum reinstall zimbra-patch
  • Re-do any customization you’ve done to zimbra core
    • https://techblog.jeppson.org/2022/08/configure-zimbra-to-use-anymxrelay/
    • In my case, it was adding these lines to the smtp-amavis section:
    -o smtp_tls_security_level=none
    -o smtp_tls_wrappermode=no
  • Restart Zimbra services
    • sudo -u zimbra zmcontrol restart

Configure Zimbra to use AnyMXRelay

It turns out if you want to configure Zimbra to use an external SMTP relay service it can be a bit of a headache if that service doesn’t use port 25 or 587 to receive encrypted relay mail. Such is the case with AnyMXRelay. I decided to use AnyMXRelay to relay my mail since my Linode box keeps getting put on weird shadow blocklists despite mxtoolbox saying everything was fine.

It took some digging but I finally found this article on Zimbra’s wiki outlining what needs to happen. There are a few manual settings that need to be put in place on the command line in order to get this to work – namely, smtp_tls_wrappermode and smtp_tls_security_level.

In addition to the steps taken in this how-to for sending mail through a relay, you must also make these changes:

postconf -e smtp_tls_wrappermode=yes   # No Zimbra setting for smtp_tls_wrappermode yet
zmprov ms `zmhostname` zimbraMtaSmtpTlsSecurityLevel encrypt
zmprov ms `zmhostname` zimbraMtaSmtpTlsCAfile /opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt
zmprov ms `zmhostname` zimbraMtaSmtpSaslSecurityOptions noanonymous
zmprov ms `zmhostname` zimbraMtaSmtpSaslAuthEnable yes
zmprov ms `zmhostname` zimbraMtaSmtpSaslPasswordMaps lmdb:/opt/zimbra/conf/relay_password

Zimbra 8.5+ periodically applies settings automatically, so once you’ve made these changes, watch /var/log/zimbra.log for these lines

zmconfigd[25662]: Fetching All configs
zmconfigd[25662]: All configs fetched in 0.07 seconds
...
zmconfigd[25662]: All restarts completed in 1.80 sec

Once you see them, you can send some test mail. Tail /var/log/zimbra.log to see if it worked or to see any error messages.

If you get these error messages:

HANGUP after 0.08 from [IP]:56518 in tests before SMTP handshake
#or#
status=deferred (Cannot start TLS: handshake failure)

It means you must also add two configuration lines to the amavis configuration file in /opt/zimbra/common/conf/master.cf.in

-o smtp_tls_security_level=none
-o smtp_tls_wrappermode=no

So the complete section looks like this:

smtp-amavis unix -      -       n       -       %%zimbraAmavisMaxServers%%   smtp
    -o smtp_tls_security_level=none
    -o smtp_tls_wrappermode=no
    -o smtp_data_done_timeout=1200 
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

Once you made the changes, save the file and restart all zimbra services with zmcontrol restart

The above disables TLS security for the antivirus piece. This could cause security issues if you Zimbra configuration is distributed to multiple hosts. In my case, this is an all-in-one server, so it does not matter.

Once I made the above changes, mail flowed through my external SMTP server successfully!

Self host postfix SMTP relay for Zimbra Mail Server

My notes for spinning up a small Debian linode server to act as an SMTP relay for my home network (note you will have to engage with linode support to enable mail ports for new accounts.)

Relay server configuration

Install postfix

sudo apt install postfix

Modify main.cf

sudo vim /etc/postfix/main.cf

Under TLS parameters, add TLS security to enable secure transfer of mail

smtp_tls_security_level = may
I decided not to open up postfix to the internet but instead my relay has a wireguard tunnel and postfix is allowed to relay only from that VPN subnet.

Add your subnets and relay restrictions further down:

mynetworks = 127.0.0.0/8 <YOUR_SERVER_SUBNET>
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated check_relay_domains
relay_domains = <MY_DOMAIN_NAME>
myhostname = <RELAYSERVER_HOSTNAME>
inet_interfaces = 127.0.0.1, <IP_OF_WIREGUARD_VPN_INTERFACE>

Zimbra configuration

In Zimmbra admin panel, edit your mail server

Configure / Servers / your_mail_server

MTA section

Add the DNS name and port of the relay system next to “Relay MTA for external deliverability”

If it won’t let you save, saying ::1 is required, you can add ::1 to MTA Trusted networks, however, on my Zimbra server this broke postfix. The symptoms were e-mails hanging and not sending. To fix, log into the Zimbra mail server and run as the zimbra user:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks ‘127.0.0.1 192.168.0.0/16’ (list of networks you had before but excluding ::1)

Then, issue postfix reload

That was it. A simple postfix SMTP relay which only accepts mail from my internal VPN (it doesn’t listen on the external interface at all.)

Troubleshooting

Realyed mail shows red unlock icon in Gmail (mail getting sent unencrypted)

Per postfix documentation I needed to enable secure transfer of mail by adding

smtp_tls_security_level = may

to main.cf

Mail does not send after adding ::1 to MTA Trusted Networks

Remove it via the CLI and reload postfix:

zmprov ms YOUR_MAIL_DOMAIN_NAME zimbraMtaMyNetworks '127.0.0.1 192.168.0.0/16' (list of networks you had before but excluding ::1)
postfix reload

Guacamole docker quick and easy

Apache Guacamole as an awesome HTML5 remote access gateway. Unfortunately it can be very frustrating to set up. They have docker images that are supposed to make the process easier, but I still ran into a lot of problems trying to get everything configured and linked.

Fortunately, a docker compose file exists to make Guacamole much easier to set up. Simply follow the instructions as laid out in the github readme:

  • Install docker & docker-compose
  • Clone their repository, run the initial prep script (for SSL keys & database initialization), and bring it up with docker-compose:
git clone "https://github.com/boschkundendienst/guacamole-docker-compose.git"
cd guacamole-docker-compose
sudo ./prepare.sh
sudo docker-compose up -d

Done! If you didn’t change anything in the docker-compose.yml file, you will have a new instance of Guacamole running on HTTPS port 8443 of your docker host. If you need to make changes (or if you forgot to run the prepare.sh file with sudo), you can run the reset.sh script which will destroy everything. You can then modify docker-compose.yml to suit your needs:

  • Whether to use nginx for HTTPS or just expose guacamole on port 8080 non-https (in case you already have a reverse proxy set up)
  • postgres password

Config files for each container are located within various folders in your guacamole-docker-compose folder. This can all be changed by editing the docker-compose.yml file.

Note this does configuration does not work with WOL, but as I do not use this feature I don’t mind.

Troubleshooting

docker ps will show running containers (docker ps -a shows all containers) If one is not running that should be, docker logs <container name> gives valuable insight as to why. In my case guacd was erroring out because I hadn’t initialized the database properly. Running the reset.sh script and starting over, this time running as sudo, did the trick.

Synchronize internet calendar to google calendar more frequently

Despite having my own e-mail server I still use Google Calendar for some things. I have an ICS file for the calendar for the Covid vaccination clinic I’m volunteering at. I ran into some frustrating sync problems when I tried to import it into my calendar. Google Calendar’s ICS sync process takes up to 12 hours, which was frustrating. I also had some mobile clients that wouldn’t even see the calendar imported from the ICS file.

I luckily found this post from Derek Antrican on stack exchange that outlines a script that you can configure to run at any given interval which will take all events in that ICS file and add/update/remove your calendar to match. It works beautifully. It’s a Google Apps script that you must copy into your own Google Scripts account to run.

First, go to the script here. Then go to Overview (i) and click “Make a Copy” in the top right (page icon.) Once the scripts are copied to your own script.google.com account, follow the instructions for configuring the script for your desired ICS URLs and other options, then click run.

My calendars are all synchronized and happy now.

Transcribe audio with Google Cloud speech-to-text api

I had a few audio files of an interview done with a late relative that I wanted to have Google transcribe for me. I wanted to supply an audio file and have it spit out the results. There are many ways to do this but I went with using the Google Cloud Platfrom speech-to-text API.

First I signed up for a GCP free trial via https://cloud.google.com/speech-to-text/ For my usage, it will remain free as 0-60 minutes of transcription per month is not charged: https://cloud.google.com/speech-to-text/pricing

Next, I needed to create GCP storage bucket as audio more than 10 minutes long cannot reliably be transcribed via the “uploading local file” option. I did this following the documentation at https://cloud.google.com/storage/docs/creating-buckets which walks you through going to their storage browser and creating a new bucket. From that screen I uploaded my audio files (FLAC in my case.)

Then I needed to create API credentials to use. I did this by going speech API console’s credentials tab and creating a service account, then saving the key to my working directory on my local computer.

Also on said computer I installed google-cloud-sdk (on Arch Linux in my case, it was as simple as yay -S google-cloud-sdk)

With service account json file downloaded & google-cloud-sdk installed I exported the GCP service account credentials into my BASH environment like so

export GOOGLE_APPLICATION_CREDENTIALS=NAME_OF_SERVICE_ACCOUNT_KEYFILE_DOWNLADED_EARLIER.json 

I created .json files following the format outlined in command line usage outlined in the quickstart documentation. I tweaked to add a line “model”: “video” to get the API to use the premium Video recognition set (as it was more accurate for this type of recording.) This is what my JSON file looked like:

{
  "config": {
      "encoding":"FLAC",
      "sampleRateHertz": 16000,
      "languageCode": "en-US",
      "enableWordTimeOffsets": false,
      "model": "video"

  },
  "audio": {
      "uri":"gs://googlestorarge-bucket-name/family-memories.flac"
  }
}

I then used CURL to send the transcription request to Google. This was my command:

curl -s -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://speech.googleapis.com/v1/speech:longrunningrecognize -d @JSON_FILE_CREATED_ABOVE.json

If all goes well you will get something like this in response:

{
  "name": "4663803355627080910"
}

You can check the status of the transcription, which usually takes half the length of the audio file to do, by running this command:

curl -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) -H "Content-Type: application/json; charset=utf-8" "https://speech.googleapis.com/v1/operations/ID_NUMBER_ACQUIRED_ABOVE"

You will either get a percent progress, or if it’s done, the output of the transcription.

Success! It took some time to figure out but was still much better than manually transcribing the audio by hand.

Zimbra expired ldap certificate fix

I started getting SSL errors with my Zimbra mail server despite having a valid SSL certificate everywhere I knew where to check. When I tried to use zmcontrol status I got this error:

Unable to start TLS: SSL connect attempt failed error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed when connecting to ldap master.

Eventually I found this blog post explaining the problem – it’s with the LDAP component in Zimbra. You have to switch it from ldap to ldaps. Why did this change? I do not know.

ZIMBRA_HOSTNAME=_your_mail_server_dns_hostname_

sudo -u zimbra bash
zmlocalconfig -e ldap_master_url=ldaps://$ZIMBRA_HOSTNAME:636
zmlocalconfig -e ldap_url=ldaps://$ZIMBRA_HOSTNAME:636
zmlocalconfig -e ldap_starttls_supported=0
zmlocalconfig -e ldap_port=636
zmcontrol stop
zmcontrol start

This did the trick. The errors went away.

apache reverse proxy with basic authentication

I have an old Apache server that’s serving as a reverse proxy for my webcam. I swapped webcams out and unfortunately the new one requires authentication. I had to figure out how to get Apache to reverse proxy with the proper authentication. The best information I found was given by user ThR37 at superuser.com

Essentially you have to use an Apache module called headers to add an HTTP header to the request. On my Debian system this was not enabled, so I had to install it (thanks to Andy over at serverfault)

sudo a2enmod headers
#if you're on ubuntu then it's mod_headers

I then needed to generate the basic authentication hash for the header to pass. This was done via a simple python script:

#replace USERNAME:PASSWORD below with your credentials
import base64
hash = base64.b64encode(b'USERNAME:PASSWORD')
print hash

Save the above script into a file hash.py and then run it by typing

python hash.py

With headers enabled and hash acquired I just needed to tweak my config by adding a RequestHeader line:

RequestHeader set Authorization "Basic <HASH>"
#Replace <HASH> with hash acquired above

After adding that one line and restarting apache, it worked!

Headless Private Internet Access with OpenVPN & Port Forwarding

I wanted to initiate a connection to my VPN provider Private Internet Access (PIA) from one of my headless CentOS 7 servers. Unfortunately PIA software requires a GUI to work. Fortunately, PIA provides ovpn files that fully work with the openVPN software, which does work headlessly.

I followed this guide from htpcguides.com to get me going. I tweaked it a bit to work with CentOS 7. I then used this guide to help me set up port forwarding to work with transmission-daemon.

First, install openvpn & iproute

sudo yum -y install openvpn

Next grab PIA’s ovpn files for their servers:

wget https://www.privateinternetaccess.com/openvpn/openvpn.zip
sudo unzip openvpn.zip -d /etc/openvpn/pia

Now create a credentials file to specify your PIA username and password. The format is simply the first line being your PIA username, and the second line being your PIA password, nothing else.

#Create file
sudo vim /etc/openvpn/pia/.secret

#Enter PIA credentials into the file, then save and exit
USERNAME
PASSWORD

#Change permissions to be readable only by root
sudo chmod 700 /etc/openvpn/pia/.secret

Test if it’s working by manually running OpenVPN

sudo openvpn --config /etc/openvpn/PIA_SITE_FILENAME.ovpn --auth-user-pass /etc/openvpn/pia/.secret

Create a systemd service file to get OpenVPN to run on startup

sudo vim /usr/lib/systemd/system/pia.service

[Unit]
Description=OpenVPN
After=syslog.target network.target

[Service]
PrivateTmp=true
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --config "/etc/openvpn/PIA_SITE_FILENAME.ovpn" --auth-user-pass /etc/openvpn/.secret

[Install]
WantedBy=multi-user.target

Reload systemd daemons and enable at boot:

sudo systemctl daemon-reload
sudo systemctl enable pia.service

Start the service (or reboot) and then use curl to verify your external IP has changed

sudo systemctl start pia.service
curl https://ipinfo.io/ip

Port Forwarding

PIA is awesome because it supports port forwarding to facilitate things like torrents. I took the script from this guide and tweaked it a bit to suit my needs (and to correct a syntax error)

Note: PIA only supports port forwarding from specific regions. Here is the current list.

Download script to pia directory

sudo wget -O /etc/openvpn/pia/portforward.sh https://raw.githubusercontent.com/HTPCGuides/pia-port-forwarding-scripts/master/transmission-pia-port-forwarding.sh
sudo chmod +x /etc/openvpn/portforward.sh

Edit the script to provide PIA username and pass, and Transmission user/pass (if you have that configured.) The variables to change are USERNAME,PASSWORD, TRANSUSER, and TRANSPASS

Test the script by running it manually (make sure both your pia service and transmission services are already running.) I’ve noticed that the script will say success even on failure which is why I suggest using bash -x so you can see actual error messages from PIA regarding port forwarding:

sudo bash -x /etc/openvpn/pia/portforward.sh

After that’s run you can check your transmission daemon to ensure it is listening on the PIA port that’s been forwarded.

Configure to run on startup and every two hours by invoking cron:

sudo crontab -e

@reboot sleep 60 && /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward
0 */2 * * * /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward

If you are paranoid about transmission running without the VPN working you can tweak your transmission service to rely on the pia service before starting. Modify /usr/lib/systemd/system/transmission-daemon.service and tweak the After line and add a Required line so it now reads this:

Required=pia.service
After=network.target pia.service

Don’t forget to run a sudo systemctl daemon-reload afterward.

Troubleshooting

If you get the error message “/etc/openvpn/pia/portforward.sh: line 35: [: : integer expression expected” it means you have to tweak the script a bit (it’s broken?)

Change line 35 to be the following:

if [ "$SPLITVPN" == 1 ]; then