Tag Archives: cron

Networking, OS

NPM reboot workaround script

Leave a comment

I had an annoying “chicken and egg” problem with nginx proxy manager. It has hosts configurations that reference DNS names that are only reachable over VPN. It also hosts headscale, which is required for VPN to establish properly for said DNS entries. Brought up from scratch (say, after a reboot) NPM fails to launch because it can’t resolve the DNS entries. Headscale therefore isn’t reachable by the other nodes, and we’re stuck.

I developed a script to be run at boot to get around this annoying behavior of nginx proxy manager. It disables all other hosts in NPM except headscale, which doesn’t need DNS resolution as it resides on the same host. It then fires up NPM, and then tailscale, and waits to confirm the VPN is established and DNS resolution is working. Then it restores all the other hosts and restarts NPM. This script was 100% done myself, no AI assistance was used.

#!/bin/bash
# Script to properly bring things up after a reboot
# Name resolution doesn't happen until vpn VIP comes up
# NPM doesn't spin up until name resolution works
# vpn headscale is part of NPM
#
# Replace all VPN DNS host entries with 127.0.0.1
# Wait for NPM to come up
# Restore original host entries to get name resolution to work
# restart npm
#
# Run on a cron job @reboot. Ensure cron service is enabled.
# @reboot /docker/npm/reboot-script.sh | tee /tmp/reboot-script.log

# Variables
DOCKER_DIR=/docker/npm
CONF_DIR=$DOCKER_DIR/data/nginx/proxy_host
TEMP_DIR=/tmp/$(basename $CONF_DIR)
VPN_IP=100.1.1.1

# Log start of script
echo "Reboot script started on $(date)"

# Backup host entries
rsync -aP --delete $CONF_DIR $(dirname $TEMP_DIR) 

# Disable all hosts except for headscale
sed -i  's/\".*.<VPN_DNS_SUFFIX>\"/\"127.0.0.1\"/g' $CONF_DIR/*.conf
sed -i  's/http:\/\/.*.<VPN_DNS_SUFFIX>/http:\/\/127.0.0.1/g' $CONF_DIR/*.conf

# Listen on 127.0.0.1 for npm
sed -i "s/$VPN_IP/127.0.0.1/g" $DOCKER_DIR/docker-compose.yml

# Stop and restart npm
cd $CONF_DIR && docker compose down && docker compose up -d

# Wait for NPM to launch
until netstat -an|grep 0.0.0.0:80 >/dev/null; do echo "Waiting for NPM"; sleep 5; done; echo "NPM up"

# Restart tailscale
systemctl restart tailscaled

# Wait until VPN pings are successful
until ping -w 1 -c 1 <VPN_DNS_IP> >/dev/null; do echo "Waiting for VPN"; sleep 5; done; echo "VPN Successful"

# Restore config files
rsync -aP $TEMP_DIR/ $CONF_DIR
sed -i "s/127.0.0.1/$VPN_IP/g" $DOCKER_DIR/docker-compose.yml

# Restart npm
cd $CONF_DIR && docker compose down && docker compose up -d

echo "Reboot script completed on $(date)"

CLI, Web

Fix cron output not being sent via e-mail

Leave a comment

I had an issue where I had cron jobs that output data to stdout, yet mail of the output was never delivered. Everything showed fine in cron.log :

Aug  3 21:21:01 mail CROND[10426]: (nicholas) CMD (echo “test”)
Aug  3 21:21:01 mail CROND[10424]: (nicholas) CMDOUT (test)

yet no e-mail was sent. I finally found out how to fix this in a roundabout way. I came across this article on cpanel.net on how to silence cron e-mails. I then thought I’d try the reverse of a suggestion and add MAILTO= variable at the top of my cron file. It worked! Example crontab:

MAILTO=”youremail@address.com”
0 * * * * /home/nicholas/queue-check.sh

This came about due to my Zimbra box not sending system e-mails. In addition to the above, I had to configure zimbra as a sendmail alternative per this Zimbra wiki post: https://wiki.zimbra.com/wiki/How_to_%22fix%22_system%27s_sendmail_to_use_that_of_zimbra

CLI

mountpoint check script

Leave a comment

I have a few NFS mounts that I want to be working at all times. If there is a power outage, sometimes NFS clients come up before the NFS server does, and thus the mounts are not there. I wrote a quick little bash script to fix this utilizing the mountpoint command.

Behold (Change the mountpoint(s) to the one you want to monitor.)

#!/bin/bash
#Simple bash script to check mount points and re-mount them if they're not mounted
#Authored by Nick Jepspon 8/11/2019

### Variables ###
# Changes these to suit your needs

MOUNTPOINTS=(/mnt/1 /mnt/2 /mnt/3)  #space separated list of mountpoints to monitor

### End Variables ###


for mount in $MOUNTPOINTS 
do
    if  ! mountpoint -q $mount
    then
        echo "$mount is not mounted, attempting to mount."
        mount $mount
    fi
    #otherwise do nothing
done

I have this set as a cronjob running every 5 minutes

*/5 * * * * /mountcheck.sh

Now the system will continually try to mount the specified folder if it isn’t already mounted.

OS

Create local CentOS 7 Repo

Leave a comment

I’ve recently needed to create a local mirror of Cent7 packages. I followed the guide posted on techmint but also made a few tweaks to get it to work to my liking.

Create local repo mirror

  • Install necessary packages
    • sudo yum -y install epel-release nginx createrepo yum-utils moreutils
  • Create directories that will host your repo
    • sudo mkdir -p /usr/share/nginx/html/repos/{base,centosplus,extras,updates,epel}
  • Use the reposync tool to synchronize to those local directories (repeat for each directory, changing repoid= value to match)
    • reposync -g -l -d -m --repoid=base --newest-only --download-metadata --download_path=/usr/share/nginx/html/repos/
  • Use the createrepo tool to create repodata
    • base & epel have a group file, other repos do not.
    • For base & epel:
      • createrepo -g comps.xml /usr/share/nginx/html/repos/<FOLDER>
    • For the rest:
      • createrepo /usr/share/nginx/html/repos/<FOLDER>

Configure daily synchronization via cron

Copy this script to /etc/cron.daily/ and give it execute rights

#!/bin/bash
##specify all local repositories in a single variable
LOCAL_REPOS="base extras updates epel centosplus"
##a loop to update repos one at a time
for REPO in ${LOCAL_REPOS}; do
reposync -g -l -d -m --repoid=$REPO --newest-only --download-metadata --download_path=/usr/share/nginx/html/repos/Cent7/
if [[ $REPO = 'base' || $REPO = 'epel' ]]; then
        createrepo -g comps.xml /usr/share/nginx/html/repos/Cent7/$REPO/
else
        createrepo /usr/share/nginx/html/repos/Cent7/$REPO/
fi
done
chmod 755 /etc/cron.daily/<script_name>

E-mails from cron became annoying. I wanted to only get e-mailed on error. The solution is to use chronic

Modify /etc/anacrontab to add “chronic” between nice and run-parts

1 5 cron.daily nice chronic run-parts /etc/cron.daily
7 25 cron.weekly nice chronic run-parts /etc/cron.weekly
@monthly 45 cron.monthly nice chronic run-parts /etc/cron.monthly

Success.

Update 4/25/19 I encountered an issue while trying to use repsync to mirror the remi repo.

warning: /usr/share/nginx/html/repos/Cent7/remi/remi/aspell-nl-0.50-1.el7.remi.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 00f97f56: NOKEY

I found out from here that it means you need to manually import the package’s key into the RPMDB like so

sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-remi

OS, Virtualization

Docker – run a cron job for a container from the host

Leave a comment

I’ve installed tiny tiny rss as a replacement for Feedly once they started inserting ads that looked like articles. Deceptive advertising. I’m not a fan.

I’ve spun up linuxserver’s version of it in docker and it works pretty well except for updating articles. I couldn’t find a great guide on configuring it for updates specifically within a docker container, so here is mine. My solution was to have a cron job running on the docker host to run the feed update script within the docker container, inspired by this post.

The trick is to use the docker exec command to run a command from the docker host but execute it within the running container.

docker exec -u 1001 -it TinyTinyRSS /usr/bin/php /config/www/tt-rss/update.php --feeds --quiet

The -u command specifies which user ID to run the command as. TinyTinyRSS is the name of my container. I’ve set this to run every 15 minutes with the following crontab syntax:

*/15 * * * * /usr/bin/docker exec -u 1001 -d TinyTinyRSS /usr/bin/php /config/www/tt-rss/update.php --feeds --quiet

edit: Modified the crontab entry to make it work properly per this post.