Tag Archives: linux

CLI

rsync to remote directory containing spaces

Leave a comment

I ran into an issue where if I ran rsync to a remote server with a path that contained spaces it would simply terminate at the first space instead of copying the whole path. I learned you need to do some quote tricks to get this to work properly (thanks to this site for helping me.)

The trick is to enclose the entire remote path, server included, in SINGLE quotes. In addition to the single quotes you still need to escape every space with a backslash ( \ )

rsync -aP <source_folder> 'REMOTE_SERVER:/full\ path\ with\ escaped\ spaces/folder/'

CLI, Hardware, OS

Manually set monitor refresh rate with xrandr

Leave a comment

My recent distro hopping has found some distros don’t let you change your monitor’s refresh rate via the GUI. After a while I found this post which explains the syntax for xrandr to force the refresh rate of 100hz (adjust to your liking)

xrandr --output DP-0 --mode 3440x1440 --rate 100

Create an auto start item with this line of code in it to get this to happen automatically on startup.

CLI, OS

KeePassRPC incompatible with the current KeePass version

Leave a comment

I keep forgetting about this snag so I’ll document it. In Debian / Ubuntu distros, once you’ve added the PPA to have the latest version of KeePass installed, if you try to install the KeePassRPC plugin (Kee) it will tell you the version is incompatible, even though it is.

The following plugin is incompatible with the current KeePass version: /usr/lib/keepass2/Plugins/KeePassRPC.plgx

Have a look at the plugin's website for an appropriate version.

The fix, as found here, is to install the mono-complete package

sudo apt install mono-complete

Restart KeePass after installation. That’s it!

CLI, Web

Headless Private Internet Access with OpenVPN & Port Forwarding

Leave a comment

I wanted to initiate a connection to my VPN provider Private Internet Access (PIA) from one of my headless CentOS 7 servers. Unfortunately PIA software requires a GUI to work. Fortunately, PIA provides ovpn files that fully work with the openVPN software, which does work headlessly.

I followed this guide from htpcguides.com to get me going. I tweaked it a bit to work with CentOS 7. I then used this guide to help me set up port forwarding to work with transmission-daemon.

First, install openvpn & iproute

sudo yum -y install openvpn

Next grab PIA’s ovpn files for their servers:

wget https://www.privateinternetaccess.com/openvpn/openvpn.zip
sudo unzip openvpn.zip -d /etc/openvpn/pia

Now create a credentials file to specify your PIA username and password. The format is simply the first line being your PIA username, and the second line being your PIA password, nothing else.

#Create file
sudo vim /etc/openvpn/pia/.secret

#Enter PIA credentials into the file, then save and exit
USERNAME
PASSWORD

#Change permissions to be readable only by root
sudo chmod 700 /etc/openvpn/pia/.secret

Test if it’s working by manually running OpenVPN

sudo openvpn --config /etc/openvpn/PIA_SITE_FILENAME.ovpn --auth-user-pass /etc/openvpn/pia/.secret

Create a systemd service file to get OpenVPN to run on startup

sudo vim /usr/lib/systemd/system/pia.service

[Unit]
Description=OpenVPN
After=syslog.target network.target

[Service]
PrivateTmp=true
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --config "/etc/openvpn/PIA_SITE_FILENAME.ovpn" --auth-user-pass /etc/openvpn/.secret

[Install]
WantedBy=multi-user.target

Reload systemd daemons and enable at boot:

sudo systemctl daemon-reload
sudo systemctl enable pia.service

Start the service (or reboot) and then use curl to verify your external IP has changed

sudo systemctl start pia.service
curl https://ipinfo.io/ip

Port Forwarding

PIA is awesome because it supports port forwarding to facilitate things like torrents. I took the script from this guide and tweaked it a bit to suit my needs (and to correct a syntax error)

Note: PIA only supports port forwarding from specific regions. Here is the current list.

Download script to pia directory

sudo wget -O /etc/openvpn/pia/portforward.sh https://raw.githubusercontent.com/HTPCGuides/pia-port-forwarding-scripts/master/transmission-pia-port-forwarding.sh
sudo chmod +x /etc/openvpn/portforward.sh

Edit the script to provide PIA username and pass, and Transmission user/pass (if you have that configured.) The variables to change are USERNAME,PASSWORD, TRANSUSER, and TRANSPASS

Test the script by running it manually (make sure both your pia service and transmission services are already running.) I’ve noticed that the script will say success even on failure which is why I suggest using bash -x so you can see actual error messages from PIA regarding port forwarding:

sudo bash -x /etc/openvpn/pia/portforward.sh

After that’s run you can check your transmission daemon to ensure it is listening on the PIA port that’s been forwarded.

Configure to run on startup and every two hours by invoking cron:

sudo crontab -e

@reboot sleep 60 && /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward
0 */2 * * * /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward

If you are paranoid about transmission running without the VPN working you can tweak your transmission service to rely on the pia service before starting. Modify /usr/lib/systemd/system/transmission-daemon.service and tweak the After line and add a Required line so it now reads this:

Required=pia.service
After=network.target pia.service

Don’t forget to run a sudo systemctl daemon-reload afterward.

Troubleshooting

If you get the error message “/etc/openvpn/pia/portforward.sh: line 35: [: : integer expression expected” it means you have to tweak the script a bit (it’s broken?)

Change line 35 to be the following:

if [ "$SPLITVPN" == 1 ]; then

CLI

Merge multiple MP4 files into one with ffmpeg concat

Leave a comment

I had a bunch of MP4 files I wanted to merge into a single file. FFMpeg came to the rescue, but it had a bit of a quirk I needed to figure out. Thanks to this site for the help.

First, create a list of files you wish to merge. In this example I want every file with a specific file pattern beginning with 2019-05-02. I take that output and run sed against it to add “file ‘” to the begging and a closing ” ‘ ” at the end. The end result is a nice list of files for ffmpeg to ingest.

ls 2019-05-02* | sed "s/^/file '/g; s/$/'/g" > filelist.txt

With our shiny new list we can now tell ffmpeg to use it to concatenate to a single file:

ffmpeg -f concat -safe 0 -i filelist.txt -c copy OUTPUT_FILENAME.mp4

CLI

Split flac files with shnsplit

Leave a comment

I had a few single FLAC files with cue files I wanted to put into Plex but to my dismay it doesn’t read the CUE files at all. Thus I needed to split the one FLAC file into multiple pieces with shnsplit. Thanks to Stack Exchange for the help.

On my Debian system:

 sudo apt install cuetools shntool flac

With the necessary tools installed you simply have to run the shnsplit command:

 shnsplit -f FILENAME.cue -t "%a - %n %t - %p" FILENAME.flac

the -t parameters formats the filename as desired per the manpage


-t fmt
Name output files in user‐specified format based on CUE sheet fields. The following formatting strings are recognized: 

%p
Performer
%a
Album
%t
Track title
%n
Track number

OS, Virtualization

Proxmox first VM boot delay workaround

2 Comments

My home lab has an NFS server for storage and a proxmox hypervisor connecting to it. If the power ever goes out for more than my UPS can handle, startup is a bit of a mess. My ProxMox server boots up much faster than my NFS server, so the result is no VMs start automatically (due to storage being unavailable) and I have to manually go in and start everything.

I found this bug report from 2015 which frustratingly doesn’t appear to have any traction to it. Ideally I could just tell the first VM to wait 5 minutes before turning on, and then trigger all the other VMs to turn on once the first one is up, but the devs don’t seem to want to address that issue. So, I got creative.

My solution was to alter the grub menu timeout before booting ProxMox. Simple but effective.

Edit /etc/default/grub and modify GRUB_TIMEOUT

#modify GRUB_TIMEOUT to your liking
GRUB_TIMEOUT=300

Then simply run update-grub

update-grub

Now my proxmox server waits 5 minutes before even booting the OS, by which time the NAS should be up and running. No more manual turning on of VMs after a power outage.

CLI, OS, Virtualization

Fix Proxmox swapping issue

1 Comment

I recently had an issue with one of my Proxmox hosts where it would max out all swap and slow down to a crawl despite having plenty of physical memory free. After digging and tweaking, I found this post which directed to set the kernel swappiness setting to 0. More reading suggested I should set it to 1, which is what I did.

Append to /etc/sysctl.conf:

#Fix excessive swap usage
vm.swappiness = 1

Apply settings with:

sysctl --system

This did the trick for me.

CLI, Hardware, OS

Rasbperry Pi as a dashboard computer

Leave a comment

Here are my raw, unpolished notes on how I set up a raspberry pi to serve as a dashboard display:

Use Raspbian OS

Autostart Chrome in kiosk mode

Eliminate Chrome crash bubble thanks to this post

mkdir -p ~/.config/lxsession/LXDE-pi/
nano ~/.config/lxsession/LXDE-pi/autostart

Add this line:
@chromium-browser --kiosk --app=<URL>

Mouse removal

sudo apt-get install unclutter

in ~/.config/lxsession/LXDE-pi/autostart add

@unclutter -idle 5

Disable screen blank:

in /etc/lightdm/lightdm.conf add

[SeatDefaults]
xserver-command=X -s 0 -dpms

Open up SSH & VNC

Pi / Preferences / Raspberry Pi Configuration: Interfaces tab

SSH: Enable
VNC: Enable

Increase swap file

sudo nano /etc/dphys-swapfile
CONF_SWAPSIZE=2048M

Configure NTP

sudo apt-get install openntpd ntpdate
sudo systemctl enable openntpd
sudo ntpdate <IP of NTP server>

edit /etc/openntpd/ntpd.conf and modify servers lines to fit your NTP server

Disable overscan

Pi / Preferences / Raspberry Pi Configuration: System tab
Overscan: Disable

CLI, OS

Recover files from ZFS snapshot of ProxMox VM

Leave a comment

I recently needed to restore a specific file from one of my ProxMox VMs that had been deleted. I didn’t want to roll back the entire VM from a previous snapshot – I just wanted a single file from the snapshot. My snapshots are handled via ZFS using FreeNAS.

Since my VM was CentOS 7 it uses XFS, which made things a bit more difficult. I couldn’t find a way to crash-mount a read-only XFS snapshot – it simply resufed to mount, so I had to make everything read/write. Below is the process I used to recover my file:

On the FreeNAS server, find the snapshot you wish to clone:

sudo zfs list -t snapshot -o name -s creation -r DATASET_NAME

Next, clone the snapshot

sudo zfs clone SNAPSHOT_NAME CLONED_SNAPSHOT_NAME

Next, on a Linux box, use SSHFS to mount the snapshot:

mkdir Snapshot
sshfs -o allow_other user@freenas:/mnt/CLONED_SNAPSHOT_NAME Snapshot/

Now create a read/write loopback device following instructions found here:

sudo -i #easy lazy way to get past permissions issues
cd /path/to/Snapshot/folder/created/above
losetup -P -f VM_DISK_FILENAME.raw
losetup 
#Take note of output, it's likely set to /dev/loop0 unless you have other loopbacks

Note if your VM files are not in RAW format, extra steps will need to be taken in order to convert it to RAW format.

Now we have an SSH-mounted loopback device ready for mounting. Things are complicated if your VM uses LVM, which mine does (CentOS 7). Once the loopback device is set, lvscan should see the image’s logical volumes. Make the desired volume active

sudo lvscan
sudo lvchange -ay /dev/VG_NAME/LV_NAME

Now you can mount your volume:

mkdir Restore
mount /dev/VG_NAME/LV_NAME Restore/

Note: for XFS you must have read/write capability on the loopback device for this to work.

When you’re done, do your steps in reverse to unmount the snaspshot:

#Unmount snapshot
umount Restore
#Deactivate LVM
lvchange -an /dev/VG_NAME/LV_NAME
Remove loopback device
losetup -d /dev/loop0 #or whatever the loopback device was
#Unmount SSHfs mount to ZFS server
umount Snapshot

Finally, on the ZFS server, delete the snapshot:

sudo zfs destroy CLONED_SNAPSHOT_NAME

Troubleshooting

When I tried to mount the LVM partition at this point I got this error message:

mount: /dev/mapper/centos_plexlocal-root: can't read superblock

It ended up being because I was accidentally creating a read-only loopback device. I destroy the loopback device and re-created with write support and all was well.