Category Archives: CLI

Headless Private Internet Access with OpenVPN & Port Forwarding

I wanted to initiate a connection to my VPN provider Private Internet Access (PIA) from one of my headless CentOS 7 servers. Unfortunately PIA software requires a GUI to work. Fortunately, PIA provides ovpn files that fully work with the openVPN software, which does work headlessly.

I followed this guide from htpcguides.com to get me going. I tweaked it a bit to work with CentOS 7. I then used this guide to help me set up port forwarding to work with transmission-daemon.

First, install openvpn & iproute

sudo yum -y install openvpn

Next grab PIA’s ovpn files for their servers:

wget https://www.privateinternetaccess.com/openvpn/openvpn.zip
sudo unzip openvpn.zip -d /etc/openvpn/pia

Now create a credentials file to specify your PIA username and password. The format is simply the first line being your PIA username, and the second line being your PIA password, nothing else.

#Create file
sudo vim /etc/openvpn/pia/.secret

#Enter PIA credentials into the file, then save and exit
USERNAME
PASSWORD

#Change permissions to be readable only by root
sudo chmod 700 /etc/openvpn/pia/.secret

Test if it’s working by manually running OpenVPN

sudo openvpn --config /etc/openvpn/PIA_SITE_FILENAME.ovpn --auth-user-pass /etc/openvpn/pia/.secret

Create a systemd service file to get OpenVPN to run on startup

sudo vim /usr/lib/systemd/system/pia.service

[Unit]
Description=OpenVPN
After=syslog.target network.target

[Service]
PrivateTmp=true
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --config "/etc/openvpn/PIA_SITE_FILENAME.ovpn" --auth-user-pass /etc/openvpn/.secret

[Install]
WantedBy=multi-user.target

Reload systemd daemons and enable at boot:

sudo systemctl daemon-reload
sudo systemctl enable pia.service

Start the service (or reboot) and then use curl to verify your external IP has changed

sudo systemctl start pia.service
curl https://ipinfo.io/ip

Port Forwarding

PIA is awesome because it supports port forwarding to facilitate things like torrents. I took the script from this guide and tweaked it a bit to suit my needs (and to correct a syntax error)

Note: PIA only supports port forwarding from specific regions. Here is the current list.

Download script to pia directory

sudo wget -O /etc/openvpn/pia/portforward.sh https://raw.githubusercontent.com/HTPCGuides/pia-port-forwarding-scripts/master/transmission-pia-port-forwarding.sh
sudo chmod +x /etc/openvpn/portforward.sh

Edit the script to provide PIA username and pass, and Transmission user/pass (if you have that configured.) The variables to change are USERNAME,PASSWORD, TRANSUSER, and TRANSPASS

Test the script by running it manually (make sure both your pia service and transmission services are already running.) I’ve noticed that the script will say success even on failure which is why I suggest using bash -x so you can see actual error messages from PIA regarding port forwarding:

sudo bash -x /etc/openvpn/pia/portforward.sh

After that’s run you can check your transmission daemon to ensure it is listening on the PIA port that’s been forwarded.

Configure to run on startup and every two hours by invoking cron:

sudo crontab -e

@reboot sleep 60 && /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward
0 */2 * * * /etc/openvpn/pia/portforward.sh | while IFS= read -r line; do echo "$(date) $line"; done >> /var/log/pia_portforward.log 2>&1 #PIA Port Forward

If you are paranoid about transmission running without the VPN working you can tweak your transmission service to rely on the pia service before starting. Modify /usr/lib/systemd/system/transmission-daemon.service and tweak the After line and add a Required line so it now reads this:

Required=pia.service
After=network.target pia.service

Don’t forget to run a sudo systemctl daemon-reload afterward.

Troubleshooting

If you get the error message “/etc/openvpn/pia/portforward.sh: line 35: [: : integer expression expected” it means you have to tweak the script a bit (it’s broken?)

Change line 35 to be the following:

if [ "$SPLITVPN" == 1 ]; then

Merge multiple MP4 files into one with ffmpeg concat

I had a bunch of MP4 files I wanted to merge into a single file. FFMpeg came to the rescue, but it had a bit of a quirk I needed to figure out. Thanks to this site for the help.

First, create a list of files you wish to merge. In this example I want every file with a specific file pattern beginning with 2019-05-02. I take that output and run sed against it to add “file ‘” to the begging and a closing ” ‘ ” at the end. The end result is a nice list of files for ffmpeg to ingest.

ls 2019-05-02* | sed "s/^/file '/g; s/$/'/g" > filelist.txt

With our shiny new list we can now tell ffmpeg to use it to concatenate to a single file:

ffmpeg -f concat -safe 0 -i filelist.txt -c copy OUTPUT_FILENAME.mp4

Split flac files with shnsplit

I had a few single FLAC files with cue files I wanted to put into Plex but to my dismay it doesn’t read the CUE files at all. Thus I needed to split the one FLAC file into multiple pieces with shnsplit. Thanks to Stack Exchange for the help.

On my Debian system:

 sudo apt install cuetools shntool flac

With the necessary tools installed you simply have to run the shnsplit command:

 shnsplit -f FILENAME.cue -t "%a - %n %t - %p" FILENAME.flac

the -t parameters formats the filename as desired per the manpage


-t fmt
Name output files in user‐specified format based on CUE sheet fields. The following formatting strings are recognized: 

%p
Performer
%a
Album
%t
Track title
%n
Track number

Fix Proxmox swapping issue

I recently had an issue with one of my Proxmox hosts where it would max out all swap and slow down to a crawl despite having plenty of physical memory free. After digging and tweaking, I found this post which directed to set the kernel swappiness setting to 0. More reading suggested I should set it to 1, which is what I did.

Append to /etc/sysctl.conf:

#Fix excessive swap usage
vm.swappiness = 1 

Apply settings with:

sysctl --system

This did the trick for me.

Rasbperry Pi as a dashboard computer

Here are my raw, unpolished notes on how I set up a raspberry pi to serve as a dashboard display:

Use Raspbian OS

Autostart Chrome in kiosk mode

Eliminate Chrome crash bubble thanks to this post

mkdir -p ~/.config/lxsession/LXDE-pi/
nano ~/.config/lxsession/LXDE-pi/autostart

Add this line:
@chromium-browser --kiosk --app=<URL>

Mouse removal

sudo apt-get install unclutter

in ~/.config/lxsession/LXDE-pi/autostart add

@unclutter -idle 5

Disable screen blank:

in /etc/lightdm/lightdm.conf add

[SeatDefaults]
xserver-command=X -s 0 -dpms

Open up SSH & VNC

Pi / Preferences / Raspberry Pi Configuration: Interfaces tab

SSH: Enable
VNC: Enable

Increase swap file

sudo nano /etc/dphys-swapfile
CONF_SWAPSIZE=2048M

Configure NTP

sudo apt-get install openntpd ntpdate
sudo systemctl enable openntpd
sudo ntpdate <IP of NTP server>

edit /etc/openntpd/ntpd.conf and modify servers lines to fit your NTP server

Disable overscan

Pi / Preferences / Raspberry Pi Configuration: System tab
Overscan: Disable

Installing Gears of War 4 in Windows 10

Installing Gears of War 4 on Windows can only be described as a hellish nightmare. Here are my notes on how I finally got it to install and run.

  • Download file with fiddler and a download accelerator, as outlined here:

[GUIDE v2.0] How to download install package for Windows Store games (bypass Store download issues or for install on another PC) from pcgaming

  • Once you have the file, begin installing / downloading the game from the Windows store. Get several hundred MB / a few GB, then pause download.
  • Open powershell (no need to be an admin) and run the following:
    • Add-AppxPackage -path "<PATH TO GEARS FILE>"

After finish, close windows store and re-open, launch from there.

You may need to repeat removal and installation process many times, but now that you have the EAPPX file, it should be much less painful.

Recover files from ZFS snapshot of ProxMox VM

I recently needed to restore a specific file from one of my ProxMox VMs that had been deleted. I didn’t want to roll back the entire VM from a previous snapshot – I just wanted a single file from the snapshot. My snapshots are handled via ZFS using FreeNAS.

Since my VM was CentOS 7 it uses XFS, which made things a bit more difficult. I couldn’t find a way to crash-mount a read-only XFS snapshot – it simply resufed to mount, so I had to make everything read/write. Below is the process I used to recover my file:

On the FreeNAS server, find the snapshot you wish to clone:

sudo zfs list -t snapshot -o name -s creation -r DATASET_NAME

Next, clone the snapshot

sudo zfs clone SNAPSHOT_NAME CLONED_SNAPSHOT_NAME

Next, on a Linux box, use SSHFS to mount the snapshot:

mkdir Snapshot
sshfs -o allow_other user@freenas:/mnt/CLONED_SNAPSHOT_NAME Snapshot/

Now create a read/write loopback device following instructions found here:

sudo -i #easy lazy way to get past permissions issues
cd /path/to/Snapshot/folder/created/above
losetup -P -f VM_DISK_FILENAME.raw
losetup 
#Take note of output, it's likely set to /dev/loop0 unless you have other loopbacks

Note if your VM files are not in RAW format, extra steps will need to be taken in order to convert it to RAW format.

Now we have an SSH-mounted loopback device ready for mounting. Things are complicated if your VM uses LVM, which mine does (CentOS 7). Once the loopback device is set, lvscan should see the image’s logical volumes. Make the desired volume active

sudo lvscan
sudo lvchange -ay /dev/VG_NAME/LV_NAME

Now you can mount your volume:

mkdir Restore
mount /dev/VG_NAME/LV_NAME Restore/

Note: for XFS you must have read/write capability on the loopback device for this to work.

When you’re done, do your steps in reverse to unmount the snaspshot:

#Unmount snapshot
umount Restore
#Deactivate LVM
lvchange -an /dev/VG_NAME/LV_NAME
Remove loopback device
losetup -d /dev/loop0 #or whatever the loopback device was
#Unmount SSHfs mount to ZFS server
umount Snapshot

Finally, on the ZFS server, delete the snapshot:

sudo zfs destroy CLONED_SNAPSHOT_NAME

Troubleshooting

When I tried to mount the LVM partition at this point I got this error message:

mount: /dev/mapper/centos_plexlocal-root: can't read superblock

It ended up being because I was accidentally creating a read-only loopback device. I destroy the loopback device and re-created with write support and all was well.

Batch crop images with imagemagick

My scanner adds annoying borders on everything it scans. I wanted to find a way to fix this with the command line. Enter Imagemagick (thanks to this site for the help.)

I found one picture and selected the area I wanted to crop it from. I used IrfanView to tell me the dimensions of the desired crop, then passed that info onto the command line. I used a bash for loop to get the job done on the entire directory:

for file in *.jpg; do convert $file -crop 4907x6561+53+75 $file; done

It worked beautifully.

Find video files in bash

I wanted a quick way to search my files for video types. I found here a quick snippet on how to do so. I augmented it after finding out how to remove some info and make it case insensitive. Here is the result:

find FULL_FOLDER_PATH -type f | grep -E "\.webm$|\.flv$|\.vob$|\.ogg$|\.ogv$|\.drc$|\.gifv$|\.mng$|\.avi$|\.mov$|\.qt$|\.wmv$|\.yuv$|\.rm$|\.rmvb$|/.asf$|\.amv$|\.mp4$|\.m4v$|\.mp*$|\.m?v$|\.svi$|\.3gp$|\.flv$|\.f4v$" -iname|sed 's/^.*://g'|sort