All posts by nicholas

Rasbperry Pi as a dashboard computer

Here are my raw, unpolished notes on how I set up a raspberry pi to serve as a dashboard display:

Use Raspbian OS

Autostart Chrome in kiosk mode

Eliminate Chrome crash bubble thanks to this post

mkdir -p ~/.config/lxsession/LXDE-pi/
nano ~/.config/lxsession/LXDE-pi/autostart

Add this line:
@chromium-browser --kiosk --app=<URL>

Mouse removal

sudo apt-get install unclutter

in ~/.config/lxsession/LXDE-pi/autostart add

@unclutter -idle 5

Disable screen blank:

in /etc/lightdm/lightdm.conf add

[SeatDefaults]
xserver-command=X -s 0 -dpms

Open up SSH & VNC

Pi / Preferences / Raspberry Pi Configuration: Interfaces tab

SSH: Enable
VNC: Enable

Increase swap file

sudo nano /etc/dphys-swapfile
CONF_SWAPSIZE=2048M

Configure NTP

sudo apt-get install openntpd ntpdate
sudo systemctl enable openntpd
sudo ntpdate <IP of NTP server>

edit /etc/openntpd/ntpd.conf and modify servers lines to fit your NTP server

Disable overscan

Pi / Preferences / Raspberry Pi Configuration: System tab
Overscan: Disable

Installing Android Auto / Apple Carplay retrofit in Mazda CX3

I just finished installing the Android Auto / Apple CarPlay retrofit kit into my 2016 Mazda CX-3 Grand Touring AWD. It’s sweet! The process to get it installed was a learning experience for me.

The CX-3 is not as popular as its bigger brother the CX-5 and therefore there wasn’t nearly as much information on how to do the retrofit. Of great help were various posts over at CX3 forum and this youtube video.

The steps to accomplish the retrofit are as follows, taken from the official Mazda guide

  1. Update Mazda firmware to latest 70.00+ version
    1. Firmware can be found here
  2. Remove glove compartment
  3. Remove Audio Panel 1
  4. Remove Audio Panel 2
  5. Remove Central display
    1. Be careful here. This will scratch up your dashboard. Do not ignore recommendations to lay down protective tape/other layer
  6. Remove Center Console Tray (or, in my case, the armrest attachment)
  7. Remove shift bezel
  8. Remove upper panel
  9. Remove/detach shift panel (Removal not necessary)
    1. I didn’t want to undo the shift knob, so I just unplugged the panel and rotated the whole assembly to the side. It was enough for the installation – no removal required.
  10. Remove console side panels
  11. Remove front console box
  12. Remove DVD/CD Player (if installed)
    1. Removal keys were not easy to find. I ended up using two small cutting knives I found in my kitchen. Insert into keyholes, angle them slightly, and pull.
  13. Remove Front Console
  14. Remove old USB hub from front console
  15. Replace USB hub in front console
  16. Wrap wiring in foam
  17. Route wiring and add zip ties
  18. Replace components in reverse order from how you removed them

The biggest challenges for me were the armrest and CD player. The rest of it was pretty straightforward. Here are some pictures of the process:

Mazda CX3 CMU removal
CX3 bezel removal
CX3 upper panel removed
CX3 front console removed
CX3 everything removed

Installing Gears of War 4 in Windows 10

Installing Gears of War 4 on Windows can only be described as a hellish nightmare. Here are my notes on how I finally got it to install and run.

  • Download file with fiddler and a download accelerator, as outlined here:

[GUIDE v2.0] How to download install package for Windows Store games (bypass Store download issues or for install on another PC) from pcgaming

  • Once you have the file, begin installing / downloading the game from the Windows store. Get several hundred MB / a few GB, then pause download.
  • Open powershell (no need to be an admin) and run the following:
    • Add-AppxPackage -path "<PATH TO GEARS FILE>"

After finish, close windows store and re-open, launch from there.

You may need to repeat removal and installation process many times, but now that you have the EAPPX file, it should be much less painful.

Recover files from ZFS snapshot of ProxMox VM

I recently needed to restore a specific file from one of my ProxMox VMs that had been deleted. I didn’t want to roll back the entire VM from a previous snapshot – I just wanted a single file from the snapshot. My snapshots are handled via ZFS using FreeNAS.

Since my VM was CentOS 7 it uses XFS, which made things a bit more difficult. I couldn’t find a way to crash-mount a read-only XFS snapshot – it simply resufed to mount, so I had to make everything read/write. Below is the process I used to recover my file:

On the FreeNAS server, find the snapshot you wish to clone:

sudo zfs list -t snapshot -o name -s creation -r DATASET_NAME

Next, clone the snapshot

sudo zfs clone SNAPSHOT_NAME CLONED_SNAPSHOT_NAME

Next, on a Linux box, use SSHFS to mount the snapshot:

mkdir Snapshot
sshfs -o allow_other user@freenas:/mnt/CLONED_SNAPSHOT_NAME Snapshot/

Now create a read/write loopback device following instructions found here:

sudo -i #easy lazy way to get past permissions issues
cd /path/to/Snapshot/folder/created/above
losetup -P -f VM_DISK_FILENAME.raw
losetup 
#Take note of output, it's likely set to /dev/loop0 unless you have other loopbacks

Note if your VM files are not in RAW format, extra steps will need to be taken in order to convert it to RAW format.

Now we have an SSH-mounted loopback device ready for mounting. Things are complicated if your VM uses LVM, which mine does (CentOS 7). Once the loopback device is set, lvscan should see the image’s logical volumes. Make the desired volume active

sudo lvscan
sudo lvchange -ay /dev/VG_NAME/LV_NAME

Now you can mount your volume:

mkdir Restore
mount /dev/VG_NAME/LV_NAME Restore/

Note: for XFS you must have read/write capability on the loopback device for this to work.

When you’re done, do your steps in reverse to unmount the snaspshot:

#Unmount snapshot
umount Restore
#Deactivate LVM
lvchange -an /dev/VG_NAME/LV_NAME
Remove loopback device
losetup -d /dev/loop0 #or whatever the loopback device was
#Unmount SSHfs mount to ZFS server
umount Snapshot

Finally, on the ZFS server, delete the snapshot:

sudo zfs destroy CLONED_SNAPSHOT_NAME

Troubleshooting

When I tried to mount the LVM partition at this point I got this error message:

mount: /dev/mapper/centos_plexlocal-root: can't read superblock

It ended up being because I was accidentally creating a read-only loopback device. I destroy the loopback device and re-created with write support and all was well.

Batch crop images with imagemagick

My scanner adds annoying borders on everything it scans. I wanted to find a way to fix this with the command line. Enter Imagemagick (thanks to this site for the help.)

I found one picture and selected the area I wanted to crop it from. I used IrfanView to tell me the dimensions of the desired crop, then passed that info onto the command line. I used a bash for loop to get the job done on the entire directory:

for file in *.jpg; do convert $file -crop 4907x6561+53+75 $file; done

It worked beautifully.

Find video files in bash

I wanted a quick way to search my files for video types. I found here a quick snippet on how to do so. I augmented it after finding out how to remove some info and make it case insensitive. Here is the result:

find FULL_FOLDER_PATH -type f | grep -E "\.webm$|\.flv$|\.vob$|\.ogg$|\.ogv$|\.drc$|\.gifv$|\.mng$|\.avi$|\.mov$|\.qt$|\.wmv$|\.yuv$|\.rm$|\.rmvb$|/.asf$|\.amv$|\.mp4$|\.m4v$|\.mp*$|\.m?v$|\.svi$|\.3gp$|\.flv$|\.f4v$" -iname|sed 's/^.*://g'|sort

Generate SuperMicro IPMI license

Thank-you, Peter Kleissner, for saving me from having to use my time machine to simply update my server’s BIOS: https://peterkleissner.com/2018/05/27/reverse-engineering-supermicro-ipmi/

Supermicro IPMI License Key (for updating BIOS) = HMAC-SHA1-96(INPUT: MAC address of BMC, SECRET KEY: 85 44 E3 B4 7E CA 58 F9 58 30 43 F8)

echo -n 'bmc-mac' | xxd -r -p | openssl dgst -sha1 -mac HMAC -macopt hexkey:8544E3B47ECA58F9583043F8 | awk '{print $2}' | cut -c 1-24

Upgrading AWX

AWX is the open source version of Ansible Tower. It’s a powerful tool, but unfortunately AWX has no in place upgrade capability. If you want to upgrade your AWX to the latest version it takes a bit of trickery (the easy way out being just to pay for Ansible Tower.)

Essentially to upgrade AWX you need to spin up a completely new instance and then migrate your data over to it. Fortunately there is a script out there that makes doing this a bit easier.

Below are my notes for how I upgraded my instance of AWX from version 1.0.6 to 2.1.0.

Create temporary AWX migration server

Spin up new server with ansible installed, then clone AWX

git clone https://github.com/ansible/awx.git 
cd awx 
git clone https://github.com/ansible/awx-logos.git

Modify AWX install to expose 5432 externally:

edit installer/roles/local_docker/tasks/standalone.yml and add

    ports:
      - "5432:5432" 

right above the when: pg_hostname is not defined or pg_hostname == '' line. Complete stanza looks like this:

- name: Activate postgres container
  docker_container:
    name: postgres
    state: started
    restart_policy: unless-stopped
    image: "{{ postgresql_image }}"
    volumes:
      - "{{ postgres_data_dir }}:/var/lib/postgresql/data:Z"
    env:
      POSTGRES_USER: "{{ pg_username }}"
      POSTGRES_PASSWORD: "{{ pg_password }}"
      POSTGRES_DB: "{{ pg_database }}"
      PGDATA: "/var/lib/postgresql/data/pgdata"
    ports:
      - "5432:5432"
  when: pg_hostname is not defined or pg_hostname == ''
  register: postgres_container_activate

Make sure you have port 5432 open on your host-based firewall.

Install AWX on the new host. Verify you can log into the empty instance and that it’s the version you want to upgrade to.

Prepare original AWX server to send

Kill the AWX postgres container on the source machine, and re-run awx installer after modifying it to expose its postgres port as described above.

Install tower-cli (this can be on either source or destination servers)

sudo pip install ansible-tower-cli

Configure tower-cli

tower-cli config username SRC_AWX_USERNAME
towercli config password SRC_AWX_PASSWORD
towercli config host SRC_AWX_HOST

Make sure to use full ansible URL as accessed from a browser for both source and destination

Install awx-migrate:

git clone https://github.com/autops/awx-migrate.git

Update awx-migrate/awx-migrate-wrapper with correct source and destination info

Run awx-migrate-wrapper. It will generate json files with your configuration.

Migrate database to temporary server

Modify tower-cli config, set host, username and password to that of the destination AWX instance

tower-cli config username DEST_AWX_USERNAME
towercli config password DEST_AWX_PASSWORD
towercli config host: DEST_AWX_HOST

Send JSON info to destination:

tower-cli send awx-data.json

You will now have a fresh new, updated AWX instance working, with imported database, on the destination host. Confirm you can log into it with the admin account you set it up with.

Prepare original AWX server to receive

Now, on the source, remove  the old AWX docker containers:

sudo docker rm -f postgres awx_task awx_web memcached rabbitmq

Move / delete the database folder the postgres docker container was using (as defined in awx installer inventory) in my case:

/var/lib/awx
/var/db/pgsqldocker

Remove and re-install AWX folder with a fresh git checkout

rm -rf awx
git clone https://github.com/ansible/awx.git
cd awx
git clone https://github.com/ansible/awx-logos.git

Re-run the AWX installer to re-create a blank database on the source host, modify the new awx/installer/inventory as needed. Also modify installer/roles/local_docker/tasks/standalone.yml as outlined above.

cd awx/installer
sudo ansible-playbook -i inventory install.yml

Migrate from temporary AWX server back to source AWX server

Once a new, empty version of awx is running on the source host,  start the awx-migrate process in reverse to migrate the database on the destination instance back to the source. Modify awx-migrate-wrapper and tower-cli to switch src and destination (the destination has become the source and the source has become the destination)

Use awx-migrate-wrapper to generate  new ansible version json files (don’t confuse them with the old json files – best to delete / move all json files before running awx-migrate-wrapper)

Modify tower-cli to point to original AWX URL

Run tower-cli send awx-data.json

Once completed, log in as the admin account. Input LDAP BIND password under settings, then delete any imported LDAP users.

Cleanup

You may want to remove the exposed postgres database ports. Simply undo the changes you made in awx/installer/roles/local_docker/tasks/standalone.yml to remove the Ports part of the first play, then remove your postgres container and re-install AWX with install.yml

Also remember to delete the JSON files generated with awx-migrate as they contain all your credentials in plaintext.

Success.

 

Export multiple resolutions in Lightroom

I needed to one-click export multiple resolutions of pictures from Lightroom. Unfortunately there isn’t any kind of plugin available to do this. Fortunately I was able to find this guide on how to get an applescript script to do it for me (Mac only, sadly.)

The trick is to write a few bits of applescript and save it as an application. Then when exporting the pictures in lightroom, make sure the word “fullsize” is in the filename, and configure lightroom to run your applescript after export.

I tweaked the script a bit to move the full size version to a different folder, then open Finder to the folder that the other resolutions were created in (thanks to this site for the guidance)

Here is my script below. It works!

on open of myFiles
	set newSizes to {5000}
	
	repeat with aFile in myFiles
		set filePath to aFile's POSIX path
		set bPath to (do shell script "dirname " & quoted form of filePath)
		tell application "System Events" to set fileName to aFile's name
		repeat with newSize in newSizes
			do shell script "sips " & quoted form of aFile's POSIX path & " -Z " & newSize & " --out " & quoted form of (bPath & "/" & rename(fileName, newSize) as text)
		end repeat
		do shell script "mv " & quoted form of aFile's POSIX path & " /path/to/folder/for/fullres/images"
	end repeat
	do shell script "open /path/to/folder/of/resized/images"
end open

on rename(fName, fSize)
	do shell script "sed 's/fullsize/" & fSize & "/' <<< " & quoted form of fName
end rename