Install Virtualbox on a Chromebook

I am really enjoying my Chromebook Pixel 2015. Recently I needed to spin up a few VMs on this box. I tried to install virtualbox but it turns out that the kernel for the chromebook does not include virtualbox headers. Fortunately it’s fairly easy to add them, thanks to divx118’s scripts.

First, enable the necessary chromebook kernel flags. Run these commands in a crosh shell (not in a chroot)

cd ~/Downloads
wget https://raw.githubusercontent.com/divx118/crouton-packages/master/change-kernel-flags
sudo sh ~/Downloads/change-kernel-flags

Note: You will need to repeat the above steps after each chromeos update.

Next, open up a chroot shell and do the following:

cd ~
wget https://raw.githubusercontent.com/divx118/crouton-packages/master/setup-headers.sh
sudo sh setup-headers.sh

Lastly, reboot the chromebook. Once you’re back up, enter your chrooted environment and install Virtualbox from their Oracle’s download page. Don’t use the virtualbox repositories – they don’t work.

Install the downloaded deb file with dpkg

dpkg -i virtualbox-5.0_5.0.10-104061~Ubuntu~trusty_amd64.deb

You might get this error:

dpkg: dependency problems prevent configuration of virtualbox-5.0:
 virtualbox-5.0 depends on libqt4-opengl (>= 4:4.7.2); however:
  Package libqt4-opengl is not installed.

The fix to that error is to run the following command to install missing dependencies:

apt-get -f install

If you get weird errors about VT-X not being enabled in the BIOS, try running the script and rebooting again.

Success!

Randomize files in a folder

I wanted to make a simple slideshow for a cheap Kindle turned photo frame in my office. Windows Movie Maker (free, already installed program) does not have a randomize function when importing photos. I had a lot of photos I wanted imported and I wanted them randomized. Movie Maker doesn’t include subfolders for some dumb reason, so I also needed a way to grab pictures from various directories and put them in a single directory.

My solution (not movie-maker specific) was to use bash combined with find, ln, and mv to get the files exactly how I want them. The process goes as follows:

  1. Create a temporary folder
  2. Use the find command to find files you want
    1. Use -type f argument to find only files (don’t replicate directory structure)
    2. Use the -exec argument to call the ln command to create links to each file found
    3. Use the -s argument of ln to create symbolic links
    4. Use the -b argument of ln to ensure duplicate filenames are not overwritten
  3. Invoke a one line bash command to randomize the filenames of those symbolic links

It worked beautifully. The commands I ended up using were as follows:

mkdir temp
cd temp
find /Pictures/2013/ -type f -exec ln -s -b {} . \;
#repeat for each subfolder as needed, unless you want all folders in which case you can just specify the directory beneath it.
find /Pictures/2014/ -type f -exec ln -s -b {} . \;
find /Pictures/2015/ -type f -exec ln -s -b {} . \;

for i in *.JPG; do mv "$i" "$RANDOM.jpg"; done
#repeat for all permutations. The -b argument of ln creates files with tildes in the extension - don't forget about them.
for i in *.jpg; do mv "$i" "$RANDOM.jpg"; done
for i in *.JPG~; do mv "$i" "$RANDOM.jpg"; done
for i in *.jpg~; do mv "$i" "$RANDOM.jpg"; done

The end result was a directory full of pictures with random filenames, ready to be dropped into any crappy slideshow making software of your choosing 🙂

Install Owncloud 8 on Centos 7

I recently needed to re-install my Owncloud VM. I’ve been on a CentOS kick lately so I decided to see if I could install OwnCloud 8 on a Centos 7 base install. It turned out to not be as easy as I thought it would be.

When I tried to install owncloud on my CentOS 7 system, I kept getting a 404 error message even though I followed the documentation outlined here.

It turns out that they changed where the RPM is held and apparently forgot to update the documentation. I discovered this by manually navigating to download.suse.org/repositories/isv:ownCloud:community and browsing the directories. The documentation has you grab a repo from Centos_Centos-7 folder, which is broken. It looks like the proper directory is just Centos_7.

I had to remove the old repo, purge the cache, and try again. To do so, remove the .repo file and purge via yum:

cd /etc/yum.repos/d/
rm isv\:ownCloud\:community.repo
yum --enablerepo=isv_ownCloud_community clean metadata
wget http://download.opensuse.org/repositories/isv:ownCloud:community/CentOS_7/isv:ownCloud:community.repo
yum install owncloud

The above procedure is what you should run if you’ve already tried to use the broken link in the documentation and failed. If you haven’t installed owncloud yet, do the following

cd /etc/yum.repos/d/
wget http://download.opensuse.org/repositories/isv:ownCloud:community/CentOS_7/isv:ownCloud:community.repo
yum install owncloud

Success.

Fix raidcheck error on Xenserver 6.5

After configuring a software RAID to host my VMs on my Xenserver 6.5 instance I began receiving odd e-mails once a week. The e-mails simply said:

/usr/sbin/raid-check: line 62: declare: -A: invalid option 
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]

It turns out /usr/sbin-raid-check is a bash script called from the file /etc/cron.d/raid-check. It’s a weekly cron job designed to “scrub” the RAID array. I was getting these e-mails because I had configured my xen server to e-mail me anything sent to root, which includes messages encountered during cron jobs.

There appears to be a typo in the raid-check script. Line 62 of raid-check reads:

declare -A check

After reading the syntax of the declare command, I believe the issue is the fact that the A is capitalized. I commented out that line and replaced it with

declare -a check

That seemed to work. No more weird errors coming from my xenserver.

Install Splunk Universal Forwarder on Linux

I do this infrequently enough that I decided I should really write this down. Below is the quick and dirty way to get the Splunk universal forwarder installed on a new Linux  system. Thanks to byteschef for the information used to create this guide.

Download the latest splunk .RPM from their site and install it via RPM -i <filename> (if RedHat based) or dpki -i <filename> if debian based.

Run the following commands as root:

cd /opt/splunkforwarder/bin
./splunk start --accept-license
./splunk enable boot-start
./splunk add forward-server <IP/hostname of splunk server>:9997 -auth admin:changeme
./splunk add monitor /var/log
./splunk edit user admin -password NEW_PASSWORD -auth admin:changeme
./splunk restart

If there are any other directories you want monitored other than /var/log (application logs, for example) then issue:

./splunk add monitor <directory to monitor>

Done.

Fix Owncloud 8.1.1 samba shares not working

It never seems to go smoothly, does it? I just upgraded my version of Owncloud from 8.0.4 to 8.1.1 on my Ubuntu Trusty Tahr 14.04 VM. After the upgrade I noticed that all my samba (SMB) shares were gone. The logs were not very helpful, full of things like these:

Exception: {"Exception":"Icewind\\SMB\\Exception\\InvalidHostException","Message":"","Code":0,"Trace":"#0 \/var\/www\/owncloud\/apps\/files_external\/3rdparty\/icewind\/smb\/src\/Connection.php(37): Icewind\\SMB\\Connection

Additionally errors like this were showing up:

Your web server is not yet set up properly to allow file synchronization because the WebDAV interface seems to be broken.

After much digging I discovered this post which had a suggestion to install libsmbclient-php. In Ubuntu 14.04 it involves this command:

sudo apt-get install php5-libsmbclient

That did the trick! After installing php5-libsmbclient my samba shares worked once more.

 

Xenserver SSH backup script

Citrix Xenserver is an amazing hypervisor with pretty much every function released to you for free. One thing they do not handle, however, is automated backups.

I have hacked together a backup script for myself that seems to work fairly well. It is my own mix of this and this script along with some logic for e-mail reporting that I came up with myself. It does not require any modification of the xenserver host at all (no need to mount anything!) with the exception of adding a public key to the xenserver’s authorized_keys file.

This script can be run on anything with BASH and the appropriate UNIX tools (even other xenservers if you want) and uses SSH to initiate and transfer the backups to a location of your choosing.

Place this script on the machine you want to be initiating / saving the backups on. It requires that you generate an SSH public/private key pair, which can be done with this command:

ssh-keygen

Add the resulting .pub file’s contents to your xenserver’s /root/.ssh/authorized_keys file (create it if it doesn’t exist.) Take note of the location of the private key file that was generated with that command and put that path in the script.

You can download the script here or view it below. This script has been working pretty well for me. Note it will not work with any VMs that have spaces in their names. I was too lazy to debug this so I just renamed the problem VMs to remove the spaces. Enjoy!

#!/bin/bash

# Modified August 30, 2015 by Nicholas Jeppson
# Taken from http://discussions.citrix.com/topic/345960-xenserver-automated-snapshots-script/ and modified to allow for ssh backups
# Additional insight taken from https://github.com/cepa/xen-backup

# [Usage & Config]
# This script involves two computers: a xenserver machine and a backup machine.
# Put this script on the backup server and run with any account that has privileges to the desired export directory.
#
# This script assumes you have already created a private and public key pair on the backup server
# as well as adding respective the public key to the xenserver authorized_keys file.
#
# [How it works]
# Step1: Snapshots each VM on the xen server
# Step2: Backs up the snapshots to specified location
# Step3: Deletes temporary snapshot created in step 1
# Step4: Deletes old VM backups as defined later in this file
#
# [Note]
# This script will only work with VMs that don't have spaces in their names
# Please make sure you have enough disk space for BACKUP_PATH, or backup will fail
#
# Tested on xenserver 6.5
#
# Modify the variables in the config section below to suit your particular environment's needs.
#

############### Config section ###############

#Location where you want backups to go
BACKUP_PATH="/mnt/backup/OS/"

#SSH configuration
SSH_CIPHER="arcfour128"

#Number of backups to keep
NUM_BACKUPS=2

#Xenserver ssh configuration
#This dictates the address and location of keyfiles as they reside on the xenserver
XEN_ADDRESS="192.168.1.1"
XEN_KEY_LOCATION="/home/backup/.ssh/backup"
XEN_USER="root"

#E-mail configuration
EMAIL_ADDRESS="youremail@provider.here"
EMAIL_SUBJECT="`hostname -s | awk '{print "["toupper($1)"]"}'` VM Backup Report: `date +"%A %b %d %Y"`"

########## End of Config section ###############

ret_code=0

#Replace any spaces found with backslashes because dd doesn't like them
BACKUP_PATH_ESCAPED="`echo $BACKUP_PATH | sed 's/ /\\\ /g'`"

# SSH command
remote_exec() {
chmod 0600 $XEN_KEY_LOCATION
ssh -i $XEN_KEY_LOCATION -o "StrictHostKeyChecking no" -c $SSH_CIPHER $XEN_USER@$XEN_ADDRESS $1
}

backup() {
echo "======================================================"
echo "VM Backup started: `date`"
begin="$(date +%s)"
echo "Backup location: ${BACKUP_PATH}"
echo

#add a slash to the end of the backup path if it doesn't exist
if [[ "$BACKUP_PATH" != */ ]]; then
BACKUP_PATH="$BACKUP_PATH/"
fi

#Build array of VM names
VMNAMES=$(remote_exec "xe vm-list is-control-domain=false | grep name-label | cut -d ':' -f 2 | tr -d ' '")

for VMNAME in $VMNAMES
do
echo "======================================================"
echo "$VMNAME backup started `date`"
echo
before="$(date +%s)"

# create snapshot
TIMESTAMP=`date '+%Y%m%d-%H%M%S'`
SNAPNAME="$VMNAME-$TIMESTAMP"
SNAPUUID=$(remote_exec "xe vm-snapshot vm=\"$VMNAME\" new-name-label=\"$SNAPNAME\"")

# export snapshot
# remote_exec "xe snapshot-export-to-template snapshot-uuid=$SNAPUUID filename= | gzip" | gunzip | dd of="$BACKUP_PATH/$SNAPNAME.xva"
remote_exec "xe snapshot-export-to-template snapshot-uuid=$SNAPUUID filename=" | dd of="$BACKUP_PATH/$SNAPNAME.xva"

#if export was unsuccessful, return error
if [ $? -ne 0 ]; then
echo "Failed to export snapshot name = $snapshot_name$backup_ext"
ret_code=1

else
#calculate backup time, print results
after="$(date +%s)"
elapsed=`bc -l <<< "$after-$before"`
elapsedMin=`bc -l <<< "$elapsed/60"`
echo "Snapshot of $VMNAME saved to $SNAPNAME.xva"
echo "Backup completed in `echo $(printf %.2f $elapsedMin)` minutes"

# destroy snapshot
remote_exec "xe snapshot-uninstall force=true snapshot-uuid=$SNAPUUID"

#remove old backups (uses num_backups variable from top)
BACKUP_COUNT=$(find $BACKUP_PATH -name "$VMNAME*.xva" | wc -l)

if [[ "$BACKUP_COUNT" -gt "$NUM_BACKUPS" ]]; then

OLDEST_BACKUP=$(find $BACKUP_PATH -name "$VMNAME*" -print0 | xargs -0 ls -tr | head -n 1)
echo
echo "Removing oldest backup: $OLDEST_BACKUP"
rm $OLDEST_BACKUP
if [ $? -ne 0 ]; then
echo "Failed to remove $OLDEST_BACKUP"
fi
fi
echo "======================================================"
fi
done

end=$"$(date +%s)"
total_time=`bc -l <<< "$end-$begin"`
total_time_min=`bc -l <<< "$total_time/60"`
echo "Backup completed: `date`"
echo "VM Backup completed in `echo $(printf %.2f $total_time_min)` minutes"
echo
}

#Run the backup function and save all output to a variable, including stderr
BACKUP_OUTPUT=$(backup 2>&1)

#Clean up the output of the backup function
#Remove records count from dd, do some basic math to make dd's numbers more human readable
BACKUP_OUTPUT_HUMANIZED=$(echo "$BACKUP_OUTPUT" | sed -r '/.*records /d' | tr -d '()' \
| awk '{sub(/.*bytes /, $1/1024/1024/1024" GB "); sub(/in .* secs/, "in "$5/60" mins "); sub(/mins .*/, "mins (" $7/1024/1024" MB/sec)"); print}')

#Send a report e-mail with the backup results
echo "$BACKUP_OUTPUT_HUMANIZED" | mail -s "$EMAIL_SUBJECT" "$EMAIL_ADDRESS"

exit $ret_code

Avoid prompts when installing FreeBSD ports

The FreeBSD ports tree is wonderful for installing software but sometimes it can be a real pain. Recently I was trying to install Emby in FreeBSD because why not? The instructions were easy enough except for when I ran

make install clean

I was constantly barraged with choices for things. I want to assume the default on all of these and not be barraged with questions.

Thanks to stack exchange I learned it’s relatively easy to bypass all these questions. Simply add:

BATCH=yes

to the end of your make install clean statement to assume the defaults to all the questions for the package. The Emby guide is pretty comprehensive, but I would add this command at the bottom:

make install clean BATCH=yes

Handy.

Measure SSH transfer speeds

SSH is a beautiful thing. In addition to remotely administering machines you can use it to transfer files. To do this one simply pipes the cat command on both ends. For example, to copy hello.txt on the source host to hi.txt on the destination host, the command would be:

ssh remote_host cat hello.txt | cat > hi.txt

The command takes the contents of hello.txt and pipes it over to the remote host. The cat command on the remote host takes what was piped to it as  input and the > sign instructs cat to take its input and output it to hi.txt.

A great way to measure transfer speeds using ssh between two hosts is to take /dev/zero on the source host and output it to /dev/null on the destination host. This bypasses any disk speed bottlenecks and only measures network throughput. Combine this with the pv command to get a nice graphical view of how fast the transfer is going.

ssh remote_host cat /dev/zero | pv | cat > /dev/null

The default options between my machines result in about a 65 megabytes a second transfer speed.

1

It turns out that the encryption cipher used makes a big difference on transfer speeds. Use the -c command to specify which cipher to use and see how much of a difference this makes. -o compression=no can also help with transfer speeds.

The fastest cipher I’ve found is arcfour. It’s touted as less secure, but for my local network I can accept the risk (thanks to slashdot for the discussion.)

ssh -c arcfour -o Compression=no remote_host cat /dev/zero | pv | cat > /dev/null

2

Using acrfour more than doubles the speed for me! Amazing.

Allow non-root users to mount disks

I came across a need today to allow a regular (non-root) user to mount disks in Ubuntu 14.04 Trusty Tahr. I usually use sudo but in this case I needed to be able to run photorec as a regular user.

The way to accomplish this is to add the regular user to the disk group. To accomplish this, run this command:

sudo usermod -a -G disk <username>

If you are logged in as that user, you will have to log out and log back in to receive the permissions. Once this is done you should be able to mount disks without using sudo or being root.