Category Archives: CLI

CLI, Virtualization

Transfer VMs between Xenserver pools

8 Comments

When Xenserver 7 came out I found myself unable to easily upgrade to it thanks to my custom RAID 1 build.  If I wanted Xenserver 7 I would have to blow the whole instance away and start from scratch. This posed a problem because I have a pool of 2 xenserver hosts. You cannot add a server with a higher xenserver version to a lower versioned pool; the pool master must always have the highest version of Xenserver installed. My decision to have an mdadm RAID 1 setup on my pool master ultimately turned into forced VM downtime for an upgrade despite having a pool of other xenserver hosts.

After transferring VMs to my secondary host and promoting it to pool master, I wiped my primary xenserver and installed 7. When it was up and running I essentially had two separate pools running. To transfer my VMs back to my primary server I had to resort to the command line.

Offline VM transfer

The xe vm-export and vm-import commands work with stdin/out and piping. This is how I accomplished transferring my VMs directly between two pools. Simply pipe xe vm-export commands with an ssh xe vm-import command like so:

xe vm-export uuid=<VM_UUID> filename= | ssh <other_server> xe vm-import filename=/dev/stdin

Note the lack of a filename – this instructs xenserver to pipe to standard output instead. Also note that transferring the VM scrambles the MAC addresses of its interfaces. If you want to keep the MAC address you’ll have to manually re-assign it after the copy is complete.

Minimal downtime

For the method above you will have to turn the VM off in order to transfer it.  I had some VMs that I didn’t want to stay down for the entire transfer. A way around this is to take a snapshot of the VM and then copy the snapshot to the other pool. Note that this method does not retain any changes made inside the VM that occurred after you took the snapshot. You will have to manually transfer any file changes that took place during the VM transfer (or be fine with losing them.)

In order to export a snapshot you must first convert it to a VM from a template (thanks to this site for outlining how.) The full procedure is as follows:

  1. Take a snapshot of the VM you want to move 
    xe vm-snapshot uuid=<VM_UUID> new-name-label=<snapshotname>
  2. Convert the snashot template to a VM (the command xe snapshot-list is a handy way to obtain UUIDs of your snapshots) 
    xe template-param-set is-a-template=false ha-always-run=false uuid=<UUID of snapshot>
  3. Transfer the template to the new pool 
    xe vm-export uuid=<UUID of snapshot> filename= | ssh <other_server> xe vm-import filename=/dev/stdin
  4. Rename VM and/or modify interface MAC addresses as needed on the new host. Stop the VM on the old host and start it on the new one.

I used both methods above to successfully move my VMs from my older 6.5 pool to the newer 7 pool. Success.

CLI, Virtualization

PCI Passthrough in Xenserver 7 “Dundee”

20 Comments

I’ve recently upgraded to the latest version of Citrix Xenserver 7 (codenamed “Dundee”.) 7 is based on CentOS 7 and has a massive amount of changes under the hood. One such change was how they handle PCI Passthrough.

It took some time to figure PCI Passthrough out. 7 uses grub instead of extlinux for the bootloader. It appears to be grub2 but they don’t use the standard update-grub tool, rather you simply edit the config file and do nothing else.

After much searching I found this post which led me in the right direction. In Xenserver 7, for pci passthrough support you must do the following:

  • Prepare the VM for PCI passthrough (this part hasn’t changed) 
    xe vm-param-set other-config:pci=0/0000:B:D.f uuid=<vm uuid>
  • Modify /boot/grub/grub.cfg and append the following to the end of the module2 line (if you boot from EFI the file to modify is /boot/efi/EFI/xenserver/grub.cfg) 
     xen-pciback.hide=(B:D.f)
  • Reboot

You will now be able to pass through hardware to your virtual machines in Xenserver 7. Hooray.

CLI

Use Irfanview in Linux with Wine

Leave a comment

IrfanView is a very versatile little program that you can use to manipulate and view image files. I am very familiar with it and often find myself wishing I could use it in Linux. Well, it turns out I can!

There are a few things you need to do. First, install wine

#For debian-based distros
sudo apt-get install wine

After wine is installed, use it to install irfanview

wine iview442_setup.exe

If you get the following error, it means you don’t have mfc42.dll installed:

err:module:import_dll Library MFC42.DLL (which is needed by L"Z:\\home\\nicholas\\Downloads\\iview442_setup.exe") not foun

The solution is to use winetricks to install mcf42.dll (thanks to winehq for the soultion to this)

winetricks -q mfc42

Optionally you can make wine your default image program (or just have it in the list of available image handling programs.) The way to do this is to write a quick script and mark it executable (thanks to linuxquestions for the answer on how to do this)

#!/bin/sh
IRFANVIEW="C:\\Program Files\Irfanview (x86)\i_view32.exe"
ROOT_DRIVE="Z:\\"
for arg
do
	wine "$IRFANVIEW" "${ROOT_DRIVE}$(echo "$arg" | sed 's/\//\\/g')"
done

You may need to tweak it a little bit if you don’t have 64bit wine installed.  Save that file somewhere you will remember and then mark it executable with chmod +x.

Lastly, when you right click a file and do open with, simply point it to the above script. The result: wine goodness.

CLI

Exclude directories in find command

Leave a comment

For some reason I had a hell of a time finding a way of excluding a directory from the ‘find’ command. I finally found a solution that works thanks to this website.

The solution for me was to add the following to my find command:

! -path "<directory to exclude>"

So to find all files modified less than 24 hours ago on my grandmother-in-law’s computer, I used the following command:

find . -mtime -1 ! -path "./AppData/*"

Grandma’s computer runs Windows 7 but I’ve configured Cygwin and ssh so I can use the tools I’m fimilar with. The above command excludes appdata and searches her user profile for all files modified today.

CLI, OS

Persistent SSH tunnel for Windows

6 Comments

Over the years I’ve needed to access family members’ machines for remote support. The problem with parents and grandparents is walking them through certain prompts for services like join.me is quite problematic. To that end I’ve devised an open source way for me to automatically remote into their machine regardless of firewalls or machine location. This is possible thanks to cygwin, autoSSH, and NSSM. As long as the machine has internet access, I can get to it.

To pull this off you’ll need to install a few cygwin packages, copy over a private key file, create a batch script, and invoke NSSM to create a service to invoke the batch script on startup.

Cygwin

Obtain cygwin from here. You’ll need to use the graphical installer for the initial setup. Install the following packages:

  • ssh
  • autossh
  • wget (not necessary, but handy to have)

If cygwin is already installed, install it again. I wasted an hour once trying to figure out why it wasn’t working when the culprit turned out to be a buggy old version of cygwin itself.

Private key

For this to work you’ll need an SSH server configured for key authentication (no password.) On your SSH server:

  • Create new user for the Windows machine
  • Execute ssh-keygen as that user
  • Copy the contents of the .pub file into ~/.ssh/authorized_keys
  • Copy the private key (the one with no extension) to the Windows computer
  • Make sure permissions for the .ssh folder and everything inside of it is 600

GatewayPorts

One option that I really enjoy on my SSH server is the GatewayPorts option. This turns your SSH server into a gateway for any port forwards. Simply edit /etc/ssh/sshd_config and add

GatewayPorts yes

Save the file and restart the SSH service. Now if you create SSH tunnels your SSH server opens those ports for you to connect from other machines.

Create batch file

On the windows machine a simple command gets us up and running. Create a one-liner .cmd file on the Windows machine in a location of your choosing with the following:

c:\cygwin\bin\autossh.exe -M <random_port_number> -i <keyfile location>  -l <user> -R<remote_port:localhost:<local_port> -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null <remote address>

Update it to reflect the path of your cygwin installation if you installed somewhere other than the default location.

I add the reverse port forward option ( -R ) so that I can simply connect to my ssh server on the specified port and the connection will tunnel through to the Windows computer. In my case, I do -R5700:localhost:5900 which instructs my ssh server to listen on port 5700, then forward that connection to the Windows machine on port 5900 for VNC.

Create service

The Non-suciking service manager is a nifty little program that lets us turn anything into a windows service. Once it’s a service it can be started automatically on startup, even if nobody has logged in yet.

Obtain NSSM from here and extract it to a location you can remember. Then, open an administrator command prompt, cd to the directory containing nssm.exe, and enter the following:

nssm.exe install autossh

A GUI will open up. Specify the location of your batch file in the Path: section, then click Install service.

Once this is done, start the service by running services.msc, looking for your service, right click and select start. Make sure the startup type is set to automatic.

That’s it! If your keys are in the right place and the permissions are correct, the computer will automatically (and silently) log into your SSH server and create a tunnel for you. Autossh will continually try to re-connect in the event of connection loss. Awesome.

Reverse SSH

You can also configure cygwin to be an SSH server for your windows host. This will allow you to SSH into the machine if you specify -R<random_port:localhost:22 in your batch file. Here are a few notes for getting ssh working

  • Open up a cygwin terminal and execute the command: 
    ssh-host-config
  • Once the SSH server is configured, tweak the SSH configuration to allow logging in with blank passwords (many of my family do not use a password to log into the machine.) Simply un-comment the line “PermitEmptyPasswords no” and change no to yes. Then, restart the ssh service. (thanks to this blog for the insight)
CLI

Crop pictures with ImageMagick’s mogrify

Leave a comment

I recently needed a quick and dirty way to crop the bottom chunk of a large batch of scanned photos. Thanks to Linux and FOSS, this is possible with a fantastic tool known as imagemagick.

Simply install imagemagick to get the necessary tools

#Assuming you have a redhat based distro
sudo yum install ImageMagick*

Once installed use the mogrify tool (part of ImageMagick) to quickly chop the bottom part off:

mogrify -chop 0x45+0+0 -gravity South *.jpg

The above example chops the bottom 45 pixels off of every picture in the directory you’re in. Thanks to this site for the info. Handy.

CLI, OS

Resizing LVM storage checklist

Leave a comment

This is a short note of what to do when you change size of the physical disk an LVM setup, such as the default configuration in CentOS 7.

  1. Modify the physical disk size
  2. Modify the partition size
    1. I used fdisk to delete the partition, then re-create with a larger size
    2. Reboot
  3. Extend the physical volume size 
    1. pvresize <path to enlarged partition>
  4. Extend the logical volume size 
    1. lvextend <lv path> -l100%FREE
  5. Extend filesystem size 
    1. resize2fs <lv path>
    2. #If you're running CentOS 7, the default filesystem is actually XFS, not ext4. In that case:
      xfs_growfs <lv path>
  6. Profit.

UPDATE 2024-12-17

I looked this trusty old article up to rezize a VM disk but kept running into the error that there was not enough free space even after doing a pvresize. In this case, I had to use -L+<increased size> instead of -l100%FREE. Not sure why 100%FREE works sometimes and doesn’t others.

CLI

Compile ffmpeg on CentOS 7

2 Comments

Recently I had to compile ffmpeg from scratch on CentOS 7. The reason? I wanted libfdk_aac support. Here are my notes on the procedure. The how-to on https://trac.ffmpeg.org/wiki/CompilationGuide/Centos was actually quite helpful and accurate.

Install necessary dependencies and set up build folder

yum install autoconf automake cmake freetype-devel gcc gcc-c++ git libtool make mercurial nasm pkgconfig zlib-devel
mkdir ~/ffmpeg_sources

Build necessary components
I only needed x264 and libfdk_aac, so that’s all I ended up doing:

#yasm
cd ~/ffmpeg_sources
git clone --depth 1 git://github.com/yasm/yasm.git
cd yasm
autoreconf -fiv
./configure --prefix="$HOME/ffmpeg_build" --bindir="$HOME/bin"
make
make install
make distclean
#libx264
cd ~/ffmpeg_sources
git clone --depth 1 git://git.videolan.org/x264
cd x264
PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure --prefix="$HOME/ffmpeg_build" --bindir="$HOME/bin" --enable-static
make
make install
make distclean
#libfdk_aac
cd ~/ffmpeg_sources
git clone --depth 1 git://git.code.sf.net/p/opencore-amr/fdk-aac
cd fdk-aac
autoreconf -fiv
./configure --prefix="$HOME/ffmpeg_build" --disable-shared
make
make install
make distclean

Compile ffmpeg
I actually specified a git mirror because the sources at ffmpeg site were glacially slow.

cd ~/ffmpeg_sources
git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg
PKG_CONFIG_PATH="$HOME/FFmpeg_build/lib/pkgconfig" ./configure --prefix="$HOME/FFmpeg_build" --extra-cflags="-I$HOME/FFmpeg_build/include" --extra-ldflags="-L$HOME/FFmpeg_build/lib" --bindir="$HOME/bin" --pkg-config-flags="--static" --enable-gpl --enable-nonfree --enable-libfdk-aac  --enable-libx264
make
make install
make distclean
hash -r

Optionally, remove existing ffmpeg

sudo yum remove ffmpeg

That was it! After a bit of compile time ffmpeg worked with the items I wanted it to.

CLI, OS

Configure HDHR Viewer XMLTV in CentOS Linux

Leave a comment

Recently I’ve accomplished the herculean task of setting up my parents’ cable connection to stream through Plex using a HD Homerun 3 cablecard network tuner. It works! This is how I got XMLTV guide working for the HDHR Viewer plugin for Plex on CentOS 7 Linux.

Required reading: http://hdhrviewer.zynine.net/hdhrviewerv2-initial-setup/xmltv-zap2xml/

First, install and configure the required perl and java packages

sudo yum install perl-Compress-Zlib perl-HTML-Parser perl-HTTP-Cookies perl-LWP-Protocol-https perl-JSON gcc cpan java-1.7.0-openjdk-headless 
sudo cpan JSON::XS 
#accept all defaults when prompted

Download the zap2xml perl module (zap2xml.pl) Place it somewhere it can be easily accessed.

Test to make sure the script will run properly:

perl zap2xml.pl -u <zap2it username> -p <zap2it password>

If you get an error like this:

Can't locate Compress/Zlib.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at zap2xml.pl line 26.

It means you haven’t installed the correct perl modules. Double check that you installed them all.

Once we know it runs properly e need to configure a cron job to run zap2xml daily (to make sure the guide data is always up to date.)

crontab -e
#press i to begin inserting
0 0 * * * perl <full path to where you downloaded zap2xml>/zap2xml.pl -u <zap2it e-mail> -p <zap2it password>
#ESC :wq to save and exit

 

Next download and unzip the Channel Guide app. I placed it in the same place I downloaded zap2xml to keep things simple.

Test it out to make sure it works:

java -jar channel-guide-app-0.0.3.jar server app-config.yml

If it starts and doesn’t crash, you know it’s working.

Now we want to configure the channel guide app to run on startup

sudo vi /etc/systemd/system/channelguide.service
[Unit]
Description=Plex Channel Guide

[Service]
TimeoutStartSec=0
ExecStart=/usr/bin/java -jar <full path to channel-guide dir>/channel-guide-app-0.0.3.jar server <full path to channel-guide dir>/app-config.yml

[Install]
WantedBy=multi-user.target

Make sure this systemd service is enabled:

sudo systemctl enable /etc/systemd/system/channelguide.service

Lastly make sure you’ve configured the HDHRViewer plugin in Plex to use xmltv and rest API as per the how-to on their site.

Success!