Category Archives: OS

Install fresh Windows 8.1 on Lenovo G50-S70

Inspired by Lenovo’s bone headed move to install the superfish malware on its machines, I decided to wipe my mother’s Lenovo G50-S70 laptop and start anew. It was supposed to be easy but I ended up running into some issues with this new fangled hardware.

Microsoft has released a very easy tool to create boot ISO images and / or USB media to install Windows 8.1. For Windows 8.1 certified devices like the Lenovo G50 this is extra nice because the key is embedded in the UEFI BIOS – no need to write down or memorize a key.

After creating a USB drive, however, I was greeted with a lovely error message:

Select the driver to install.

It seemed that the install media didn’t see the G50’s hard drive. I could not get past this error message. All drivers on Lenovo’s website are .EXE files which don’t extract well – even when extracted, the installer didn’t like them.

The solution is to boot into a Windows PE environment and run the Windows installer from there. I chose this PE image, which worked quite nicely. Once booted from this PE disk, I was able to mount the install media and run setup.exe manually. This time the installer saw the hard drives and installed Windows 8.1 as you would expect. Success.

Make Notepad++ open files in separate windows

I love Notepad++. When working in Windows it’s my go-to text editor. One thing I don’t like about it, though, is that it seems to only work in one window by default.

It turns out there is a way to change Notepad++ to work more like Notepad – that is, each file you open opens up in a new window instead of a tab in the same window. There doesn’t appear to be a menu option to enable this functionality; however it is still possible to get Notepad++ to behave more like Notepad that way, thanks to this post.

The trick is to create an empty file named asNotepad.xml and to place that file in the directory where Notepad++ is installed (C:\Program Files (x86)\Notepad++ in my case.)

That’s it! once the empty asNotepad.xml file is in the Notepad++ program directory, it acts more like notepad in the sense that each file is opened in a new window. Handy.

Fix Xen VGA Passthrough in Linux Mint 17.1

I wrote in my last post about how I upgraded from Linux Mint 16 to 17.1. I thought everything went smoothly, but it turns out one feature did break: VGA passthrough via Xen. For the past year or so I’ve had a Windows 8.1 gaming VM with direct access to my video card. It’s worked out nicely in Linux Mint 16 but broke completely in 17.1.

I followed the advice of powerhouse on the Linux Mint forums on how to get things up and running, but it wasn’t quite enough. After much banging of my head against the wall I read on the Xen mailing list that there was a regression in VGA passthrough functionality with Xen 4.4.1, which is the version of Xen Mint 17.1 uses.

I finally came to a solution to my problem today – upgrade to Xen 4.5. I couldn’t find any pre-built packages for Ubuntu 14.04 (the base of Mint 17.1) so I ended up compiling Xen 4.5 from source. Below is what I did to make it all work.

Fix broken symlink for /usr/lib/xen-default

sudo rm /usr/lib/xen-default
sudo ln -s /usr/lib/xen-4.4/ /usr/lib/xen-default

Update the DomU CFG file

A couple things needed tweaking. Here is my working cfg:

builder='hvm'
memory = '8192'
name = 'win8.1'
vcpus=6
cpus="2-7"
pae=1
acpi=1
apic=1
vif = [ 'mac=3a:82:47:2a:51:20,bridge=xenbr0,model=virtio' ]
disk = [ 'phy:/dev/mapper/desktop--xen-Win8.1,xvda,w' ]
device_model_version = 'qemu-xen-traditional'
boot='c'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '01:00.0', '01:00.1' , '00:1d.0' ]
localtime=1
pci_power_mgmt=1
on_xend_stop = "shutdown"
xen_platform_pci=1
pci_power_mgmt=1

For some, that’s all they had to do. For me, I had to do a few more things.

Compile Xen 4.5

This step was thanks to two different sites, this one and this one.

Install necessary packages

sudo apt-get install build-essential bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial libjpeg-dev make gcc libc6-dev-i386 zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libpixman-1-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils libyajl-dev gettext markdown libaio-dev pandoc

Checkout Xen source

git clone git://xenbits.xen.org/xen.git xen-4.5.0
cd xen-4.5.0
git checkout RELEASE-4.5.0

Build from source

./configure --libdir=/usr/lib
 make world -j8

When I tried this the make failed with this error:

/usr/include/linux/errno.h:1:23: fatal error: asm/errno.h: No such file or directory
 #include <asm/errno.h>

The fix (thanks to askubuntu)  was to install linux-libc-dev and make a symlink for it:

sudo apt-get install linux-libc-dev
sudo ln -s /usr/include/asm-generic /usr/include/asm

It then compiled successfully.

Install freshly compiled Xen 4.5

sudo make install
sudo update-rc.d xencommons defaults
sudo update-rc.d xendomains defaults
sudo ldconifg

Set grub to boot from new Xen kernel

sudo update-grub
sudo vim /etc/default/grub

Edit GRUB_DEFAULT to match wherever update-grub put your new Xen kernel (in my case it was the second entry, so my GRUB_DEFAULT=1), then run update-grub again

sudo update-grub

Reboot

Success at last. Enjoy your VM gaming once more with Xen 4.5.

Upgrade Linux Mint 16 to 17.1

I realized recently that my desktop system is quite out of date. It has worked so well for so long that I didn’t realize for a while that it was end of support. I was running Linux Mint 16 – Petra.

Thanks to this site the upgrade was fairly painless – a few repository updates, upgrade, and reboot. Simple! The steps I took are below

Update all repositories

Use sed in conjunction with find to quickly and easily update all your repository files from saucy to trusty, and from petra to rebecca, making a backup of files modified.

sudo find /etc/apt/sources.list.d/ -type f -exec sed --in-place=.bak 's/saucy/trusty/' {} \;
sudo find /etc/apt/sources.list.d/ -type f -exec sed --in-place=.bak 's/petra/rebecca/' {} \;

Update your system

This took a while. It had to download 1.5GB of data and install it.

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get upgrade

Cleanup

After running the upgrade I had a notice that I had many packages that were installed but no longer required. To remove unnecessary packages after the upgrade:

sudo apt-get autoremove

Install new language settings:

sudo apt-get install mintlocale

Install gvfs-backends (for Thunar)

sudo apt-get install gvfs-backends

Reboot

Flawless! It worked on the first try. Awesome.

Slow Linux VM performance in VMware vSphere

Recently I’ve been scratching my head over a particular performance issue with Linux VMs hosted on VMWare vSphere. Everything seemed to move at a glacial pace.

vmstat gave a few clues as to what was happening, although depending on what I read it still wasn’t clear:

vmstat

It became apparent that I was suffering from some kind of queuing problem. I wasn’t sure if it was CPU or disk related. I came across this post which has a lot of good performance tuning guides.This tip caught my eye:


 

7. Set your disk scheduling algorithm to ‘noop’

The Linux kernel has different ways to schedule disk I/O, using schedulers like deadline, cfq, and noop. The ‘noop’ — No Op — scheduler does nothing to optimize disk I/O. So why is this a good thing? Because ESX is also doing I/O optimization and queuing! It’s better for a guest OS to just hand over all the I/O requests to the hypervisor to sort out than to try optimizing them itself and potentially defeating the more global optimizations.

You can change the kernel’s disk scheduler at boot time by appending:

elevator=noop

to the kernel parameters in /etc/grub.conf.


Sure enough, I modified /boot/grub/grub.conf on my Centos 6 boxes and appended elevator=noop to the kernel line, then rebooted. It helped a lot! Performance no longer was pitiful. I’m not nearly as familiar with vmware as I am with Xenserver so this was a good hint.

Troubleshoot RSA SecurID in CentOS 6

Unexpected error from ACE/Agent API.

In following this guide for configuring a CentOS 6 system to authenticate with RSA SecurID I came across an unusual error message that had me scratching my head:

Unexpected error from ACE/Agent API.

The problem stemmed from having an incorrect value in the /var/ace/sdopts.rec file for CLIENT_IP. For some reason I had put the IP address of the RSA authentication server in there. CLIENT_IP is the IP address of the RSA client, or rather, the machine you’re working on. The client uses whatever’s in that file to report to the RSA server what its IP address is. If the RSA server gets an invalid IP response from the client, it won’t authenticate.

SELinux issues

Much blood and tears were shed in dealing with getting SELinux to exist harmoniously with RSA SecurID. The problem was exacerbated my the fact that there is a lot of half solutions and misinformation floating out there on the internet. This will hopefully help fix that.

The message entry does not exist for Message ID: 1001

At this point acetest worked beautifully but I could not use an RSA passcode to SSH into the system. Digging into the log revealed this error message:

sshd[2135]: ACEAGENT: The message entry does not exist for Message ID: 1001

Thanks to this post, I realized it was due to selinux. Modifying the selinux config information to allow /var/ace to be read, per the commands below, seemed to fix the issue.

setenforce 0
chcon -Rv --type=sshd_t /var/ace/
setenforce 1

But alas! The solution was not a very good one. The commands above have two problems with them: first, the chcon command is temporary and does not survive selinux policy relabels; second, it assigns the type sshd_t, which does allow SSH to access it, but revokes RSA SecurID’s ability to write to the directory. This is a problem if you ever need to clear node secrets. The server will initiate the wipe but the client will not be able to modify that directory, resulting in node secret mismatches.

I finally decided to RTFM and landed on this documentation page, which explained the issue I was having: selinux mislabeling. The proper solution to this problem is use a label that both SecurID and SSHD can write / read to. Thanks to this SELinux Manpage (it really pays to RTFM!) I discovered that the label I want is var_auth_t (the default label applied when creating /var/ace is var_t, which SSH can’t read.) 

To survive relabeling, use the semanage command, which is not installed by default. Thanks to this site I learned I must install policycoreutils-pithon:

yum install policycoreutils-python

Once semanage is installed, use it to change the label for /var/ace and everything inside it to var_auth_t, then apply the changes with restorecon:

semanage fcontext -a -t var_auth_t "/var/ace(/.*)?"
restorecon -R -v /var/ace

Finally, both RSA SecurID and OpenSSH can read what they need to and authentication is successful.

First acetest succeeds but subsequent ones fail

If you followed the bad advice of relabeling /var/ace to sshd_t you might run across a very frustrating issue where acetest would succeed, but any attempts to SSH into the box or even run acetest again would fail. The error message on the RSA SecurID server was

Node secret mismatch: cleared on server but not on agent

The problem is due to the improper SELinux labeling mentioned above. The fix is the same:

yum install policycoreutils-python
semanage fcontext -a -t var_auth_t "/var/ace(/.*)?"
restorecon -R -v /var/ace

SSH access denied even with successful acetest

If acetest succeeds and you’ve loaded the module into PAM but still get access denied, it could be due to your SSH configuration. Ensure the following options are set:

ChallengeResponseAuthentication yes 
UsePrivilegeSeparation no

Victory.

Join a CentOS machine to an AD domain

I ran into enough snags when attempting to join an CentOS 6.6 machine to a Microsoft domain that I thought I would document them here. Hopefully it is of use to someone. The majority of the experience is thanks to this site.

Update 03/16/2015: I came across this site which makes things a little easier when it comes to initial configuration – messing with other config files is no longer necessary. The authconfig command to do this is below:

authconfig --disablecache --enablelocauthorize --enablewinbind --enablewinbindusedefaultdomain --enablewinbindauth        --smbsecurity=ads --enablekrb5 --enablekrb5kdcdns --enablekrb5realmdns --enablemkhomedir --enablepamaccess --updateall        --smbidmapuid=100000-1000000 --smbidmapgid=100000-1000000 --disablewinbindoffline --winbindjoin=Admin_account --winbindtemplateshell=/bin/bash --smbworkgroup=DOMAIN --smbrealm=FQDN --krb5realm=FQDN

Replace DOMAIN with short domain name, FQDN with your fully qualified domain name, and Admin_account with an account with domain admin privileges, then skip to the Reboot section, as it covers everything before that.

Install the necessary packages

yum -y install authconfig krb5-workstation pam_krb5 samba-common oddjob-mkhomedir

Configure kerberos auth with authconfig

There is a curses-based GUI you can use to do this in but I opted for the command line.

authconfig --disablecache --enablewinbind --enablewinbindauth --smbsecurity=ads --smbworkgroup=DOMAIN --smbrealm=DOMAIN.COM.AU --enablewinbindusedefaultdomain --winbindtemplatehomedir=/home/DOMAIN/%U --winbindtemplateshell=/bin/bash --enablekrb5 --krb5realm=DOMAIN.COM.AU --enablekrb5kdcdns --enablekrb5realmdns --enablelocauthorize --enablemkhomedir --enablepamaccess --updateall

Add your domain to kerberos configuration

Kerberos information is stored in /etc/krb5.conf. Append your domain in the realms configuration, like below

vi /etc/krb5.conf
[realms]
 EXAMPLE.COM = {
 kdc = kerberos.example.com
 admin_server = kerberos.example.com
 }
 
DOMAIN.COM.AU = {
admin_server = DOMAIN.COM.AU
kdc = DC1.DOMAIN.COM.AU
kdc = DC2.DOMAIN.COM.AU
}
 
[domain_realm]
 .example.com = EXAMPLE.COM
 example.com = EXAMPLE.COM
 domain.com.au = DOMAIN.COM.AU
 .domain.com.au = DOMAIN.COM.AU

 Test your configuration

Use the kinit command with a valid AD user to ensure a good connection with the domain controllers:

kinit <AD user account>
It should return you to the prompt with no error messages. You can further make sure it worked by issuing the klist command to show open Kerberos tickets
klist

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: someaduser@DOMAIN.COM.AU
Valid starting Expires Service principal
02/27/14 12:23:21 02/27/14 22:23:21 krbtgt/DOMAIN.COM.AU@DOMAIN.COM.AU
renew until 03/06/14 12:23:19
When I tried the kinit command it returned an error:
kinit: KDC reply did not match expectations while getting initial credentials
 After scratching my head for a while I came across this site, which explains that your krb5.conf is case sensitive – it must all be all upper case. Fixing my krb5.conf to be all caps for my domain resolved that issue.

Join the domain

net ads join domain.com.au -U someadadmin
When I tried to join the domain I received this lovely message:
Our netbios name can be at most 15 chars long, "EXAMPLEMACHINE01" is 16 chars long
Invalid configuration. Exiting....
Failed to join domain: The format of the specified computer name is invalid.
Thanks to Ubuntu forms I learned I needed to edit my samba configuration to assign an abbreviated NETBIOS name to my machine.
vi /etc/samba/smb.conf
Uncomment the “netbios name =” line and fill it in with a shorter (max 15 characters) NETBIOS name.
netbios name = EXAMPLE01
You can test to ensure the join was successful with this command
net ads testjoin

Configure home directories

The authconfig command above included a switch for home directories. Make sure you create a matching directory and set appropriate permissions for it.

mkdir /home/DOMAIN
setfacl -m group:"Domain Users":rwx /home/DOMAIN #the article calls to do this, this command doesn't work for me but home directories still appear to be created properly

Reboot

To really test everything the best way is to reboot the machine. When it comes back up, log in with Active Directory credentials. It should work!

Account lockout issues

I ran into a very frustrating problem where everything works dandy if you get the password correct on the first try, but if you mess up even once it results in your Active Directory account being locked. You were locked out after the first try. Each login, even when successful, had this in the logs:

winbind pam_unix(sshd:auth): authentication failure

This problem took a few days to solve. Ultimately it involved modifying two files:

vi /etc/pam.d/system-auth
vi /etc/pam.d/password-auth

As far as I can tell, the problem was a combination of pam_unix being first (which always failed when using AD login), as well as having both winbind and kerberos enabled. The fix was to change the order of each mention of pam_unix to be below any mention of pam_winbind. The other fix I had to do was to comment out mentions of pam_krb5 completely.

#auth        sufficient    pam_krb5.so use_first_pass

Restrict logins

The current configuration allows any domain account to log into the machine. You will probably want to restrict who can log in to the machine to certain security groups. The problem: many Active Directory security groups contain spaces in their name, which Linux doesn’t like.

How do you add a security group that contains a space? Escape characters don’t seem to work in the pam config files.  I found out thanks to this site that it is easier to just not use spaces at all. Get the SID of the group instead.

Use wbcinfo -n to query the group in question, using the backslash to escape the space. It will return the SID we desire.

wbinfo -n Domain\ Users
S-1-5-21-464601995-1902203606-794563710-513 Domain Group (2)

Next, modify /etc/pam.d/password-auth and add the require_membership_of argument to pam_winbind.so:

auth        sufficient    pam_winbind.so require_membership_of=S-1-5-21-464601995-1902203606-794563710-513

That’s it! Logins are now restricted to the security group listed.

Configure sudo access

Sudo uses a different list for authorization, which amusingly, handles escaped spaces just fine.  Simply add the active directory group in sudo as you a local one, eg using a % and then group name, escaping spaces with a backslash:

%Domain\ Users ALL=(ALL) ALL

Rejoice

You’ve just gone through a long and painful battle. Hopefully this article helped you to achieve victory.

Configure iSCSI initiator in CentOS

Below are my notes for configuring a CentOS box to connect to an iSCSI target. This assumes you have already configured an iSCSI target on another machine / NAS. Much of this information comes thanks to this very helpful website.

Install the software package

1
yum -y install iscsi-initiator-utils

Configure the iqn name for the initiator

1
vi /etc/iscsi/initiatorname.iscsi
1
2
InitiatorName=iqn.2012-10.net.cpd:san.initiator01
InitiatorAlias=initiator01

Edit the iSCSI initiator configuration

1
vi /etc/iscsi/iscsid.conf
1
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = initiator_user
node.session.auth.password = initiator_pass
#The next two lines are for mutual CHAP authentication
node.session.auth.username_in = target_user
node.session.auth.password_in = target_password

Start iSCSI initiator daemon

1
2
/etc/init.d/iscsid start
chkconfig --levels 235 iscsid on

Discover targets in the iSCSI server:

1
2
iscsiadm --mode discovery -t sendtargets --portal 172.16.201.200 the portal's IP address
172.16.201.200:3260,1 iqn.2012-10.net.cpd:san.target01

Try to log in with the iSCSI LUN:

1
2
3
iscsiadm --mode node --targetname iqn.2012-10.net.cpd:san.target01 --portal 172.16.201.200 --login
Logging in to [iface: default, target: iqn.2012-10.net.cpd:san.target01, portal: 172.16.201.200,3260] (multiple)
Login to [iface: default, target: iqn.2012-10.net.cpd:san.target01, portal: 172.16.201.200,3260] successful.

Verify configuration

This command shows what is put into the  iSCSI targets database  (the files located in /var/lib/iscsi/)

1
cat /var/lib/iscsi/send_targets/172.16.201.200,3260/st_config
1
2
3
4
5
6
7
8
9
10
11
12
discovery.startup = manual
discovery.type = sendtargets
discovery.sendtargets.address = 172.16.201.200
discovery.sendtargets.port = 3260
discovery.sendtargets.auth.authmethod = None
discovery.sendtargets.timeo.login_timeout = 15
discovery.sendtargets.use_discoveryd = No
discovery.sendtargets.discoveryd_poll_inval = 30
discovery.sendtargets.reopen_max = 5
discovery.sendtargets.timeo.auth_timeout = 45
discovery.sendtargets.timeo.active_timeout = 30
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768

Verify session is established

1
2
iscsiadm --mode session --op show
tcp: [2] 172.16.201.200:3260,1 iqn.2012-10.net.cpd:san.target01

Create LVM volume and mount

Add our iSCSI disk to a new LVM physical volume, volume group, and logical volume

1
2
3
4
5
6
7
fdisk -l
Disk /dev/sdb: 17.2 GB, 17171480576 bytes
64 heads, 32 sectors/track, 16376 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
1
Disk /dev/sdb doesn't contain a valid partition table
1
2
pvcreate /dev/sdb
vgcreate iSCSI /dev/sdb
lvcreate iSCSI -n volume_name -l100%FREE
mkfs.ext4 /dev/iSCSI/volume_name

Add the logical volume to fstab

Make sure to use the mount option _netdev.  Without this option, Linux will try to mount this device before it loads network support.

1
vi /etc/fstab
/dev/mapper/iSCSI-volume_name    /mnt   ext4   _netdev  0 0

Success.

Convert xenserver 6.5 to software RAID 1

I have written previously about how to convert Citrix Xenserver 6.2 to a software RAID 1. When I upgraded to Xenserver 6.5 I found I had to re-install the xenserver instance because the upgrade didn’t recognize the software RAID. When trying to follow my own guide I found that I couldn’t create the array – it gave the following error message:

mdadm: unexpected failure opening /dev/md0

It turns out 6.5 handles RAID differently. You have to manually load the RAID kernel modules before you can create arrays. I was able to get this running successfully thanks to guidance from this site, specifically comments on it by Olli.

The majority of this can simply be copy/pasted into the command window, once drive paths have been updated for your specific setup.

# Prepare /dev/sdd
sgdisk --zap-all /dev/sdd
sgdisk --mbrtogpt --clear /dev/sdd
sgdisk -R/dev/sdd /dev/sdc # Replicate partion table from /dev/sdc to /dev/sdd with unique identifier
sleep 5 # Sleep 5 seconds here if you script this…
sgdisk --typecode=1:fd00 /dev/sdd
sgdisk --typecode=2:fd00 /dev/sdd
sgdisk --typecode=3:fd00 /dev/sdd
sleep 5 # Sleep 5 seconds here if you script this…
modprobe md_mod # load raid, because it isn't load by default (XS6.5 only)
yes|mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdd1 missing # Create md0 (root)
yes|mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdd2 missing # Create md0 (swap)
yes|mdadm --create /dev/md2 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdd3 missing # Create md0 (storage)
sleep 5 # Sleep 5 seconds here if you script this…
mkfs.ext3 /dev/md0 # Create root FS
mount /dev/md0 /mnt # Mount root FS
cp -xR --preserve=all / /mnt # Replicate root files
mdadm --detail --scan > /mnt/etc/mdadm.conf #generate RAID configuration
sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab # Update fstab for new RAID device
mount --bind /dev /mnt/dev
mount -t sysfs none /mnt/sys
mount -t proc none /mnt/proc
chroot /mnt /sbin/extlinux --install /boot
dd if=/mnt/usr/share/syslinux/gptmbr.bin of=/dev/sdd
chroot /mnt
mkinitrd -v -f --theme=/usr/share/splash --without-multipath /boot/initrd-`uname -r`.img `uname -r`
exit
sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/boot/extlinux.conf # Update extlinux for new RAID device
cd /mnt && extlinux --raid -i boot/
sgdisk /dev/sdd --attributes=1:set:2

#Unmount filesystems and reboot
cd
umount /mnt/dev
umount /mnt/sys
umount /mnt/proc
umount /mnt
sync
reboot

Tell BIOS to use disk B
After reboot to disk B…

sgdisk -R/dev/sdc /dev/sdd # Replicate partition table from /dev/sdd to /dev/sdc with unique identifier
sgdisk /dev/sdc --attributes=1:set:2
sleep 5 # Sleep 5 seconds here if you script this…
mdadm -a /dev/md0 /dev/sdc1
mdadm -a /dev/md1 /dev/sdc2
mdadm -a /dev/md2 /dev/sdc3 # If this command gives error, you need to forget/destroy an active SR first
#This next command is the only command you have to manually update before pasting in. Find the UUID of your xenserver host and paste it between the <> below
xe sr-create content-type=user device-config:device=/dev/md2 host-uuid=<UUID of xenserver host> name-label="RAID 1" shared=false type=lvm
# Watch rebuild progress and wait until no arrays are rebuilding before proceeding with any reboot
watch “mdadm --detail /dev/md* | grep rebuild”

Done!

Fix Apache Permission Denied errors

The other day I ran the rsync command to migrate files from an old webserver to a new one. What I didn’t notice right away was that the rsync changed the permissions of the folder I was copying into.

The problem presented itself with a very lovely 403 forbidden error message when trying to access any website that server hosted. Checking the logs (/var/log/apache2/error.log on my Debian system) revealed this curious message:

[error] [client 192.168.22.22] (13)Permission denied: access to / denied

This made it look like apache was denying access for some reason. I verified apache config and confirmed it shouldn’t be denying anything. After some head scratching I came across this site which explained that Apache throws that error when it encounters filesystem access denied error messages.

I was confused because /var/www, where the websites live, had the appropriate permissions. After some digging I found that the culprit in my case was not /var/www, but rather the /var directory underneath /var/www. For some reason the rsync changed /var to not have any execute permissions (necessary for folder access.)  A simple

chmod o+rx /var/

resolved my problem. Next time you get 403 it could be underlying filesystem issues and not apache at all.