Web

Block bad networks from sites behind Sophos WAF

Leave a comment

Recently I have noticed some odd traffic coming to one of my blogs. This particular blog is set to NOT be indexed by search engines b(robots.txt deny.) Every bot that’s touched that site has honored that file… until now.

Periodically I will get huge spikes of traffic (huge for my small site, anyway.) The culprit is always the same: Apple! Why are they crawling my site? I can’t find a definitive reason. A couple searches reveals articles like this one speculating that Apple is starting a search engine. The problem is the traffic I’m seeing from Apple shows just a safari user agent, nothing about being a bot. A discussion on Reddit talks about Apple crawling sites, but they also list a user agent I’m not seeing.

The user agent reported by the bot that’s been crawling me (ignoring robots.txt file) is:

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36

The IPs rotate randomly from Apple’s IP space, with the biggest offender being 17.142.152.102.

x_forwarded_for count
17.142.152.102 1680
17.142.151.205 982
17.142.151.80 444
17.142.152.14 174
17.142.151.134 36
17.142.152.78 28
17.142.151.182 26
17.142.151.239 26
17.142.150.250 24
17.142.152.101 24
17.142.152.151 24
17.142.151.198 22
17.142.149.55 21
17.142.147.58 7
17.142.148.75 7
17.142.151.49 6
17.142.148.12 4
17.142.151.197 4
17.149.228.59 4
17.142.152.118 3
17.142.149.167 2
17.142.151.179 2
17.142.151.79 2
17.142.151.92 2
17.142.144.105 1

 

I e-mailed Apple at abuse@apple.com requesting they stop this action. I didn’t expect anything from it, and indeed nothing happened. I kept getting crawled.

So, now to the title of this post. I had to tell my Web Application Firewall to block Apple’s address space. Sophos UTM 9.3 makes this easier, although the option is somewhat hidden for some reason. The option is in the “Site Path Routing” tab within the Web Application Firewall context. Once there, edit your site path and check the “Access Control” checkbox.

Capture

In my case I decided to block the entire subnet – 17.0.0.0/8. No more Apple crawling.. at least from the 17 network.

Hardware, OS

Add folders to libraries in Windows 8.1

Leave a comment

I recently purchased a shiny new Microsoft Surface Pro 3. I must say so far I am quite impressed with it. I love the form factor. It’s a laptop or tablet depending on what I want to do with it.

When I’m in tablet mode using “Metro” apps I noticed that many of them require the use of Libraries. It took me longer than I care to admit to figure out how to add folders to libraries so I’m including that here.

In Windows 7 it was pretty easy – right click on the library and do properties, go to folders and add. The default explorer view in Windows 8.1 does not have a Libraries option.. so how do you do it?

Thanks to this guide I discovered it’s a simple matter of telling Explorer to show Libraries again. Open Explorer, go to the View tab, then click on Navigation Pane (top left), then select Libraries.

libraries

Note: There is no Print Screen key on the surface, press Fn + Space instead.

Once that’s done you can the the Library in the Navigation pane just like you can in Windows 7, and you can add folders to those libraries to your heart’s content.

Web

Two factor authentication in WordPress with Authy

Leave a comment

With data breaches as rampant as they are I’ve decided to get more serious about security and implement two factor authentication. Authy is a great way to add this to WordPress, and it’s free (or at least most of its features are.) This information comes from their blog.

  1. Install the Authy plugin from here
  2. Create an account at https://dashboard.authy.com
  3. Add an application for your blog to the Authy dashboard and copy the API key given to you
  4. Activate the Authy wordpress plugin, go into settings and paste in the API key
  5. Activate two factor authentication for your user by mousing over the top right corner and selcting “Edit my profile”, scroll down to the bottom, and click “Enable/Disable Authy”

When I did this I had forgotten that I had a different login plugin running – Login Lockdown. With both these enabled I could no longer log in! There was some sort of conflict between the two plugins. I had to disable both plugins by following this guide.

  1. Navigate to your wordpress directory and go to wp-content/plugins
  2. Rename the offending plugin directory to something like pluginname-disabled
  3. Log into WordPress and go to your plugins page, it will generate an error
  4. Now that you’re logged in, you can rename those folders back to their original name to either re-activate or delete those plugins.

Now you are much more secure. Even if someone has your password they will not be able to log in unless they also have your phone.

OS, Web

Mythweb broken after upgrading to Ubuntu 14.04

4 Comments

I recently upgraded my mythbuntu installation from 12.04 to 14.04. For some reason the distribution upgrade tool failed on me. I had to upgrade manually by updating everything in /etc/apt/sources* to point to trusty instead of precise.

After a reboot I was surprised to find out that everything upgraded beautifully except for one thing – mythweb. When I tried to start Apache I was greeted with this lovely message:

* The apache2 configtest failed.
Output of config test was:
AH00526: Syntax error on line 30 of /etc/apache2/sites-enabled/mythweb.conf:
Illegal option AllowAll
Action 'configtest' failed.
The Apache error log may have more information.

It turns out Ubuntu 14.04 uses a different version of Apache with different syntax, which breaks the configuration. Thanks to this post I found the fix to be relatively easy:

sudo rm /etc/apache2/sites-available/mythweb.conf
sudo dpkg-reconfigure mythweb
sudo /etc/init.d/apache2 start

After that was done, all was well and upgraded.

Mobile

Restore Verizon Galaxy S4 to factory firmware

2 Comments

My new job has a BYOD policy, which means I can use my personal phone for work use. Before I ascribed to their policy I wanted to make sure my phone was in pristine condition. I wrote earlier about how I rooted and flashed a custom ROM for my phone. This will explain how to un-do all of that and restore it to pristine factory condition.

I got my information from here, which is a great guide on how to do this.  I will mirror the files mentioned there as it’s well known that XDA mirrors come and go.

Here is the rundown:

  1. Download Samsung USB drivers from here and install them.
  2. Download ODIN 3.09 here (alternate)
  3. Download PIT file for the S4 here (alternate)
  4. Download full wipe stock firmware here (make sure you’ve backed up anything important) (alternate)
  5. Unzip everything into a common directory
  6. Run ODIN and flash back to factory
    1. Put your phone in ODIN mode by turning the phone off, then pressing and holding power and volume down at the same time. A message will appear, press volume up to confirm and enter ODIN mode.
    2. Plug your phone into USB and run ODIN on your PC. Make sure the odin log says Added!! If it doesn’t, drivers are not installed properly.
    3. Check AP and then press the AP button to specify the firmware file (wait a minute for md5 verification)
    4. Press the PIT button and select the PIT file downloaded
    5. Press start. It will take about ten minutes.
  7. Profit!

pass

When I first tried to do this I soft bricked my phone. I was very confused because everything seemed to be going fine and then suddenly FAIL

...
<ID:0/003> sbl1.mbn
<ID:0/003> sbl2.mbn
<ID:0/003> sbl3.mbn
Complete(Write) operation failed.
<OSM> All threads completed. (succeeded 0 / failed 1)
<ID:0/003> Removed!!
...

I scratched my head for quite some time before I came across came across this post suggesting it’s a bad USB cable / port. Sure enough, I switched out the cable I was using with the official cable the phone came with, and it worked beautifully!

Hardware

Manually reproduce flux on your monitor

23 Comments

I recently got a new job which uses a VDI infrastructure. We don’t have individual workstations, but rather terminal into a central server which serves us individual desktops. One unfortunate side effect of this configuration is that f.lux (which I’ve written about before) doesn’t appear to do anything. Research suggests that f.lux must talk directly to display hardware to work – no remote desktops.

A co-worker suggested fiddling with the monitor’s color settings to try and reproduce what f.lux does. I hadn’t thought of that before!

It turns out my monitors have  pre-built color temperatures, but the lowest they go is 5400k. My color temperature comfort level is more like 3400k, which as it turns out what most office lighting is.

The monitors allow me to manually select RGB percentages. The trick was translating 3400k (f.lux setting) to percentages of red, green, and blue. Searching Google for the RGB values of 3400k revealed this page, which had some helpful information. 3400k translates to the hex values #ffc184.

The last step was translating that hex to percentages. Googling “ffc183 in percentage rgb” revealed this link, which is what I wanted!

In short: 3400k in flux roughly equates to:

100% red
76% green
51% blue

Success! My eyes are much more comfortable now.

OS

Troubleshooting high CPU usage in Windows 7

Leave a comment

My mother’s laptop has been behaving very strangely lately. CPU usage suddenly would spike to 100% usage inexplicably. It seemed to happen more often when Firefox / Thunderbird were launched, but that wasn’t necessarily the case. The issue would often persist across reboots.

My first thought, of course, was malware. An extensive scan via rootkit scanner, malware scanner, spyware scanner, and anti-virus revealed nothing. Observing running processes with process explorer and startup programs with autoruns revealed nothing suspicious. I installed all Windows updates, updated all typical applications with the help of ninite, and even ran sfc /scannow for good measure. The issue remained!

Even more confusing is that when running task manager, process explorer, or performance monitor they all reported 100% CPU usage and with a single process hogging the CPU – themselves! It seemed that whatever was the last process to execute was taking all CPU. It was truly baffling.

A suggestion on superuser.com was to remove the laptop battery and try again. They reported that as the magic bullet for their problem. I had my father remove and re-seat the battery (I was doing all of this remotely.) Magic! Everything worked normally again. Perhaps there was some sort of static electricity buildup causing problems. Truly bizarre.

 

Gaming, Networking

Fix battle.net 2600 error

Leave a comment

Recently I tried to install the latest patch for Heroes of the Storm when I got a nasty error code 2600, Whoops! something broke.

wrong

Re-installing battle.net didn’t fix the issue. After much frustration I came across this post, which describes situations when you’re behind a caching proxy (which I am.)

I did as it directed, which is to disable the caching function of my proxy and delete anything Blizzard-related from my %temp% folder.

That did the trick. All is well now!

Update: I decided that rather than disabling caching / virus checkings completely I would create an exception in Sophos UTM web access policy. Thanks to the guidance from here I added the following exception:

blizzard
Skipping: Authentication / Caching / Block by download size / Antivirus / Extension blocking / MIME type blocking / URL Filter / Content Removal / SSL scanning / Certificate Trust Check / Certificate Date Check
Matching these URLs: ^https?://([A-Za-z0-9.-]*\.)?blizzard\.com/
^https?://([A-Za-z0-9.-]*\.)?blizzard\.vo\.llnwd\.net/
^https?://([A-Za-z0-9.-]*\.)?blizzard.com\.edgesuite\.net/
^https?://([A-Za-z0-9.-]*\.)?battle\.net/
CLI

Use batch script to continually check site status

Leave a comment

Recently my blog went down (the ISP running it had downtime.) I wanted to see when it came back up. As a result I wrote a little Windows batch script to continually poll my address in order to do just that.

The script issues a query to the default DNS server as well as pings the address of the blog. I used both since sometimes in Windows a ping will simply use internal system cache, which may be wrong if the IP address hosting my blog changes (it’s address is dynamic.)

The script is below:

@ECHO OFF
:loop
 cls
 nslookup jeppson.org | findstr "Address" | findstr /V 10.97.160.160
 ping -n 1 jeppson.org
 timeout /t 3
goto loop

I use the /V argument to take out the first bit spit out from the nslookup command, namely the IP address of the nameserver being used.

A simpler version of the script only issues one ping, waits a second, and then repeats the command. This is different from doing ping -t because it forces ping to do a new lookup for the domain name, whereas ping -t only resolves the IP once, then just pings the IP address. That wouldn’t work in my case as the IP of the domain name changes when it comes back online.

@ECHO OFF
:loop
 ping -n 1 jeppson.org
 timeout /t 1
goto loop

Thanks to Stack Overflow for educating me on how to write a quick loop to emulate the Linux Watch command,  ping only once, and use an application similar to grep to clean up output.

 

OS

Fix DNS issues after Yosemite upgrade

Leave a comment

My wife’s macbook pro started behaving strangely after upgrading to Yosemite from Mavericks. The initial upgrade went smoothly but over time certain applications began to quit working. The Pandora desktop client suddenly could never connect. Tunnelbick completely broke no matter what I did. Dnslookups all were fine but pings hung forever, eventually saying they couldn’t resolve a hostname.

It turns out that Yosemite changed the way Mac OSX resolves DNS records from mDNSresponder to dnsdiscoveryd. The issue I had only happened from and upgrade – clean installs didn’t have the issue

The fix for this madness, taken from here, is to remove a few network configuration files and reboot.

First, disconnect from any networks you’re connected to. Then, go to /Library/Preferences/SystemConfiguration/ and remove the following files (if they exist)

com.apple.airport.preferences.plist

com.apple.network.identification.plist
com.apple.wifi.message-tracer.plist

NetworkInterfaces.plist

preferences.plist

After removing those files, reboot. That should fix your problem! (At least, it did for me.)