Backup and restore docker container configurations

I came across a need to start afresh with my docker setup. I didn’t want to re-create all the port and volume mappings for my various containers. Fortunately I found a way around this by using docker-autocompose to create .yml files with all my settings and docker-compose to restore them to my new docker host.

Backup

Docker-autocompose source: https://github.com/Red5d/docker-autocompose

git clone https://github.com/Red5d/docker-autocompose.git
cd docker-autocompose
docker build -t red5d/docker-autocompose .

With docker-autocompose created you can then use it to create .yml files for each of your running containers by utilizing a simple BASH for loop:

for image in $(docker ps --format '{{.Names}}'); do docker run -v /var/run/docker.sock:/var/run/docker.sock red5d/docker-autocompose $image > $image.yml; done

Simple.

Restore

To restore, install and use docker-compose:

sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Next we use another simple for loop to go through each .yml file and import them into Docker. The sed piece escapes any $ characters in the .yml files so they will import properly.

for file in *.yml; do sed 's/\$/\$\$/g' -i $file;
docker-compose -f $file up --force-recreate -d; done

You can safely ignore the warnings about orphans.

That’s it!

Troubleshooting

ERROR: Invalid interpolation format for “environment” option in service “Transmission”: “PS1=$(whoami)@$(hostname):$(pwd)$ “

This is due to .yml files which contain unescaped $ characters.

Escape any $ with another $ using sed

sed 's/\$/\$\$/g' -i <filename>.yml

ERROR: The Compose file ‘./MariaDB.yml’ is invalid because:
MariaDB.user contains an invalid type, it should be a string

My MariaDB docker .yml file had a user: environment variable that was a number, which docker compose interpreted as a number instead of a string. I had to modify that particular .yml file and add quotes around the value that I had for the User environment variable.

Sabrent USB AU-MMSA microphone not working in Windows 10

I recently installed Windows 10 for my gaming VM and discovered that my microphone was no longer working. All the drivers were properly installed and sound worked fine, but there was nothing coming from the microphone.

My gaming VM uses a Sabrent USB External Stereo Sound Adapter model  AU-MMSA passed through for sound. This was most perplexing because it worked in other OSes, but not Windows 10.

After much digging I finally found this youtube video which outlined the problem: Microphone permissions to the system. The hybrid that Windows 10 is between Store apps / permissions and regular desktop apps reminds me of Windows ME. An unholy union.. terrible.

At any rate, the fix is to grant the system permission to use its own microphone, un-granting it first if necessary.

Go to Start / Settings (little gear icon in bottom left) then search for Microphone Privacy Settings. Click the big Change button beneath “Microphone access for this device is on”  at the top of that screen. Change the toggle to “off”, then change it back to “on” again. This fixed my microphone.

 

Accept multiple SSH RSA keys with ssh-keyscan

I came across a new machine that needed to connect to many SSH hosts via ansible. I had a problem where ansible was prompting me for each post if I wanted to accept the RSA key. As I had dozens of hosts I didn’t want to type yes for every single one; furthermore the yes command didn’t appear to work. I needed a way to automatically accept all SSH RSA keys from a list of server names. I know you can disable RSA key checking but I didn’t want to do that.

I eventually found this site which suggested a small for loop, which did the trick beautifully. I modified it to suit my needs.

This little two-liner takes a file (in my case, my ansible hosts file) and then runs ssh-keyscan against it and adds the results to the .ssh/known_hosts file. The end result is an automated way to accept many SSH keys.

SERVER_LIST=$(cat /etc/ansible/hosts)
for host in $SERVER_LIST; do ssh-keyscan -H $host >> ~/.ssh/known_hosts; done

Using a Bus Pirate to fix Seagate drives

I wrote these notes almost three years ago but never published them. Since I’ve now referenced them again I’ll publish them albeit in a crude state.

7200.11 BSY bug

I had a need to fix the firmware of a Seagate  7200.11 BSY bug, which involved connecting to the RS232 serial ports on the drive and issuing a few commands to clear SMART data. Details here:

http://www.arvydas.co.uk/2012/07/fixing-a-seagate-7200-11-hard-drive-with-arduino/

http://hackaday.com/2012/07/30/recovering-from-a-seagate-hdd-firmware-bug/

https://plus.google.com/u/0/+BillFarrow/posts/ir1xnfu46TE

http://webcache.googleusercontent.com/search?q=cache:F1J2P5E3mrIJ:haquesprojects.com/embedded-device-hacking/using-a-bus-pirate-as-a-usb-ttl-serial-converter/+&cd=4&hl=en&ct=clnk&gl=us

http://fillwithcoolblogname.blogspot.com/2011/02/fixing-seagate-720011-bsy-0-lba-fw-bug.html

Using a Bus Pirate:

Find out what device the bus pirate is given:

dmesg | tail

usb 1-1.6.3: FTDI USB Serial Device converter now attached to ttyUSB0

Next, add your user to the dialout group (thanks to here for the hint)

usermod -a -G dialout $USER

You may need to log out and log back in after issuing the above command for it to take effect.

Fire up a terminal editor (I used screen after learning about my options from here.)

screen /dev/ttyUSB0 115200 8N1

Press Enter and you should be greeted with the Bus Pirate’s HiZ> prompt. Next, enter the following:

1. m – to change the mode
2. 3 – for UART mode
3. 7 – for 38400 bps
4. 1 – for 8 bits of data, no parity control
5. 1 – for 1 stop bit
6. 1 – for Idle 1 receive polarity
7. 2 – for Normal output type

At the “UART>” prompt. Enter “(0)” to show available macros:

UART>(0)
0.Macro menu
1.Transparent bridge
2.Live monitor
3.Bridge with flow control

Now enter “(3)”  (don’t forget the parenthesis – this burned me) to enter bridge mode with flow control and hit “y” at the “Are you sure?” prompt. The terminal will receive input from your device.

UART>(3)
UART bridge
Reset to exit
Are you sure?

Now plug in pins to hard drive. Use this site as a guide for which pins to use. The drive should be upside down to expose the controller board.
BP Gnd (top left) to Gnd on drive (Second pin from the left)
BP MISO (UART RX – bottom right) to TX on drive (far right pin)
BP MOSI (UART TX) to RX on drive (Seconf from the right pin)

I only ended up needing MISO & MOSI, ground wasn’t required.

Un-screw hard drive, add shim to prevent electrical contact

Power on drive

CTRL+Z

/2

(wait 30 seconds)

Z

(un-shim, re-screw hard drive)

U

/1

N1

Power down drive, wait few seconds, power back up

CTRL + Z

m0,2,2,0,0,0,0,22 (enter)

Clear SMART data

A couple years later I came across some old NAS drives that I wanted to use. I ran a full battery of burn-in tests using badblocks and the drives passed with flying colors. The only problem is they had SMART data saying Reallocated_Sector_Ct was past the threshold. Barely. I decided to roll the dice with these drives anyway given their proven performance currently and over the years.

The problem is FreeNAS will e-mail spam you about that SMART attribute. I couldn’t find a good way to suppress those alerts yet have them alert if that number gets worse, so I decided to cheat and clear all SMART data from those drives, thus getting FreeNAS happy with me yet alerting me if the reallocated sector count increases in the future.

I read a few sources to accomplish this with my bus pirate.

https://blog.zencoffee.org/2011/07/bus-pirate-as-ftdi-cable/

https://forum.hddguru.com/viewtopic.php?f=1&t=33886&start=20&mobile=mobile

https://forum.hddguru.com/viewtopic.php?f=1&t=33886&start=20

Use the same instructions as above for hooking up the bus pirate to the drive’s RS232 ports (to the right of the SATA port.)

Once you’ve serial connected to the drive, it’s three simple commands to clear the SMART data:

CTRL + Z
/1
N1

Docker – run a cron job for a container from the host

I’ve installed tiny tiny rss as a replacement for Feedly once they started inserting ads that looked like articles. Deceptive advertising. I’m not a fan.

I’ve spun up linuxserver’s version of it in docker and it works pretty well except for updating articles. I couldn’t find a great guide on configuring it for updates specifically within a docker container, so here is mine. My solution was to have a cron job running on the docker host to run the feed update script within the docker container, inspired by this post.

The trick is to use the docker exec command to run a command from the docker host but execute it within the running container.

docker exec -u 1001 -it TinyTinyRSS /usr/bin/php /config/www/tt-rss/update.php --feeds --quiet

The -u command specifies which user ID to run the command as. TinyTinyRSS is the name of my container. I’ve set this to run every 15 minutes with the following crontab syntax:

*/15 * * * * /usr/bin/docker exec -u 1001 -d TinyTinyRSS /usr/bin/php /config/www/tt-rss/update.php --feeds --quiet

edit: Modified the crontab entry to make it work properly per this post.