Quick little bash snippet: I needed to prepend all files in a directory with their respective modified times. This was to make sorting in various applications easier. This is what I came up with.
for file in *.mp3; do mv "$file" $(date +%Y%m%d-%H%M%S -r $file")_"$file"; done
I wanted a quick way to search my files for video types. I found here a quick snippet on how to do so. I augmented it after finding out how to remove some info and make it case insensitive. Here is the result:
find FULL_FOLDER_PATH -type f | grep -E "\.webm$|\.flv$|\.vob$|\.ogg$|\.ogv$|\.drc$|\.gifv$|\.mng$|\.avi$|\.mov$|\.qt$|\.wmv$|\.yuv$|\.rm$|\.rmvb$|/.asf$|\.amv$|\.mp4$|\.m4v$|\.mp*$|\.m?v$|\.svi$|\.3gp$|\.flv$|\.f4v$" -iname|sed 's/^.*://g'|sortThank-you, Peter Kleissner, for saving me from having to use my time machine to simply update my server’s BIOS: https://peterkleissner.com/2018/05/27/reverse-engineering-supermicro-ipmi/
https://twitter.com/kleissner/status/996955400787423232?lang=en
Supermicro IPMI License Key (for updating BIOS) = HMAC-SHA1-96(INPUT: MAC address of BMC, SECRET KEY: 85 44 E3 B4 7E CA 58 F9 58 30 43 F8)
echo -n 'bmc-mac' | xxd -r -p | openssl dgst -sha1 -mac HMAC -macopt hexkey:8544E3B47ECA58F9583043F8 | awk '{print $2}' | cut -c 1-24
AWX is the open source version of Ansible Tower. It’s a powerful tool, but unfortunately AWX has no in place upgrade capability. If you want to upgrade your AWX to the latest version it takes a bit of trickery (the easy way out being just to pay for Ansible Tower.)
Essentially to upgrade AWX you need to spin up a completely new instance and then migrate your data over to it. Fortunately there is a script out there that makes doing this a bit easier.
Below are my notes for how I upgraded my instance of AWX from version 1.0.6 to 2.1.0.
Spin up new server with ansible installed, then clone AWX
git clone https://github.com/ansible/awx.git cd awx git clone https://github.com/ansible/awx-logos.git
Modify AWX install to expose 5432 externally:
edit installer/roles/local_docker/tasks/standalone.yml and add
ports:
- "5432:5432" right above the when: pg_hostname is not defined or pg_hostname == '' line. Complete stanza looks like this:
- name: Activate postgres container
docker_container:
name: postgres
state: started
restart_policy: unless-stopped
image: "{{ postgresql_image }}"
volumes:
- "{{ postgres_data_dir }}:/var/lib/postgresql/data:Z"
env:
POSTGRES_USER: "{{ pg_username }}"
POSTGRES_PASSWORD: "{{ pg_password }}"
POSTGRES_DB: "{{ pg_database }}"
PGDATA: "/var/lib/postgresql/data/pgdata"
ports:
- "5432:5432"
when: pg_hostname is not defined or pg_hostname == ''
register: postgres_container_activate
Make sure you have port 5432 open on your host-based firewall.
Install AWX on the new host. Verify you can log into the empty instance and that it’s the version you want to upgrade to.
Kill the AWX postgres container on the source machine, and re-run awx installer after modifying it to expose its postgres port as described above.
Install tower-cli (this can be on either source or destination servers)
sudo pip install ansible-tower-cli
Configure tower-cli
tower-cli config username SRC_AWX_USERNAME towercli config password SRC_AWX_PASSWORD towercli config host SRC_AWX_HOST
Make sure to use full ansible URL as accessed from a browser for both source and destination
Install awx-migrate:
git clone https://github.com/autops/awx-migrate.git
Update awx-migrate/awx-migrate-wrapper with correct source and destination info
Run awx-migrate-wrapper. It will generate json files with your configuration.
Modify tower-cli config, set host, username and password to that of the destination AWX instance
tower-cli config username DEST_AWX_USERNAME towercli config password DEST_AWX_PASSWORD towercli config host: DEST_AWX_HOST
Send JSON info to destination:
tower-cli send awx-data.json
You will now have a fresh new, updated AWX instance working, with imported database, on the destination host. Confirm you can log into it with the admin account you set it up with.
Now, on the source, remove the old AWX docker containers:
sudo docker rm -f postgres awx_task awx_web memcached rabbitmq
Move / delete the database folder the postgres docker container was using (as defined in awx installer inventory) in my case:
/var/lib/awx /var/db/pgsqldocker
Remove and re-install AWX folder with a fresh git checkout
rm -rf awx git clone https://github.com/ansible/awx.git cd awx git clone https://github.com/ansible/awx-logos.git
Re-run the AWX installer to re-create a blank database on the source host, modify the new awx/installer/inventory as needed. Also modify installer/roles/local_docker/tasks/standalone.yml as outlined above.
cd awx/installer sudo ansible-playbook -i inventory install.yml
Once a new, empty version of awx is running on the source host, start the awx-migrate process in reverse to migrate the database on the destination instance back to the source. Modify awx-migrate-wrapper and tower-cli to switch src and destination (the destination has become the source and the source has become the destination)
Use awx-migrate-wrapper to generate new ansible version json files (don’t confuse them with the old json files – best to delete / move all json files before running awx-migrate-wrapper)
Modify tower-cli to point to original AWX URL
Run tower-cli send awx-data.json
Once completed, log in as the admin account. Input LDAP BIND password under settings, then delete any imported LDAP users.
You may want to remove the exposed postgres database ports. Simply undo the changes you made in awx/installer/roles/local_docker/tasks/standalone.yml to remove the Ports part of the first play, then remove your postgres container and re-install AWX with install.yml
Also remember to delete the JSON files generated with awx-migrate as they contain all your credentials in plaintext.
I needed to one-click export multiple resolutions of pictures from Lightroom. Unfortunately there isn’t any kind of plugin available to do this. Fortunately I was able to find this guide on how to get an applescript script to do it for me (Mac only, sadly.)
The trick is to write a few bits of applescript and save it as an application. Then when exporting the pictures in lightroom, make sure the word “fullsize” is in the filename, and configure lightroom to run your applescript after export.
I tweaked the script a bit to move the full size version to a different folder, then open Finder to the folder that the other resolutions were created in (thanks to this site for the guidance)
Here is my script below. It works!
on open of myFiles
set newSizes to {5000}
repeat with aFile in myFiles
set filePath to aFile's POSIX path
set bPath to (do shell script "dirname " & quoted form of filePath)
tell application "System Events" to set fileName to aFile's name
repeat with newSize in newSizes
do shell script "sips " & quoted form of aFile's POSIX path & " -Z " & newSize & " --out " & quoted form of (bPath & "/" & rename(fileName, newSize) as text)
end repeat
do shell script "mv " & quoted form of aFile's POSIX path & " /path/to/folder/for/fullres/images"
end repeat
do shell script "open /path/to/folder/of/resized/images"
end open
on rename(fName, fSize)
do shell script "sed 's/fullsize/" & fSize & "/' <<< " & quoted form of fName
end rename
chmod g+s -R <folder> setfacl -d -m "g:<group name>:<permissions>" -R <directory>
setfacl -d -m "g:testgrouprw:rwX" -R /var/www/html/wordpress/
WordPress has an extremely annoying issue with reading EXIF data when posting photos and videos. There is a plugin to fix rotated pictures, but I couldn’t find one for rotated videos. If you happen to upload a video from your phone that was shot in portait orientation, it will likely upload to wordpress in landscape orientation (sideways.) Very annoying.
My fix to this is to re-encode the video so it is at the proper orientation before uploading to wordpress. Thanks to this site for the information – ffmpeg does this automatically for you. So the syntax is simple:
ffmpeg -i SOURCE_VIDEO_FILENAME -c:a copy FIXED_VIDEO_FILENAME.mp4
Success.
I have a lot of applications at work which do not support Active Directory but instead rely on LDAP queries for granting user access. A problem we have is much of our access is granted to a security group (known as a ROLE) and users are granted to that single security group to get access to things. This allows easier access granting to new hires / transfers. The problem is it makes LDAP queries much more difficult. Things are further complicated by the fact that sometimes users are directly granted access to resources instead of going through their ROLE security group.
I spent a lot of time researching LDAP nested group queries. I now have a functional way of doing semi-nested LDAP group searches. The scenario: a user could be directly added to a security group granting access to a resource, or could be a member of a security group which has access to the resource. I want the LDAP group search string to account for both. I accomplish this by combining these two queries:
Search groups beginning with the name ROLE for a specific member, then return what that ROLE group has access to
(&(objectClass=group)(DisplayName=ROLE*)(member=FQDN_OF_USER_IN_QUESTION)(memberOf=*))
Search for all groups a specified member is a member of
(&(objectClass=user)(sAMAccountName=USERNAME_OF_USER_IN_QUESTION)(memberOf=*))
I combine these two queries by separating them out with an OR operator (|)
Return the group membership of the user in question, as well as the group membership of the group beginning with the name ROLE that the user is a member of
(|(&(objectClass=group)(DisplayName=ROLE*)(member=FQDN_OF_USER_IN_QUESTION)(memberOf=*))(&(objectClass=user)(sAMAccountName=USERNAME_OF_USER_IN_QUESTION)(memberOf=*)))
It has three main parts:
This works for our organization because ROLE groups are not nested within themselves and each user can only have one ROLE group assigned to them.
This combined query allows me to not have to “flatten” security groups for LDAP-bound applications. It makes me so happy.
This was made possible by a flurry of stack overflow posts:
https://stackoverflow.com/questions/32829104/ldap-query-with-wildcard
https://stackoverflow.com/questions/9564120/using-wildcards-in-ldap-search-filters-queries
https://stackoverflow.com/questions/6195812/ldap-nested-group-membership
I struggled for a while to get WordPress to use Active Directory credentials on CentOS 7. Below is how I finally got it to work.
First, install necessary packages:
sudo yum -y install openldap-clients php-ldap
If you use self-signed certificate for ldaps, you’ll need to modify /etc/openldap/ldap.conf
HOST <HOSTNAME_OF_LDAP_SERVER> PORT 636 TLS_CACERT <PATH_TO_CA_CERT> TLS_REQCERT demand
With the above settings you can test your ldap string with ldapsearch
ldapsearch -x -D "<BIND USERNAME>" -b "<BASE_DN>" -H ldaps://<LDAP_SERVER_HOSTNAME> -W sAMAccountName=<USER_TO_QUERY>
Once ldapsearch works properly, install your AD integration plugin. I use AuthLDAP by Andreas Heigl
I struggled with which LDAP strings and filters to use. This is what finally got everything working with our Active Directory environment:
LDAP URI: ldaps://<BIND_USERNAME>:<BIND_PASSWORD>@<AD_SERVER_ADDRESS>:636/<BASE DN>
Filter: (sAMAccountName=%s)
Name-Attribute: givenName
User-ID Attribute: sAMAccountName
Second Name Attribute: sn
Group-Attribute:memberOf
Group-Separator: _
Group-Filter: (&(objectClass=user)(sAMAccountName=%s)(memberOf=*))
I had to change Group-Separator to _ above, because in Role – group mapping for active directory, you must put the FQDN, which includes commas. Put an underscore separated list of FQDNS for each of these fields you want.
In one of my ansible playbooks I need to obtain a file from a Windows share. I can’t find a module that handles this so I’m using the shell module to call the smbclient command to do what I need. The problem with this solution is that smbclient prompts for a password (and I don’t want to supply it on the command itself for security reasons.)
I tried using ansible’s built-in expect module, but frustratingly it only works on systems that have pexpect >= 3.3 , which CentOS 7 & Ubuntu 14.04 do not have.
My solution to this is to install the expect command on the host, and then use the ansible shell module to call it, following the example given in Ansible’s shell module page
Part of the process in my playbook is registering stdout from that command for later use. I then ran into a problem where I would run smbclient -c “ls <filename>” but ansible would register nothing. After some digging I found I also need to include the interact command after sending the password. Without it, anything after sending the password is not registered to stdout. Thanks to rostyslav-fridman on Stack Overflow for the answer.
My final problem was I was sending a password that had a ] character in it. It was causing this error on run:
missing close-bracket\n while executing\n\"send
I found here (thanks glenn-jackman) it was due to the fact that the expect syntax uses tcl language, which treats those brackets as special characters. To get around this I had to use an ansible filter, specifically regex_escape()
Lastly I ran into an issue specifically with how I was spawning smbclient. I kept getting this message:
"stderr": "send: spawn id exp4 not open\n while executing\n\"send
It boiled down to single vs double quotes. If I put my -c arguments in single quotes it failed; with double quotes, it worked.
My completed play is below. Finally, success!
- name: Get RSA filename
shell: |
set timeout 300
spawn smbclient {{standards_location}} -W DOMAIN -U {{username}} -c "ls {{file_location}}*.tar"
expect "password:"
send {{ password | regex_escape() }}\r
interact
exit 0
args:
executable: /usr/bin/expect
changed_when: false
no_log: true
register: RSA_filename_raw