I wanted a quick way to search my files for video types. I found here a quick snippet on how to do so. I augmented it after finding out how to remove some info and make it case insensitive. Here is the result:
find FULL_FOLDER_PATH -type f | grep -E "\.webm$|\.flv$|\.vob$|\.ogg$|\.ogv$|\.drc$|\.gifv$|\.mng$|\.avi$|\.mov$|\.qt$|\.wmv$|\.yuv$|\.rm$|\.rmvb$|/.asf$|\.amv$|\.mp4$|\.m4v$|\.mp*$|\.m?v$|\.svi$|\.3gp$|\.flv$|\.f4v$" -iname|sed 's/^.*://g'|sortI recently found myself on a Windows 10 system needing to do the equivalent of “find . -name *.mdi -exec mdiconvert -source {} -log log.txt \;” I knew what to do instantly on a Unix system, not so on a Windows system
I finally figured it out and am now writing it down because I know I’ll forget! Thanks to these several sites for pointing me in the right direction.
The command I ended up using is below (find any .mdi files and use Microsoft’s mdi2tif utility to convert the result to .tif files)
Get-ChildItem "C:\Users" -Recurse -Filter *.mdi | % { & 'C:\Program
Files (x86)\modiconv\MDI2TIF.EXE' -source $_.FullName -log log.txt }
Recently I needed to find folders with two different things in the path – mysql and DB. I toyed around with a bunch of options but finally settled on using xargs. I don’t use it much. I should use it more.
The command below takes output from find, greps it twice (thus looking for things that have both terms in them) and then creates a symbolic link of the results.
find . -type d | grep mysql | grep DB | xargs -n 1 ln -s
This accomplished what I needed quite well. In a huge stash of folders there were a subset that contained both the words mysql and DB in their paths that I was interested in. Find, grep, and xargs to the rescue.
I’ve recently had to do some bash-fu and thought I’d document it in case I come across the need again. It involved an if statement inside a for loop. The if statement looked at the result of an external command and acted if conditions were met.
The scenario: An application created folders beginning with a series of digits. Later it was decided to add a prefix to new folders. A problem occurred where there were folders with the same numeric sequence – corresponding to the same user – but the program was saving things in both prefixed and non-prefixed folders at random. We needed a way to copy information from the numeric only folders into the prefix folders, then backup and delete the numeric-only folders. We also wanted to be warned about any file overwrites in the process.
After a bunch of research and experimentation I came up with the following one-line bash script:
for d in [0-9]*; do BN=$(basename "$d"); if [[ $(find . -maxdepth 1 -type d -name "*$d" | grep -o $d | wc -l) = 2 ]]; then cp -i -p -r "$d" ../archive/"$d"; cp -i -p -r "$d"/* "PREFIX_$BN"; rm -rf "$d"; fi; done
It does the following:
The big learning moment for me was embedding a bash command into an if statement. The if statement runs the find command, pipes to wc -l to count the number of results, and then compares that result to something else. Pretty handy.
Thanks to these sites for helping me in my journey:
If statement inside for loop: https://unix.stackexchange.com/questions/52800/how-to-do-an-if-statement-from-the-result-of-a-executed-command
Find results only in current directory: https://unix.stackexchange.com/questions/162411/find-maxdepth-0-not-returning-me-any-output
Count results from find command: https://stackoverflow.com/questions/6181324/counting-regex-pattern-matches-in-one-line-using-sed-or-grep
Warn before overwriting files: https://askubuntu.com/questions/236478/how-do-i-make-bash-warn-me-when-overwriting-an-existing-file
For some reason I had a hell of a time finding a way of excluding a directory from the ‘find’ command. I finally found a solution that works thanks to this website.
The solution for me was to add the following to my find command:
! -path "<directory to exclude>"
So to find all files modified less than 24 hours ago on my grandmother-in-law’s computer, I used the following command:
find . -mtime -1 ! -path "./AppData/*"
Grandma’s computer runs Windows 7 but I’ve configured Cygwin and ssh so I can use the tools I’m fimilar with. The above command excludes appdata and searches her user profile for all files modified today.
I came across a need to install multiple patches manually (via SSH) on one of my xenservers. It’s quite tedious to do this manually so I found a way to here.)
Download all the patch .zip files to a directory your xenserver can access. Then, extract them all with this command:
find *.zip -exec unzip {} \;
Next, upload all the .xsupdates:
find *.xsupdate -exec xe patch-upload file-name={} \;
This spits out a bunch of UUIDs. Make note of these. You will also need to get your host-uuid by using the
xe host-list
command.
Lastly, a quick for loop applies the patches we want (replace the UUIDs with those of the patches uploaded earlier and the host-uuid with yours)
for file in c3520494-be00-4133-afb3-adf8ab5edb11 7fea2d85-7ce1-428c-a92f-57e37551d6f1 d9862b7f-9be6-4672-b9a8-4f52f776fd03 a424dfe5-8be8-4bd6-a49e-62620e369a43 e28bb0ae-e43f-46d9-9147-c7dc712508eb; do xe patch-apply uuid=$file host-uuid=46f8ef28-8ee1-44b5-967c-b8e48585094b; done
That did the trick for me. After applying the patches I came across this post which appears to have a much better script. Whatever works.
Below are the tips and tricks I’ve learned in my quest to get all my family home movies working properly in Plex. I ended up having to re-encode most of them to have a proper codec.
ffmpeg is what I ended up using. Thanks to here and here I found a quality setting that I liked: x264 video with crf of 18 and veryslow encoder preset, AAC audio at 192kbps, mp4 container.
ffmpeg -i <file> -acodec libfdk_aac -b:a 192k -ac 2 -vcodec libx264 -crf 18 -preset veryslow <filename>.mp4
I tried to use exiftools to add metadata to the file but Plex wouldn’t read it. Instead I discovered you can use ffmpeg to encode things like title and date taken directly into the file. The syntax for ffmpeg is
-metadata "key=value"
The metadata values I ended up using were:
My first pass at encoding combined find with ffmpeg to re-encode all my avi files. I used this command:
find . -name *.avi -exec ffmpeg -i {} -acodec libfdk_aac -b:a 192k -ac 2 -vcodec libx264 -crf 18 -preset veryslow {}.mp4 \;
I bit off more than I could chew with the above command. I had to kill the process because it was chewing up too much CPU. There are two ways to figure out which file ffmpeg is on:
ps aux|grep ffmpeg
will list the file that ffmpeg is currently working on. Take note of this because you’ll want to remove it so ffmpeg will re-encode it.
You can also run this command to see the last 10 most recently modified files, taken from here:
find $1 -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
Use the above command to remove what ffmpeg was working on when killed. Append -n to tell ffmpeg to exit and not overwrite existing files
find . -name *.avi -exec ffmpeg -n -i {} -acodec libfdk_aac -b:a 192k -ac 2 -vcodec libx264 -crf 18 -preset veryslow {}.mp4 \;
The command above gave me a bunch of .avi.mp4 files. To clean them up to just be .mp4 files combine find with some bash-fu (thanks to this site for the info)
find . -name “*.avi.mp4” -exec sh -c ‘mv “$1” “${1%.avi.mp4}.mp4″‘ _ {} \;
Finally, remove original .avi files:
find . -name "*.avi" -exec rm {} \;
Update 2/3/2016: Added a verification step.
Well, it looks like the conversion I did was not without problems. I only detected them when I tried to change the audio codec I used. You can use ffmpeg to analyze your file to see if there are any errors with it. I learned this thanks to this thread.
The command is:
ffmpeg -v error -i <file to check> -f null -
This will have ffmpeg go through each frame and log and tell you any errors it finds. You can have it dump them to a logfile instead by redirecting the output like so:
ffmpeg -v error -i <file to scan> -f null - >error.log 2>&1
What if you want to check the integrity of multiple files? Use find:
find . -type f -exec sh -c 'ffmpeg -v error -i "{}" -f null - > "{}".log 2>&1' \;
The above command searches for any file in the current directory and subdirectories, uses ffmpeg to analyze the file, and will output what it finds to a log file of the same name as the file that was analyzed. If nothing was found, a 0 byte file was created. You can use find to remove all 0 byte files so all that’s left are logs of files with errors:
find . -type f -size 0 -exec rm {} \;
You can open each .log file to see what was wrong, or do what I did and see that the presence of the log file at all means that the file needs to be re-encoded from source.
Checking the integrity of the generated files themselves made me realize I had to re-encode part of my home movie library. After re-encoding, there were no more errors. Peace of mind. Hooray!
I wanted to make a simple slideshow for a cheap Kindle turned photo frame in my office. Windows Movie Maker (free, already installed program) does not have a randomize function when importing photos. I had a lot of photos I wanted imported and I wanted them randomized. Movie Maker doesn’t include subfolders for some dumb reason, so I also needed a way to grab pictures from various directories and put them in a single directory.
My solution (not movie-maker specific) was to use bash combined with find, ln, and mv to get the files exactly how I want them. The process goes as follows:
It worked beautifully. The commands I ended up using were as follows:
mkdir temp
cd temp
find /Pictures/2013/ -type f -exec ln -s -b {} . \;
#repeat for each subfolder as needed, unless you want all folders in which case you can just specify the directory beneath it.
find /Pictures/2014/ -type f -exec ln -s -b {} . \;
find /Pictures/2015/ -type f -exec ln -s -b {} . \;
for i in *.JPG; do mv "$i" "$RANDOM.jpg"; done
#repeat for all permutations. The -b argument of ln creates files with tildes in the extension - don't forget about them.
for i in *.jpg; do mv "$i" "$RANDOM.jpg"; done
for i in *.JPG~; do mv "$i" "$RANDOM.jpg"; done
for i in *.jpg~; do mv "$i" "$RANDOM.jpg"; done
The end result was a directory full of pictures with random filenames, ready to be dropped into any crappy slideshow making software of your choosing 🙂
I realized recently that my desktop system is quite out of date. It has worked so well for so long that I didn’t realize for a while that it was end of support. I was running Linux Mint 16 – Petra.
Thanks to this site the upgrade was fairly painless – a few repository updates, upgrade, and reboot. Simple! The steps I took are below
Use sed in conjunction with find to quickly and easily update all your repository files from saucy to trusty, and from petra to rebecca, making a backup of files modified.
sudo find /etc/apt/sources.list.d/ -type f -exec sed --in-place=.bak 's/saucy/trusty/' {} \;
sudo find /etc/apt/sources.list.d/ -type f -exec sed --in-place=.bak 's/petra/rebecca/' {} \;
This took a while. It had to download 1.5GB of data and install it.
sudo apt-get update sudo apt-get dist-upgrade sudo apt-get upgrade
After running the upgrade I had a notice that I had many packages that were installed but no longer required. To remove unnecessary packages after the upgrade:
sudo apt-get autoremove
sudo apt-get install mintlocale
sudo apt-get install gvfs-backends
Flawless! It worked on the first try. Awesome.
Occasionally I have a need to unzip multiple zip files into a single directory, renaming any files with duplicate names so all zip contents end up in the same directory. I accomplish this in a lazy fashion with find and unzip.
First, put all the zip files you need to extract in the same folder. I used find with the -ctime flag to find files created today (as those are the ones I want.) I use the -exec flag to execute a command on the resulting files; in this case, the unzip program with the -B flag, which doesn’t overwrite files with duplicate names, and the -d flag to specify which folder to extract to:
find *.zip -ctime 1 unzip -B {} -d example/ \;
This finds and unzips all my files, but there is a catch: files with the same filename have been renamed with tildes at the end of each file, for example pic1.jpg~. I do another quick find operation to simply tack .jpg to the end of each of these files
find example/ -name "*~*" -exec mv {} {}.jpg \;
The result is a directory full of the files you desire. My case is very simplistic as it assumes that all files in each zip file are of the same extension. You could use something like awk to parse the result of find command and re-add appropriate extensions, but I won’t detail that here (see laziness reference above.)