Terminal Tips: creating a Spotlight-based gallery

As sexy as Leopard is to me, I'm just as enamored with the powerful UNIX underpinnings of the operating system. I've been toying with a few ways to combine a couple of my favorite command line utilities into something that might prove useful. I won't claim I've achieved a truly practical usage yet, but I thought I'd offer some possibilities. Even for the Terminal-timid, you just might find the potential intriguing enough to slap on some binary camouflage and go all "Code Warrior" for a bit.

I'm going to elaborate on some image processing techniques using results from Spotlight searches. In order to make use of the examples without modification, you'll need to have a collection of images containing IPTC keywords (or very descriptive filenames). With a little modification, the searches can be expanded to other criteria. But if you can't run a spotlight search and find at least 8 jpegs with one keyword, you'll have to accept this as a proof of concept. But, that being said, let's get started.

An old acquaintance

First, a quick reminder about mdfind, a command line interface for Spotlight which we've covered before. It allows for superb control over the search predicate (the combination of parameters that define the search). While I love the concept of it, I hadn't put it to very good use because the Spotlight interface, at least in Leopard, was far easier and sufficient for most applications. However, the advanced boolean search possibilities (AND, OR, NOT), and the fact that it can be scripted leads to another realm of possibilities not available in the Spotlight GUI.

I'm not going to get too crazy with mdfind tricks yet. If you find yourself intrigued, the man page is an almost complete reference (and great bedtime story material), with a couple of omissions that we've mentioned before. For now, let's move on to mdfind's dance partner for this round.

Have you met sips?

The Scriptable Image Processing System, or sips, is by no means a new kid on the block. You may already be using it for batch resizing, format conversion and more. If you're not, it's a great tool, included with OS X, with myriad possibilities. Take a look at all of the things sips can do. One of the many talents of sips is generating the icon previews that make your JPEG and other bitmap images easier to identify in the GUI. The command sips -i filename.jpg will take filename.jpg, add the icon preview and save it with the original file. If your computer is anything like mine, you probably have a few JPEGs laying around that, for various reasons, were never awarded with a custom icon.

Note: the Finder has a setting, "Show icon preview," that will create previews for the folder even for JPEGs without custom icons. To see the real custom icon state, turn off this setting in View View Options. Also, if you plan to experiment with this, CocoThumbX can make it easy to add and remove icons. As a side note, CocoThumbX can also do a bang-up job of creating icons for vector and movie files.

A semi-practical application

What if we could scan a folder - or even an entire drive - for all of the files missing their custom icons and generate them in one fell swoop? Thanks to our Spotlight index and a little shell prompt action, we can. Here's how it works:

You can use mdfind to locate the files in need of icons. The spotlight schema includes the boolean "kMDItemFSHasCustomIcon," which is set to 1 if the file has a custom icon, or 0 if it doesn't. But we're going to want to narrow that list down significantly; there are going to be a lot of files in that search. The first thing we can do is narrow the spread of the search with the "-onlyin" option. For now, we can use "." to represent the current directory, limiting the search to whatever location we run the command from, which gives us the flexibility to change directories and run the command from the history without editing. So far, the command looks like this:

mdfind -onlyin . 'kMDItemFSHasCustomIcon == 0'

That will find all of the files without custom icons, starting from the directory in which we run the command. Now, we also want to narrow down the list to certain image types. sips can't, for example, create preview icons for EPS files (why, Apple, have you forsaken the designers?), and many file formats get no preview icon at all. For our purposes, we're going to work solely with JPEG files, although sips can work with a range of raster images. Using "kMDItemContentType" in the predicate, we'll specify "public.jpeg" as the file type. One way to find out how to reference these identifiers is to manually locate a file that matches the search you want to perform (for now, any JPEG) and run mdls filename.jpg. That will show you all of the metadata associated with the file, the way Spotlight sees it. If you're curious, you can also try mdimport -X or mdimport -A to get full listings of the schema and available attributes, respectively. With that information, you can find keys unique to that file type or values that will narrow the search results. If we modify our predicate to target JPEG files, we get this:

mdfind -onlyin . '((kMDItemFSHasCustomIcon == 0) && (kMDItemContentType == "public.jpeg"))'

As we start combining the options, we build a bit of a logic maze using parenthesis - which are processed just like you learned in High School math class - and operators like || (or) and && (and). If you trace the parenthetic pairs in the command above, you'll see we're asking first that the matched files do not have a custom icon (kMDItemFSHasCustomIcon == 0), and that their content type is JPEG (kMDItemContentType == "public.jpeg"). You can go ahead and run that command to get a feel for the size of the list it might return. At this point, it will just output the results to Terminal, no harm, no foul. If you get no response, it worked but it didn't find anything in the current search. You can try running it without the "-onlyin ..." argument, which will search your entire system. If that doesn't return anything, you're sitting pretty with no icon-less JPEGs in the Spotlight index (again, you could use a utility to roll your own). If you're getting way too many results to experiment with, try running the command from a specific folder such as Documents, Pictures, or even Desktop to narrow down the field.

Assuming you found something to work with, what we need to do now is extract the filenames and run them through the aforementioned "sips -i [filename]" command. I'm going to do this using awk and a pipe (|) which will send the mdfind results to the awk command. When a list of file paths is directed to it, the command awk '{sub(/ /,"\\ ");system("sips -i " $0);}' will escape any spaces in the path and call "sips -i" to generate the icon for each file listed. Here it is all together:

mdfind -onlyin . '((kMDItemFSHasCustomIcon == 0) && (kMDItemContentType == "public.jpeg"))'|awk '{gsub(/ /,"\\ ");system("sips -i " $0);}'

You can run that command from any directory to fix any missing icon previews on files in it and all of its subdirectories. You can also turn this into a shell script or, being a one-liner, an alias.

The makings of a "Spotlight Gallery"

Just for fun (this is fun, right?), let's look at one more. This one creates a directory of thumbnails from the result of an mdfind search for JPEGs matching a certain keyword (in this case, "texture"). Because it's not overly advanced, it requires the creation of an output folder (Texture, in this example) on your desktop before running it:

mdfind -onlyin . '(((* == "texture"cdw) || (kMDItemTextContent == "texture"cdw)) && (kMDItemContentType == "public.jpeg"))'|awk '{gsub(/ /,"\\ ");system("sips --resampleHeightWidthMax 250 --out ~/Desktop/Texture " $0);}'

The "cdw" means the keyword is case insensitive, to ignore diacritical marks and that the search is "word based," respectively. More information on advanced search syntax can be found in the Spotlight Overview at Apple's developer site. You can easily modify the "-out" argument to point to any output folder you like, just make sure it exists before running the command. In practice, though, it would be important to set an output directory that was excluded from Spotlight indexing using the Privacy section of the Spotlight preferences panel, otherwise each time the script creates new images they'd be indexed again in Spotlight, creating duplicate results. Note that the gsub in the awk statement only escapes spaces, if you have a lot of reserved characters popping up, you'll need a more detailed regular expression, or you could potentially incorporate something like perl's quotemeta function. Speaking of perl...

Grand finale: proof of concept

I'm including a simple perl script to take this concept one step further and actually generate a full gallery with thumbnails, resized full images and some HTML/CSS to show off the results. Everything is generated with relative paths so the final output could be moved, uploaded, etc.. Again, it's important to keep the resulting gallery outside of Spotlight's indexing to avoid duplicate results later.

You can download the script, unzip it and open it in a text editor to get a feel for what it's doing. The current incarnation of this script only takes one single-word term as an argument, but has several editable parameters at the top of the script which you can change with any text editor. If you save the script to your Desktop (or any folder), you can locate it in Terminal and make it executable (chmod a+x albumgenerator.pl). Assuming your perl executable is in the standard location, you can then run the script with:

./albumgenerator.pl keyword

Substitute your favorite keyword and see what it comes up with. This works really well on my system, which is loaded with keyworded stock images. Depending on your image collection, you may have more depressing results, but I'm offering this as a proof of concept that shows one potential use for mdfind. The end goal of this project could be to have a web gallery (or galleries) that automatically updated every time a matching file showed up in your system. Like I said in the intro, I'm not sure about the ultimate "usefulness" of that, but I'm intrigued. And, if nothing else, I learned some new tricks and hopefully shared something a wiz like you hadn't already figured out.