Document Metadata and the Finder: Part I

January 27, 2009 iTunes and the Finder: Information Software

Problem: Each update to Mac OS X integrates metadata more deeply into the Finder. Yet, outside of Spotlight, document metadata remains largely inaccessible without a specialized application.[1] How deeply will Apple integrate metadata into the Finder’s interface, and what happens when functionality in the Finder overlaps functionality in Apple’s core applications?1 iTunes, Mail, Address Book, iCal, iPhoto, and so on.

Motivation: Information consumption and management habits are highly divergent across user demographics. As the importance of medium ceases to be the defining categorization of content – as radio, television, books, newspapers, and the internet coalesce – the perceptual bias of the individual becomes central to the effective management and networking of information; metadata is the best available tool to capture this bias. Today, digital media is liberating information from physical locality at an unprecedented pace;[2] it is time that the information stored on one’s personal computer is freed from hierarchical file system locality in a similar manner.2 Though formerly physical entities – audio records, newspapers, etc. – still inform the structure & boundaries of virtual medias.

Thesis: Document metadata cannot coexist with directory metadata[3] in file-management interfaces without greatly complicating file management interface usability. This leads to a necessary separation; in Mac OS X Leopard the solution has been to create two interfaces, one that allows the management of files via directory-based file-system metadata (the traditional Finder) while another (Spotlight and “Smart Folders”) allows the browsing and management of files via document metadata.[4] 3 Document metadata stores information specific to a file type (such as “Artist” and “Album” in an mp3). File system metadata, of which directory metadata is part, stores information about a file’s place within a system, such as its timestamp and location.
4 Note that temporal file metadata transcends this division.

In addition to Spotlight and “Smart Folders”, Leopard provides numerous specialized document metadata interfaces, such as Mail, iTunes, Address Book, iCal, and so on. These file-type-specific interfaces demonstrate that document metadata actually renders directory file-system metadata useless; in these interfaces, file location is hidden in favor of individualized document metadata, of “tags” that allow for synchronicity of locality across playlists, Smart Folders, Groups, and so on. Accordingly, Apple has chosen to organize the files that these interfaces manage via the Application itself, not via the Finder; these interfaces create “flat file systems” in which directories are irrelevant to file-management.

The demonstrated success of such interfaces poses a shocking question: hardware concerns and precedence aside, what would prevent Apple from making future versions of the entire file-system itself flat? Thus freed from the limitations of directory hierarchies, file-management interfaces such as the Finder would gain the ability to manage files across systems, devices, and the cloud simultaneously while removing the need for the specialized file-management software we use today. In such a system, the only location that would matter is which system or device the file lives on; users basic and expert are otherwise freed of the omnipresent concern, “In what folder is my file located?”.

Case Study: the Success of iTunes

iTunes is arguably the most successful piece of software that Apple has ever released. Along with Quicktime, it is the only Apple desktop software to successfully penetrate the massive Windows consumer market, and its popularity builds success throughout the iPod/iTunes/iTunes Music Store ecosystem; it now looks as though the iTunes Music Store is poised to capture a quarter of the worldwide music market by 2012.

Obviously, the popularity of iTunes is vital to Apple’s place in the entertainment industry. Yet, iTunes’ success is not simply a result of the iPod halo effect. What has compelled millions of iPods users to use iTunes as the hub for their digital files over Windows Media Player, WinAMP, or MusicMatch Jukebox, beyond syncing to the iPod? In my view, the software’s success derives from three key features:4 Though file-management is vital to the interface, apparently its less so to the marketing: “iTunes is a free application for Mac and PC. It plays all your digital music and video. It syncs content to your iPod, iPhone, and Apple TV. And it’s an entertainment superstore that stays open 24/7.” -Apple.com

  1. Its integration with Apple’s popular peripheral hardware; and,
  2. Its role as a media-file management application; and,
  3. It’s ability to connect to information across local networks and the internet.

All three of these features share a commonality: the grounding of the iTunes interface in file-management,[4] in information design. It is file management, it is iTunes’ ability to traverse, manage, and categorize one’s media files that allowed it to capture the market upon its release, and that is largely responsible for the continued success of the iPod and iTunes Music Store today.

This file-management core is deeply grounded in the history of iTunes’ development and its place in Mac OS X. As a file-management application, iTunes is an audio-visual specific extension of the Finder, a program designed to manage music and movie files the way the Finder manages the rest of a computer’s files. All of iTunes’ other functions – file playback, online browsing, ripping and burning compact discs, audio visualization, audio quality optimization, and so forth – all rely on this first, core purpose.[5] iTunes’ file-management functionality is the reason for its success.5 Online store? Impossible to reach the mass-user market without integration into a users’s media file-management application. Ripping or burning compact discs? Highly dependent on the user’s ability to locate & organize the necessary files.

History of the iTunes Interface

Much of the coverage devoted to the iTunes interface in the past has centered on how the interface is skinned, and whether or not the current iteration conforms to Apple’s own Human Interface Guidelines (the controversies over the “Brushed Metal” phase come to mind). The importance of its skin, however, only penetrates so deep: in fact, it was Apple’s decision to focus iTunes’ interface on file-management that differentiated iTunes from its competitors.

At the time of its release, this emphasis was peculiarly different from its primary competitors, whose interfaces often used separate windows for audio controls and file browsing, and were highly focused on the skinning of these windows with user-made designs.

Screenshot of Winamp 1.0
Winamp Then

These interfaces had their foundations in early audio players, designed to play compact discs and navigate a limited number of tracks. While an emphasis on the controlling window gave rise to some gorgeously designed skins, they were also a real usability problems: finding, organizing, and manipulating a directory of music necessarily took place in a second, separate window, and jumping between the two was universally awkward and unintuitive.

Early audio players and organizers made by Apple, such as AppleCD Audio Player and the Quicktime Drawer in Macintosh Systems 7-9, as well as their first MP3 application, the Music Player in the Mac OS X Public Beta, all took this partitioned approach, emphasizing controls over browsing, and regulating browsing to a separate window.

Screenshot of Apple Music Player
Apple Music Player in Mac OS X Public Beta

iTunes, in contrast, took the product it was built on – Cassidy & Greene’s Soundjam MP – and actually stripped the skinning functionality out, eschewing visual diversity for Apple’s simple-yet-intuitive file-browsing interface.

Screenshot of iTunes 1.0
iTunes Version 1.0

The story of iTunes’ evolution out of Soundjam (and not Audion) has been well documented, as has the versioned iterations iTunes has gone through over the past 5 years. As far as I know, however, little has been published about the actual development of the application. Was the interface of iTunes driven by the necessity that the software function as the file-browser for the iPod? Or, was this emphasis on file-management a prescient awareness of the growing size of digital media libraries? Either way, this decision has helped shape the course of the iTunes/iPod/Music Store economy since the beginning, and will continue to do so in the future with the iPhone, Apple TV, and whatever else is incubating in Apple’s design laboratories.

Screenshot of Winamp 1.0
Winamp Now

Though competitors develop APIs to harness the information of the cloud and add functionality, iTunes, as the center of Apple’s digital hub, continues to perform its function more elegantly and with greater usability than its competitors, despite increasingly complex functionality and an outdated name. In iTunes, the functionality of the software has been carefully edited to maintain the usability of the interface for the vast range of demographics that make up iTunes’ user base.

Increasing Complexity

iTunes stands apart[6] in Apple’s lineup of media applications as one devoted primarily to file location and organization, instead of file creation and/or manipulation. This makes sense: file-management is something that manipulation software often integrates, but rarely succeeds at entirely. Adobe’s solution, for example, has been to break this functionality out into Adobe Bridge, a separate application.6 iPhoto, Aperture, and others also to manage files, but unlike iTunes they are also manipulation software.

This may seem like a trivial distinction, but with the release of Mac OS X Leopard iTunes’ file-management functionality now represents a possible next step in metadata implementation within the Finder itself. Indeed, Leopard’s new metadata features – the ability to create “Smart Folders”, search with metadata filters, and open files via file type Smart Folders in the Open Dialog Box – all allow the Finder to manipulate file system metadata the way that iTunes has been manipulating document metadata since its very first version.[7]

Screenshot of Leopard Open Dialog
Leopard’s “Open File Dialog” Incorporates Document Metadata

So, while Leopard’s Finder has gained the ability to search & collate a file’s document metadata, it continues to lack the ability to manipulate document metadata, functionality that remains delegated to special-purpose applications like Mail, iPhoto, and iTunes.7 “Now browsing the files on your Mac is as easy as browsing music in iTunes. That’s the idea behind the new Finder in Leopard.”-Apple.com

Yet, as these applications balloon – as Mail takes on RSS, as iPhoto adds complex manipulation and publishing features, and as iTunes has expanded beyond music into movies (though not DVDs), TV shows, and documents like PDFs – the need for a slimmer, quicker, and unified interface solution to the searching and manipulation of document metadata in Mac OS X has become apparent. What is the purpose of withholding document metadata manipulation from the Finder when the location, organization, and launching of a file is the Finder’s very purpose? Document metadata is a natural extension of the file organization and file traversing processes, as evidenced by iTunes, which by default organizes music files into folders by Artist and Album, attributes that are both ID3 metadata tags – document metadata tags.

Or if, alternately, the goal is to break document metadata file management out into specialized applications, why not create similar applications to manage PDFs, word processing documents; even applications themselves? The ability to add and manipulate customized document metadata in files beyond audio and video would bring desktop management & search more inline with tagging and RSS functionalities found online, and would de-solidify the data stored on a personal computer for even basic users. The ability to manipulate and group files without the risk of permanently destroying their hierarchy is as vital for libraries of applications, office documents, and preferences as it is for email, music, and images.

Ultimately, this issue goes beyond the personal computer: it is also vital to the Mac OS’s ability to integrate with the cloud. The iTunes Music Store was Apple’s first mass step into making documents in the cloud available to consumers,[8] and while its success is a testament to Apple’s interface decisions, it is also an unsustainable direction for the Finder in a future based in the integration of Desktop Software with cloud computing and online services.8 iDisk lives in the cloud, but only stores files generated locally.

Unfortunately, iTunes’ success on Windows increasingly limits Apple’s options to re-envision iApps and the Finder as one ecosystem, as the necessity to keep feature parity across operating systems makes Apple dependent on an interface that averages its integration with both systems.

Imagining A Future in the Finder

As I stated in the beginning, the release of Mac OS X Leopard brought vast improvements to the integration of file system metadata into the Finder. Specifically, Spotlight, “Smart Folders”, and advanced metadata search filters all allow quick and powerful access to file system metadata. These make the Finder more powerful, and in doing so raise questions about the role of iTunes in Mac OS X itself.

Specifically they ask: what is it about music & video files (and photos in iPhoto, and email in Mail, and vcards in Address Book, and so on) that requires special treatment? Why can’t users of Mac OS X organize the other documents on their hard drives – office documents, browser bookmarks, applications, and so on – with the same metadata enriched power? If the purpose of the Finder is to find, organize, and launch files and applications, why do users have to open an Application bloated with extraneous functionality – iTunes – to find, organize, and launch media files? The Finder already recognizes ID3 tags – get info on an .mp3 file and you’ll see information like artist, genre, etc. – it just doesn’t fully integrate this sort of metadata into its file browser.

These problems are compounded by the inconsistent treatment of media files across iApps. iTunes stores music files organized in folders by artist > album, but iPhoto by default stores my photos in one inaccessible “library” file? It is obvious to anyone who’s tried to manage their own media files that Apple clearly intends iTunes and iPhoto to be Finder replacements when it comes to their respective file types, not Finder supplements. So, what keeps Apple from going a step farther and turning a “Music” smart folder into a primary interface for accessing music files? If Apple implements document metadata into the Finder, the “All Music” Smart Folder will essentially have become the Source List in iTunes, and any other Smart Folders pointing to audio files will have become Smart Playlists.

I am not advocating the dissolution of iTunes; its increasingly complex function in the iPod/iTunes/Music Store ecosystem and the necessity of keeping file-management functionality in an application on Windows means doing away with iTunes would make little market sense. That doesn’t, however, prevent Apple from adding the functionality I’m talking about into the Mac OS Finder, set up in such a way that changes in one application would automatically be reflected in the other.

How Would this Integration Work?

Part II of this article will investigate possible Finder interfaces for Mac OS 11 that would treat your files as a flat file system.

In the short-term, however, the answers to these problems can be found in how the Finder currently handles metadata; the obvious choice for duplicating iTunes functionality in the finder would be to limit metadata-enabled listing and filtering to Leopard’s “Smart Folders”, leaving regular directory folders safe and secure with their file system metadata as always. This way the users most likely to be confused by these additions are sheltered from the storms of change, as they are the least likely to be using the “Smart Folder” functionality anyway

Indeed, enhancing Smart Folder would go a long way toward moving document metadata functionality – such as iTunes’ – into the Finder, the core of which is laid out by Apple here; I’ve added/subtracted some things to get the following list of features available on the Desktop version of the application:

  1. Jukebox
    1. Finding files (source list)
    2. Browsing files
    3. Playing files
    4. Organizing files into Playlists
    5. Editing file metadata (Artist, Album, etc.)
    6. CD Burning
    7. CD Ripping
    8. Visualization
    9. Audio Optimization (Equalizer, Gapless Playback, etc.)
  2. Store
    1. Music
    2. Movies
    3. TV Shows
    4. Applications
    5. Podcasts
    6. Audiobooks
    7. iPod Games
    8. Ringtones
    9. iPhone App Store
  3. Synching
    1. iPod
    2. iPhone
    3. Apple TV

The “Jukebox” functionality of iTunes is the easiest of this list to imagine in the Finder, as it is the most similar to Finder’s role with other files in the Mac OS.

  1. 1a. Finding files (source list)
  2. 1b. Browsing files
  3. 1c. Playing files
  4. 1d. Organizing your files into Playlists

Of these four things, only one was possible in Mac OS 10.4 – finding files when searching by file type. The inclusion of “Smart Folders” in Mac OS X Leopard, however, has changed that; this new feature easily duplicates these four functions. Using Smart Folders, one can easily create a folder that looks for all audio files (1a), then browse the results and open the files you want to open (1b). What Leopard doesn’t support yet is displaying audio- and video-file specific metadata: so, no browsing by album or artist.

With the inclusion of document metadata support in the Finder, however, the ability to create and maintain playlists is as simple as creating a Smart Folder that filters for the specific File Names or File Paths that you drop into the playlist. This would be a more specialized, less search-related use of Smart Folders, but should prove as equally useful (1c).

Actually playing audio files in the Finder itself, however, would be a monumental conceptual step for the Finder, and is one that is far less relevant to the current investigation, as it would upset the entire document / application paradigm. However, a change of this type wouldn’t be without precedent: Mac OS X’s Dashboard is one example of application functionality within the system; the inclusion in Leopard of “Quick Look”, or a more accessible and fully-featured extension of the preview capabilities previously available in “column view”, gave the Finder the ability to playback audio files, video files, and even display digital images fullscreen and in slideshow mode without launching a separate application. This expansion of the Finder, which is otherwise wholly dedicated to organizing and traversing file directories, grants it the functionality needed to emulate iTunes’ playback abilities (1d) if it so desired. What would remain to be seen, however, is how the Finder’s interface could incorporate such functionality without radical concessions to usability.

  1. 1e. Editing file metadata

Along with the ability to show document metadata, the ability to edit document metadata is another core functionality needed to allow the Finder the same power as iTunes (and iPhoto, Mail, Address Book, etc.).

  1. 1f. CD Burning
  2. 1g. CD Ripping

These two functions have been in the Mac OS since OS 8; tying them in automatically to the proper folders via the OS would be the only change necessary to achieve feature parity with iTunes 7.

  1. 1h. Visualization
  2. 1i. Audio Optimization

Visualization isn’t currently supported outside of iTunes; however, I’d image it would be simple to implement into the interface as a button, perhaps similar to the Quick Look button. Similarly, application-specific audio manipulation is not currently supported by Leopard; Microsoft has included this in Vista, however, so it’s not a stretch to image Apple doing the same in an update.

  1. 2a-h. Store

The iTunes Music Store is perhaps the biggest hurdle, beyond designing a usable interface, standing between iTunes functionality and the Finder. However, it’s relatively easy to imagine Apple making the store browser-accessible (finally), and building some sort of plug-in that would intercept iTunes Music Store purchases and route them to the appropriate folder. Alternately, and preferrably for the OS’s long-term health, Apple could implement it as a “cloud service”, similar to iDisk or Networked Drives, and it’s interface could more closely mimic that of the Finder’s.

  1. 3a-c. Synching

With media files being organized primarily in the Finder, synching to an iPod or iPhone would be almost identical to how it operates today.

So, running through the features, there are a few changes that need to happen in order to see this sort of powerful functionality integrated into the Finder:

  1. The ability to browse document metadata directly in the Finder
  2. The ability to edit file-type specific metadata directly in the Finder
  3. A User Interface that combines metadata browsing with basic media playback functionality – perhaps a second toolbar, collapsible like the current toolbar is with the White button in the upper-right-hand corner of every Finder window.
  4. A built-in file-type specific directory pointer: the ability to tell the Finder and all Mac OS X applications where audio, video, and image files should be stored, and what part of their metadata should be used to set the containing folder’s name. This would have to be migrated out of the applications, where it currently resides, into the Finder and made a universally accessible system preference.

What Would it Look Like?

Though the Finder interface is primarily for organizing files, it also performs a limited number of actions: creating a new folder, opening a file, burning discs, emptying the trash, etc. While the interfaces for these actions are spread out and inconsistent – and so problematic precedents for migrating further actions (such as file playback) into the Finder – one minor but interesting change in Leopard was the addition of an “Empty Trash” button directly in the Trash finder window itself (a change mimicked in Firefox 3′s pop-up and add-on notification bars).

Screenshot of Trash
10.5 Trash Window, w/Empty Bar & Button

As far as I know, this is a largely unprecedented exception to the otherwise uniform Finder window design, and its addition along with Apple’s in-window “Find” UI, signals a small but welcome change in approach to the Finder. First, functionality crept from the menubar into contextual menus. Then, search functionality crept from a separate application – Sherlock – into Finder windows. Now, contextual actions are poised to creep into relevant windows themselves.

Screenshot of Find
10.5 Find Window, w/Search Filters Interface

Such contextual, non-modal controls are extremely useful, and their proliferation in select places have improved the user experience in the Finder dramatically.

With such success, why not explore integrating other file actions into the Finder? What, for example, would file playback functionality – such as that in Quick Look – look like in a Finder window, using Music/Photo/Video specific Smart Folders as a launching point?

Screenshot possible iTunes/Finder Mashup
Rough Mashup of basic iTunes Controls (already in QuickLook) and a Finder Window

Then, we can take the concept a step further and start trying to roughly integrate more advanced iTunes and iPhoto functionality – like playlists and Events, for example – and move them into the Finder as Smart Folders accessible from any Finder window.

Rough Mashup of iTunes and the Finder
Rough Mashup of iTunes and the Finder
Rough Mashup of iPhoto and the Finder
Rough Mashup of iPhoto and the Finder
Rough Mashup of iTunes Video and the Finder
Rough Mashup of iTunes Video and the Finder

Is this a Good Idea?

Improvements to iTunes’, Aperture’s, and other Apple created applications’ file-browsing interfaces have already bled down into the interface in the Finder: Cover Flow, zebra-striping, smart playlists, and the finder sidebar are all examples. With these past successes as a guide, why not tie the integration in more tightly? iTunes currently duplicates features already in the Finder or other Core Mac OS X applications: device management, download-management, internet-browsing, and movie-playback. The difference is that iTunes manages files with document metadata.

It is obvious that Apple would like to consolidate most of it’s media-management services into one application, and that iTunes has become the focus of this effort. Instead, why not move this functionality into the file-management application itself – the Finder – and then offer a set of lightweight, core media playing applications, or perhaps one light-weight pan-media-playing application (combining Quicktime, iTunes playback, DVD Player, and the browsing and viewing capabilities of iPhoto) into one place? Then, separately, Apple could offer a set of file-manipulation applications: iMovie, the photo-manipulation features of iPhoto, iDVD, etc., all of which focus around the Finder as primary-file browser and file interface.

Ultimately, as long as Mac OS X remains a directory-based file system document metadata will always be in conflict with directory metadata in the Finder interface. While the two-view paradigm can be tweaked and improved dramatically to better take advantage of document metadata, it is my opinion that only the total dissolution of inter-system directories will free the true power of custom document metadata and bring about the next revloution in operating system interfaces – the revolution that will bring equality to all devices locally, in the cloud, and beyond.

An interface that confronts the a directory-less Mac OS will be the focus of Part II of this article.